Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649) (#3827)

* Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649)

RawSqlBuilder.parse() uses keyword-based scanning to locate SELECT
columns and the WHERE/HAVING injection points. This fails for CTEs,
window functions and other complex SQL where "select"/"from" keywords
appear in places the scanner doesn't expect.

withPlaceholders(sql) skips SELECT/FROM column parsing entirely and
only locates the ${where}, ${andWhere}, ${having} and ${andHaving}
placeholder positions, requiring explicit columnMapping() calls (as
with unparsed()). This lets dynamic where()/having() expressions be
injected into otherwise unparseable SQL.

Also fixes two bugs in the underlying placeholder-position splitting:
- static SQL following a ${having}/${andHaving} placeholder (e.g. a
  trailing ORDER BY) was silently dropped when both a where and a
  having placeholder were present
- using only ${having}/${andHaving} (no where placeholder) caused a
  dynamically added HAVING clause to be appended after trailing
  static SQL, producing invalid SQL

Changes:
- RawSqlBuilder.withPlaceholders(sql) + SpiRawSqlService.withPlaceholders()
- DRawSqlParser.parseAsTemplate() / parseTemplate() - placeholder-position
  only parsing, correctly splitting preWhere/preHaving/trailing SQL
- CQueryBuilderRawSql - skip column-list and "select" prefix handling
  in template mode (signalled by an empty preFrom)
- Unit tests in DRawSqlServiceTest covering where/andWhere/having/andHaving
  placeholder combinations, including the two fixed edge cases
- Integration tests in TestRawSqlWithPlaceholders (ebean-test) covering
  CTE queries with dynamic where/having and verifying generated SQL

* Add examples test using query beans

* Add docs guides for RawSql

* Add ${orderBy} ${andOrderBy}

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
This commit is contained in:
Rob Bygrave
2026-07-03 18:22:42 +12:00
committed by GitHub
co-authored by robin.bygrave
parent 29647ebe1a
commit fc78b2af80
13 changed files with 1385 additions and 17 deletions
+1
View File
@@ -48,6 +48,7 @@ existing Maven project. Complete the steps in order.
|-------|-------------|
| [Write Ebean queries with query beans](writing-ebean-query-beans.md) | Step-by-step guidance for AI agents to write type-safe Ebean queries; choose the right terminal method; tune `select()` / `fetch()` / `fetchQuery()`; and project to DTOs when entity beans are not the right output |
| [Immutable bean cache for read-only references](immutable-bean-cache.md) | Use `ImmutableBeanCache` and `ImmutableBeanCaches.loading(...)` to resolve assoc-one references in read-only/unmodifiable queries, including secondary `fetchQuery`/`fetchLazy` loads |
| [Using `RawSql` with Ebean](using-rawsql-with-ebean.md) | Choose between `RawSqlBuilder.parse()`, `unparsed()`, and `withPlaceholders()`; the `${where}`/`${andWhere}`/`${having}`/`${andHaving}` placeholder reference for CTEs, window functions, and subqueries; column mapping; and using `RawSql` with query beans |
## Persisting & transactions
+464
View File
@@ -0,0 +1,464 @@
# Guide: Using `RawSql` with Ebean
## Purpose
`RawSql` lets you back an Ebean bean with a **hand-written SQL query** instead of
Ebean generating the SQL from the entity mapping. Ebean still handles object
mapping (result set columns → bean properties), lazy loading of associated beans,
and - depending on how the `RawSql` is built - dynamic `WHERE`/`HAVING` predicates
added through the normal query API.
Use this guide when you need to:
- run vendor-specific SQL, complex aggregation, or reporting queries that don't
map cleanly to an ORM query
- reuse a hand-tuned query but still want typed/dynamic predicates, paging, or
`ORDER BY` added by the caller
- back a query bean (`Q*`) or DTO-like bean with SQL containing a CTE, window
function, or subquery in the `FROM` clause
Prefer ordinary query bean queries first - see
[Write Ebean queries with query beans](writing-ebean-query-beans.md), Step 9,
for the full decision order (query bean → `asDto()` → DTO query → raw SQL).
This guide covers raw SQL once you've decided it's the right tool.
---
## The bean behind a `RawSql` query
A bean queried with `RawSql` is not necessarily backed by a physical table. Annotate
it `@Entity @Sql` to tell Ebean it is mapped via `RawSql` rather than table DDL:
```java
@Entity
@Sql
public class OrderAggregate {
@OneToOne
Order order;
Double totalAmount;
Long totalItems;
// getters/setters
}
```
`@Sql` beans still get a generated query bean (`QOrderAggregate`) if the
querybean-generator annotation processor is configured - see
[Using `RawSql` with query beans](#using-rawsql-with-query-beans) below.
You can also query an ordinary table-backed `@Entity` with `RawSql` - the column
mapping just needs to line up with that entity's properties.
---
## Building a `RawSql` - three factory methods
`RawSqlBuilder` has three ways to construct a `RawSql`, depending on how much of
the SQL Ebean needs to understand:
| Method | SELECT columns parsed? | Dynamic WHERE/HAVING/ORDER BY? | Use for |
|--------|------------------------|------------------------|---------|
| `RawSqlBuilder.parse(sql)` | Yes | Yes | Ordinary `SELECT ... FROM ... WHERE ...` statements |
| `RawSqlBuilder.unparsed(sql)` | No | No | Fixed SQL that never needs additional predicates |
| `RawSqlBuilder.withPlaceholders(sql)` | No (explicit `columnMapping()` required) | Yes, via `${where}` / `${andWhere}` / `${having}` / `${andHaving}` / `${orderBy}` / `${andOrderBy}` | CTEs, window functions, subqueries - SQL that keyword-based parsing can't handle |
### `parse(sql)` - the common case
`parse(sql)` scans the SQL text for the `select` / `from` / `where` / `group by`
/ `having` / `order by` keywords to work out the SELECT column list (so it can
validate your column mappings) and the injection points for dynamic `WHERE`/
`HAVING` expressions.
```java
RawSql rawSql = RawSqlBuilder.parse(
"select c.id, c.name, c.status from customer c")
.columnMapping("c.id", "id")
.columnMapping("c.name", "name")
.columnMapping("c.status", "status")
.create();
List<Customer> customers = DB.find(Customer.class)
.setRawSql(rawSql)
.where().eq("status", Customer.Status.ACTIVE)
.orderBy("name")
.findList();
```
Because the SQL is parsed, mistakes in `columnMapping()` (unknown column, wrong
order for `unparsed`-style mappings) are caught early. **This fails on SQL the
keyword parser can't make sense of** - a `WITH` CTE, a window function, a
subquery in `FROM`, etc. - because the keyword positions found don't correspond
to the outer query's real structure. Use `withPlaceholders(sql)` for that SQL
instead (see below).
### `unparsed(sql)` - fixed queries
`unparsed(sql)` skips all parsing. The SQL is used exactly as written, and **no
further `WHERE`/`HAVING`/`ORDER BY` can be added** by the caller - useful for a
completely fixed reporting query with no caller-supplied filtering.
```java
RawSql rawSql = RawSqlBuilder.unparsed(
"select id, name, status from customer where status = 'ACTIVE'")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = DB.find(Customer.class)
.setRawSql(rawSql)
.findList();
```
Column mappings for `unparsed(sql)` must be supplied **in the same order** as
the columns appear in the SQL, since there's no parsing to match them by name.
### `withPlaceholders(sql)` - complex SQL (CTEs, window functions, subqueries)
`withPlaceholders(sql)` avoids keyword scanning entirely. You mark exactly where
a dynamic `WHERE`/`HAVING`/`ORDER BY` expression should be injected using
placeholder tokens, and column mappings are always explicit (as with `unparsed`).
#### Placeholder reference
| Placeholder | Meaning | Use when |
|-------------|---------|----------|
| `${where}` | Insert a new `WHERE <expr>` clause here | No static `WHERE` clause exists yet at this point in the SQL |
| `${andWhere}` | Insert `AND <expr>` here | A static `WHERE ...` clause already exists in the SQL and you want to append to it |
| `${having}` | Insert a new `HAVING <expr>` clause here | No static `HAVING` clause exists yet at this point in the SQL |
| `${andHaving}` | Insert `AND <expr>` here | A static `HAVING ...` clause already exists in the SQL and you want to append to it |
| `${orderBy}` | Insert a new `ORDER BY <expr>` clause here | No static `ORDER BY` clause exists yet at this point in the SQL, and callers may supply `.orderBy(...)` |
| `${andOrderBy}` | Insert `, <expr>` here | A static `ORDER BY ...` clause already exists in the SQL and you want callers to be able to append extra sort columns to it |
Rules:
- At least one placeholder is required - `withPlaceholders(sql)` throws
`IllegalArgumentException` if none of the six tokens are present.
- Use only the placeholders you need. Omit `${where}`/`${andWhere}` entirely if
the query never needs a dynamic `WHERE` (e.g. only a dynamic `HAVING` on an
aggregate). Omit `${having}`/`${andHaving}` if there's no dynamic `HAVING`.
Omit `${orderBy}`/`${andOrderBy}` if the ordering is always fixed.
- Explicit `columnMapping()` is required for every returned column - there is no
column-list parsing to infer names from.
- **A caller-supplied `.orderBy(...)`/`.order(...)` is only applied if the SQL
contains an `${orderBy}` or `${andOrderBy}` placeholder.** Without one of
those placeholders there is no defined injection point for dynamic ordering,
so any `.orderBy(...)` call on the query is safely ignored rather than risk
producing invalid SQL - even if the template has a static trailing
`ORDER BY ...` of its own. If you need callers to be able to influence
ordering, add `${orderBy}` (no existing static order by) or `${andOrderBy}`
(append after an existing static order by).
- Any other static SQL that follows a `${where}`/`${having}` placeholder (e.g.
a trailing `GROUP BY`) is preserved and correctly positioned **after**
whatever dynamic expression gets injected at that placeholder.
#### Example - CTE with `${where}`
```java
String sql = """
with order_totals as (
select o.id as order_id, sum(d.qty * d.unit_price) as total_amount
from o_order o
join o_order_detail d on d.order_id = o.id
group by o.id
)
select order_id, total_amount
from order_totals
${where}
order by order_id
""";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 100)
.findList();
```
`total_amount` is a genuine column of the `order_totals` CTE here, so it's valid
to filter on it in the outer `WHERE` - this only works because the aggregate is
computed inside the CTE rather than as a same-level `SELECT` alias.
#### Example - static `WHERE` already present, append with `${andWhere}`
```java
String sql = "... from order_totals where total_amount > 0 ${andWhere} order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... where total_amount > 0 and total_amount > ? order by order_id
DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 100)
.findList();
```
#### Example - `${having}` only, filtering on an aggregate directly
No `WHERE` placeholder is needed if you only ever filter on the aggregate value:
```java
String sql =
"select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
" ${having}" +
" order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.having().gt("totalAmount", 100)
.findList();
```
The dynamic `HAVING` clause is injected before the static trailing `ORDER BY`,
even though `${having}` is the only placeholder present. Because there's no
`${orderBy}`/`${andOrderBy}` placeholder here, a caller-supplied `.orderBy(...)`
would be ignored - the ordering stays fixed as `order by order_id`.
#### Example - both `${where}` and `${having}`
```java
String sql =
"select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" ${where}" +
" group by o.id" +
" ${having}" +
" order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("order.id", 0)
.having().gt("totalAmount", 50)
.findList();
```
Both the dynamic `WHERE` and dynamic `HAVING` are injected at their respective
placeholders, and the trailing `order by order_id` is preserved after the
`HAVING` clause.
#### Example - `${orderBy}`, fully dynamic ordering
Use `${orderBy}` when there's no static default ordering and you want the
caller's `.orderBy(...)` to control it entirely:
```java
String sql =
"with order_totals as (" +
" select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount from order_totals" +
" ${where}" +
" ${orderBy}";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... where total_amount > ? order by total_amount desc
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 0)
.orderBy("totalAmount desc")
.findList();
```
If the caller doesn't call `.orderBy(...)`, nothing is injected at `${orderBy}`
and no `ORDER BY` clause is emitted at all.
#### Example - `${andOrderBy}`, appending to a static default ordering
Use `${andOrderBy}` when there's a sensible static default ordering but you
want callers to be able to add extra tie-breaker sort columns:
```java
String sql =
"... from order_totals" +
" ${where}" +
" order by total_amount desc ${andOrderBy}";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... order by total_amount desc , order_id
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 0)
.orderBy("order.id")
.findList();
```
---
## Using `fetchQuery()` to build out more of the graph
A `RawSql` query can be the **root query** and still use `fetchQuery(path)` the
same way an ordinary ORM query does - Ebean runs the raw SQL for the root rows,
then runs additional secondary ORM queries to populate the requested paths. This
lets you hand-write only the part of the query that needs raw SQL (e.g. an
aggregate/CTE) and let the ORM build out the rest of the object graph normally.
```java
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql) // root query - runs the CTE/aggregate SQL
.fetchQuery("order") // secondary query - loads the full Order
.fetchQuery("order.details") // secondary query - loads Order.details
.where().gt("totalAmount", 50)
.findList();
```
This executes **three** queries: the raw SQL root query, then one secondary
query per `fetchQuery(path)` call.
**Important**: if the raw SQL's column mapping only populates part of an
association (e.g. only `order.id`, as in the examples above), that association
is a *partial reference*. To load a nested to-many under it (e.g.
`order.details`), you must add an explicit `fetchQuery(...)` (or `fetch(...)`)
for the **intermediate path** (`order`) as well as the nested path
(`order.details`) - `fetchQuery("order.details")` alone will leave `details` as
a deferred/lazy collection, because Ebean doesn't otherwise have a fetch node
for `order` to hang the secondary query off. If the raw SQL already selects the
full set of columns for an association directly (no partial reference), this
extra step isn't needed.
This is the same `fetchQuery()` mechanism used for ordinary query bean queries -
see [Use `fetchQuery()` for to-many paths](writing-ebean-query-beans.md#step-7---use-fetchquery-for-to-many-paths-and-fetchgroup-for-reusable-query-shapes)
for background on why to-many paths are loaded via secondary queries rather than
a single joined query.
---
## Column mapping
Every `RawSqlBuilder` (except a bare `unparsed(sql)` with implicit positional
mapping) uses `columnMapping(dbColumn, propertyName)` to map SQL result columns
to bean properties:
```java
.columnMapping("order_id", "order.id") // maps to the "order" association's "id" property
.columnMapping("total_amount", "totalAmount")
```
- Dotted property paths (e.g. `"order.id"`) map a column into a nested/associated
bean property.
- `columnMappingIgnore(dbColumn)` marks a selected column as intentionally unmapped
(present in the SQL but not needed on the bean).
- `tableAliasMapping(tableAlias, path)` bulk-renames every mapping using a given
SQL table alias to be prefixed with a bean property path - handy when a `parse()`
query selects many columns from a joined table (e.g. alias `c` → path `customer`)
and you don't want to repeat the prefix in every `columnMapping()` call.
---
## Using `RawSql` with query beans
`RawSql` is not limited to the plain `Query<T>` API - it also works with a
generated query bean, giving type-safe `where()`/`having()`-equivalent
expressions (as bean properties) over hand-written SQL. Every generated query
bean exposes `setRawSql(...)`:
```java
RawSql rawSql = RawSqlBuilder.parse("select id, name, status from customer")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = new QCustomer()
.setRawSql(rawSql)
.status.equalTo(Customer.Status.ACTIVE) // typed expression, injected into the parsed WHERE clause
.findList();
```
This also works with `withPlaceholders(sql)` and an `@Sql` query bean:
```java
List<OrderAggregate> list = new QOrderAggregate()
.setRawSql(rawSql) // built with withPlaceholders() as shown above
.totalAmount.gt(100)
.findList();
```
The typed property expression (`.totalAmount.gt(100)`) is translated to a bound
predicate and injected at the `${where}`/`${having}` placeholder position, exactly
as `.where().gt("totalAmount", 100)` would be on the plain `Query<T>` API.
---
## Common anti-patterns
### Anti-pattern 1 - reaching for raw SQL before trying a query bean
Complex-looking joins are often just ordinary association traversal in a query
bean. Don't use raw SQL just because a query touches several tables - see
[Write Ebean queries with query beans](writing-ebean-query-beans.md).
### Anti-pattern 2 - using `parse(sql)` on a CTE or window-function query
`parse(sql)` will throw a parsing exception (or silently mis-locate the WHERE
injection point) on SQL it can't understand structurally. If your SQL starts
with `WITH ...` or has a subquery in `FROM`, use `withPlaceholders(sql)` instead.
### Anti-pattern 3 - filtering on a same-level SELECT alias
You cannot add a dynamic `WHERE` predicate on a `SELECT`-clause alias in the
same query level (e.g. `select sum(x) as total ... ${where}` - `total` isn't a
real column yet at the `WHERE` stage of that query level). Either:
- move the aggregation into a CTE and filter on the CTE's output column in the
outer query (`WHERE` case), or
- use `${having}`/`${andHaving}` to filter on the aggregate at the `HAVING` stage
of the same query level, where the aggregate expression is valid.
### Anti-pattern 4 - forgetting `columnMapping()` with `unparsed()`/`withPlaceholders()`
Both `unparsed(sql)` and `withPlaceholders(sql)` require **every** returned
column to be explicitly mapped (or explicitly ignored via
`columnMappingIgnore(...)`) - there's no column-list parsing to infer them.
---
## Troubleshooting
| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| `RuntimeException: Error parsing sql, can not find ... keyword` | `parse(sql)` used on SQL with a CTE, window function, or subquery in `FROM` | Use `RawSqlBuilder.withPlaceholders(sql)` instead |
| `IllegalArgumentException: withPlaceholders() requires at least one of ${where}, ${andWhere}, ${having}, ${andHaving}, ${orderBy}, ${andOrderBy}...` | None of the six placeholder tokens were found in the SQL | Add the appropriate placeholder token at the injection point |
| Dynamic `WHERE`/`HAVING` predicate silently has no effect, or query throws | Used `unparsed(sql)` and then tried to add a predicate | `unparsed(sql)` queries cannot be modified - switch to `parse(sql)` or `withPlaceholders(sql)` |
| Generated SQL is invalid / clauses appear in the wrong order | Predicates added via `.where()`/`.having()` don't match the placeholders actually present in the SQL | Make sure `${where}`/`${having}` (or the `and` variants) exist at the point you expect predicates to be injected |
| `.orderBy(...)`/`.order(...)` on the query silently has no effect | The SQL has no `${orderBy}`/`${andOrderBy}` placeholder | This is by design - without one of those placeholders there's no defined injection point, so the ordering is ignored rather than corrupting the SQL. Add `${orderBy}` or `${andOrderBy}` if you need caller-controlled ordering |
| `Unknown column` / unmapped property error | Missing `columnMapping()` for a selected column | Add a `columnMapping(...)` or `columnMappingIgnore(...)` for every SQL column |
| `fetchQuery("a.b")` collection stays deferred/lazy | `a` is a partial reference from the raw SQL column mapping (e.g. only `a.id` mapped), and there's no fetch node for `a` itself | Add `fetchQuery("a")` (or `fetch("a")`) alongside `fetchQuery("a.b")` |
---
## Related documentation
- [Write Ebean queries with query beans](writing-ebean-query-beans.md)
- [Derived / formula properties (`@Formula`, `@Formula2`)](derived-formula-properties.md)
- [Ebean query docs](https://ebean.io/docs/query/)
+25
View File
@@ -483,6 +483,30 @@ Prefer the following order:
Do **not** jump to raw SQL just because the query joins multiple tables. Query
beans already handle ordinary relationship traversal well.
### Using `RawSql` with query beans
`RawSql` is not limited to the plain `Query<T>` API - it also works with a
generated query bean, giving type-safe `where()`/`having()` expressions over
hand-written SQL. Every generated query bean exposes `setRawSql(...)`:
```java
RawSql rawSql = RawSqlBuilder.parse("select id, name, status from customer")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = new QCustomer()
.setRawSql(rawSql)
.status.equalTo(Customer.Status.ACTIVE) // typed expression, injected into the parsed WHERE clause
.findList();
```
For the full guide to building `RawSql` - including `unparsed()`,
`withPlaceholders()` for CTEs/window functions, the `${where}` / `${andWhere}`
/ `${having}` / `${andHaving}` placeholder reference, and column mapping - see
[Using `RawSql` with Ebean](using-rawsql-with-ebean.md).
---
## Common anti-patterns
@@ -570,4 +594,5 @@ When asked to add or modify an Ebean query:
- [Add Ebean Postgres Maven POM](add-ebean-postgres-maven-pom.md)
- [Entity Bean Creation](entity-bean-creation.md)
- [Immutable bean cache for read-only references](immutable-bean-cache.md)
- [Using `RawSql` with Ebean](using-rawsql-with-ebean.md)
- [Ebean query docs](https://ebean.io/docs/query/)
@@ -41,6 +41,49 @@ public interface RawSqlBuilder {
return XBootstrapService.rawSql().unparsed(sql);
}
/**
* Return a RawSqlBuilder for SQL containing {@code ${where}}, {@code ${having}} and/or
* {@code ${orderBy}} placeholder(s). Unlike {@link #parse(String)} this does NOT attempt to parse
* the SELECT columns, so it supports complex SQL such as CTEs, subqueries, and window functions.
* <p>
* Explicit column mappings must be provided (as with {@link #unparsed(String)}), but
* WHERE, HAVING and ORDER BY expressions can be added dynamically via the query API - provided
* the corresponding placeholder is present in the SQL. If a query calls {@code .orderBy(...)}
* on a template with no {@code ${orderBy}}/{@code ${andOrderBy}} placeholder, that ordering is
* ignored (there is no injection point for it) rather than producing invalid SQL.
* </p>
* <p>
* Available placeholders:
* </p>
* <ul>
* <li>{@code ${where}} / {@code ${andWhere}} - inject "where &lt;expr&gt;" / "and &lt;expr&gt;"</li>
* <li>{@code ${having}} / {@code ${andHaving}} - inject "having &lt;expr&gt;" / "and &lt;expr&gt;"</li>
* <li>{@code ${orderBy}} / {@code ${andOrderBy}} - inject "order by &lt;expr&gt;" / ", &lt;expr&gt;"</li>
* </ul>
* <h3>Example:</h3>
* <pre>{@code
*
* String sql = """
* with agg as (
* select company_id, sum(amount) as total
* from orders
* ${where}
* group by company_id
* )
* select company_id, total from agg ${orderBy}
* """;
*
* RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
* .columnMapping("company_id", "companyId")
* .columnMapping("total", "total")
* .create();
*
* }</pre>
*/
static RawSqlBuilder withPlaceholders(String sql) {
return XBootstrapService.rawSql().withPlaceholders(sql);
}
/**
* Return a RawSqlBuilder parsing the sql.
* <p>
@@ -27,6 +27,13 @@ public interface SpiRawSqlService extends BootstrapService {
*/
RawSqlBuilder unparsed(String sql);
/**
* SQL with ${where}/${having} placeholder(s) but no SELECT column parsing.
* Supports complex SQL (CTEs, window functions) where keyword parsing would fail.
* Explicit column mapping is required (as with unparsed).
*/
RawSqlBuilder withPlaceholders(String sql);
/**
* Create based on a JDBC ResultSet.
*
@@ -48,9 +48,12 @@ final class CQueryBuilderRawSql {
// wrap with a limit offset or ROW_NUMBER() etc
return sqlLimiter.limit(new OrmQueryLimitRequest(sql, orderBy, query, dbPlatform, rsql.isDistinct() || query.isDistinct()));
} else {
// add back select keyword (it was removed to support sqlQueryLimiter)
String prefix = "select " + (rsql.isDistinct() ? "distinct " : "");
sql = prefix + sql;
if (hasValue(rsql.getPreFrom())) {
// add back select keyword (it was removed to support sqlQueryLimiter)
String prefix = "select " + (rsql.isDistinct() ? "distinct " : "");
sql = prefix + sql;
}
// else: template mode — SQL is already complete (no keyword stripping was done)
return new SqlLimitResponse(sql);
}
}
@@ -67,10 +70,12 @@ final class CQueryBuilderRawSql {
sb.append(selectProperty);
first = false;
}
} else {
sb.append(sql.getPreFrom());
sb.append(' ');
} else if (hasValue(sql.getPreFrom())) {
// standard parsed mode: column list with "select" prefix added in buildSql()
sb.append(sql.getPreFrom()).append(' ');
}
sb.append(' ');
// else: template mode (preFrom empty) — the full SQL is in preWhere/preHaving, no prefix needed
String s = sql.getPreWhere();
BindParams bindParams = request.query().bindParams();
@@ -126,9 +131,18 @@ final class CQueryBuilderRawSql {
}
sb.append(dbHaving).append(' ');
}
String preOrderBy = sql.getPreOrderBy();
if (hasValue(preOrderBy)) {
sb.append(preOrderBy).append(' ');
}
if (hasValue(orderBy)) {
sb.append(' ').append(sql.getOrderByPrefix()).append(' ').append(orderBy);
}
String postOrderBy = sql.getPostOrderBy();
if (hasValue(postOrderBy)) {
sb.append(' ').append(postOrderBy);
}
return sb.toString().trim();
}
@@ -137,6 +151,12 @@ final class CQueryBuilderRawSql {
}
private String orderBy(CQueryPredicates predicates, SpiRawSql.Sql sql) {
if (!hasValue(sql.getPreFrom()) && !sql.isOrderByPlaceholder()) {
// template mode (withPlaceholders()) without an explicit ${orderBy}/${andOrderBy} placeholder -
// there is no defined injection point for a dynamic order by, so ignore any caller-supplied
// order by rather than risk emitting it at an undefined (and likely invalid) position.
return sql.getOrderBy();
}
String orderBy = predicates.dbOrderBy();
if (orderBy != null) {
return orderBy;
@@ -3,6 +3,10 @@ package io.ebeaninternal.server.rawsql;
import io.ebeaninternal.server.querydefn.SimpleTextParser;
import io.ebeaninternal.server.rawsql.SpiRawSql.Sql;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
/**
* Parses sql-select queries to try and determine the location where WHERE and
* HAVING clauses can be added dynamically to the sql.
@@ -13,6 +17,12 @@ final class DRawSqlParser {
private static final String $_HAVING = "${having}";
private static final String $_AND_WHERE = "${andWhere}";
private static final String $_WHERE = "${where}";
private static final String $_AND_ORDER_BY = "${andOrderBy}";
private static final String $_ORDER_BY = "${orderBy}";
private static final int KIND_WHERE = 0;
private static final int KIND_HAVING = 1;
private static final int KIND_ORDER_BY = 2;
private final SimpleTextParser textParser;
private String sql;
@@ -20,6 +30,8 @@ final class DRawSqlParser {
private int placeHolderAndWhere;
private int placeHolderHaving;
private int placeHolderAndHaving;
private int placeHolderOrderBy;
private int placeHolderAndOrderBy;
private final boolean hasPlaceHolders;
private int selectPos = -1;
@@ -35,11 +47,24 @@ final class DRawSqlParser {
private int whereExprPos = -1;
private boolean havingExprAnd;
private int havingExprPos = -1;
private boolean orderByExprAnd;
private int orderByExprPos = -1;
public static Sql parse(String sql) {
return new DRawSqlParser(sql).parse();
}
/**
* Parse for template mode: finds ${where} / ${having} placeholder positions without
* attempting SELECT/FROM keyword parsing. This supports complex SQL (CTEs, window functions,
* subqueries) where keyword-based parsing would fail.
* <p>
* The caller is expected to provide manual column mappings (like unparsed mode).
*/
public static Sql parseAsTemplate(String sql) {
return new DRawSqlParser(sql).parseTemplate();
}
private DRawSqlParser(String sqlString) {
sqlString = sqlString.trim();
sqlString = sqlString.replace('\n', ' ');
@@ -74,6 +99,8 @@ final class DRawSqlParser {
placeHolderAndWhere = removePlaceHolder($_AND_WHERE);
placeHolderHaving = removePlaceHolder($_HAVING);
placeHolderAndHaving = removePlaceHolder($_AND_HAVING);
placeHolderOrderBy = removePlaceHolder($_ORDER_BY);
placeHolderAndOrderBy = removePlaceHolder($_AND_ORDER_BY);
return hasPlaceHolders();
}
@@ -91,7 +118,8 @@ final class DRawSqlParser {
}
private boolean hasPlaceHolders() {
return placeHolderWhere > -1 || placeHolderAndWhere > -1 || placeHolderHaving > -1 || placeHolderAndHaving > -1;
return placeHolderWhere > -1 || placeHolderAndWhere > -1 || placeHolderHaving > -1 || placeHolderAndHaving > -1
|| placeHolderOrderBy > -1 || placeHolderAndOrderBy > -1;
}
/**
@@ -246,6 +274,73 @@ final class DRawSqlParser {
return -1;
}
/**
* Find the ${orderBy}/${andOrderBy} placeholder position (template mode only).
* Returns -1 if neither placeholder is present in the SQL.
*/
private int findOrderByExprPosition() {
if (placeHolderOrderBy > -1) {
return placeHolderOrderBy;
}
if (placeHolderAndOrderBy > -1) {
orderByExprAnd = true;
return placeHolderAndOrderBy;
}
return -1;
}
private Sql parseTemplate() {
if (!hasPlaceHolders) {
throw new IllegalArgumentException("withPlaceholders() requires at least one of "
+ "${where}, ${andWhere}, ${having}, ${andHaving}, ${orderBy}, ${andOrderBy} in the SQL");
}
whereExprPos = findWhereExprPosition();
havingExprPos = findHavingExprPosition();
orderByExprPos = findOrderByExprPosition();
// Order the placeholder positions found (where/having/orderBy may each be absent) and slice the
// placeholder-stripped SQL into the static text segments that sit between them. Each segment is the
// static SQL that must be emitted immediately after the *previous* placeholder's dynamic expression
// (or as the query prefix, for the very first segment).
List<int[]> markers = new ArrayList<>(3);
if (whereExprPos > -1) markers.add(new int[]{whereExprPos, KIND_WHERE});
if (havingExprPos > -1) markers.add(new int[]{havingExprPos, KIND_HAVING});
if (orderByExprPos > -1) markers.add(new int[]{orderByExprPos, KIND_ORDER_BY});
markers.sort(Comparator.comparingInt(m -> m[0]));
String preWhere;
String preHaving = null;
String preOrderBy = null;
String postOrderBy = null;
if (markers.isEmpty()) {
preWhere = sql.trim();
} else {
preWhere = sql.substring(0, markers.get(0)[0]).trim();
for (int i = 0; i < markers.size(); i++) {
int kind = markers.get(i)[1];
int startPos = markers.get(i)[0];
int endPos = (i + 1 < markers.size()) ? markers.get(i + 1)[0] : sql.length();
String segment = sql.substring(startPos, endPos).trim();
if (kind == KIND_WHERE) {
preHaving = segment;
} else if (kind == KIND_HAVING) {
preOrderBy = segment;
} else {
postOrderBy = segment;
}
}
}
boolean orderByPlaceholder = orderByExprPos > -1;
// preFrom is empty — signals template mode to CQueryBuilderRawSql (no "select" prefix handling).
// For the dynamic order-by prefix/value: only set when a ${orderBy}/${andOrderBy} placeholder was
// actually found - there is no static default order-by value at that placeholder (the placeholder
// is purely a dynamic injection point), so orderBySql is left null.
String orderByPrefix = orderByPlaceholder ? (orderByExprAnd ? "," : "order by") : null;
return new Sql(sql, "", preWhere, whereExprAnd, preHaving, havingExprAnd, orderByPrefix, null, false,
preOrderBy, postOrderBy, orderByPlaceholder);
}
private String removeWhitespace(String sql) {
if (sql == null) {
return "";
@@ -32,6 +32,12 @@ public final class DRawSqlService implements SpiRawSqlService {
return new DRawSqlBuilder(s, new SpiRawSql.ColumnMapping());
}
@Override
public RawSqlBuilder withPlaceholders(String sql) {
SpiRawSql.Sql s = DRawSqlParser.parseAsTemplate(sql);
return new DRawSqlBuilder(s, new SpiRawSql.ColumnMapping());
}
@Override
public SqlRow sqlRow(ResultSet resultSet, String dbTrueValue, boolean binaryOptimizedUUID) throws SQLException {
ResultSetMetaData meta = resultSet.getMetaData();
@@ -37,24 +37,18 @@ public interface SpiRawSql extends RawSql {
private static final long serialVersionUID = 1L;
private final boolean parsed;
private final String unparsedSql;
private final String preFrom;
private final String preWhere;
private final boolean andWhereExpr;
private final String preHaving;
private final boolean andHavingExpr;
private final String orderByPrefix;
private final String orderBy;
private final boolean distinct;
private final String preOrderBy;
private final String postOrderBy;
private final boolean orderByPlaceholder;
/**
* Construct for unparsed SQL.
@@ -70,13 +64,26 @@ public interface SpiRawSql extends RawSql {
this.orderByPrefix = null;
this.orderBy = null;
this.distinct = false;
this.preOrderBy = null;
this.postOrderBy = null;
this.orderByPlaceholder = false;
}
/**
* Construct for parsed SQL.
* Construct for parsed SQL (normal keyword-parsed mode - no ${orderBy}/${andOrderBy} placeholder support).
*/
Sql(String unparsedSql, String preFrom, String preWhere, boolean andWhereExpr,
String preHaving, boolean andHavingExpr, String orderByPrefix, String orderBy, boolean distinct) {
this(unparsedSql, preFrom, preWhere, andWhereExpr, preHaving, andHavingExpr, orderByPrefix, orderBy, distinct,
null, null, false);
}
/**
* Construct for parsed SQL, including template mode's ${orderBy}/${andOrderBy} placeholder support.
*/
Sql(String unparsedSql, String preFrom, String preWhere, boolean andWhereExpr,
String preHaving, boolean andHavingExpr, String orderByPrefix, String orderBy, boolean distinct,
String preOrderBy, String postOrderBy, boolean orderByPlaceholder) {
this.unparsedSql = unparsedSql;
this.parsed = true;
@@ -88,6 +95,9 @@ public interface SpiRawSql extends RawSql {
this.orderByPrefix = orderByPrefix;
this.orderBy = orderBy;
this.distinct = distinct;
this.preOrderBy = preOrderBy;
this.postOrderBy = postOrderBy;
this.orderByPlaceholder = orderByPlaceholder;
}
@Override
@@ -172,6 +182,32 @@ public interface SpiRawSql extends RawSql {
return orderBy;
}
/**
* Return the static SQL to emit immediately before the dynamic order-by injection point
* (template / withPlaceholders() mode only, e.g. static SQL between a ${having} and ${orderBy}
* placeholder).
*/
public String getPreOrderBy() {
return preOrderBy;
}
/**
* Return the static SQL to emit after the dynamic order-by injection point
* (template / withPlaceholders() mode only - typically empty since ORDER BY is usually last).
*/
public String getPostOrderBy() {
return postOrderBy;
}
/**
* Return true if a ${orderBy}/${andOrderBy} placeholder was found (template / withPlaceholders()
* mode only). When false, any dynamic order by set on the query is ignored rather than risk
* producing invalid SQL by injecting it at an undefined position.
*/
public boolean isOrderByPlaceholder() {
return orderByPlaceholder;
}
}
/**
@@ -2,6 +2,8 @@ package io.ebeaninternal.server.rawsql;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class DRawSqlServiceTest {
@@ -16,4 +18,110 @@ public class DRawSqlServiceTest {
assertEquals("myschema.mytable.mycol", dRawSqlService.combine("myschema", "mytable", "mycol"));
assertEquals("myschema.mycol", dRawSqlService.combine("myschema", null, "mycol"));
}
@Test
void withPlaceholders_where() {
String sql = "with cte as (select a, b from t ${where} group by a) select a, b from cte order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.isParsed()).isTrue();
assertThat(result.getPreFrom()).isEmpty();
assertThat(result.getPreWhere()).isEqualTo("with cte as (select a, b from t");
assertThat(result.getPreHaving()).isEqualTo("group by a) select a, b from cte order by a");
assertThat(result.isAndWhereExpr()).isFalse();
}
@Test
void withPlaceholders_andWhere() {
String sql = "with cte as (select a from t where x=1 ${andWhere} group by a) select a from cte";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("with cte as (select a from t where x=1");
assertThat(result.isAndWhereExpr()).isTrue();
}
@Test
void withPlaceholders_requiresPlaceholder() {
assertThatThrownBy(() -> DRawSqlParser.parseAsTemplate("select a from t"))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("${where}");
}
@Test
void withPlaceholders_havingOnly_noWherePlaceholder() {
String sql = "select a, sum(b) as total from t group by a ${having} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t group by a");
assertThat(result.getPreHaving()).isNull();
assertThat(result.isAndHavingExpr()).isFalse();
// trailing static SQL after the placeholder is preserved and emitted after any dynamic having.
// There is no ${orderBy}/${andOrderBy} placeholder so no dynamic order-by injection point exists -
// the static text is carried as preOrderBy and orderBy remains null/unused (getOrderByPrefix()
// falls back to its "order by" default but that value is never used - the gating in
// CQueryBuilderRawSql.orderBy() means no dynamic order by is ever appended in this case).
assertThat(result.getOrderBy()).isNull();
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_andHavingOnly_noWherePlaceholder() {
String sql = "select a, sum(b) as total from t group by a having total > 0 ${andHaving} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t group by a having total > 0");
assertThat(result.getPreHaving()).isNull();
assertThat(result.isAndHavingExpr()).isTrue();
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_whereAndHaving_bothPresent() {
String sql = "select a, sum(b) as total from t ${where} group by a ${having} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t");
assertThat(result.getPreHaving()).isEqualTo("group by a");
// no data loss - trailing "order by a" preserved and emitted after the dynamic having clause
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_orderBy() {
String sql = "select a, b from t ${where} ${orderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, b from t");
assertThat(result.getPreHaving()).isEmpty();
assertThat(result.getOrderByPrefix()).isEqualTo("order by");
assertThat(result.getOrderBy()).isNull();
assertThat(result.isOrderByPlaceholder()).isTrue();
}
@Test
void withPlaceholders_andOrderBy() {
String sql = "select a, b from t ${where} order by a ${andOrderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, b from t");
assertThat(result.getPreHaving()).isEqualTo("order by a");
assertThat(result.getOrderByPrefix()).isEqualTo(",");
assertThat(result.getOrderBy()).isNull();
assertThat(result.isOrderByPlaceholder()).isTrue();
}
@Test
void withPlaceholders_whereHavingAndOrderBy_allThreePresent() {
String sql = "select a, sum(b) as total from t ${where} group by a ${having} ${orderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t");
assertThat(result.getPreHaving()).isEqualTo("group by a");
assertThat(result.getPreOrderBy()).isEmpty();
assertThat(result.getOrderByPrefix()).isEqualTo("order by");
assertThat(result.isOrderByPlaceholder()).isTrue();
}
}
@@ -0,0 +1,46 @@
package org.example.domain;
import io.ebean.annotation.Sql;
import jakarta.persistence.Entity;
import jakarta.persistence.OneToOne;
/**
* An example of an Aggregate object populated via RawSql rather than a table.
* <p>
* Note the &#064;Sql indicates to Ebean that this bean is not based on a table but
* instead uses RawSql. As it is a normal &#064;Entity a query bean (QOrderAggregate)
* is generated for it, so RawSql (including RawSqlBuilder.withPlaceholders()) can be
* used together with the generated, type-safe query bean API.
* </p>
*/
@Entity
@Sql
public class OrderAggregate {
@OneToOne
Order order;
Double totalAmount;
public Order getOrder() {
return order;
}
public void setOrder(Order order) {
this.order = order;
}
public Double getTotalAmount() {
return totalAmount;
}
public void setTotalAmount(Double totalAmount) {
this.totalAmount = totalAmount;
}
@Override
public String toString() {
return order.getId() + " totalAmount:" + totalAmount;
}
}
@@ -0,0 +1,123 @@
package org.querytest;
import io.ebean.DB;
import io.ebean.RawSql;
import io.ebean.RawSqlBuilder;
import io.ebean.test.LoggedSql;
import org.example.domain.Customer;
import org.example.domain.Order;
import org.example.domain.OrderAggregate;
import org.example.domain.OrderDetail;
import org.example.domain.Product;
import org.example.domain.query.QOrderAggregate;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
/**
* Demonstrates that RawSql (including RawSqlBuilder.withPlaceholders()) can be used
* together with a generated, type-safe query bean (QOrderAggregate) - not just the
* plain Query&lt;T&gt; API.
*/
class QOrderAggregateTest {
/** CTE with a ${where} placeholder so it can't be parsed by RawSqlBuilder.parse(). */
private static final String SQL =
"with order_totals as (" +
" select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount" +
" from order_totals" +
" ${where}" +
" order by order_id";
private static Order order1;
private static Order order2;
@BeforeAll
static void setup() {
Customer customer = new Customer();
customer.setName("QOrderAggregateTest customer");
customer.save();
Product product = new Product();
product.setName("prod1");
product.setSku("qoat-1");
product.save();
order1 = new Order();
order1.setCustomer(customer);
order1.getDetails().add(new OrderDetail(product, 2, 20.0)); // total 40.00
order1.save();
order2 = new Order();
order2.setCustomer(customer);
order2.getDetails().add(new OrderDetail(product, 5, 20.0)); // total 100.00
order2.save();
}
@AfterAll
static void tearDown() {
DB.delete(order1);
DB.delete(order2);
}
@Test
void queryBean_setRawSql_noFilter_returnsAllRows() {
RawSql rawSql = RawSqlBuilder.withPlaceholders(SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = new QOrderAggregate()
.setRawSql(rawSql)
.findList();
assertThat(list).hasSizeGreaterThanOrEqualTo(2);
}
@Test
void queryBean_setRawSql_withTypedWhereExpression_filtersRows() {
RawSql rawSql = RawSqlBuilder.withPlaceholders(SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// typed property expression (totalAmount.gt(...)) rather than the string based where().gt(...)
List<OrderAggregate> list = new QOrderAggregate()
.setRawSql(rawSql)
.totalAmount.gt(50.0)
.findList();
assertThat(list).extracting(OrderAggregate::getOrder)
.extracting(Order::getId)
.contains(order2.getId())
.doesNotContain(order1.getId());
}
@Test
void queryBean_setRawSql_verifySqlStructure() {
RawSql rawSql = RawSqlBuilder.withPlaceholders(SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
LoggedSql.start();
new QOrderAggregate()
.setRawSql(rawSql)
.totalAmount.gt(50.0)
.findList();
List<String> sql = LoggedSql.stop();
assertThat(sql).hasSize(1);
assertThat(sql.get(0)).containsIgnoringCase("where total_amount > ?");
}
}
@@ -0,0 +1,394 @@
package org.tests.rawsql;
import io.ebean.DB;
import io.ebean.RawSql;
import io.ebean.RawSqlBuilder;
import io.ebean.test.LoggedSql;
import io.ebean.xtest.BaseTestCase;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.tests.model.basic.OrderAggregate;
import org.tests.model.basic.ResetBasicData;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/**
* Integration tests for RawSqlBuilder.withPlaceholders() — complex SQL (CTEs,
* window functions) with ${where} / ${andWhere} / ${having} / ${andHaving}
* placeholders for dynamic WHERE and HAVING injection.
*/
class TestRawSqlWithPlaceholders extends BaseTestCase {
/** CTE with ${where} in the outer SELECT — column names match the CTE output aliases. */
private static final String CTE_SQL =
"with order_totals as (" +
" select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount" +
" from order_totals" +
" ${where}" +
" order by order_id";
/** Same CTE with ${andWhere} — a static WHERE clause is already present. */
private static final String CTE_AND_WHERE_SQL =
"with order_totals as (" +
" select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount" +
" from order_totals" +
" where total_amount > 0 ${andWhere}" +
" order by order_id";
/** Direct aggregate (no CTE) with only a ${having} placeholder, and static ORDER BY after it. */
private static final String HAVING_ONLY_SQL =
"select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
" ${having}" +
" order by order_id";
/** Both ${where} and ${having} placeholders present, with static ORDER BY after the having. */
private static final String WHERE_AND_HAVING_SQL =
"select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" ${where}" +
" group by o.id" +
" ${having}" +
" order by order_id";
/** ${where} plus ${orderBy} — no static ORDER BY, ordering is entirely dynamic. */
private static final String WHERE_AND_ORDER_BY_SQL =
"with order_totals as (" +
" select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount" +
" from order_totals" +
" ${where}" +
" ${orderBy}";
/** Static ORDER BY already present, with ${andOrderBy} to append additional dynamic sort columns. */
private static final String AND_ORDER_BY_SQL =
"with order_totals as (" +
" select o.id as order_id," +
" sum(d.order_qty * d.unit_price) as total_amount" +
" from o_order o" +
" join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount" +
" from order_totals" +
" ${where}" +
" order by total_amount desc ${andOrderBy}";
private static RawSql cteSql;
private static RawSql cteAndWhereSql;
private static RawSql havingOnlySql;
private static RawSql whereAndHavingSql;
private static RawSql whereAndOrderBySql;
private static RawSql andOrderBySql;
@BeforeAll
static void setup() {
ResetBasicData.reset();
cteSql = RawSqlBuilder.withPlaceholders(CTE_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
cteAndWhereSql = RawSqlBuilder.withPlaceholders(CTE_AND_WHERE_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
havingOnlySql = RawSqlBuilder.withPlaceholders(HAVING_ONLY_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
whereAndHavingSql = RawSqlBuilder.withPlaceholders(WHERE_AND_HAVING_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
whereAndOrderBySql = RawSqlBuilder.withPlaceholders(WHERE_AND_ORDER_BY_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
andOrderBySql = RawSqlBuilder.withPlaceholders(AND_ORDER_BY_SQL)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
}
@Test
void withPlaceholders_noFilter_returnsAllRowsWithDetails() {
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.findList();
// orders 1, 2, 3 have details; orders 4 and 5 do not
assertThat(list).hasSize(3);
assertThat(list).extracting(OrderAggregate::getTotalAmount).doesNotContainNull();
}
@Test
void withPlaceholders_withWhereFilter_returnsFilteredRows() {
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.where().gt("totalAmount", 50)
.findList();
// order 1 ≈ 57.80, order 3 ≈ 165.50; order 2 = 42.00 is filtered out
assertThat(list).hasSize(2);
assertThat(list).extracting(OrderAggregate::getTotalAmount)
.allMatch(amount -> amount > 50.0);
}
@Test
void withPlaceholders_withStrongWhereFilter_returnsSingleRow() {
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.where().gt("totalAmount", 100)
.findList();
// only order 3 has total > 100 (≈ 165.50)
assertThat(list).hasSize(1);
assertThat(list.get(0).getTotalAmount()).isGreaterThan(100.0);
}
@Test
void withPlaceholders_andWhere_appendsToExistingWhereClause() {
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteAndWhereSql)
.where().gt("totalAmount", 100)
.findList();
// ${andWhere} appends "and total_amount > 100" to the existing "where total_amount > 0"
assertThat(list).hasSize(1);
assertThat(list.get(0).getTotalAmount()).isGreaterThan(100.0);
}
@Test
void withPlaceholders_verifySqlStructure() {
LoggedSql.start();
DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.where().gt("totalAmount", 50)
.findList();
List<String> sql = LoggedSql.stop();
assertThat(sql).hasSize(1);
String executed = sql.get(0);
// SQL starts with the CTE — no spurious "select" prefix prepended
assertThat(executed).containsIgnoringCase("with order_totals as");
// WHERE is injected at the ${where} position (in the outer SELECT, before ORDER BY)
assertThat(executed).containsIgnoringCase("where total_amount > ?");
assertThat(executed).containsIgnoringCase("order by order_id");
// WHERE appears after the CTE body
int wherePos = executed.toLowerCase().lastIndexOf("where total_amount");
int orderByPos = executed.toLowerCase().indexOf("order by order_id");
assertThat(wherePos).isLessThan(orderByPos);
}
@Test
void withPlaceholders_findCount() {
int count = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.where().gt("totalAmount", 50)
.findCount();
assertThat(count).isEqualTo(2);
}
@Test
void withPlaceholders_fetchQuery_loadsAssociatedGraphViaSecondaryQuery() {
// the RawSql query is the root query; fetchQuery() adds secondary ORM queries
// to build out more of the object graph rather than hand-writing it into the
// raw SQL itself. Because the raw SQL only maps "order.id" (a partial "order"
// reference), we explicitly fetchQuery("order") as well as fetchQuery("order.details") -
// fetchQuery("order.details") alone would leave "details" as a deferred/lazy
// collection since the intermediate "order" fetch node isn't otherwise requested.
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.fetchQuery("order")
.fetchQuery("order.details")
.where().gt("totalAmount", 50)
.findList();
List<String> sql = LoggedSql.stop();
// one query for the raw-sql root, plus one secondary (ORM) query per fetchQuery() path
assertThat(sql).hasSize(3);
assertThat(sql.get(0)).containsIgnoringCase("with order_totals as");
assertThat(sql.get(1)).containsIgnoringCase("from o_order ");
assertThat(sql.get(2)).containsIgnoringCase("from o_order_detail");
assertThat(list).hasSize(2);
for (OrderAggregate orderAggregate : list) {
// order.details was populated by the secondary query - no further lazy loading needed
assertThat(orderAggregate.getOrder().getDetails()).isNotEmpty();
}
}
@Test
void parse_failsOnCteSql() {
// Demonstrates why withPlaceholders() is needed — parse() cannot handle CTEs
assertThatThrownBy(() -> RawSqlBuilder.parse(CTE_SQL).create())
.isInstanceOf(RuntimeException.class);
}
@Test
void withPlaceholders_havingOnly_appliesBeforeStaticOrderBy() {
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(havingOnlySql)
.having().gt("totalAmount", 100)
.findList();
List<String> sql = LoggedSql.stop();
// only order 3 has aggregate total > 100 (≈ 165.50)
assertThat(list).hasSize(1);
assertThat(list.get(0).getTotalAmount()).isGreaterThan(100.0);
// the dynamically injected HAVING must appear before the static trailing ORDER BY,
// otherwise the generated SQL would be invalid
String executed = sql.get(0).toLowerCase();
int havingPos = executed.indexOf("having");
int orderByPos = executed.indexOf("order by");
assertThat(havingPos).isGreaterThan(-1);
assertThat(orderByPos).isGreaterThan(havingPos);
}
@Test
void withPlaceholders_whereAndHaving_orderByTailNotLost() {
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(whereAndHavingSql)
.where().gt("order.id", 0)
.having().gt("totalAmount", 50)
.findList();
List<String> sql = LoggedSql.stop();
// orders 1 (≈57.80) and 3 (≈165.50) pass the having filter; order 2 (=42.00) does not
assertThat(list).hasSize(2);
// both dynamic where and having are injected, and the static "order by order_id" tail
// (positioned after ${having} in the template) is preserved rather than dropped
String executed = sql.get(0).toLowerCase();
assertThat(executed).contains("where");
assertThat(executed).contains("having");
assertThat(executed).contains("order by order_id");
int havingPos = executed.indexOf("having");
int orderByPos = executed.indexOf("order by");
assertThat(orderByPos).isGreaterThan(havingPos);
}
@Test
void withPlaceholders_orderBy_appliesDynamicOrdering() {
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(whereAndOrderBySql)
.where().gt("totalAmount", 0)
.orderBy("totalAmount desc")
.findList();
List<String> sql = LoggedSql.stop();
assertThat(list).hasSize(3);
// order 3 (≈165.50) first, then order 1 (≈57.80), then order 2 (=42.00)
assertThat(list).extracting(OrderAggregate::getTotalAmount)
.isSortedAccordingTo((a, b) -> Double.compare(b, a));
String executed = sql.get(0).toLowerCase();
assertThat(executed).contains("order by total_amount desc");
}
@Test
void withPlaceholders_orderBy_noExplicitOrderBy_noOrderByClauseEmitted() {
LoggedSql.start();
DB.find(OrderAggregate.class)
.setRawSql(whereAndOrderBySql)
.where().gt("totalAmount", 0)
.findList();
List<String> sql = LoggedSql.stop();
// ${orderBy} placeholder present but caller supplied no .orderBy() - nothing injected
assertThat(sql.get(0).toLowerCase()).doesNotContain("order by");
}
@Test
void withPlaceholders_andOrderBy_appendsToStaticOrderBy() {
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(andOrderBySql)
.where().gt("totalAmount", 0)
.orderBy("order.id")
.findList();
List<String> sql = LoggedSql.stop();
assertThat(list).hasSize(3);
String executed = sql.get(0).toLowerCase().replaceAll("\\s+", " ");
// static "order by total_amount desc" is kept, and the dynamic order by is appended after a comma
assertThat(executed).contains("order by total_amount desc , order_id");
}
@Test
void withPlaceholders_explicitOrderBy_ignoredWhenNoOrderByPlaceholder_havingOnly() {
// regression test: before ${orderBy}/${andOrderBy} placeholder support was added, calling
// .orderBy() on a template whose static "order by order_id" tail followed a ${having}
// placeholder (with no dedicated order-by placeholder) produced invalid SQL missing the
// "order by" keyword entirely. Now the explicit ordering is safely ignored instead.
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(havingOnlySql)
.having().gt("totalAmount", 0)
.orderBy("totalAmount desc")
.findList();
List<String> sql = LoggedSql.stop();
assertThat(list).hasSize(3);
String executed = sql.get(0).toLowerCase();
assertThat(executed).contains("order by order_id");
assertThat(executed).doesNotContain("totalamount desc");
}
@Test
void withPlaceholders_explicitOrderBy_ignoredWhenNoOrderByPlaceholder_whereOnly() {
// regression test: before ${orderBy}/${andOrderBy} placeholder support was added, calling
// .orderBy() on a where-only template with a static trailing "order by order_id" produced a
// duplicate "order by ... order by ..." clause (a SQL syntax error). Now it is safely ignored.
LoggedSql.start();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(cteSql)
.where().gt("totalAmount", 0)
.orderBy("totalAmount desc")
.findList();
List<String> sql = LoggedSql.stop();
assertThat(list).hasSize(3);
String executed = sql.get(0).toLowerCase();
long orderByCount = executed.split("order by", -1).length - 1;
assertThat(orderByCount).isEqualTo(1);
assertThat(executed).contains("order by order_id");
}
}