Compare commits

..
27 Commits
Author SHA1 Message Date
Rob Bygrave e525d4e159 Version 18.2.0 2026-07-03 19:38:05 +12:00
4e639cb88a Add deletePermanent() - hard delete for soft delete capable beans (#3828)
Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 18:23:18 +12:00
fc78b2af80 Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649) (#3827)
* Add RawSqlBuilder.withPlaceholders() for CTEs and other complex SQL (#3649)

RawSqlBuilder.parse() uses keyword-based scanning to locate SELECT
columns and the WHERE/HAVING injection points. This fails for CTEs,
window functions and other complex SQL where "select"/"from" keywords
appear in places the scanner doesn't expect.

withPlaceholders(sql) skips SELECT/FROM column parsing entirely and
only locates the ${where}, ${andWhere}, ${having} and ${andHaving}
placeholder positions, requiring explicit columnMapping() calls (as
with unparsed()). This lets dynamic where()/having() expressions be
injected into otherwise unparseable SQL.

Also fixes two bugs in the underlying placeholder-position splitting:
- static SQL following a ${having}/${andHaving} placeholder (e.g. a
  trailing ORDER BY) was silently dropped when both a where and a
  having placeholder were present
- using only ${having}/${andHaving} (no where placeholder) caused a
  dynamically added HAVING clause to be appended after trailing
  static SQL, producing invalid SQL

Changes:
- RawSqlBuilder.withPlaceholders(sql) + SpiRawSqlService.withPlaceholders()
- DRawSqlParser.parseAsTemplate() / parseTemplate() - placeholder-position
  only parsing, correctly splitting preWhere/preHaving/trailing SQL
- CQueryBuilderRawSql - skip column-list and "select" prefix handling
  in template mode (signalled by an empty preFrom)
- Unit tests in DRawSqlServiceTest covering where/andWhere/having/andHaving
  placeholder combinations, including the two fixed edge cases
- Integration tests in TestRawSqlWithPlaceholders (ebean-test) covering
  CTE queries with dynamic where/having and verifying generated SQL

* Add examples test using query beans

* Add docs guides for RawSql

* Add ${orderBy} ${andOrderBy}

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 18:22:42 +12:00
29647ebe1a Fire BeanPersistController when only ManyToMany collection changes (#3652) (#3826)
When only a @ManyToMany collection is modified and saved, the parent bean has no dirty scalar properties so BeanPersistController preUpdate/postUpdate were never invoked — only the junction table rows were written.

Fix mirrors the existing preElementCollectionUpdate() mechanism: add
preManyToManyUpdate() on PersistRequestBean and call it from
SaveManyBeans.saveAssocManyIntersection() when the intersection actually
changes (vanillaCollection, forcedUpdate, or a tracked BeanCollection
with non-empty additions or removals). The !dirty guard in
preManyToManyUpdate() prevents double-firing when the bean itself is
also dirty.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-03 11:43:16 +12:00
38146fdce5 Support option to have always non-null @Embeddable(s) refereneces (#3825)
(#3702)

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 22:02:15 +12:00
864aff6c3b Implement query.exists() via select exists(...) (#3824)
* Implement query.exists() via select exists(...)

Previously query.exists() was implemented by running a findIds() limited to 1 row and checking whether the list was empty. This PR replaces that with a dedicated select exists(select 1 ...) execution path.

## Changes:

• CQueryExists — new query executor, analogous to CQueryCount, that runs select exists(...) and returns the boolean result directly from the JDBC ResultSet
• CQueryBuilder.buildExistsQuery() — builds select exists(select 1 ...) SQL, reusing the query plan cache on repeat calls
• CQueryEngine.findExists() / DefaultOrmQueryEngine / OrmQueryEngine interface — wire the new executor through the standard query engine stack, including SQL logging, summary logging, and query cache put
• DefaultServer.exists() — delegate to request.findExists() instead of findIds()

## Tests added to TestQueryExists:

• testExistsBoolean_returnsFalse — verifies false is returned when no rows match
• testExistsBoolean_withJoin — exercises the join SQL path in buildExistsQuery; also checks false when the join yields no match
• testExistsBoolean_queryPlanReuse — runs the same query twice and asserts the generated SQL is identical, confirming the plan cache is hit on the second call

* Fix TestQueryExists only

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 18:32:50 +12:00
a161a42742 #3676 Add DbMigration.setAddForeignKeySkipCheck(true) (#3822)
Provides an option to include IDE -- @formatter:off style comments into the db migration generated sql.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 17:45:32 +12:00
24da374aa8 Fix #3821 fetchQuery secondary tables not included in query cache dep… (#3823)
* Fix #3821 fetchQuery secondary tables not included in query cache dependent tables

When a query uses fetchQuery("path"), the secondary table(s) must be registered as dependent tables in the query cache entry so that updates to those tables properly invalidate the cache.

## Fix:
In CQueryPlan, after building the SQL-tree dependent tables, walk the query's OrmQueryDetail for any fetchQuery paths and resolve each path through the root BeanDescriptor to its target entity's baseTable(). These tables are merged into dependentTables at plan construction time (once, then cached in the plan).

This means the fix applies uniformly whether paths were specified via fetchQuery("path") calls or via query.select(fetchGroup) — both populate the same OrmQueryDetail.

### Tests added to TestQueryCacheTableDependency:

• fetchQuery_oneToMany_invalidatesQueryCacheOnSecondaryTableUpdate — updating a contact invalidates a Customer query cache that includes fetchQuery("contacts"); asserts the refreshed result contains the updated phone number

• fetchQuery_manyToOne_invalidatesQueryCacheOnSecondaryTableUpdate — updating a root record invalidates an ECacheChild query cache that includes fetchQuery("root")

* Ensure tests cleanup their test data

* query.secondaryQuery() (called during prepareQuery()) removes fetchQuery paths from OrmQueryDetail before CQueryPlan is built. The existing buildDependentTables iterated detail.entries() looking for isQueryFetch() entries — but they were already gone.

Changes:

1. OrmQueryRequest.java — added secondaryQueries() getter to expose the already-stored SpiQuerySecondary field.
2. CQueryPlan.java — rewrote buildDependentTables to accept SpiQuerySecondary instead of OrmQueryDetail. It now iterates secondary.queryJoins() (the paths already removed from detail) and adds each path's base table to the dependent tables set.

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 17:44:10 +12:00
398848378d Fix asOf() queries bypassing bean cache isolation (#3820)
* Fix asOf() queries bypassing bean cache isolation

asOf() queries must never read from or write to the bean cache since the cache holds only current state. Previously, lazy loads and secondary queries spawned by an asOf parent could return stale  current data from the cache instead of the historical snapshot.

- DefaultServer.findId(): skip bean cache check when isAsOfQuery()
  so the findById fast-path doesn't return current state for temporal queries

- DLoadContext: force useBeanCache=CacheMode.OFF when asOf != null so hitCache=false for all lazy load batch cache checks, and propagate CacheMode.OFF to secondary (+query) joins

- LoadBeanRequest.configureQuery(): when !loadCache (parent context disabled cache), explicitly set CacheMode.OFF on the lazy-load SQL query — previously left at CacheMode.AUTO which caused cache reads/writes even when the parent had cache disabled

Fixes #3713

* Simplify asOf to CacheMode.OFF

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 09:06:36 +12:00
688eee532c Fix #3817 incorrect type deserialization for @DbJson field in generic superclass (#3819)
In createJsonObjectMapperType(), remove the instanceof DeployBeanProperty
branch that overrode ownerType() with getField().getDeclaringClass().

For a generic superclass B<T>, getDeclaringClass() returns B with T
unresolved, so Jackson treated the field as Object and deserialized to
LinkedHashMap. DeployBeanProperty.ownerType() already returns
desc.getBeanType() (the concrete subclass), which gives Jackson the full
supertype context needed to resolve T correctly.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-02 00:08:47 +12:00
88d7b5e85a Support @ManyToOne inside @Embeddable for query predicates and joins (#3816)
* Make embeddable diggable

* WIP

* WIP

* WIP

* WIP

* WIP

* Support @ManyToOne inside @Embeddable for query predicates and joins

 Fix path resolution for @ManyToOne associations nested inside @Embeddable
 classes. Previously, a query like .where().eq("address.country.name", "NZ")
 on an entity with an @Embedded EAddr (containing @ManyToOne Country country)
 would throw PersistenceException: Embedded Property country.name not found.

 Changes:
 - BeanPropertyAssocOne.buildElPropertyValue: correctly resolve @ManyToOne
   paths inside embedded by extracting first segment, validating via
   embeddedPropsMap, then delegating deeper traversal to the overridden
   property; restore embedded flag on ElPropertyChain for scalar leaves
 - BeanPropertyAssoc.targetDescriptor(): lazy-init fix for override copies
   created by BeanEmbeddedMetaFactory (initialise() is not called on them)
 - BeanDescriptor.extraJoin(): restore !assocProp.isEmbedded() guard to
   prevent null-table ExtraJoin for composite keys and other embeddables
 - ElPropertyChainBuilder/ElPropertyChain: restore embedded field and prefix
   computation so alias resolution works correctly for paths through embedded
   (e.g. outer.datePeriod.date1 → prefix "outer" not "outer.datePeriod")
 - SqlTreeBuilder.buildExtraJoins: remove erroneous removeAll on
   orderByIncludes that broke Formula2 placeholder resolution and distinct
   on aggregation queries with many-side order-by joins
 - SqlTreeAlias.addJoin: fix indentation
 - Tests: add three test cases to TestEmbeddedManyToOne covering WHERE
   predicate through embedded FK column, through association property
   (requiring a JOIN), and combined with fetch

* Throw PersistenceException with unknown path

---------

Co-authored-by: Iliya Ivanov <i.ivanov@proforge.org>
2026-07-01 23:33:40 +12:00
985e3b12b1 Fix ON_CONFLICT_NOTHING cascade causing FK violation (#3712) (#3818)
When inserting a bean with InsertOptions.ON_CONFLICT_NOTHING that has cascade children (OneToMany / exported OneToOne), a unique constraint conflict caused the parent insert to be silently skipped (0 rows), but Ebean still cascaded and attempted to insert the children — resulting in a FK violation.

Changes

• BeanDescriptor.hasCascadeChildren() — returns true when the bean has save-cascade children (OneToMany or exported OneToOne) that hold a FK back to this bean.
• PersistRequestBean.setInsertOptions() — when ON_CONFLICT_NOTHING is used and the bean has cascade children, sets skipBatchForTopLevel = true so the parent INSERT executes immediately (non-batched). This ensures the row count is known before any cascade runs. Beans without cascade children are unaffected and continue to batch normally.
• PersistRequestBean.checkRowCount() — when the parent INSERT returns 0 rows under ON_CONFLICT_NOTHING, sets insertConflictSkipped = true and returns early, leaving the bean unmarked as loaded/persisted.
• DmlHandler.checkRowCount() — skips postExecute() when the insert was conflict-skipped.
• DefaultPersister.insert() — guards saveAssocMany() with !request.isInsertConflictSkipped(), preventing cascade saves when the parent was not actually inserted.

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-01 23:32:02 +12:00
Andrey GlushkovandGitHub 942bab5efb Fix: DuplicateKeyException when cascade-saving through a @ManyToOne to an unmodifiable bean (address #3813) (#3814)
* Fix DuplicateKeyException with unmodifiable cascaded bean

* Fix broken test TestDeleteByQuery.maxDelete_when_beanCaching_expect_selectThenDelete
2026-07-01 22:43:33 +12:00
Rob BygraveandGitHub c114ebaec5 Merge pull request #3815 from ebean-orm/feature/add-sqlite-ioc
Sqlite: Add Insert On Conflict support (#3770)
2026-07-01 22:02:22 +12:00
robin.bygrave 9acc555e8e Sqlite: Add Insert On Conflict support (#3770) 2026-07-01 21:58:16 +12:00
Rob BygraveandGitHub 73a646e7dc Merge pull request #3812 from dragkes/feature/invalidate-one-to-one-and-collection-cache
Invalidate one to one and collection cache (address #3811)
2026-07-01 21:21:19 +12:00
Rob BygraveandGitHub bdda502d55 Merge pull request #3810 from ebean-orm/tenant-support-for-query-plans
Tenant support for query plans
2026-07-01 21:08:33 +12:00
robin.bygrave 6d447a6c1a Add back the rollback() into CQueryBindCapture 2026-07-01 21:04:59 +12:00
robin.bygrave 16872fdb30 Add back the rollback() into CQueryBindCapture 2026-07-01 21:00:19 +12:00
robin.bygrave ec9303762e Fix the CQueryPlanManager conflict edit? 2026-07-01 20:50:34 +12:00
Andrey Glushkov a59c70054b Tests 2026-07-01 11:48:28 +03:00
Andrey Glushkov 25bce83afc Merge remote-tracking branch 'origin/master' into feature/invalidate-one-to-one-and-collection-cache
# Conflicts:
#	ebean-core/src/main/java/io/ebeaninternal/server/deploy/BeanDescriptorCacheHelp.java
2026-07-01 11:46:30 +03:00
Rob BygraveandGitHub e372579b15 Merge branch 'master' into tenant-support-for-query-plans 2026-07-01 20:33:31 +12:00
robin.bygrave f470782481 Whitespace and remove excess comments only 2026-07-01 20:29:36 +12:00
Andrey Glushkov d7b9f6688a Invalidate inverse association caches on owning-side FK change 2026-07-01 11:25:27 +03:00
66bcfd107d Add tenant-partitioned L2 caches (#2956) (#3809)
* Add tenant-partitioned L2 caches (#2956)

 When `tenantPartitionedCache` is enabled, each tenant gets its own
 cache namespace (keys include the tenant id), improving cache-hit
 ratio by preventing cross-tenant key collisions.

 Refactor BeanDescriptorCacheHelp to abstract base class with two
 concrete subclasses:
 - BeanDescriptorCacheHelpFixed - static cache refs (original behaviour)
 - BeanDescriptorCacheHelpPartitioned - per-request Supplier<> lookup
   so the correct tenant-scoped cache is resolved on each access

 Fix background-thread invalidation: clear(name) in partitioned mode
 now scans all matching cache entries by key prefix rather than calling
 tenantProvider.currentId() which is null/wrong on background executor
 threads.

 Add SpiCacheManager.clearTenant(tenantId) to allow removal of all
 cache entries for a deactivated tenant, preventing unbounded memory
 growth in long-running multi-tenant deployments.

 Also load cache settings (cacheMaxSize, cacheMaxIdleTime, etc.) from
 application properties - these were previously missing from loadSettings().

* Fix versions in test-java16

---------

Co-authored-by: robin.bygrave <robin.bygrave@eroad.com>
2026-07-01 20:04:08 +12:00
Roland Praml 1c0a811c01 Tenant support for query plans 2024-10-07 15:07:38 +02:00
151 changed files with 4080 additions and 567 deletions
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-clickhouse</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-db2</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-h2</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-hana</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-mariadb</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-mysql</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+6 -6
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -22,13 +22,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -47,19 +47,19 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-net-postgis-types</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-nuodb</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-oracle</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+6 -6
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -22,13 +22,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -47,19 +47,19 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-pgvector-types</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
+6 -6
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -22,13 +22,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -47,19 +47,19 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-postgis-types</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-sqlite</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-sqlserver</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -42,13 +42,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+6 -6
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
<relativePath>../..</relativePath>
</parent>
@@ -17,13 +17,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -41,7 +41,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-jackson-mapper</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -60,13 +60,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-all</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+1 -1
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>composites</artifactId>
+1
View File
@@ -48,6 +48,7 @@ existing Maven project. Complete the steps in order.
|-------|-------------|
| [Write Ebean queries with query beans](writing-ebean-query-beans.md) | Step-by-step guidance for AI agents to write type-safe Ebean queries; choose the right terminal method; tune `select()` / `fetch()` / `fetchQuery()`; and project to DTOs when entity beans are not the right output |
| [Immutable bean cache for read-only references](immutable-bean-cache.md) | Use `ImmutableBeanCache` and `ImmutableBeanCaches.loading(...)` to resolve assoc-one references in read-only/unmodifiable queries, including secondary `fetchQuery`/`fetchLazy` loads |
| [Using `RawSql` with Ebean](using-rawsql-with-ebean.md) | Choose between `RawSqlBuilder.parse()`, `unparsed()`, and `withPlaceholders()`; the `${where}`/`${andWhere}`/`${having}`/`${andHaving}` placeholder reference for CTEs, window functions, and subqueries; column mapping; and using `RawSql` with query beans |
## Persisting & transactions
+464
View File
@@ -0,0 +1,464 @@
# Guide: Using `RawSql` with Ebean
## Purpose
`RawSql` lets you back an Ebean bean with a **hand-written SQL query** instead of
Ebean generating the SQL from the entity mapping. Ebean still handles object
mapping (result set columns → bean properties), lazy loading of associated beans,
and - depending on how the `RawSql` is built - dynamic `WHERE`/`HAVING` predicates
added through the normal query API.
Use this guide when you need to:
- run vendor-specific SQL, complex aggregation, or reporting queries that don't
map cleanly to an ORM query
- reuse a hand-tuned query but still want typed/dynamic predicates, paging, or
`ORDER BY` added by the caller
- back a query bean (`Q*`) or DTO-like bean with SQL containing a CTE, window
function, or subquery in the `FROM` clause
Prefer ordinary query bean queries first - see
[Write Ebean queries with query beans](writing-ebean-query-beans.md), Step 9,
for the full decision order (query bean → `asDto()` → DTO query → raw SQL).
This guide covers raw SQL once you've decided it's the right tool.
---
## The bean behind a `RawSql` query
A bean queried with `RawSql` is not necessarily backed by a physical table. Annotate
it `@Entity @Sql` to tell Ebean it is mapped via `RawSql` rather than table DDL:
```java
@Entity
@Sql
public class OrderAggregate {
@OneToOne
Order order;
Double totalAmount;
Long totalItems;
// getters/setters
}
```
`@Sql` beans still get a generated query bean (`QOrderAggregate`) if the
querybean-generator annotation processor is configured - see
[Using `RawSql` with query beans](#using-rawsql-with-query-beans) below.
You can also query an ordinary table-backed `@Entity` with `RawSql` - the column
mapping just needs to line up with that entity's properties.
---
## Building a `RawSql` - three factory methods
`RawSqlBuilder` has three ways to construct a `RawSql`, depending on how much of
the SQL Ebean needs to understand:
| Method | SELECT columns parsed? | Dynamic WHERE/HAVING/ORDER BY? | Use for |
|--------|------------------------|------------------------|---------|
| `RawSqlBuilder.parse(sql)` | Yes | Yes | Ordinary `SELECT ... FROM ... WHERE ...` statements |
| `RawSqlBuilder.unparsed(sql)` | No | No | Fixed SQL that never needs additional predicates |
| `RawSqlBuilder.withPlaceholders(sql)` | No (explicit `columnMapping()` required) | Yes, via `${where}` / `${andWhere}` / `${having}` / `${andHaving}` / `${orderBy}` / `${andOrderBy}` | CTEs, window functions, subqueries - SQL that keyword-based parsing can't handle |
### `parse(sql)` - the common case
`parse(sql)` scans the SQL text for the `select` / `from` / `where` / `group by`
/ `having` / `order by` keywords to work out the SELECT column list (so it can
validate your column mappings) and the injection points for dynamic `WHERE`/
`HAVING` expressions.
```java
RawSql rawSql = RawSqlBuilder.parse(
"select c.id, c.name, c.status from customer c")
.columnMapping("c.id", "id")
.columnMapping("c.name", "name")
.columnMapping("c.status", "status")
.create();
List<Customer> customers = DB.find(Customer.class)
.setRawSql(rawSql)
.where().eq("status", Customer.Status.ACTIVE)
.orderBy("name")
.findList();
```
Because the SQL is parsed, mistakes in `columnMapping()` (unknown column, wrong
order for `unparsed`-style mappings) are caught early. **This fails on SQL the
keyword parser can't make sense of** - a `WITH` CTE, a window function, a
subquery in `FROM`, etc. - because the keyword positions found don't correspond
to the outer query's real structure. Use `withPlaceholders(sql)` for that SQL
instead (see below).
### `unparsed(sql)` - fixed queries
`unparsed(sql)` skips all parsing. The SQL is used exactly as written, and **no
further `WHERE`/`HAVING`/`ORDER BY` can be added** by the caller - useful for a
completely fixed reporting query with no caller-supplied filtering.
```java
RawSql rawSql = RawSqlBuilder.unparsed(
"select id, name, status from customer where status = 'ACTIVE'")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = DB.find(Customer.class)
.setRawSql(rawSql)
.findList();
```
Column mappings for `unparsed(sql)` must be supplied **in the same order** as
the columns appear in the SQL, since there's no parsing to match them by name.
### `withPlaceholders(sql)` - complex SQL (CTEs, window functions, subqueries)
`withPlaceholders(sql)` avoids keyword scanning entirely. You mark exactly where
a dynamic `WHERE`/`HAVING`/`ORDER BY` expression should be injected using
placeholder tokens, and column mappings are always explicit (as with `unparsed`).
#### Placeholder reference
| Placeholder | Meaning | Use when |
|-------------|---------|----------|
| `${where}` | Insert a new `WHERE <expr>` clause here | No static `WHERE` clause exists yet at this point in the SQL |
| `${andWhere}` | Insert `AND <expr>` here | A static `WHERE ...` clause already exists in the SQL and you want to append to it |
| `${having}` | Insert a new `HAVING <expr>` clause here | No static `HAVING` clause exists yet at this point in the SQL |
| `${andHaving}` | Insert `AND <expr>` here | A static `HAVING ...` clause already exists in the SQL and you want to append to it |
| `${orderBy}` | Insert a new `ORDER BY <expr>` clause here | No static `ORDER BY` clause exists yet at this point in the SQL, and callers may supply `.orderBy(...)` |
| `${andOrderBy}` | Insert `, <expr>` here | A static `ORDER BY ...` clause already exists in the SQL and you want callers to be able to append extra sort columns to it |
Rules:
- At least one placeholder is required - `withPlaceholders(sql)` throws
`IllegalArgumentException` if none of the six tokens are present.
- Use only the placeholders you need. Omit `${where}`/`${andWhere}` entirely if
the query never needs a dynamic `WHERE` (e.g. only a dynamic `HAVING` on an
aggregate). Omit `${having}`/`${andHaving}` if there's no dynamic `HAVING`.
Omit `${orderBy}`/`${andOrderBy}` if the ordering is always fixed.
- Explicit `columnMapping()` is required for every returned column - there is no
column-list parsing to infer names from.
- **A caller-supplied `.orderBy(...)`/`.order(...)` is only applied if the SQL
contains an `${orderBy}` or `${andOrderBy}` placeholder.** Without one of
those placeholders there is no defined injection point for dynamic ordering,
so any `.orderBy(...)` call on the query is safely ignored rather than risk
producing invalid SQL - even if the template has a static trailing
`ORDER BY ...` of its own. If you need callers to be able to influence
ordering, add `${orderBy}` (no existing static order by) or `${andOrderBy}`
(append after an existing static order by).
- Any other static SQL that follows a `${where}`/`${having}` placeholder (e.g.
a trailing `GROUP BY`) is preserved and correctly positioned **after**
whatever dynamic expression gets injected at that placeholder.
#### Example - CTE with `${where}`
```java
String sql = """
with order_totals as (
select o.id as order_id, sum(d.qty * d.unit_price) as total_amount
from o_order o
join o_order_detail d on d.order_id = o.id
group by o.id
)
select order_id, total_amount
from order_totals
${where}
order by order_id
""";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 100)
.findList();
```
`total_amount` is a genuine column of the `order_totals` CTE here, so it's valid
to filter on it in the outer `WHERE` - this only works because the aggregate is
computed inside the CTE rather than as a same-level `SELECT` alias.
#### Example - static `WHERE` already present, append with `${andWhere}`
```java
String sql = "... from order_totals where total_amount > 0 ${andWhere} order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... where total_amount > 0 and total_amount > ? order by order_id
DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 100)
.findList();
```
#### Example - `${having}` only, filtering on an aggregate directly
No `WHERE` placeholder is needed if you only ever filter on the aggregate value:
```java
String sql =
"select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
" ${having}" +
" order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.having().gt("totalAmount", 100)
.findList();
```
The dynamic `HAVING` clause is injected before the static trailing `ORDER BY`,
even though `${having}` is the only placeholder present. Because there's no
`${orderBy}`/`${andOrderBy}` placeholder here, a caller-supplied `.orderBy(...)`
would be ignored - the ordering stays fixed as `order by order_id`.
#### Example - both `${where}` and `${having}`
```java
String sql =
"select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" ${where}" +
" group by o.id" +
" ${having}" +
" order by order_id";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("order.id", 0)
.having().gt("totalAmount", 50)
.findList();
```
Both the dynamic `WHERE` and dynamic `HAVING` are injected at their respective
placeholders, and the trailing `order by order_id` is preserved after the
`HAVING` clause.
#### Example - `${orderBy}`, fully dynamic ordering
Use `${orderBy}` when there's no static default ordering and you want the
caller's `.orderBy(...)` to control it entirely:
```java
String sql =
"with order_totals as (" +
" select o.id as order_id, sum(d.qty * d.unit_price) as total_amount" +
" from o_order o join o_order_detail d on d.order_id = o.id" +
" group by o.id" +
")" +
" select order_id, total_amount from order_totals" +
" ${where}" +
" ${orderBy}";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... where total_amount > ? order by total_amount desc
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 0)
.orderBy("totalAmount desc")
.findList();
```
If the caller doesn't call `.orderBy(...)`, nothing is injected at `${orderBy}`
and no `ORDER BY` clause is emitted at all.
#### Example - `${andOrderBy}`, appending to a static default ordering
Use `${andOrderBy}` when there's a sensible static default ordering but you
want callers to be able to add extra tie-breaker sort columns:
```java
String sql =
"... from order_totals" +
" ${where}" +
" order by total_amount desc ${andOrderBy}";
RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
.columnMapping("order_id", "order.id")
.columnMapping("total_amount", "totalAmount")
.create();
// executed SQL: ... order by total_amount desc , order_id
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql)
.where().gt("totalAmount", 0)
.orderBy("order.id")
.findList();
```
---
## Using `fetchQuery()` to build out more of the graph
A `RawSql` query can be the **root query** and still use `fetchQuery(path)` the
same way an ordinary ORM query does - Ebean runs the raw SQL for the root rows,
then runs additional secondary ORM queries to populate the requested paths. This
lets you hand-write only the part of the query that needs raw SQL (e.g. an
aggregate/CTE) and let the ORM build out the rest of the object graph normally.
```java
List<OrderAggregate> list = DB.find(OrderAggregate.class)
.setRawSql(rawSql) // root query - runs the CTE/aggregate SQL
.fetchQuery("order") // secondary query - loads the full Order
.fetchQuery("order.details") // secondary query - loads Order.details
.where().gt("totalAmount", 50)
.findList();
```
This executes **three** queries: the raw SQL root query, then one secondary
query per `fetchQuery(path)` call.
**Important**: if the raw SQL's column mapping only populates part of an
association (e.g. only `order.id`, as in the examples above), that association
is a *partial reference*. To load a nested to-many under it (e.g.
`order.details`), you must add an explicit `fetchQuery(...)` (or `fetch(...)`)
for the **intermediate path** (`order`) as well as the nested path
(`order.details`) - `fetchQuery("order.details")` alone will leave `details` as
a deferred/lazy collection, because Ebean doesn't otherwise have a fetch node
for `order` to hang the secondary query off. If the raw SQL already selects the
full set of columns for an association directly (no partial reference), this
extra step isn't needed.
This is the same `fetchQuery()` mechanism used for ordinary query bean queries -
see [Use `fetchQuery()` for to-many paths](writing-ebean-query-beans.md#step-7---use-fetchquery-for-to-many-paths-and-fetchgroup-for-reusable-query-shapes)
for background on why to-many paths are loaded via secondary queries rather than
a single joined query.
---
## Column mapping
Every `RawSqlBuilder` (except a bare `unparsed(sql)` with implicit positional
mapping) uses `columnMapping(dbColumn, propertyName)` to map SQL result columns
to bean properties:
```java
.columnMapping("order_id", "order.id") // maps to the "order" association's "id" property
.columnMapping("total_amount", "totalAmount")
```
- Dotted property paths (e.g. `"order.id"`) map a column into a nested/associated
bean property.
- `columnMappingIgnore(dbColumn)` marks a selected column as intentionally unmapped
(present in the SQL but not needed on the bean).
- `tableAliasMapping(tableAlias, path)` bulk-renames every mapping using a given
SQL table alias to be prefixed with a bean property path - handy when a `parse()`
query selects many columns from a joined table (e.g. alias `c` → path `customer`)
and you don't want to repeat the prefix in every `columnMapping()` call.
---
## Using `RawSql` with query beans
`RawSql` is not limited to the plain `Query<T>` API - it also works with a
generated query bean, giving type-safe `where()`/`having()`-equivalent
expressions (as bean properties) over hand-written SQL. Every generated query
bean exposes `setRawSql(...)`:
```java
RawSql rawSql = RawSqlBuilder.parse("select id, name, status from customer")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = new QCustomer()
.setRawSql(rawSql)
.status.equalTo(Customer.Status.ACTIVE) // typed expression, injected into the parsed WHERE clause
.findList();
```
This also works with `withPlaceholders(sql)` and an `@Sql` query bean:
```java
List<OrderAggregate> list = new QOrderAggregate()
.setRawSql(rawSql) // built with withPlaceholders() as shown above
.totalAmount.gt(100)
.findList();
```
The typed property expression (`.totalAmount.gt(100)`) is translated to a bound
predicate and injected at the `${where}`/`${having}` placeholder position, exactly
as `.where().gt("totalAmount", 100)` would be on the plain `Query<T>` API.
---
## Common anti-patterns
### Anti-pattern 1 - reaching for raw SQL before trying a query bean
Complex-looking joins are often just ordinary association traversal in a query
bean. Don't use raw SQL just because a query touches several tables - see
[Write Ebean queries with query beans](writing-ebean-query-beans.md).
### Anti-pattern 2 - using `parse(sql)` on a CTE or window-function query
`parse(sql)` will throw a parsing exception (or silently mis-locate the WHERE
injection point) on SQL it can't understand structurally. If your SQL starts
with `WITH ...` or has a subquery in `FROM`, use `withPlaceholders(sql)` instead.
### Anti-pattern 3 - filtering on a same-level SELECT alias
You cannot add a dynamic `WHERE` predicate on a `SELECT`-clause alias in the
same query level (e.g. `select sum(x) as total ... ${where}` - `total` isn't a
real column yet at the `WHERE` stage of that query level). Either:
- move the aggregation into a CTE and filter on the CTE's output column in the
outer query (`WHERE` case), or
- use `${having}`/`${andHaving}` to filter on the aggregate at the `HAVING` stage
of the same query level, where the aggregate expression is valid.
### Anti-pattern 4 - forgetting `columnMapping()` with `unparsed()`/`withPlaceholders()`
Both `unparsed(sql)` and `withPlaceholders(sql)` require **every** returned
column to be explicitly mapped (or explicitly ignored via
`columnMappingIgnore(...)`) - there's no column-list parsing to infer them.
---
## Troubleshooting
| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| `RuntimeException: Error parsing sql, can not find ... keyword` | `parse(sql)` used on SQL with a CTE, window function, or subquery in `FROM` | Use `RawSqlBuilder.withPlaceholders(sql)` instead |
| `IllegalArgumentException: withPlaceholders() requires at least one of ${where}, ${andWhere}, ${having}, ${andHaving}, ${orderBy}, ${andOrderBy}...` | None of the six placeholder tokens were found in the SQL | Add the appropriate placeholder token at the injection point |
| Dynamic `WHERE`/`HAVING` predicate silently has no effect, or query throws | Used `unparsed(sql)` and then tried to add a predicate | `unparsed(sql)` queries cannot be modified - switch to `parse(sql)` or `withPlaceholders(sql)` |
| Generated SQL is invalid / clauses appear in the wrong order | Predicates added via `.where()`/`.having()` don't match the placeholders actually present in the SQL | Make sure `${where}`/`${having}` (or the `and` variants) exist at the point you expect predicates to be injected |
| `.orderBy(...)`/`.order(...)` on the query silently has no effect | The SQL has no `${orderBy}`/`${andOrderBy}` placeholder | This is by design - without one of those placeholders there's no defined injection point, so the ordering is ignored rather than corrupting the SQL. Add `${orderBy}` or `${andOrderBy}` if you need caller-controlled ordering |
| `Unknown column` / unmapped property error | Missing `columnMapping()` for a selected column | Add a `columnMapping(...)` or `columnMappingIgnore(...)` for every SQL column |
| `fetchQuery("a.b")` collection stays deferred/lazy | `a` is a partial reference from the raw SQL column mapping (e.g. only `a.id` mapped), and there's no fetch node for `a` itself | Add `fetchQuery("a")` (or `fetch("a")`) alongside `fetchQuery("a.b")` |
---
## Related documentation
- [Write Ebean queries with query beans](writing-ebean-query-beans.md)
- [Derived / formula properties (`@Formula`, `@Formula2`)](derived-formula-properties.md)
- [Ebean query docs](https://ebean.io/docs/query/)
+25
View File
@@ -483,6 +483,30 @@ Prefer the following order:
Do **not** jump to raw SQL just because the query joins multiple tables. Query
beans already handle ordinary relationship traversal well.
### Using `RawSql` with query beans
`RawSql` is not limited to the plain `Query<T>` API - it also works with a
generated query bean, giving type-safe `where()`/`having()` expressions over
hand-written SQL. Every generated query bean exposes `setRawSql(...)`:
```java
RawSql rawSql = RawSqlBuilder.parse("select id, name, status from customer")
.columnMapping("id", "id")
.columnMapping("name", "name")
.columnMapping("status", "status")
.create();
List<Customer> customers = new QCustomer()
.setRawSql(rawSql)
.status.equalTo(Customer.Status.ACTIVE) // typed expression, injected into the parsed WHERE clause
.findList();
```
For the full guide to building `RawSql` - including `unparsed()`,
`withPlaceholders()` for CTEs/window functions, the `${where}` / `${andWhere}`
/ `${having}` / `${andHaving}` placeholder reference, and column mapping - see
[Using `RawSql` with Ebean](using-rawsql-with-ebean.md).
---
## Common anti-patterns
@@ -570,4 +594,5 @@ When asked to add or modify an Ebean query:
- [Add Ebean Postgres Maven POM](add-ebean-postgres-maven-pom.md)
- [Entity Bean Creation](entity-bean-creation.md)
- [Immutable bean cache for read-only references](immutable-bean-cache.md)
- [Using `RawSql` with Ebean](using-rawsql-with-ebean.md)
- [Ebean query docs](https://ebean.io/docs/query/)
+1 -1
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean api</name>
@@ -887,6 +887,13 @@ public interface DatabaseBuilder {
@Deprecated
DatabaseBuilder setBackgroundExecutorWrapper(BackgroundExecutorWrapper backgroundExecutorWrapper);
/**
* Enable tenant-partitioned caches. When enabled each tenant gets its own cache namespace,
* improving cache-hit ratio by preventing cross-tenant key collisions.
* Use {@link SpiCacheManager#clearTenant(Object)} when a tenant is deactivated.
*/
DatabaseBuilder tenantPartitionedCache(boolean tenantPartitionedCache);
/**
* Set the L2 cache default max size.
*/
@@ -2567,6 +2574,11 @@ public interface DatabaseBuilder {
*/
boolean isAutoPersistUpdates();
/**
* Return true if caches are partitioned by tenant.
*/
boolean isTenantPartitionedCache();
/**
* Return the L2 cache default max size.
*/
@@ -205,6 +205,22 @@ public interface ExpressionList<T> {
*/
int delete();
/**
* Execute as a delete query permanently deleting the 'root level' beans that match the
* predicates in the query without soft delete.
* <p>
* This is the same as {@link #delete()} except that when the bean type uses soft delete
* (e.g. {@code @SoftDelete}) the matching rows are permanently (hard) deleted rather than
* being marked as deleted.
* <p>
* Note that if the query includes joins then the generated delete statement may not be
* optimal depending on the database platform.
* </p>
*
* @return the number of rows that were permanently deleted.
*/
int deletePermanent();
/**
* Execute as a update query.
*
@@ -607,6 +607,21 @@ public interface QueryBuilder<SELF extends QueryBuilder<SELF, T>, T> extends Que
*/
int delete();
/**
* Execute as a delete query permanently deleting the 'root level' beans that match the
* predicates in the query without soft delete.
* <p>
* This is the same as {@link #delete()} except that when the bean type uses soft delete
* (e.g. {@code @SoftDelete}) the matching rows are permanently (hard) deleted rather than
* being marked as deleted.
* <p>
* Note that if the query includes joins then the generated delete statement may not be
* optimal depending on the database platform.
*
* @return the number of beans/rows that were permanently deleted.
*/
int deletePermanent();
/**
* Execute the query returning true if a row is found.
* <p>
@@ -41,6 +41,49 @@ public interface RawSqlBuilder {
return XBootstrapService.rawSql().unparsed(sql);
}
/**
* Return a RawSqlBuilder for SQL containing {@code ${where}}, {@code ${having}} and/or
* {@code ${orderBy}} placeholder(s). Unlike {@link #parse(String)} this does NOT attempt to parse
* the SELECT columns, so it supports complex SQL such as CTEs, subqueries, and window functions.
* <p>
* Explicit column mappings must be provided (as with {@link #unparsed(String)}), but
* WHERE, HAVING and ORDER BY expressions can be added dynamically via the query API - provided
* the corresponding placeholder is present in the SQL. If a query calls {@code .orderBy(...)}
* on a template with no {@code ${orderBy}}/{@code ${andOrderBy}} placeholder, that ordering is
* ignored (there is no injection point for it) rather than producing invalid SQL.
* </p>
* <p>
* Available placeholders:
* </p>
* <ul>
* <li>{@code ${where}} / {@code ${andWhere}} - inject "where &lt;expr&gt;" / "and &lt;expr&gt;"</li>
* <li>{@code ${having}} / {@code ${andHaving}} - inject "having &lt;expr&gt;" / "and &lt;expr&gt;"</li>
* <li>{@code ${orderBy}} / {@code ${andOrderBy}} - inject "order by &lt;expr&gt;" / ", &lt;expr&gt;"</li>
* </ul>
* <h3>Example:</h3>
* <pre>{@code
*
* String sql = """
* with agg as (
* select company_id, sum(amount) as total
* from orders
* ${where}
* group by company_id
* )
* select company_id, total from agg ${orderBy}
* """;
*
* RawSql rawSql = RawSqlBuilder.withPlaceholders(sql)
* .columnMapping("company_id", "companyId")
* .columnMapping("total", "total")
* .create();
*
* }</pre>
*/
static RawSqlBuilder withPlaceholders(String sql) {
return XBootstrapService.rawSql().withPlaceholders(sql);
}
/**
* Return a RawSqlBuilder parsing the sql.
* <p>
@@ -175,7 +175,7 @@ public final class InterceptReadOnly extends InterceptBase {
@Override
public boolean isUpdate() {
return false;
return true;
}
@Override
@@ -432,6 +432,8 @@ public class DatabaseConfig implements DatabaseBuilder.Settings {
private int backgroundExecutorShutdownSecs = 30;
private BackgroundExecutorWrapper backgroundExecutorWrapper = new MdcBackgroundExecutorWrapper();
private boolean tenantPartitionedCache;
// defaults for the L2 bean caching
private int cacheMaxSize = 10000;
@@ -1175,6 +1177,17 @@ public class DatabaseConfig implements DatabaseBuilder.Settings {
return cacheMaxSize;
}
@Override
public boolean isTenantPartitionedCache() {
return tenantPartitionedCache;
}
@Override
public DatabaseConfig tenantPartitionedCache(boolean tenantPartitionedCache) {
this.tenantPartitionedCache = tenantPartitionedCache;
return this;
}
@Override
public DatabaseConfig setCacheMaxSize(int cacheMaxSize) {
this.cacheMaxSize = cacheMaxSize;
@@ -2228,6 +2241,15 @@ public class DatabaseConfig implements DatabaseBuilder.Settings {
ddlPlaceholders = p.get("ddl.placeholders", ddlPlaceholders);
ddlHeader = p.get("ddl.header", ddlHeader);
tenantPartitionedCache = p.getBoolean("tenantPartitionedCache", tenantPartitionedCache);
cacheMaxSize = p.getInt("cacheMaxSize", cacheMaxSize);
cacheMaxIdleTime = p.getInt("cacheMaxIdleTime", cacheMaxIdleTime);
cacheMaxTimeToLive = p.getInt("cacheMaxTimeToLive", cacheMaxTimeToLive);
queryCacheMaxSize = p.getInt("queryCacheMaxSize", queryCacheMaxSize);
queryCacheMaxIdleTime = p.getInt("queryCacheMaxIdleTime", queryCacheMaxIdleTime);
queryCacheMaxTimeToLive = p.getInt("queryCacheMaxTimeToLive", queryCacheMaxTimeToLive);
// read tenant-configuration from config:
// tenant.mode = NONE | DB | SCHEMA | CATALOG | PARTITION
String mode = p.get("tenant.mode");
@@ -44,6 +44,11 @@ public interface MetaQueryPlan {
*/
String plan();
/**
* The tenant ID of the plan.
*/
Object tenantId();
/**
* Return the query execution time associated with the bind values capture.
*/
@@ -27,6 +27,13 @@ public interface SpiRawSqlService extends BootstrapService {
*/
RawSqlBuilder unparsed(String sql);
/**
* SQL with ${where}/${having} placeholder(s) but no SELECT column parsing.
* Supports complex SQL (CTEs, window functions) where keyword parsing would fail.
* Explicit column mapping is required (as with unparsed).
*/
RawSqlBuilder withPlaceholders(String sql);
/**
* Create based on a JDBC ResultSet.
*
+1 -1
View File
@@ -6,7 +6,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>ebean-bench</artifactId>
+28 -28
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean bom</name>
@@ -89,25 +89,25 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core-type</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -125,13 +125,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-jackson-mapper</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-ddl-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -155,37 +155,37 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>querybean-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>kotlin-querybean-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-redis</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-spring-txn</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<!-- platforms -->
@@ -193,91 +193,91 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-clickhouse</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-db2</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-h2</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-hana</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-mariadb</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-mysql</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-nuodb</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-oracle</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-postgis</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-postgis-types</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-pgvector</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-pgvector-types</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-sqlite</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-sqlserver</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
</dependencies>
+2 -2
View File
@@ -3,7 +3,7 @@
<parent>
<groupId>io.ebean</groupId>
<artifactId>ebean-parent</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>ebean-core-json</artifactId>
<name>ebean-core-json</name>
@@ -16,7 +16,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
+2 -2
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>ebean-core-type</artifactId>
@@ -16,7 +16,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
+7 -7
View File
@@ -3,7 +3,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>ebean-core</artifactId>
@@ -22,13 +22,13 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-api</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core-json</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -52,7 +52,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core-type</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<dependency>
@@ -157,21 +157,21 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-h2</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-sqlserver</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
@@ -100,7 +100,9 @@ public final class LoadBeanRequest extends LoadRequest {
query.setLoadDescription(mode(), description());
if (lazy) {
query.setLazyLoadBatchSize(loadBuffer.batchSize());
if (alreadyLoaded) {
if (alreadyLoaded || !loadCache) {
// alreadyLoaded: bean is being re-loaded, skip the cache to avoid a stale hit
// !loadCache: parent context disabled cache (e.g. CacheMode.OFF or asOf query),
query.setBeanCacheMode(CacheMode.OFF);
}
} else {
@@ -12,6 +12,6 @@ public interface SpiDbQueryPlan extends MetaQueryPlan {
/**
* Extend with queryTimeMicros, captureCount, captureMicros and when the bind values were captured.
*/
SpiDbQueryPlan with(long queryTimeMicros, long captureCount, long captureMicros, Instant whenCaptured);
SpiDbQueryPlan with(long queryTimeMicros, long captureCount, long captureMicros, Instant whenCaptured, Object tenantId);
}
@@ -374,6 +374,8 @@ public interface SpiEbeanServer extends SpiServer, BeanCollectionLoader {
<T> int delete(SpiQuery<T> query);
<T> int deletePermanent(SpiQuery<T> query);
<T> int update(SpiQuery<T> query);
List<SqlRow> findList(SpiSqlQuery query);
@@ -60,6 +60,6 @@ public interface SpiTransactionManager {
/**
* Return a connection used for query plan collection.
*/
Connection queryPlanConnection() throws SQLException;
Connection queryPlanConnection(Object tenantId) throws SQLException;
}
@@ -13,8 +13,9 @@ import io.ebeaninternal.server.cluster.ClusterManager;
public final class CacheManagerOptions {
private final ClusterManager clusterManager;
private final DatabaseBuilder.Settings databaseBuilder;
private final String serverName;
private final boolean localL2Caching;
private final boolean tenantPartitionedCache;
private CurrentTenantProvider currentTenantProvider;
private QueryCacheEntryValidate queryCacheEntryValidate;
private ServerCacheFactory cacheFactory = new DefaultServerCacheFactory();
@@ -24,7 +25,8 @@ public final class CacheManagerOptions {
CacheManagerOptions() {
this.localL2Caching = true;
this.clusterManager = null;
this.databaseBuilder = null;
this.serverName = "db";
this.tenantPartitionedCache = false;
this.cacheFactory = new DefaultServerCacheFactory();
this.beanDefault = new ServerCacheOptions();
this.queryDefault = new ServerCacheOptions();
@@ -32,9 +34,10 @@ public final class CacheManagerOptions {
public CacheManagerOptions(ClusterManager clusterManager, DatabaseBuilder.Settings config, boolean localL2Caching) {
this.clusterManager = clusterManager;
this.databaseBuilder = config;
this.serverName = config.getName();
this.localL2Caching = localL2Caching;
this.currentTenantProvider = config.getCurrentTenantProvider();
this.tenantPartitionedCache = config.isTenantPartitionedCache();
}
public CacheManagerOptions with(ServerCacheOptions beanDefault, ServerCacheOptions queryDefault) {
@@ -55,7 +58,7 @@ public final class CacheManagerOptions {
}
public String getServerName() {
return (databaseBuilder == null) ? "db" : databaseBuilder.getName();
return serverName;
}
public boolean isLocalL2Caching() {
@@ -85,4 +88,8 @@ public final class CacheManagerOptions {
public QueryCacheEntryValidate getQueryCacheEntryValidate() {
return queryCacheEntryValidate;
}
public boolean isTenantPartitionedCache() {
return tenantPartitionedCache;
}
}
@@ -9,6 +9,7 @@ import io.ebean.config.CurrentTenantProvider;
import io.ebean.meta.MetricVisitor;
import io.ebean.util.AnnotationUtil;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentSkipListSet;
@@ -32,6 +33,7 @@ final class DefaultCacheHolder {
private final ServerCacheOptions queryDefault;
private final CurrentTenantProvider tenantProvider;
private final QueryCacheEntryValidate queryCacheEntryValidate;
private final boolean tenantPartitionedCache;
DefaultCacheHolder(CacheManagerOptions builder) {
this.cacheFactory = builder.getCacheFactory();
@@ -39,6 +41,7 @@ final class DefaultCacheHolder {
this.queryDefault = builder.getQueryDefault();
this.tenantProvider = builder.getCurrentTenantProvider();
this.queryCacheEntryValidate = builder.getQueryCacheEntryValidate();
this.tenantPartitionedCache = builder.isTenantPartitionedCache();
}
void visitMetrics(MetricVisitor visitor) {
@@ -56,16 +59,31 @@ final class DefaultCacheHolder {
return getCacheInternal(beanType, ServerCacheType.COLLECTION_IDS, collectionProperty);
}
private String tenantKey(String beanName) {
if (tenantPartitionedCache) {
return beanName + '.' + tenantProvider.currentId();
}
return beanName;
}
private String key(String beanName, ServerCacheType type) {
if (tenantPartitionedCache) {
return beanName + '.' + tenantProvider.currentId() + type.code();
}
return beanName + type.code();
}
private String key(String beanName, String collectionProperty, ServerCacheType type) {
if (collectionProperty != null) {
return beanName + "." + collectionProperty + type.code();
} else {
return beanName + type.code();
StringBuilder sb = new StringBuilder(beanName.length() + 64);
sb.append(beanName);
if (tenantPartitionedCache) {
sb.append('.').append(tenantProvider.currentId());
}
if (collectionProperty != null) {
sb.append('.').append(collectionProperty);
}
sb.append(type.code());
return sb.toString();
}
/**
@@ -82,12 +100,15 @@ final class DefaultCacheHolder {
if (type == ServerCacheType.COLLECTION_IDS) {
lock.lock();
try {
collectIdCaches.computeIfAbsent(beanType.getName(), s -> new ConcurrentSkipListSet<>()).add(key);
collectIdCaches.computeIfAbsent(tenantKey(beanType.getName()), s -> new ConcurrentSkipListSet<>()).add(key);
} finally {
lock.unlock();
}
}
return cacheFactory.createCache(new ServerCacheConfig(type, key, shortName, options, tenantProvider, queryCacheEntryValidate));
// in partitioned mode each ServerCache instance is already tenant-scoped via its key,
// so the tenantProvider is not needed inside the cache itself
CurrentTenantProvider cacheProvider = tenantPartitionedCache ? null : tenantProvider;
return cacheFactory.createCache(new ServerCacheConfig(type, key, shortName, options, cacheProvider, queryCacheEntryValidate));
}
void clearAll() {
@@ -100,17 +121,75 @@ final class DefaultCacheHolder {
public void clear(String name) {
log.log(DEBUG, "clear {0}", name);
clearIfExists(key(name, ServerCacheType.QUERY));
clearIfExists(key(name, ServerCacheType.BEAN));
clearIfExists(key(name, ServerCacheType.NATURAL_KEY));
Set<String> keys = collectIdCaches.get(name);
if (keys != null) {
for (String collectionIdKey : keys) {
clearIfExists(collectionIdKey);
if (tenantPartitionedCache) {
// In partitioned mode, tenantProvider.currentId() may be null/wrong on a background
// invalidation thread. Scan all cache entries for this entity type across all tenants.
clearAllTenantsFor(name);
} else {
clearIfExists(key(name, ServerCacheType.QUERY));
clearIfExists(key(name, ServerCacheType.BEAN));
clearIfExists(key(name, ServerCacheType.NATURAL_KEY));
Set<String> keys = collectIdCaches.get(name);
if (keys != null) {
for (String collectionIdKey : keys) {
clearIfExists(collectionIdKey);
}
}
}
}
private void clearAllTenantsFor(String name) {
// Keys in partitioned mode: beanName.tenantId_X or beanName.tenantId.prop_X
// collectIdCaches keys: beanName.tenantId
// Both start with beanName + '.' so we can use prefix scan.
String prefix = name + '.';
for (Map.Entry<String, ServerCache> entry : allCaches.entrySet()) {
if (entry.getKey().startsWith(prefix)) {
log.log(TRACE, "clear cache {0}", entry.getKey());
entry.getValue().clear();
}
}
lock.lock();
try {
for (Map.Entry<String, Set<String>> entry : collectIdCaches.entrySet()) {
if (entry.getKey().startsWith(prefix)) {
for (String collectionIdKey : entry.getValue()) {
clearIfExists(collectionIdKey);
}
}
}
} finally {
lock.unlock();
}
}
/**
* Remove all cache entries belonging to the given tenant.
* Call this when a tenant is deactivated to prevent unbounded memory growth.
*/
void clearTenant(Object tenantId) {
String tenantIdStr = String.valueOf(tenantId);
// Keys look like: beanName.tenantId_X or beanName.tenantId.prop_X
String segmentBeforeTypeCode = '.' + tenantIdStr + '_';
String segmentBeforeProperty = '.' + tenantIdStr + '.';
allCaches.entrySet().removeIf(e -> {
String k = e.getKey();
return k.contains(segmentBeforeTypeCode) || k.contains(segmentBeforeProperty);
});
// collectIdCaches keys look like: beanName.tenantId
String collectKeySuffix = '.' + tenantIdStr;
lock.lock();
try {
collectIdCaches.entrySet().removeIf(e -> e.getKey().endsWith(collectKeySuffix));
} finally {
lock.unlock();
}
}
boolean isTenantPartitionedCache() {
return tenantPartitionedCache;
}
private void clearIfExists(String fullKey) {
ServerCache cache = allCaches.get(fullKey);
if (cache != null) {
@@ -154,4 +154,14 @@ public final class DefaultServerCacheManager implements SpiCacheManager {
return cacheHolder.getCache(beanType, ServerCacheType.BEAN);
}
@Override
public boolean isTenantPartitionedCache() {
return cacheHolder.isTenantPartitionedCache();
}
@Override
public void clearTenant(Object tenantId) {
cacheHolder.clearTenant(tenantId);
}
}
@@ -88,4 +88,17 @@ public interface SpiCacheManager {
*/
void clearLocal(Class<?> beanType);
/**
* Returns true if this cache manager runs in tenant-partitioned mode.
* In this mode caches are namespaced per tenant to improve cache-hit ratio.
*/
boolean isTenantPartitionedCache();
/**
* Remove all cache entries belonging to the given tenant.
* Call this when a tenant is deactivated to prevent unbounded memory growth
* in tenant-partitioned mode.
*/
void clearTenant(Object tenantId);
}
@@ -1197,15 +1197,14 @@ public final class DefaultServer implements SpiServer, SpiEbeanServer {
@Override
public <T> boolean exists(SpiQuery<T> ormQuery) {
SpiQuery<T> ormQueryCopy = ormQuery.copy();
ormQueryCopy.setMaxRows(1);
SpiOrmQueryRequest<?> request = createQueryRequest(Type.EXISTS, ormQueryCopy);
List<Object> ids = request.getFromQueryCache();
if (ids != null) {
return !ids.isEmpty();
Object cached = request.getFromQueryCache();
if (cached != null) {
return (Boolean) cached;
}
try {
request.initTransIfRequired();
return !request.findIds().isEmpty();
return request.findExists();
} finally {
request.endTransIfRequired();
}
@@ -1234,6 +1233,15 @@ public final class DefaultServer implements SpiServer, SpiEbeanServer {
@Override
public <T> int delete(SpiQuery<T> query) {
return delete(query, false);
}
@Override
public <T> int deletePermanent(SpiQuery<T> query) {
return delete(query, true);
}
private <T> int delete(SpiQuery<T> query, boolean permanent) {
SpiOrmQueryRequest<T> request = createQueryRequest(Type.DELETE, query);
try {
request.initTransIfRequired();
@@ -1247,7 +1255,7 @@ public final class DefaultServer implements SpiServer, SpiEbeanServer {
if (ids.isEmpty()) {
return 0;
} else {
return persister.deleteByIds(request.descriptor(), ids, request.transaction(), false);
return persister.deleteByIds(request.descriptor(), ids, request.transaction(), permanent);
}
}
} finally {
@@ -589,7 +589,8 @@ public final class InternalConfiguration {
return QueryPlanManager.NOOP;
}
long threshold = config.getQueryPlanThresholdMicros();
return new CQueryPlanManager(transactionManager, threshold, queryPlanLogger(databasePlatform.platform(), config), extraMetrics);
return new CQueryPlanManager(transactionManager, config.getCurrentTenantProvider(),
threshold, queryPlanLogger(databasePlatform.platform(), config), extraMetrics);
}
/**
@@ -49,6 +49,11 @@ public interface OrmQueryEngine {
*/
<T> int findCount(OrmQueryRequest<T> request);
/**
* Execute the exists query using SELECT EXISTS(...).
*/
<T> boolean findExists(OrmQueryRequest<T> request);
/**
* Execute the find id's query.
*/
@@ -127,6 +127,13 @@ public final class OrmQueryRequest<T> extends BeanRequest implements SpiOrmQuery
}
}
/**
* Return the secondary queries (fetchQuery and lazy joins) extracted from the query detail.
*/
public SpiQuerySecondary secondaryQueries() {
return secondaryQueries;
}
/**
* For use with QueryIterator and secondary queries this returns the minimum
* batch size that should be loaded before executing the secondary queries.
@@ -355,6 +362,11 @@ public final class OrmQueryRequest<T> extends BeanRequest implements SpiOrmQuery
return queryEngine.findCount(this);
}
@Override
public boolean findExists() {
return queryEngine.findExists(this);
}
@Override
public <A> List<A> findIds() {
return queryEngine.findIds(this);
@@ -56,6 +56,11 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
private Object idValue;
private boolean statelessUpdate;
private boolean notifyCache;
/**
* Snapshot of origValues taken before intercept.setLoaded() clears them.
* Used so that cache update code can access old FK values after setLoaded().
*/
private Object[] capturedOrigValues;
/**
* Flag used to detect when only many properties where updated via a cascade. Used to ensure
* appropriate caches are updated in that case.
@@ -122,6 +127,11 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
*/
private List<SaveMany> saveMany;
private InsertOptions insertOptions;
/**
* Set true when an ON CONFLICT NOTHING insert is skipped (0 rows affected).
* Cascade to children must be suppressed to avoid FK violations.
*/
private boolean insertConflictSkipped;
public PersistRequestBean(SpiEbeanServer server, T bean, Object parentBean, BeanManager<T> mgr, SpiTransaction t,
PersistExecute persistExecute, PersistRequest.Type type, int flags) {
@@ -710,7 +720,11 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
* Return the original / old value for the given property.
*/
public Object origValue(BeanProperty prop) {
return intercept.origValue(prop.propertyIndex());
int idx = prop.propertyIndex();
if (capturedOrigValues != null && idx < capturedOrigValues.length) {
return capturedOrigValues[idx];
}
return intercept.origValue(idx);
}
@Override
@@ -801,6 +815,9 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
throw new OptimisticLockException("Data has changed. updated row count " + rowCount, null, bean);
} else if (rowCount == 0 && type == Type.UPDATE) {
throw new EntityNotFoundException("No rows updated");
} else if (rowCount == 0 && type == Type.INSERT && isConflictNothingInsert()) {
insertConflictSkipped = true;
return;
}
}
switch (type) {
@@ -815,6 +832,14 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
}
}
private boolean isConflictNothingInsert() {
return insertOptions != null && insertOptions.key().charAt(0) == 'N';
}
public boolean isInsertConflictSkipped() {
return insertConflictSkipped;
}
/**
* Clear the bean from the PersistenceContext (L1 cache) for stateless updates.
*/
@@ -867,6 +892,15 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
if (type == Type.UPDATE) {
// get the dirty properties for notify cache & orphanRemoval of vanilla collection detection
dirtyProperties = intercept.dirtyProperties();
// snapshot orig values before setLoaded() clears them - needed by cache FK-eviction logic
if (dirtyProperties != null && notifyCache) {
capturedOrigValues = new Object[dirtyProperties.length];
for (int i = 0; i < dirtyProperties.length; i++) {
if (dirtyProperties[i]) {
capturedOrigValues[i] = intercept.origValue(i);
}
}
}
}
if (isChangeLog) {
changeLog();
@@ -903,6 +937,20 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
}
}
/**
* Ensure the preUpdate event fires (for case where only a ManyToMany collection has changed).
*/
public void preManyToManyUpdate() {
if (controller != null && !dirty) {
// fire preUpdate notification when only ManyToMany intersection updated
controller.preUpdate(this);
pendingPostUpdateNotify = true;
}
if (!dirty) {
setNotifyCache();
}
}
public boolean isNotifyCache() {
return notifyCache;
}
@@ -1415,7 +1463,12 @@ public final class PersistRequestBean<T> extends PersistRequest implements BeanP
}
public void setInsertOptions(InsertOptions insertOptions) {
this.insertOptions = insertOptions;
this.insertOptions = insertOptions;
if (insertOptions != null && insertOptions.key().charAt(0) == 'N' && beanDescriptor.hasCascadeChildren()) {
// force immediate (non-batch) execution of this insert so we know whether it
// was actually inserted before attempting cascade saves on children.
skipBatchForTopLevel = true;
}
}
public InsertOptions insertOptions() {
@@ -75,6 +75,11 @@ public interface SpiOrmQueryRequest<T> extends BeanQueryRequest<T>, DocQueryRequ
*/
int findCount();
/**
* Execute the find exists query using select exists(...).
*/
boolean findExists();
/**
* Execute the find ids query.
*/
@@ -10,8 +10,11 @@ import java.sql.SQLException;
*/
final class AssocOneHelpEmbedded extends AssocOneHelp {
AssocOneHelpEmbedded(BeanPropertyAssocOne<?> property) {
private final boolean allowEmpty;
AssocOneHelpEmbedded(BeanPropertyAssocOne<?> property, boolean allowEmpty) {
super(property);
this.allowEmpty = allowEmpty;
}
@Override
@@ -40,7 +43,7 @@ final class AssocOneHelpEmbedded extends AssocOneHelp {
notNull = true;
}
}
if (notNull) {
if (notNull || allowEmpty) {
return embeddedBean;
} else {
return null;
@@ -69,7 +72,7 @@ final class AssocOneHelpEmbedded extends AssocOneHelp {
notNull = true;
}
}
return notNull ? embeddedBean : null;
return (notNull || allowEmpty) ? embeddedBean : null;
}
@Override
@@ -320,7 +320,7 @@ public class BeanDescriptor<T> implements BeanType<T>, STreeType, SpiBeanType {
this.idOnlyReference = isIdOnlyReference(propertiesBaseScalar);
boolean noRelationships = propertiesOne.length + propertiesMany.length == 0;
this.cacheSharableBeans = noRelationships && deploy.getCacheOptions().isReadOnly();
this.cacheHelp = new BeanDescriptorCacheHelp<>(this, owner.cacheManager(), deploy.getCacheOptions(), cacheSharableBeans, propertiesOneImported);
this.cacheHelp = BeanDescriptorCacheHelp.create(this, owner.cacheManager(), deploy.getCacheOptions(), cacheSharableBeans, propertiesOneImported);
this.jsonHelp = initJsonHelp();
this.draftHelp = new BeanDescriptorDraftHelp<>(this);
this.docStoreAdapter = owner.createDocStoreBeanAdapter(this, deploy);
@@ -2402,6 +2402,12 @@ public class BeanDescriptor<T> implements BeanType<T>, STreeType, SpiBeanType {
if (assocProp == null) {
return null;
}
// this method is an entry-point, although it introduces recursive calls via
// buildElPropertyValue -> createElPropertyValue -> buildElGetValue (back to here)
// it seems we can initialize ElPropertyChainBuilder at this point and skip further checks.
if (chain == null) {
chain = new ElPropertyChainBuilder(propName);
}
String remainder = propName.substring(basePos + 1);
return assocProp.buildElPropertyValue(propName, remainder, chain, propertyDeploy);
}
@@ -2413,9 +2419,7 @@ public class BeanDescriptor<T> implements BeanType<T>, STreeType, SpiBeanType {
if (property == null) {
throw new PersistenceException("No property found for [" + propName + "] in expression " + chain.expression());
}
if (property.containsMany()) {
chain.setContainsMany();
}
return chain.add(property).build();
}
@@ -3337,6 +3341,15 @@ public class BeanDescriptor<T> implements BeanType<T>, STreeType, SpiBeanType {
return propertiesManySave;
}
/**
* Return true if this bean has cascade-save children (OneToMany or exported OneToOne)
* that hold a FK back to this bean. Used to decide whether an ON CONFLICT NOTHING
* insert must be executed immediately so the row-count is known before cascading.
*/
public boolean hasCascadeChildren() {
return propertiesManySave.length > 0 || propertiesOneExportedSave.length > 0;
}
/**
* Assoc Many's with delete cascade.
*/
@@ -27,7 +27,7 @@ import static java.lang.System.Logger.Level.*;
*
* @param <T> The entity bean type
*/
final class BeanDescriptorCacheHelp<T> {
abstract class BeanDescriptorCacheHelp<T> {
private static final System.Logger log = CoreLog.internal;
@@ -36,7 +36,7 @@ final class BeanDescriptorCacheHelp<T> {
private static final System.Logger manyLog = AppLog.getLogger("io.ebean.cache.COLL");
private static final System.Logger natLog = AppLog.getLogger("io.ebean.cache.NATKEY");
private final BeanDescriptor<T> desc;
final BeanDescriptor<T> desc;
private final SpiCacheManager cacheManager;
private final CacheOptions cacheOptions;
/**
@@ -44,13 +44,10 @@ final class BeanDescriptorCacheHelp<T> {
*/
private final boolean cacheSharableBeans;
private final boolean invalidateQueryCache;
private final Class<?> beanType;
final Class<?> beanType;
private final String cacheName;
private final BeanPropertyAssocOne<?>[] propertiesOneImported;
private final String[] naturalKey;
private final ServerCache beanCache;
private final ServerCache naturalKeyCache;
private final ServerCache queryCache;
private final boolean noCaching;
private final SpiCacheControl cacheControl;
private final SpiCacheRegion cacheRegion;
@@ -63,6 +60,21 @@ final class BeanDescriptorCacheHelp<T> {
* Set to true if delete changes need to notify cache.
*/
private boolean cacheNotifyOnDelete;
/**
* Set to true if this bean type has owning OneToOne properties pointing to cached targets.
* When true, all persist events must evict the target bean caches regardless of whether
* this bean type itself has a bean cache (bypasses the cacheRegion.isEnabled() gate).
*/
private boolean cacheNotifyOneToOneOwner;
static <T> BeanDescriptorCacheHelp<T> create(BeanDescriptor<T> desc, SpiCacheManager cacheManager, CacheOptions cacheOptions,
boolean cacheSharableBeans, BeanPropertyAssocOne<?>[] propertiesOneImported) {
if ((cacheOptions.isEnableQueryCache() || cacheOptions.isEnableBeanCache()) && cacheManager.isTenantPartitionedCache()) {
return new BeanDescriptorCacheHelpPartitioned<>(desc, cacheManager, cacheOptions, cacheSharableBeans, propertiesOneImported);
} else {
return new BeanDescriptorCacheHelpFixed<>(desc, cacheManager, cacheOptions, cacheSharableBeans, propertiesOneImported);
}
}
BeanDescriptorCacheHelp(BeanDescriptor<T> desc, SpiCacheManager cacheManager, CacheOptions cacheOptions,
boolean cacheSharableBeans, BeanPropertyAssocOne<?>[] propertiesOneImported) {
@@ -75,48 +87,54 @@ final class BeanDescriptorCacheHelp<T> {
this.cacheSharableBeans = cacheSharableBeans;
this.propertiesOneImported = propertiesOneImported;
this.naturalKey = cacheOptions.getNaturalKey();
if (!cacheOptions.isEnableQueryCache()) {
this.queryCache = null;
} else {
this.queryCache = cacheManager.getQueryCache(beanType);
}
if (cacheOptions.isEnableBeanCache()) {
this.beanCache = cacheManager.getBeanCache(beanType);
if (cacheOptions.getNaturalKey() != null) {
this.naturalKeyCache = cacheManager.getNaturalKeyCache(beanType);
} else {
this.naturalKeyCache = null;
}
} else {
this.beanCache = null;
this.naturalKeyCache = null;
}
this.noCaching = (beanCache == null && queryCache == null);
this.noCaching = !cacheOptions.isEnableQueryCache() && !cacheOptions.isEnableBeanCache();
if (noCaching) {
this.cacheControl = DCacheControlNone.INSTANCE;
this.cacheRegion = (invalidateQueryCache) ? cacheManager.getRegion(cacheOptions.getRegion()) : DCacheRegionNone.INSTANCE;
} else {
this.cacheRegion = cacheManager.getRegion(cacheOptions.getRegion());
this.cacheControl = new DCacheControl(cacheRegion, (beanCache != null), (naturalKeyCache != null), (queryCache != null));
this.cacheControl = new DCacheControl(cacheRegion,
cacheOptions.isEnableBeanCache(),
cacheOptions.isEnableBeanCache() && cacheOptions.getNaturalKey() != null,
cacheOptions.isEnableQueryCache());
}
}
abstract boolean hasBeanCache();
abstract boolean hasQueryCache();
abstract ServerCache queryCache();
abstract ServerCache naturalKeyCache();
abstract ServerCache beanCache();
/**
* Derive the cache notify flags.
*/
void deriveNotifyFlags() {
cacheNotifyOnAll = (invalidateQueryCache || beanCache != null || queryCache != null);
cacheNotifyOnAll = (invalidateQueryCache || hasBeanCache() || hasQueryCache());
cacheNotifyOnDelete = !cacheNotifyOnAll && isNotifyOnDeletes();
cacheNotifyOneToOneOwner = hasOwningOneToOneWithCachedTarget();
if (log.isLoggable(DEBUG)) {
if (cacheNotifyOnAll || cacheNotifyOnDelete) {
String notifyMode = cacheNotifyOnAll ? "All" : "Delete";
if (cacheNotifyOnAll || cacheNotifyOnDelete || cacheNotifyOneToOneOwner) {
String notifyMode = cacheNotifyOnAll ? "All" : (cacheNotifyOnDelete ? "Delete" : "OneToOneOwner");
log.log(DEBUG, "l2 caching on {0} - beanCaching:{1} queryCaching:{2} notifyMode:{3} ",
desc.fullName(), isBeanCaching(), isQueryCaching(), notifyMode);
}
}
}
private boolean hasOwningOneToOneWithCachedTarget() {
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
if (imported.isCacheNotifyOwningOneToOne()) {
return true;
}
}
return false;
}
/**
* Return true if there is an imported bi-directional relationship to a bea
* that does have bean caching enabled.
@@ -137,6 +155,9 @@ final class BeanDescriptorCacheHelp<T> {
if (hasImmutableCaches()) {
return true;
}
if (cacheNotifyOneToOneOwner) {
return true;
}
return cacheRegion.isEnabled()
&& (cacheNotifyOnAll || cacheNotifyOnDelete && (type == PersistRequest.Type.DELETE || type == PersistRequest.Type.DELETE_PERMANENT));
}
@@ -184,11 +205,11 @@ final class BeanDescriptorCacheHelp<T> {
* Clear the query cache.
*/
void queryCacheClear() {
if (queryCache != null) {
if (hasQueryCache()) {
if (queryLog.isLoggable(DEBUG)) {
queryLog.log(DEBUG, " CLEAR {0}", cacheName);
}
queryCache.clear();
queryCache().clear();
}
}
@@ -196,7 +217,7 @@ final class BeanDescriptorCacheHelp<T> {
* Add query cache clear to the changeSet.
*/
private void queryCacheClear(CacheChangeSet changeSet) {
if (queryCache != null) {
if (hasQueryCache()) {
changeSet.addClearQuery(desc);
}
}
@@ -205,10 +226,10 @@ final class BeanDescriptorCacheHelp<T> {
* Get a query result from the query cache.
*/
Object queryCacheGet(Object id) {
if (queryCache == null) {
if (!hasQueryCache()) {
throw new IllegalStateException("No query cache enabled on " + desc + ". Need explicit @Cache(enableQueryCache=true)");
}
Object queryResult = queryCache.get(id);
Object queryResult = queryCache().get(id);
if (queryLog.isLoggable(DEBUG)) {
if (queryResult == null) {
queryLog.log(DEBUG, " GET {0}({1}) - cache miss", cacheName, id);
@@ -223,13 +244,13 @@ final class BeanDescriptorCacheHelp<T> {
* Put a query result into the query cache.
*/
void queryCachePut(Object id, QueryCacheEntry entry) {
if (queryCache == null) {
if (!hasQueryCache()) {
throw new IllegalStateException("No query cache enabled on " + desc + ". Need explicit @Cache(enableQueryCache=true)");
}
if (queryLog.isLoggable(DEBUG)) {
queryLog.log(DEBUG, " PUT {0}({1})", cacheName, id);
}
queryCache.put(id, entry);
queryCache().put(id, entry);
}
void manyPropRemove(String propertyName, String parentKey) {
@@ -300,7 +321,7 @@ final class BeanDescriptorCacheHelp<T> {
*/
void manyPropPut(BeanPropertyAssocMany<?> many, Object details, String parentKey) {
if (many.isElementCollection()) {
CachedBeanData data = (CachedBeanData) beanCache.get(parentKey);
CachedBeanData data = (CachedBeanData) beanCache().get(parentKey);
if (data != null) {
try {
// add as JSON to bean cache
@@ -312,7 +333,7 @@ final class BeanDescriptorCacheHelp<T> {
if (beanLog.isLoggable(DEBUG)) {
beanLog.log(DEBUG, " UPDATE {0}({1}) changes:{2}", cacheName, parentKey, changes);
}
beanCache.put(parentKey, newData);
beanCache().put(parentKey, newData);
} catch (IOException e) {
log.log(ERROR, "Error updating L2 cache", e);
}
@@ -358,7 +379,7 @@ final class BeanDescriptorCacheHelp<T> {
if (ids.isEmpty()) {
return new BeanCacheResult<>();
}
Map<Object, Object> beanDataMap = beanCache.getAll(keys);
Map<Object, Object> beanDataMap = beanCache().getAll(keys);
if (beanLog.isLoggable(TRACE)) {
beanLog.log(TRACE, " MGET {0}({1}) - hits:{2}", cacheName, ids, beanDataMap.keySet());
}
@@ -380,7 +401,7 @@ final class BeanDescriptorCacheHelp<T> {
}
// naturalKey -> Id map
Map<Object, Object> naturalKeyMap = naturalKeyCache.getAll(keys);
Map<Object, Object> naturalKeyMap = naturalKeyCache().getAll(keys);
if (natLog.isLoggable(TRACE)) {
natLog.log(TRACE, " MLOOKUP {0}({1}) - hits:{2}", cacheName, keys, naturalKeyMap);
}
@@ -397,7 +418,7 @@ final class BeanDescriptorCacheHelp<T> {
}
Set<Object> ids = new HashSet<>(naturalKeyMap.values());
Map<Object, Object> beanDataMap = beanCache.getAll(ids);
Map<Object, Object> beanDataMap = beanCache().getAll(ids);
if (beanLog.isLoggable(TRACE)) {
beanLog.log(TRACE, " MGET {0}({1}) - hits:{2}", cacheName, ids, beanDataMap.keySet());
}
@@ -428,25 +449,15 @@ final class BeanDescriptorCacheHelp<T> {
desc.contextPut(context, id, bean);
}
/**
* Return the beanCache creating it if necessary.
*/
private ServerCache getBeanCache() {
if (beanCache == null) {
throw new IllegalStateException("No bean cache enabled for " + desc + ". Add the @Cache annotation.");
}
return beanCache;
}
/**
* Clear the bean cache.
*/
void beanCacheClear() {
if (beanCache != null) {
if (hasBeanCache()) {
if (beanLog.isLoggable(DEBUG)) {
beanLog.log(DEBUG, " CLEAR {0}", cacheName);
}
beanCache.clear();
beanCache().clear();
}
}
@@ -516,13 +527,13 @@ final class BeanDescriptorCacheHelp<T> {
if (beanLog.isLoggable(DEBUG)) {
beanLog.log(DEBUG, " MPUT {0}({1})", cacheName, map.keySet());
}
getBeanCache().putAll(map);
beanCache().putAll(map);
if (natKeys != null && !natKeys.isEmpty()) {
if (natLog.isLoggable(DEBUG)) {
natLog.log(DEBUG, " MPUT {0}({1}, {2})", cacheName, Arrays.toString(naturalKey), natKeys.keySet());
}
naturalKeyCache.putAll(natKeys);
naturalKeyCache().putAll(natKeys);
}
}
@@ -535,14 +546,14 @@ final class BeanDescriptorCacheHelp<T> {
if (beanLog.isLoggable(DEBUG)) {
beanLog.log(DEBUG, " PUT {0}({1}) data:{2}", cacheName, key, beanData);
}
getBeanCache().put(key, beanData);
beanCache().put(key, beanData);
if (naturalKey != null) {
String naturalKey = calculateNaturalKey(beanData);
if (naturalKey != null) {
if (natLog.isLoggable(DEBUG)) {
natLog.log(DEBUG, " PUT {0}({1}, {2})", cacheName, naturalKey, key);
}
naturalKeyCache.put(naturalKey, key);
naturalKeyCache().put(naturalKey, key);
}
}
}
@@ -564,7 +575,7 @@ final class BeanDescriptorCacheHelp<T> {
}
CachedBeanData beanCacheGetData(String key) {
return (CachedBeanData) getBeanCache().get(key);
return (CachedBeanData) beanCache().get(key);
}
T beanCacheGet(String key, boolean unmodifiable, PersistenceContext context) {
@@ -579,7 +590,7 @@ final class BeanDescriptorCacheHelp<T> {
* Return a bean from the bean cache.
*/
private T beanCacheGetInternal(String key, boolean unmodifiable, PersistenceContext context) {
CachedBeanData data = (CachedBeanData) getBeanCache().get(key);
CachedBeanData data = (CachedBeanData) beanCache().get(key);
if (data == null) {
if (beanLog.isLoggable(TRACE)) {
beanLog.log(TRACE, " GET {0}({1}) - cache miss", cacheName, key);
@@ -684,11 +695,11 @@ final class BeanDescriptorCacheHelp<T> {
* Remove a bean from the cache given its Id.
*/
void beanCacheApplyInvalidate(Collection<String> keys) {
if (beanCache != null) {
if (hasBeanCache()) {
if (beanLog.isLoggable(DEBUG)) {
beanLog.log(DEBUG, " MREMOVE {0}({1})", cacheName, keys);
}
beanCache.removeAll(new HashSet<>(keys));
beanCache().removeAll(new HashSet<>(keys));
}
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
imported.cacheClear();
@@ -704,7 +715,7 @@ final class BeanDescriptorCacheHelp<T> {
ebis.put(desc.cacheKeyForBean(ebi.owner()), ebi);
}
Map<Object, Object> hits = getBeanCache().getAll(ebis.keySet());
Map<Object, Object> hits = beanCache().getAll(ebis.keySet());
if (beanLog.isLoggable(TRACE)) {
beanLog.log(TRACE, " MLOAD {0}({1}) - got hits ({2})", cacheName, ebis.keySet(), hits.size());
}
@@ -737,7 +748,7 @@ final class BeanDescriptorCacheHelp<T> {
* Returns true if it managed to populate/load the single bean from the cache.
*/
boolean beanCacheLoad(EntityBean bean, EntityBeanIntercept ebi, String key, PersistenceContext context) {
CachedBeanData cacheData = (CachedBeanData) getBeanCache().get(key);
CachedBeanData cacheData = (CachedBeanData) beanCache().get(key);
if (cacheData == null) {
if (beanLog.isLoggable(TRACE)) {
beanLog.log(TRACE, " LOAD {0}({1}) - cache miss", cacheName, key);
@@ -759,7 +770,7 @@ final class BeanDescriptorCacheHelp<T> {
}
void cacheUpdateQuery(boolean update, SpiTransaction transaction) {
if (invalidateQueryCache || cacheNotifyOnAll || (!update && cacheNotifyOnDelete)) {
if (invalidateQueryCache || cacheNotifyOnAll || cacheNotifyOneToOneOwner || (!update && cacheNotifyOnDelete)) {
transaction.event().add(desc.baseTable(), false, update, !update);
}
}
@@ -775,10 +786,11 @@ final class BeanDescriptorCacheHelp<T> {
changeSet.addInvalidate(desc);
} else {
queryCacheClear(changeSet);
if (beanCache != null) {
if (hasBeanCache()) {
changeSet.addBeanRemoveMany(desc, ids);
}
cacheDeleteImported(true, null, changeSet);
cacheClearImportedOneToOne(changeSet);
}
}
@@ -793,10 +805,11 @@ final class BeanDescriptorCacheHelp<T> {
changeSet.addInvalidate(desc);
} else {
queryCacheClear(changeSet);
if (beanCache != null) {
if (hasBeanCache()) {
changeSet.addBeanRemove(desc, id);
}
cacheDeleteImported(true, deleteRequest.entityBean(), changeSet);
cacheDeleteImportedOneToOne(deleteRequest.entityBean(), changeSet);
}
}
@@ -809,6 +822,7 @@ final class BeanDescriptorCacheHelp<T> {
} else {
queryCacheClear(changeSet);
cacheDeleteImported(false, insertRequest.entityBean(), changeSet);
cacheDeleteImportedOneToOne(insertRequest.entityBean(), changeSet);
changeSet.addBeanInsert(desc.baseTable());
}
}
@@ -819,6 +833,42 @@ final class BeanDescriptorCacheHelp<T> {
}
}
/**
* Evict the target bean cache for each owning OneToOne property on insert or delete.
*/
private void cacheDeleteImportedOneToOne(EntityBean entityBean, CacheChangeSet changeSet) {
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
imported.cacheDeleteOneToOneOwned(entityBean, changeSet);
}
}
/**
* Clear the target bean cache for each owning OneToOne property on bulk delete-by-ids.
*/
private void cacheClearImportedOneToOne(CacheChangeSet changeSet) {
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
imported.cacheClearOneToOneOwned(changeSet);
}
}
/**
* Evict inverse caches for any imported FK properties whose value changed in this update.
* Handles collection-ids caches (ManyToOne FK reassignment) and bean caches (owning OneToOne FK change).
*/
private void cacheUpdateImportedFKs(PersistRequestBean<T> updateRequest, CacheChangeSet changeSet) {
boolean[] dirty = updateRequest.dirtyProperties();
if (dirty == null) {
return;
}
EntityBean entityBean = updateRequest.entityBean();
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
int propIdx = imported.propertyIndex();
if (propIdx < dirty.length && dirty[propIdx]) {
imported.cacheUpdateFKchange(entityBean, updateRequest.origValue(imported), changeSet);
}
}
}
/**
* Add appropriate changes to support update.
*/
@@ -831,7 +881,8 @@ final class BeanDescriptorCacheHelp<T> {
} else {
queryCacheClear(changeSet);
if (beanCache == null) {
cacheUpdateImportedFKs(updateRequest, changeSet);
if (!hasBeanCache()) {
// query caching only
return;
}
@@ -861,15 +912,17 @@ final class BeanDescriptorCacheHelp<T> {
changeSet.addInvalidate(desc);
return;
}
if (noCaching) {
if (noCaching && !cacheNotifyOneToOneOwner) {
return;
}
changeSet.addClearQuery(desc);
// inserts don't invalidate the bean cache
if (tableIUD.isUpdateOrDelete()) {
changeSet.addClearBean(desc);
if (!noCaching) {
changeSet.addClearQuery(desc);
// inserts don't invalidate the bean cache
if (tableIUD.isUpdateOrDelete()) {
changeSet.addClearBean(desc);
}
}
// any change invalidates the collection IDs cache
// any change invalidates the collection IDs cache and owning OneToOne target bean caches
for (BeanPropertyAssocOne<?> imported : propertiesOneImported) {
imported.cacheClear(changeSet);
}
@@ -877,7 +930,7 @@ final class BeanDescriptorCacheHelp<T> {
void cacheNaturalKeyPut(String key, String newKey) {
if (newKey != null) {
naturalKeyCache.put(newKey, key);
naturalKeyCache().put(newKey, key);
}
}
@@ -885,7 +938,7 @@ final class BeanDescriptorCacheHelp<T> {
* Apply changes to the bean cache entry.
*/
void cacheBeanUpdate(String key, Map<String, Object> changes, boolean updateNaturalKey, long version) {
ServerCache cache = getBeanCache();
ServerCache cache = beanCache();
CachedBeanData existingData = (CachedBeanData) cache.get(key);
if (existingData != null) {
long currentVersion = existingData.getVersion();
@@ -910,7 +963,7 @@ final class BeanDescriptorCacheHelp<T> {
if (natLog.isLoggable(DEBUG)) {
natLog.log(DEBUG, ".. update {0} REMOVE({1}) - old key for ({2})", cacheName, oldKey, key);
}
naturalKeyCache.remove(oldKey);
naturalKeyCache().remove(oldKey);
}
}
}
@@ -0,0 +1,63 @@
package io.ebeaninternal.server.deploy;
import io.ebean.cache.ServerCache;
import io.ebeaninternal.server.cache.SpiCacheManager;
import io.ebeaninternal.server.core.CacheOptions;
/**
* BeanDescriptorCacheHelp implementation for non-tenant-partitioned (fixed) caches.
* Cache instances are obtained once at construction time and reused for all requests.
*
* @param <T> The entity bean type
*/
final class BeanDescriptorCacheHelpFixed<T> extends BeanDescriptorCacheHelp<T> {
private final ServerCache beanCache;
private final ServerCache naturalKeyCache;
private final ServerCache queryCache;
BeanDescriptorCacheHelpFixed(BeanDescriptor<T> desc, SpiCacheManager cacheManager, CacheOptions cacheOptions,
boolean cacheSharableBeans, BeanPropertyAssocOne<?>[] propertiesOneImported) {
super(desc, cacheManager, cacheOptions, cacheSharableBeans, propertiesOneImported);
if (!cacheOptions.isEnableQueryCache()) {
this.queryCache = null;
} else {
this.queryCache = cacheManager.getQueryCache(beanType);
}
if (cacheOptions.isEnableBeanCache()) {
this.beanCache = cacheManager.getBeanCache(beanType);
this.naturalKeyCache = (cacheOptions.getNaturalKey() != null) ? cacheManager.getNaturalKeyCache(beanType) : null;
} else {
this.beanCache = null;
this.naturalKeyCache = null;
}
}
@Override
boolean hasBeanCache() {
return beanCache != null;
}
@Override
boolean hasQueryCache() {
return queryCache != null;
}
@Override
ServerCache queryCache() {
return queryCache;
}
@Override
ServerCache naturalKeyCache() {
return naturalKeyCache;
}
@Override
ServerCache beanCache() {
if (beanCache == null) {
throw new IllegalStateException("No bean cache enabled for " + desc + ". Add the @Cache annotation.");
}
return beanCache;
}
}
@@ -0,0 +1,62 @@
package io.ebeaninternal.server.deploy;
import io.ebean.cache.ServerCache;
import io.ebeaninternal.server.cache.SpiCacheManager;
import io.ebeaninternal.server.core.CacheOptions;
import java.util.function.Supplier;
/**
* BeanDescriptorCacheHelp implementation for tenant-partitioned caches.
* Cache instances are looked up on every access via the cacheManager (which resolves the
* correct tenant-namespaced cache using the current tenant id from the tenant provider).
*
* @param <T> The entity bean type
*/
final class BeanDescriptorCacheHelpPartitioned<T> extends BeanDescriptorCacheHelp<T> {
private final Supplier<ServerCache> beanCacheSupplier;
private final Supplier<ServerCache> naturalKeyCacheSupplier;
private final Supplier<ServerCache> queryCacheSupplier;
BeanDescriptorCacheHelpPartitioned(BeanDescriptor<T> desc, SpiCacheManager cacheManager, CacheOptions cacheOptions,
boolean cacheSharableBeans, BeanPropertyAssocOne<?>[] propertiesOneImported) {
super(desc, cacheManager, cacheOptions, cacheSharableBeans, propertiesOneImported);
this.queryCacheSupplier = cacheOptions.isEnableQueryCache() ? () -> cacheManager.getQueryCache(beanType) : null;
if (cacheOptions.isEnableBeanCache()) {
this.beanCacheSupplier = () -> cacheManager.getBeanCache(beanType);
this.naturalKeyCacheSupplier = (cacheOptions.getNaturalKey() != null) ? () -> cacheManager.getNaturalKeyCache(beanType) : null;
} else {
this.beanCacheSupplier = null;
this.naturalKeyCacheSupplier = null;
}
}
@Override
boolean hasBeanCache() {
return beanCacheSupplier != null;
}
@Override
boolean hasQueryCache() {
return queryCacheSupplier != null;
}
@Override
ServerCache queryCache() {
return queryCacheSupplier.get();
}
@Override
ServerCache naturalKeyCache() {
return naturalKeyCacheSupplier.get();
}
@Override
ServerCache beanCache() {
if (beanCacheSupplier == null) {
throw new IllegalStateException("No bean cache enabled for " + desc + ". Add the @Cache annotation.");
}
return beanCacheSupplier.get();
}
}
@@ -161,13 +161,7 @@ public abstract class BeanPropertyAssoc<T> extends BeanProperty implements STree
ElPropertyValue createElPropertyValue(String propName, String remainder, ElPropertyChainBuilder chain, boolean propertyDeploy) {
// associated or embedded bean
BeanDescriptor<?> embDesc = targetDescriptor();
if (chain == null) {
chain = new ElPropertyChainBuilder(isEmbedded(), propName);
}
chain.add(this);
if (containsMany()) {
chain.setContainsMany();
}
return embDesc.buildElGetValue(remainder, chain, propertyDeploy);
}
@@ -217,6 +211,10 @@ public abstract class BeanPropertyAssoc<T> extends BeanProperty implements STree
* Return the BeanDescriptor of the target.
*/
public BeanDescriptor<T> targetDescriptor() {
if (targetDescriptor == null) {
// lazily resolve for association properties inside an @Embeddable used as override copy
targetDescriptor = descriptor.descriptor(targetType);
}
return targetDescriptor;
}
@@ -45,6 +45,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
private final boolean orphanRemoval;
private final boolean primaryKeyExport;
private final boolean primaryKeyJoin;
private final boolean embeddedAllowEmpty;
private AssocOneHelp localHelp;
final BeanProperty[] embeddedProps;
@@ -54,6 +55,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
private String deleteByParentIdInSql;
private BeanPropertyAssocMany<?> relationshipProperty;
private boolean cacheNotifyRelationship;
private boolean cacheNotifyOwningOneToOne;
/**
* Create based on deploy information of an EmbeddedId.
@@ -73,6 +75,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
oneToOne = deploy.isOneToOne();
oneToOneExported = deploy.isOneToOneExported();
orphanRemoval = deploy.isOrphanRemoval();
embeddedAllowEmpty = deploy.isEmbeddedAllowEmpty();
if (embedded) {
// Overriding of the columns and use table alias of owning BeanDescriptor
BeanEmbeddedMeta overrideMeta = BeanEmbeddedMetaFactory.create(owner, deploy);
@@ -99,6 +102,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
oneToOne = source.oneToOne;
oneToOneExported = source.oneToOneExported;
orphanRemoval = source.orphanRemoval;
embeddedAllowEmpty = source.embeddedAllowEmpty;
embeddedProps = null;
embeddedPropsMap = null;
}
@@ -143,6 +147,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
*/
void initialisePostTarget() {
this.cacheNotifyRelationship = isCacheNotifyRelationship();
this.cacheNotifyOwningOneToOne = oneToOne && !oneToOneExported && targetDescriptor.isBeanCaching();
}
/**
@@ -165,18 +170,31 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
}
/**
* Clear the L2 relationship cache for this property.
* Return true if this owning OneToOne property requires target bean cache eviction on change.
*/
boolean isCacheNotifyOwningOneToOne() {
return cacheNotifyOwningOneToOne;
}
/**
* Clear the L2 relationship cache for this property (used from beanCacheApplyInvalidate).
*/
void cacheClear() {
if (cacheNotifyRelationship) {
targetDescriptor.cacheManyPropClear(relationshipProperty.name());
}
if (cacheNotifyOwningOneToOne) {
targetDescriptor.clearBeanCache();
}
}
void cacheClear(CacheChangeSet changeSet) {
if (cacheNotifyRelationship) {
changeSet.addManyClear(targetDescriptor, relationshipProperty.name());
}
if (cacheNotifyOwningOneToOne) {
changeSet.addClearBean(targetDescriptor);
}
}
/**
@@ -199,6 +217,66 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
}
}
/**
* Evict the target's bean cache entry when this owning OneToOne bean is inserted or deleted.
*/
void cacheDeleteOneToOneOwned(EntityBean bean, CacheChangeSet changeSet) {
if (cacheNotifyOwningOneToOne) {
Object assocBean = getValue(bean);
if (assocBean != null) {
Object targetId = targetDescriptor.id(assocBean);
if (targetId != null) {
changeSet.addBeanRemove(targetDescriptor, targetId);
}
}
}
}
/**
* Clear the target's entire bean cache when owning OneToOne beans are deleted by IDs.
*/
void cacheClearOneToOneOwned(CacheChangeSet changeSet) {
if (cacheNotifyOwningOneToOne) {
changeSet.addClearBean(targetDescriptor);
}
}
/**
* Evict the inverse caches for BOTH old and new targets when the FK changes on update.
* Handles collection-ids caches (ManyToOne) and bean caches (owning OneToOne).
*/
void cacheUpdateFKchange(EntityBean bean, Object origAssocBean, CacheChangeSet changeSet) {
Object newAssocBean = getValue(bean);
if (cacheNotifyRelationship) {
if (origAssocBean != null) {
Object oldId = targetDescriptor.id(origAssocBean);
if (oldId != null) {
changeSet.addManyRemove(targetDescriptor, relationshipProperty.name(), targetDescriptor.cacheKey(oldId));
}
}
if (newAssocBean != null) {
Object newId = targetDescriptor.id(newAssocBean);
if (newId != null) {
changeSet.addManyRemove(targetDescriptor, relationshipProperty.name(), targetDescriptor.cacheKey(newId));
}
}
}
if (cacheNotifyOwningOneToOne) {
if (origAssocBean != null) {
Object oldId = targetDescriptor.id(origAssocBean);
if (oldId != null) {
changeSet.addBeanRemove(targetDescriptor, oldId);
}
}
if (newAssocBean != null) {
Object newId = targetDescriptor.id(newAssocBean);
if (newId != null) {
changeSet.addBeanRemove(targetDescriptor, newId);
}
}
}
}
@Override
Object naturalKeyVal(Map<String, Object> values) {
EntityBean bean = (EntityBean) values.get(name);
@@ -211,15 +289,31 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
@Override
public ElPropertyValue buildElPropertyValue(String propName, String remainder, ElPropertyChainBuilder chain, boolean propertyDeploy) {
if (embedded) {
BeanProperty embProp = embeddedPropsMap.get(remainder);
String embName = remainder;
String embRemainder = null;
int basePos = remainder.indexOf('.');
if (basePos > -1) {
embName = remainder.substring(0, basePos);
embRemainder = remainder.substring(basePos + 1);
}
BeanProperty embProp = embeddedPropsMap.get(embName);
if (embProp == null) {
String msg = "Embedded Property " + remainder + " not found in " + fullName();
String msg = "Embedded Property " + embName + " not found in " + fullName();
throw new PersistenceException(msg);
}
if (chain == null) {
chain = new ElPropertyChainBuilder(true, propName);
}
chain.add(this);
if (embRemainder != null) {
if (embProp instanceof BeanPropertyAssocOne) {
// @ManyToOne using the overridden property, e.g. "ma_country_code" instead of "country_code")
return embProp.buildElPropertyValue(propName, embRemainder, chain, propertyDeploy);
}
// scalar (or non-navigable) leaf with a further path segment - invalid path
String msg = "Embedded Property " + embName + "." + embRemainder + " not found in " + fullName();
throw new PersistenceException(msg);
}
// direct scalar/assoc access within embedded — mark as embedded so the prefix
// strips the embedded segment (keeps parent table alias, not the embedded segment)
chain.setEmbedded(true);
return chain.add(embProp).build();
}
@@ -749,7 +843,7 @@ public class BeanPropertyAssocOne<T> extends BeanPropertyAssoc<T> implements STr
private AssocOneHelp createHelp(boolean embedded, boolean oneToOneExported, String embeddedPrefix) {
if (embedded) {
return new AssocOneHelpEmbedded(this);
return new AssocOneHelpEmbedded(this, embeddedAllowEmpty);
} else if (oneToOneExported) {
return new AssocOneHelpRefExported(this);
} else {
@@ -13,6 +13,7 @@ public final class DeployBeanPropertyAssocOne<T> extends DeployBeanPropertyAssoc
private boolean oneToOneExported;
private boolean primaryKeyJoin;
private boolean primaryKeyExport;
private boolean embeddedAllowEmpty;
private DeployBeanEmbedded deployEmbedded;
private String columnPrefix;
@@ -103,6 +104,14 @@ public final class DeployBeanPropertyAssocOne<T> extends DeployBeanPropertyAssoc
}
}
public void setEmbeddedAllowEmpty(boolean allowEmpty) {
this.embeddedAllowEmpty = allowEmpty;
}
public boolean isEmbeddedAllowEmpty() {
return embeddedAllowEmpty;
}
public void setColumnPrefix(String columnPrefix) {
this.columnPrefix = columnPrefix;
}
@@ -245,6 +245,13 @@ final class AnnotationAssocOnes extends AnnotationAssoc {
} catch (NoSuchMethodError e) {
// using standard JPA API without prefix option, maybe in EE container
}
try {
if (!embedded.nullable()) {
prop.setEmbeddedAllowEmpty(true);
}
} catch (NoSuchMethodError e) {
// older persistence-api without nullable() extension
}
readEmbeddedAttributeOverrides(prop);
}
@@ -42,17 +42,19 @@ public final class ElPropertyChain implements ElPropertyValue {
private final ScalarType<?> scalarType;
private final ElPropertyValue lastElPropertyValue;
public ElPropertyChain(boolean containsMany, boolean embedded, String expression, ElPropertyValue[] chain) {
this.containsMany = containsMany;
public ElPropertyChain(String expression, boolean containsMany, boolean embedded, ElPropertyValue[] chain) {
this.chain = chain;
this.expression = expression;
this.containsMany = containsMany;
int dotPos = expression.lastIndexOf('.');
if (dotPos > -1) {
this.name = expression.substring(dotPos + 1);
if (embedded) {
// embedded segments are transparent (share parent table) — strip the embedded
// segment from the prefix so the alias points to the parent join, not the embedded
int embPos = expression.lastIndexOf('.', dotPos - 1);
this.prefix = embPos == -1 ? null : expression.substring(0, embPos);
} else {
this.prefix = expression.substring(0, dotPos);
}
@@ -61,19 +63,18 @@ public final class ElPropertyChain implements ElPropertyValue {
this.name = expression;
}
this.assocId = chain[chain.length - 1].isAssocId();
this.last = chain.length - 1;
this.lastBeanProperty = chain[chain.length - 1].beanProperty();
this.last = this.chain.length - 1;
this.lastElPropertyValue = this.chain[this.last];
this.assocId = this.lastElPropertyValue.isAssocId();
this.lastBeanProperty = lastElPropertyValue.beanProperty();
if (lastBeanProperty != null) {
this.scalarType = lastBeanProperty.scalarType();
} else {
// case for nested compound type (non-scalar)
this.scalarType = null;
}
this.lastElPropertyValue = chain[chain.length - 1];
this.placeHolder = placeHolder(prefix, lastElPropertyValue, false);
this.placeHolderEncrypted = placeHolder(prefix, lastElPropertyValue, true);
this.placeHolder = placeHolder(this.prefix, lastElPropertyValue, false);
this.placeHolderEncrypted = placeHolder(this.prefix, lastElPropertyValue, true);
}
@Override
@@ -17,14 +17,13 @@ public final class ElPropertyChainBuilder {
private final String expression;
private final List<ElPropertyValue> chain = new ArrayList<>();
private boolean embedded;
private boolean containsMany;
private boolean containsMany = false;
private boolean embedded = false;
/**
* Create with the original expression.
*/
public ElPropertyChainBuilder(boolean embedded, String expression) {
this.embedded = embedded;
public ElPropertyChainBuilder(String expression) {
this.expression = expression;
}
@@ -32,14 +31,18 @@ public final class ElPropertyChainBuilder {
return containsMany;
}
public void setContainsMany() {
this.containsMany = true;
}
public String expression() {
return expression;
}
/**
* Mark the chain as going through an embedded property.
* This affects how the prefix (table alias) is computed for SQL generation.
*/
public void setEmbedded(boolean embedded) {
this.embedded = embedded;
}
/**
* Add a ElGetValue element to the chain.
*/
@@ -48,6 +51,9 @@ public final class ElPropertyChainBuilder {
throw new NullPointerException("element null in expression " + expression);
}
chain.add(element);
if (element.containsMany()) {
containsMany = true;
}
return this;
}
@@ -55,13 +61,6 @@ public final class ElPropertyChainBuilder {
* Build the immutable ElGetChain from the build information.
*/
public ElPropertyChain build() {
return new ElPropertyChain(containsMany, embedded, expression, chain.toArray(new ElPropertyValue[0]));
}
/**
* Permits to set whole chain as embedded when the leaf is embedded
*/
public void setEmbedded(boolean embedded) {
this.embedded = embedded;
return new ElPropertyChain(expression, containsMany, embedded, chain.toArray(new ElPropertyValue[0]));
}
}
@@ -328,6 +328,11 @@ public class DefaultExpressionList<T> implements SpiExpressionList<T> {
return query.delete();
}
@Override
public int deletePermanent() {
return query.deletePermanent();
}
@Override
public int update() {
return query.update();
@@ -342,6 +342,11 @@ final class JunctionExpression<T> implements SpiJunction<T>, SpiExpression, Expr
return exprList.delete();
}
@Override
public int deletePermanent() {
return exprList.deletePermanent();
}
@Override
public int update() {
return exprList.update();
@@ -481,7 +481,7 @@ public final class DefaultPersister implements Persister {
saveAssocOne(request);
}
request.executeOrQueue();
if (request.isPersistCascade()) {
if (request.isPersistCascade() && !request.isInsertConflictSkipped()) {
// save any associated List held beans
saveAssocMany(request);
}
@@ -292,6 +292,17 @@ final class SaveManyBeans extends SaveManyBase {
manyValue.modifyReset();
}
if (!insertedParent) {
// only fire controller when the intersection actually changes:
// vanillaCollection/forcedUpdate always do a full replace,
// tracked BeanCollection fires only when additions or removals exist
boolean intersectionChanged = vanillaCollection || forcedUpdate
|| (additions != null && !additions.isEmpty())
|| (deletions != null && !deletions.isEmpty());
if (intersectionChanged) {
request.preManyToManyUpdate();
}
}
transaction.depth(+1);
if (deletions != null && !deletions.isEmpty()) {
for (Object other : deletions) {
@@ -97,7 +97,9 @@ public abstract class DmlHandler implements PersistHandler, BindableRequest {
void checkRowCount(int rowCount) throws OptimisticLockException {
try {
persistRequest.checkRowCount(rowCount);
persistRequest.postExecute();
if (!persistRequest.isInsertConflictSkipped()) {
persistRequest.postExecute();
}
} catch (OptimisticLockException e) {
// add the SQL and bind values to error message
final String m = e.getMessage() + " sql[" + sql + "] bind[" + bindLog + "]";
@@ -0,0 +1,114 @@
package io.ebeaninternal.server.persist.dml;
import io.ebean.InsertOptions;
import io.ebeaninternal.server.deploy.BeanDescriptor;
import io.ebeaninternal.server.deploy.BeanProperty;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;
/**
* SQLite specific generation of insert on conflict.
* <p>
* SQLite supports the same upsert syntax as Postgres but does not support
* ON CONFLICT ON CONSTRAINT - only ON CONFLICT (column-list) is supported.
*/
final class InsertMetaOptionsSqlite implements InsertMetaOptions {
private final InsertMeta meta;
private final BeanDescriptor<?> desc;
private final String baseTable;
private final Map<String, String> sqlCache = new ConcurrentHashMap<>();
InsertMetaOptionsSqlite(InsertMeta meta, BeanDescriptor<?> desc) {
this.meta = meta;
this.desc = desc;
this.baseTable = desc.baseTable();
}
@Override
public String sql(boolean withId, InsertOptions options) {
String key = withId + options.key();
return sqlCache.computeIfAbsent(key, k -> generate(withId, options));
}
private String generate(boolean withId, InsertOptions options) {
char type = options.key().charAt(0);
switch (type) {
case 'U':
return generate(withId, false, options);
case 'N':
return generate(withId, true, options);
default:
return meta.sqlFor(withId);
}
}
private String generate(boolean withId, boolean doNothing, InsertOptions options) {
if (options.constraint() != null) {
throw new UnsupportedOperationException("SQLite does not support ON CONFLICT ON CONSTRAINT - use uniqueColumns() instead");
}
GenerateDmlRequest request = new GenerateDmlRequest();
meta.sql(request, !withId, baseTable, false);
request.append(" on conflict (");
List<String> uniqueColumns = desc.uniqueProps().stream()
.flatMap(Arrays::stream)
.map(BeanProperty::dbColumn)
.collect(Collectors.toList());
String cols = options.uniqueColumns();
if (cols != null) {
request.append(cols);
} else {
appendUniqueColumns(uniqueColumns, request);
}
request.append(")");
if (doNothing) {
request.append(" do nothing");
return request.toString();
}
request.append(" do update set ");
String updateSet = options.updateSet();
if (updateSet != null) {
request.append(updateSet);
} else {
setColumns(withId, request, uniqueColumns);
}
return request.toString();
}
private void setColumns(boolean withId, GenerateDmlRequest request, List<String> uniqueColumns) {
List<String> columns = request.columns();
columns.removeAll(uniqueColumns);
if (withId) {
BeanProperty idProperty = desc.idProperty();
if (idProperty != null && !idProperty.isEmbedded()) {
columns.remove(idProperty.dbColumn());
}
}
for (int i = 0; i < columns.size(); i++) {
if (i > 0) {
request.append(", ");
}
String col = columns.get(i);
request.append(col).append("=excluded.").append(col);
}
}
private static void appendUniqueColumns(List<String> uniqueColumns, GenerateDmlRequest request) {
if (uniqueColumns.isEmpty()) {
throw new IllegalStateException("Unable to identify unique columns for INSERT ON CONFLICT - Add mapping like @Column(unique=true) or @Index(unique=true)");
}
for (int i = 0; i < uniqueColumns.size(); i++) {
if (i > 0) {
request.append(", ");
}
request.append(uniqueColumns.get(i));
}
}
}
@@ -14,6 +14,8 @@ final class InsertMetaPlatform {
case YUGABYTE:
case COCKROACH:
return new InsertMetaOptionsPostgres(meta, desc);
case SQLITE:
return new InsertMetaOptionsSqlite(meta, desc);
default:
return NOT_SUPPORTED;
}
@@ -1,14 +1,17 @@
package io.ebeaninternal.server.query;
import io.ebeaninternal.api.SpiDbQueryPlan;
import io.ebeaninternal.api.SpiQueryBindCapture;
import io.ebeaninternal.api.SpiQueryPlan;
import io.ebean.config.CurrentTenantProvider;
import io.ebeaninternal.api.*;
import io.ebeaninternal.server.bind.capture.BindCapture;
import java.sql.Connection;
import java.sql.SQLException;
import java.time.Instant;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
import static java.lang.System.Logger.Level.ERROR;
final class CQueryBindCapture implements SpiQueryBindCapture {
private static final double multiplier = 1.5d;
@@ -16,23 +19,22 @@ final class CQueryBindCapture implements SpiQueryBindCapture {
private final ReentrantLock lock = new ReentrantLock();
private final CQueryPlanManager manager;
private final SpiQueryPlan queryPlan;
private final CurrentTenantProvider tenantProvider;
private BindCapture bindCapture;
private long queryTimeMicros;
private long thresholdMicros;
private long captureCount;
private Object tenantId;
private long lastBindCapture;
CQueryBindCapture(CQueryPlanManager manager, SpiQueryPlan queryPlan, long thresholdMicros) {
CQueryBindCapture(CQueryPlanManager manager, SpiQueryPlan queryPlan, long thresholdMicros, CurrentTenantProvider tenantProvider) {
this.manager = manager;
this.queryPlan = queryPlan;
this.thresholdMicros = thresholdMicros;
this.tenantProvider = tenantProvider;
}
/**
* Return true if we should capture the bind values for this query.
*/
@Override
public boolean collectFor(long timeMicros) {
return timeMicros > thresholdMicros && captureCount < 10;
@@ -45,6 +47,7 @@ final class CQueryBindCapture implements SpiQueryBindCapture {
this.thresholdMicros = Math.round(queryTimeMicros * multiplier);
this.captureCount++;
this.bindCapture = bindCapture;
this.tenantId = tenantProvider == null ? null : tenantProvider.currentId();
this.queryTimeMicros = queryTimeMicros;
lastBindCapture = System.currentTimeMillis();
manager.notifyBindCapture(this, startNanos);
@@ -68,7 +71,7 @@ final class CQueryBindCapture implements SpiQueryBindCapture {
/**
* Collect the query plan using already captured bind values.
*/
public boolean collectQueryPlan(CQueryPlanRequest request) {
boolean collectQueryPlan(CQueryPlanRequest request, SpiTransactionManager transactionManager) {
if (bindCapture == null || request.since() < lastBindCapture) {
// no bind capture since the last capture
return false;
@@ -76,11 +79,25 @@ final class CQueryBindCapture implements SpiQueryBindCapture {
final Instant whenCaptured = Instant.ofEpochMilli(this.lastBindCapture);
final BindCapture last = this.bindCapture;
final Object tenantId = this.tenantId;
final long startNanos = System.nanoTime();
SpiDbQueryPlan queryPlan = manager.collectPlan(request.connection(), this.queryPlan, last);
SpiDbQueryPlan queryPlan;
try (Connection connection = transactionManager.queryPlanConnection(tenantId)) {
try {
queryPlan = manager.collectPlan(connection, this.queryPlan, last);
} finally {
if (!connection.getAutoCommit()) {
connection.rollback();
}
}
} catch (SQLException e) {
CoreLog.log.log(ERROR, "Error during query plan collection", e);
return false;
}
if (queryPlan != null) {
final long captureMicros = TimeUnit.MICROSECONDS.convert(System.nanoTime() - startNanos, TimeUnit.NANOSECONDS);
request.add(queryPlan.with(queryTimeMicros, captureCount, captureMicros, whenCaptured));
request.add(queryPlan.with(queryTimeMicros, captureCount, captureMicros, whenCaptured, tenantId));
// effectively turn off bind capture for this plan
thresholdMicros = Long.MAX_VALUE;
return true;
@@ -336,6 +336,43 @@ final class CQueryBuilder {
return sql;
}
private String wrapSelectExists(String sql) {
return "select exists(" + sql + ")";
}
/**
* Build the exists query using select exists(...) for efficient boolean result.
*/
<T> CQueryExists buildExistsQuery(OrmQueryRequest<T> request) {
SpiQuery<T> query = request.query();
query.setOrderBy(null);
query.setFirstRow(0);
query.setMaxRows(0);
if (request.descriptor().hasId()) {
query.setSelectId();
}
CQueryPredicates predicates = new CQueryPredicates(binder, request);
CQueryPlan queryPlan = request.queryPlan();
if (queryPlan != null) {
predicates.prepare(false);
return new CQueryExists(queryPlan, request, predicates);
}
predicates.prepare(true);
SqlTree sqlTree = createSqlTree(request, predicates, false);
if (SpiQuery.TemporalMode.CURRENT == query.temporalMode()) {
sqlTree.addSoftDeletePredicate(query);
}
SqlLimitResponse s = buildSql("select 1", request, predicates, sqlTree);
String sql = wrapSelectExists(s.getSql());
queryPlan = new CQueryPlan(request, sql, sqlTree.plan(), predicates.logWhereSql());
request.putQueryPlan(queryPlan);
return new CQueryExists(queryPlan, request, predicates);
}
/**
* Return the SQL Select statement as a String. Converts logical property
* names to physical deployment column names.
@@ -48,9 +48,12 @@ final class CQueryBuilderRawSql {
// wrap with a limit offset or ROW_NUMBER() etc
return sqlLimiter.limit(new OrmQueryLimitRequest(sql, orderBy, query, dbPlatform, rsql.isDistinct() || query.isDistinct()));
} else {
// add back select keyword (it was removed to support sqlQueryLimiter)
String prefix = "select " + (rsql.isDistinct() ? "distinct " : "");
sql = prefix + sql;
if (hasValue(rsql.getPreFrom())) {
// add back select keyword (it was removed to support sqlQueryLimiter)
String prefix = "select " + (rsql.isDistinct() ? "distinct " : "");
sql = prefix + sql;
}
// else: template mode — SQL is already complete (no keyword stripping was done)
return new SqlLimitResponse(sql);
}
}
@@ -67,10 +70,12 @@ final class CQueryBuilderRawSql {
sb.append(selectProperty);
first = false;
}
} else {
sb.append(sql.getPreFrom());
sb.append(' ');
} else if (hasValue(sql.getPreFrom())) {
// standard parsed mode: column list with "select" prefix added in buildSql()
sb.append(sql.getPreFrom()).append(' ');
}
sb.append(' ');
// else: template mode (preFrom empty) — the full SQL is in preWhere/preHaving, no prefix needed
String s = sql.getPreWhere();
BindParams bindParams = request.query().bindParams();
@@ -126,9 +131,18 @@ final class CQueryBuilderRawSql {
}
sb.append(dbHaving).append(' ');
}
String preOrderBy = sql.getPreOrderBy();
if (hasValue(preOrderBy)) {
sb.append(preOrderBy).append(' ');
}
if (hasValue(orderBy)) {
sb.append(' ').append(sql.getOrderByPrefix()).append(' ').append(orderBy);
}
String postOrderBy = sql.getPostOrderBy();
if (hasValue(postOrderBy)) {
sb.append(' ').append(postOrderBy);
}
return sb.toString().trim();
}
@@ -137,6 +151,12 @@ final class CQueryBuilderRawSql {
}
private String orderBy(CQueryPredicates predicates, SpiRawSql.Sql sql) {
if (!hasValue(sql.getPreFrom()) && !sql.isOrderByPlaceholder()) {
// template mode (withPlaceholders()) without an explicit ${orderBy}/${andOrderBy} placeholder -
// there is no defined injection point for a dynamic order by, so ignore any caller-supplied
// order by rather than risk emitting it at an undefined (and likely invalid) position.
return sql.getOrderBy();
}
String orderBy = predicates.dbOrderBy();
if (orderBy != null) {
return orderBy;
@@ -170,6 +170,29 @@ public final class CQueryEngine {
}
}
/**
* Build and execute the exists query using select exists(...).
*/
public <T> boolean findExists(OrmQueryRequest<T> request) {
CQueryExists rcQuery = queryBuilder.buildExistsQuery(request);
request.setCancelableQuery(rcQuery);
try {
boolean exists = rcQuery.findExists();
if (request.logSql()) {
logGeneratedSql(request, rcQuery.generatedSql(), rcQuery.bindLog(), rcQuery.micros());
}
if (request.logSummary()) {
request.transaction().logSummary(rcQuery.summary());
}
if (request.isQueryCachePut()) {
request.putToQueryCache(exists);
}
return exists;
} catch (SQLException e) {
throw translate(request, rcQuery.bindLog(), rcQuery.generatedSql(), e);
}
}
/**
* Read many beans using an iterator (except you need to close() the iterator
* when you have finished).
@@ -0,0 +1,131 @@
package io.ebeaninternal.server.query;
import io.ebean.CancelableQuery;
import io.ebean.util.JdbcClose;
import io.ebeaninternal.api.SpiProfileTransactionEvent;
import io.ebeaninternal.api.SpiQuery;
import io.ebeaninternal.api.SpiTransaction;
import io.ebeaninternal.server.core.OrmQueryRequest;
import io.ebeaninternal.server.deploy.BeanDescriptor;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.concurrent.locks.ReentrantLock;
/**
* Executes a select exists(...) query returning a boolean result.
*/
final class CQueryExists implements SpiProfileTransactionEvent, CancelableQuery {
private final CQueryPlan queryPlan;
private final OrmQueryRequest<?> request;
private final BeanDescriptor<?> desc;
private final SpiQuery<?> query;
private final CQueryPredicates predicates;
private final String sql;
private ResultSet rset;
private PreparedStatement pstmt;
private String bindLog;
private long executionTimeMicros;
private boolean result;
private long profileOffset;
private final ReentrantLock lock = new ReentrantLock();
CQueryExists(CQueryPlan queryPlan, OrmQueryRequest<?> request, CQueryPredicates predicates) {
this.queryPlan = queryPlan;
this.request = request;
this.query = request.query();
this.sql = queryPlan.sql();
this.desc = request.descriptor();
this.predicates = predicates;
query.setGeneratedSql(sql);
}
public String summary() {
return "FindExists exeMicros[" + executionTimeMicros
+ "] result[" + result
+ "] type[" + desc.fullName()
+ "] predicates[" + predicates.logWhereSql()
+ "] bind[" + bindLog + ']';
}
public String bindLog() {
return bindLog;
}
public String generatedSql() {
return sql;
}
long micros() {
return executionTimeMicros;
}
/**
* Execute the query returning the exists boolean result.
*/
public boolean findExists() throws SQLException {
long startNano = System.nanoTime();
try {
SpiTransaction t = transaction();
profileOffset = t.profileOffset();
Connection conn = t.internalConnection();
lock.lock();
try {
query.checkCancelled();
pstmt = conn.prepareStatement(sql);
if (query.timeout() > 0) {
pstmt.setQueryTimeout(query.timeout());
}
bindLog = predicates.bind(pstmt, conn);
} finally {
lock.unlock();
}
rset = pstmt.executeQuery();
query.checkCancelled();
if (!rset.next()) {
throw new jakarta.persistence.PersistenceException("Expecting 1 row from exists query but got none?");
}
result = rset.getBoolean(1);
executionTimeMicros = (System.nanoTime() - startNano) / 1000L;
request.slowQueryCheck(executionTimeMicros, result ? 1 : 0);
if (queryPlan.executionTime(executionTimeMicros)) {
queryPlan.captureBindForQueryPlan(predicates, executionTimeMicros);
}
t.profileEvent(this);
return result;
} finally {
close();
}
}
private SpiTransaction transaction() {
return request.transaction();
}
private void close() {
JdbcClose.close(rset);
JdbcClose.close(pstmt);
rset = null;
pstmt = null;
}
@Override
public void profile() {
transaction()
.profileStream()
.addQueryEvent(query.profileEventId(), profileOffset, desc.name(), result ? 1 : 0, query.profileId(), queryPlan.hash(), query.getGeneratedSql());
}
@Override
public void cancel() {
lock.lock();
try {
JdbcClose.cancel(pstmt);
} finally {
lock.unlock();
}
}
}
@@ -10,17 +10,22 @@ import io.ebean.metric.TimedMetric;
import io.ebeaninternal.api.*;
import io.ebeaninternal.server.core.OrmQueryRequest;
import io.ebeaninternal.server.core.timezone.DataTimeZone;
import io.ebeaninternal.server.deploy.BeanDescriptor;
import io.ebeaninternal.server.query.CQueryPlanStats.Snapshot;
import io.ebeaninternal.server.bind.DataBind;
import io.ebeaninternal.server.bind.DataBindCapture;
import io.ebeaninternal.server.querydefn.OrmQueryProperties;
import io.ebeaninternal.server.type.RsetDataReader;
import io.ebeaninternal.server.util.Md5;
import io.ebeaninternal.server.util.Str;
import jakarta.persistence.PersistenceException;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import static java.lang.System.Logger.Level.ERROR;
@@ -96,7 +101,7 @@ public class CQueryPlan implements SpiQueryPlan {
this.logWhereSql = logWhereSql;
this.encryptedProps = sqlTree.encryptedProps();
this.stats = new CQueryPlanStats(this);
this.dependentTables = sqlTree.dependentTables();
this.dependentTables = buildDependentTables(request.descriptor(), request.secondaryQueries(), sqlTree.dependentTables());
this.bindCapture = initBindCapture(query);
this.hash = Md5.hash(sql, name, location);
}
@@ -121,7 +126,7 @@ public class CQueryPlan implements SpiQueryPlan {
this.logWhereSql = logWhereSql;
this.encryptedProps = sqlTree.encryptedProps();
this.stats = new CQueryPlanStats(this);
this.dependentTables = sqlTree.dependentTables();
this.dependentTables = buildDependentTables(request.descriptor(), request.secondaryQueries(), sqlTree.dependentTables());
this.bindCapture = initBindCaptureRaw(sql, query);
this.hash = Md5.hash(sql, name, location);
}
@@ -130,6 +135,34 @@ public class CQueryPlan implements SpiQueryPlan {
return deriveName(label, query.loadMode() != null, query.label() != null, query.type().label(), simpleName);
}
/**
* Merge the SQL-tree dependent tables with any additional tables from fetchQuery paths.
* fetchQuery paths fire secondary SQL queries whose results are included in the query
* cache entry, so the cache must be invalidated when those tables are modified.
*/
private static Set<String> buildDependentTables(BeanDescriptor<?> desc, SpiQuerySecondary secondary, Set<String> sqlTables) {
if (secondary == null) {
return sqlTables;
}
List<OrmQueryProperties> queryJoins = secondary.queryJoins();
if (queryJoins == null || queryJoins.isEmpty()) {
return sqlTables;
}
Set<String> merged = null;
for (OrmQueryProperties join : queryJoins) {
try {
String table = desc.descriptor(join.getPath()).baseTable();
if (merged == null) {
merged = new LinkedHashSet<>(sqlTables);
}
merged.add(table);
} catch (PersistenceException ignore) {
// invalid path — ignore
}
}
return merged != null ? merged : sqlTables;
}
/**
* Derive the query plan / metric name.
*
@@ -1,5 +1,6 @@
package io.ebeaninternal.server.query;
import io.ebean.config.CurrentTenantProvider;
import io.ebean.meta.MetaQueryPlan;
import io.ebean.meta.QueryPlanRequest;
import io.ebean.metric.TimedMetric;
@@ -8,11 +9,9 @@ import io.ebeaninternal.server.transaction.TransactionManager;
import io.ebeaninternal.server.bind.capture.BindCapture;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.List;
import java.util.concurrent.ConcurrentHashMap;
import static java.lang.System.Logger.Level.ERROR;
import static java.util.Collections.emptyList;
public final class CQueryPlanManager implements QueryPlanManager {
@@ -21,13 +20,17 @@ public final class CQueryPlanManager implements QueryPlanManager {
private final ConcurrentHashMap<CQueryBindCapture, Object> plans = new ConcurrentHashMap<>();
private final TransactionManager transactionManager;
private final CurrentTenantProvider tenantProvider;
private final QueryPlanLogger planLogger;
private final TimedMetric timeCollection;
private final TimedMetric timeBindCapture;
private long defaultThreshold;
public CQueryPlanManager(TransactionManager transactionManager, long defaultThreshold, QueryPlanLogger planLogger, ExtraMetrics extraMetrics) {
public CQueryPlanManager(TransactionManager transactionManager,
CurrentTenantProvider tenantProvider,
long defaultThreshold, QueryPlanLogger planLogger, ExtraMetrics extraMetrics) {
this.transactionManager = transactionManager;
this.tenantProvider = tenantProvider;
this.defaultThreshold = defaultThreshold;
this.planLogger = planLogger;
this.timeCollection = extraMetrics.planCollect();
@@ -41,7 +44,7 @@ public final class CQueryPlanManager implements QueryPlanManager {
@Override
public SpiQueryBindCapture createBindCapture(SpiQueryPlan queryPlan) {
return new CQueryBindCapture(this, queryPlan, defaultThreshold);
return new CQueryBindCapture(this, queryPlan, defaultThreshold, tenantProvider);
}
public void notifyBindCapture(CQueryBindCapture planBind, long startNanos) {
@@ -58,23 +61,11 @@ public final class CQueryPlanManager implements QueryPlanManager {
}
private List<MetaQueryPlan> collectPlans(QueryPlanRequest request) {
try (Connection connection = transactionManager.queryPlanConnection()) {
CQueryPlanRequest req = new CQueryPlanRequest(connection, request, plans.keySet().iterator());
while (req.hasNext()) {
req.nextCapture();
}
if (!connection.getAutoCommit()) {
// CHECKME: commit or rollback here?
// arguments for rollback: the collecting should never modify data.
// if there are collectors that may copy the plan into tables, it's up to the collector to
// commit the transaction.
connection.rollback();
}
return req.plans();
} catch (SQLException e) {
CoreLog.log.log(ERROR, "Error during query plan collection", e);
return emptyList();
CQueryPlanRequest req = new CQueryPlanRequest(transactionManager, request, plans.keySet().iterator());
while (req.hasNext()) {
req.nextCapture();
}
return req.plans();
}
public SpiDbQueryPlan collectPlan(Connection connection, SpiQueryPlan queryPlan, BindCapture last) {
@@ -2,8 +2,8 @@ package io.ebeaninternal.server.query;
import io.ebean.meta.MetaQueryPlan;
import io.ebean.meta.QueryPlanRequest;
import io.ebeaninternal.api.SpiTransactionManager;
import java.sql.Connection;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
@@ -14,15 +14,14 @@ import java.util.List;
final class CQueryPlanRequest {
private final List<MetaQueryPlan> plans = new ArrayList<>();
private final Connection connection;
private final SpiTransactionManager transactionManager;
private final long since;
private final int maxCount;
private final long maxTime;
private final Iterator<CQueryBindCapture> iterator;
CQueryPlanRequest(Connection connection, QueryPlanRequest req, Iterator<CQueryBindCapture> iterator) {
this.connection = connection;
CQueryPlanRequest(SpiTransactionManager transactionManager, QueryPlanRequest req, Iterator<CQueryBindCapture> iterator) {
this.transactionManager = transactionManager;
this.iterator = iterator;
this.maxCount = req.maxCount();
long reqSince = req.since();
@@ -31,13 +30,6 @@ final class CQueryPlanRequest {
this.maxTime = maxTimeMillis > 0 ? System.currentTimeMillis() + maxTimeMillis : 0;
}
/**
* Return the connection used to collect the db query plan.
*/
Connection connection() {
return connection;
}
/**
* Add the collected query plan.
*/
@@ -71,7 +63,7 @@ final class CQueryPlanRequest {
*/
void nextCapture() {
final CQueryBindCapture next = iterator.next();
if (next.collectQueryPlan(this)) {
if (next.collectQueryPlan(this, transactionManager)) {
iterator.remove();
}
}
@@ -23,6 +23,7 @@ public final class DQueryPlanOutput implements MetaQueryPlan, SpiDbQueryPlan {
private long captureCount;
private long captureMicros;
private Instant whenCaptured;
private Object tenantId;
public DQueryPlanOutput(Class<?> beanType, String label, String hash, String sql, ProfileLocation profileLocation, String bind, String plan) {
this.beanType = beanType;
@@ -39,17 +40,11 @@ public final class DQueryPlanOutput implements MetaQueryPlan, SpiDbQueryPlan {
return hash;
}
/**
* Return the associated bean.
*/
@Override
public Class<?> beanType() {
return beanType;
}
/**
* Return the query label if set.
*/
@Override
public String label() {
return label;
@@ -60,42 +55,31 @@ public final class DQueryPlanOutput implements MetaQueryPlan, SpiDbQueryPlan {
return profileLocation;
}
/**
* Return the sql of query.
*/
@Override
public String sql() {
return sql;
}
/**
* Return a description of the bind values used.
*/
@Override
public String bind() {
return bind;
}
/**
* Return the query plan.
*/
@Override
public String plan() {
return plan;
}
/**
* Return the query execution time associated with the capture of bind values used
* to build the query plan.
*/
@Override
public Object tenantId() {
return tenantId;
}
@Override
public long queryTimeMicros() {
return queryTimeMicros;
}
/**
* Return the total count of times bind capture has occurred.
*/
@Override
public long captureCount() {
return captureCount;
@@ -113,18 +97,24 @@ public final class DQueryPlanOutput implements MetaQueryPlan, SpiDbQueryPlan {
@Override
public String toString() {
return " BeanType:" + ((beanType == null) ? "" : beanType.getSimpleName()) + " planHash:" + hash + " label:" + label + " queryTimeMicros:" + queryTimeMicros + " captureCount:" + captureCount + "\n SQL:" + sql + "\nBIND:" + bind + "\nPLAN:" + plan;
return " BeanType:" + ((beanType == null) ? "" : beanType.getSimpleName())
+ " planHash:" + hash
+ " label:" + label
+ " queryTimeMicros:" + queryTimeMicros
+ " captureCount:" + captureCount
+ (tenantId == null ? "" : (" tenant:" + tenantId))
+ "\n SQL:" + sql
+ "\nBIND:" + bind
+ "\nPLAN:" + plan;
}
/**
* Additionally set the query execution time and the number of bind captures.
*/
@Override
public DQueryPlanOutput with(long queryTimeMicros, long captureCount, long captureMicros, Instant whenCaptured) {
public DQueryPlanOutput with(long queryTimeMicros, long captureCount, long captureMicros, Instant whenCaptured, Object tenantId) {
this.queryTimeMicros = queryTimeMicros;
this.captureCount = captureCount;
this.captureMicros = captureMicros;
this.whenCaptured = whenCaptured;
this.tenantId = tenantId;
return this;
}
}
@@ -350,6 +350,11 @@ final class DefaultFetchGroupQuery<T> implements SpiFetchGroupQuery<T>, SpiQuery
throw new RuntimeException("EB102: Only select() and fetch() clause is allowed on FetchGroup");
}
@Override
public int deletePermanent() {
throw new RuntimeException("EB102: Only select() and fetch() clause is allowed on FetchGroup");
}
@Override
public int update() {
throw new RuntimeException("EB102: Only select() and fetch() clause is allowed on FetchGroup");
@@ -88,6 +88,12 @@ public final class DefaultOrmQueryEngine implements OrmQueryEngine {
return queryEngine.findCount(request);
}
@Override
public <T> boolean findExists(OrmQueryRequest<T> request) {
flushJdbcBatchOnQuery(request);
return queryEngine.findExists(request);
}
@Override
public <A> List<A> findIds(OrmQueryRequest<?> request) {
flushJdbcBatchOnQuery(request);
@@ -34,41 +34,44 @@ final class SqlTreeAlias {
/**
* Add joins to support where predicates
*/
void addManyWhereJoins(Set<String> manyWhereJoins) {
void addManyWhereJoins(Set<String> manyWhereJoins, STreeType desc) {
if (manyWhereJoins != null) {
for (String include : manyWhereJoins) {
addPropertyJoin(include, manyWhereJoinProps);
addPropertyJoin(include, manyWhereJoinProps, desc);
}
}
}
private void addEmbeddedPropertyJoin(String embProp) {
private boolean addEmbeddedPropertyJoin(String embProp) {
if (embeddedPropertyJoins == null) {
embeddedPropertyJoins = new HashSet<>();
}
embeddedPropertyJoins.add(embProp);
return embeddedPropertyJoins.add(embProp);
}
/**
* Add joins.
*/
public void addJoin(Set<String> propJoins, STreeType desc) {
if (propJoins != null) {
for (String propJoin : propJoins) {
if (desc.isEmbeddedPath(propJoin)) {
addEmbeddedPropertyJoin(propJoin);
} else {
addPropertyJoin(propJoin, joinProps);
}
}
if (propJoins == null) {
return;
}
for (String propJoin : propJoins) {
addPropertyJoin(propJoin, joinProps, desc);
}
}
private void addPropertyJoin(String include, TreeSet<String> set) {
if (set.add(include)) {
private void addPropertyJoin(String include, TreeSet<String> set, STreeType desc) {
boolean added = false;
if (desc.isEmbeddedPath(include)) {
added = addEmbeddedPropertyJoin(include);
} else {
added = set.add(include);
}
if (added) {
String[] split = SplitName.split(include);
if (split[0] != null) {
addPropertyJoin(split[0], set);
addPropertyJoin(split[0], set, desc);
}
}
}
@@ -91,7 +94,7 @@ final class SqlTreeAlias {
for (String propJoin : embeddedPropertyJoins) {
String[] split = SplitName.split(propJoin);
// the table alias of the parent path
String alias = tableAlias(split[0]);
String alias = tableAliasManyWhere(split[0]);
aliasMap.put(propJoin, alias);
}
}
@@ -227,7 +227,7 @@ public final class SqlTreeBuilder {
alias.addJoin(queryDetail.getFetchPaths(), desc);
alias.addJoin(predicates.predicateIncludes(), desc);
alias.addJoin(formula2JoinIncludes, desc);
alias.addManyWhereJoins(manyWhereJoins.propertyNames());
alias.addManyWhereJoins(manyWhereJoins.propertyNames(), desc);
// build set of table alias
alias.buildAlias();
predicates.parseTableAlias(alias);
@@ -398,7 +398,7 @@ public final class SqlTreeBuilder {
// the 'select' part of the query. We may need to add other joins to
// support the predicates or order by clauses.
// remove ManyWhereJoins from the predicateIncludes
// remove ManyWhereJoins from the predicateIncludes (they are handled as manyWhere correlated joins)
predicateIncludes.removeAll(manyWhereJoins.propertyNames());
predicateIncludes.addAll(predicates.orderByIncludes());
@@ -778,11 +778,15 @@ public final class SqlTreeBuilder {
* Create a SqlTreeNodeExtraJoin, register and return it.
*/
private SqlTreeNodeExtraJoin createJoinLeaf(String propertyName) {
SqlTreeNodeExtraJoin extraJoin = joinRegister.get(propertyName);
if (extraJoin != null) {
return extraJoin;
}
ExtraJoin extra = desc.extraJoin(propertyName);
if (extra == null) {
return null;
} else {
SqlTreeNodeExtraJoin extraJoin = new SqlTreeNodeExtraJoin(propertyName, extra.property(), extra.isContainsMany(), temporalMode);
extraJoin = new SqlTreeNodeExtraJoin(propertyName, extra.property(), extra.isContainsMany(), temporalMode);
joinRegister.put(propertyName, extraJoin);
return extraJoin;
}
@@ -803,30 +807,30 @@ public final class SqlTreeBuilder {
// no parent possible(parent is root)
return childJoin;
} else {
// look in register ...
String parentPropertyName = includeProp.substring(0, dotPos);
if (selectIncludes.contains(parentPropertyName)) {
// parent already handled by select
return childJoin;
}
SqlTreeNodeExtraJoin parentJoin = joinRegister.get(parentPropertyName);
if (parentJoin == null) {
// we need to create this the parent implicitly...
parentJoin = createJoinLeaf(parentPropertyName);
}
// formula2 dependency joins must appear before formula2 property joins that reference
// their table aliases — use addChildFirst to push dependencies ahead
if (isFormula2Dependency(childJoin.prefix())) {
parentJoin.addChildFirst(childJoin);
} else {
parentJoin.addChild(childJoin);
}
childJoin = parentJoin;
includeProp = parentPropertyName;
}
String parentPropertyName = includeProp.substring(0, dotPos);
if (desc.isEmbeddedPath(parentPropertyName)) {
// digging in embedded property, skip to parent
includeProp = parentPropertyName;
continue;
}
// look in register ...
if (selectIncludes.contains(parentPropertyName)) {
// parent already handled by select
return childJoin;
}
SqlTreeNodeExtraJoin parentJoin = createJoinLeaf(parentPropertyName);
// formula2 dependency joins must appear before formula2 property joins that reference
// their table aliases — use addChildFirst to push dependencies ahead
if (isFormula2Dependency(childJoin.prefix())) {
parentJoin.addChildFirst(childJoin);
} else {
parentJoin.addChild(childJoin);
}
childJoin = parentJoin;
includeProp = parentPropertyName;
}
}
@@ -83,6 +83,9 @@ final class SqlTreeNodeManyWhereJoin implements SqlTreeNode {
* intersection table if this is a ManyToMany node.
*/
private void appendFromBaseTable(DbSqlContext ctx, SqlJoinType joinType) {
if (nodeBeanProp.isEmbedded()) {
return;
}
String alias = ctx.tableAliasManyWhere(prefix);
String parentAlias = ctx.tableAliasManyWhere(parentPrefix);
@@ -371,6 +371,10 @@ public class DefaultOrmQuery<T> extends AbstractQuery implements SpiQuery<T> {
public final Query<T> asOf(Timestamp asOfDateTime) {
this.temporalMode = (asOfDateTime != null) ? TemporalMode.AS_OF : TemporalMode.CURRENT;
this.asOf = asOfDateTime;
if (asOfDateTime != null) {
// bean cache holds current state only — asOf queries must bypass it entirely
this.useBeanCache = CacheMode.OFF;
}
return this;
}
@@ -1523,6 +1527,11 @@ public class DefaultOrmQuery<T> extends AbstractQuery implements SpiQuery<T> {
return server.delete(this);
}
@Override
public final int deletePermanent() {
return server.deletePermanent(this);
}
@Override
public final int update() {
return server.update(this);
@@ -3,6 +3,10 @@ package io.ebeaninternal.server.rawsql;
import io.ebeaninternal.server.querydefn.SimpleTextParser;
import io.ebeaninternal.server.rawsql.SpiRawSql.Sql;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
/**
* Parses sql-select queries to try and determine the location where WHERE and
* HAVING clauses can be added dynamically to the sql.
@@ -13,6 +17,12 @@ final class DRawSqlParser {
private static final String $_HAVING = "${having}";
private static final String $_AND_WHERE = "${andWhere}";
private static final String $_WHERE = "${where}";
private static final String $_AND_ORDER_BY = "${andOrderBy}";
private static final String $_ORDER_BY = "${orderBy}";
private static final int KIND_WHERE = 0;
private static final int KIND_HAVING = 1;
private static final int KIND_ORDER_BY = 2;
private final SimpleTextParser textParser;
private String sql;
@@ -20,6 +30,8 @@ final class DRawSqlParser {
private int placeHolderAndWhere;
private int placeHolderHaving;
private int placeHolderAndHaving;
private int placeHolderOrderBy;
private int placeHolderAndOrderBy;
private final boolean hasPlaceHolders;
private int selectPos = -1;
@@ -35,11 +47,24 @@ final class DRawSqlParser {
private int whereExprPos = -1;
private boolean havingExprAnd;
private int havingExprPos = -1;
private boolean orderByExprAnd;
private int orderByExprPos = -1;
public static Sql parse(String sql) {
return new DRawSqlParser(sql).parse();
}
/**
* Parse for template mode: finds ${where} / ${having} placeholder positions without
* attempting SELECT/FROM keyword parsing. This supports complex SQL (CTEs, window functions,
* subqueries) where keyword-based parsing would fail.
* <p>
* The caller is expected to provide manual column mappings (like unparsed mode).
*/
public static Sql parseAsTemplate(String sql) {
return new DRawSqlParser(sql).parseTemplate();
}
private DRawSqlParser(String sqlString) {
sqlString = sqlString.trim();
sqlString = sqlString.replace('\n', ' ');
@@ -74,6 +99,8 @@ final class DRawSqlParser {
placeHolderAndWhere = removePlaceHolder($_AND_WHERE);
placeHolderHaving = removePlaceHolder($_HAVING);
placeHolderAndHaving = removePlaceHolder($_AND_HAVING);
placeHolderOrderBy = removePlaceHolder($_ORDER_BY);
placeHolderAndOrderBy = removePlaceHolder($_AND_ORDER_BY);
return hasPlaceHolders();
}
@@ -91,7 +118,8 @@ final class DRawSqlParser {
}
private boolean hasPlaceHolders() {
return placeHolderWhere > -1 || placeHolderAndWhere > -1 || placeHolderHaving > -1 || placeHolderAndHaving > -1;
return placeHolderWhere > -1 || placeHolderAndWhere > -1 || placeHolderHaving > -1 || placeHolderAndHaving > -1
|| placeHolderOrderBy > -1 || placeHolderAndOrderBy > -1;
}
/**
@@ -246,6 +274,73 @@ final class DRawSqlParser {
return -1;
}
/**
* Find the ${orderBy}/${andOrderBy} placeholder position (template mode only).
* Returns -1 if neither placeholder is present in the SQL.
*/
private int findOrderByExprPosition() {
if (placeHolderOrderBy > -1) {
return placeHolderOrderBy;
}
if (placeHolderAndOrderBy > -1) {
orderByExprAnd = true;
return placeHolderAndOrderBy;
}
return -1;
}
private Sql parseTemplate() {
if (!hasPlaceHolders) {
throw new IllegalArgumentException("withPlaceholders() requires at least one of "
+ "${where}, ${andWhere}, ${having}, ${andHaving}, ${orderBy}, ${andOrderBy} in the SQL");
}
whereExprPos = findWhereExprPosition();
havingExprPos = findHavingExprPosition();
orderByExprPos = findOrderByExprPosition();
// Order the placeholder positions found (where/having/orderBy may each be absent) and slice the
// placeholder-stripped SQL into the static text segments that sit between them. Each segment is the
// static SQL that must be emitted immediately after the *previous* placeholder's dynamic expression
// (or as the query prefix, for the very first segment).
List<int[]> markers = new ArrayList<>(3);
if (whereExprPos > -1) markers.add(new int[]{whereExprPos, KIND_WHERE});
if (havingExprPos > -1) markers.add(new int[]{havingExprPos, KIND_HAVING});
if (orderByExprPos > -1) markers.add(new int[]{orderByExprPos, KIND_ORDER_BY});
markers.sort(Comparator.comparingInt(m -> m[0]));
String preWhere;
String preHaving = null;
String preOrderBy = null;
String postOrderBy = null;
if (markers.isEmpty()) {
preWhere = sql.trim();
} else {
preWhere = sql.substring(0, markers.get(0)[0]).trim();
for (int i = 0; i < markers.size(); i++) {
int kind = markers.get(i)[1];
int startPos = markers.get(i)[0];
int endPos = (i + 1 < markers.size()) ? markers.get(i + 1)[0] : sql.length();
String segment = sql.substring(startPos, endPos).trim();
if (kind == KIND_WHERE) {
preHaving = segment;
} else if (kind == KIND_HAVING) {
preOrderBy = segment;
} else {
postOrderBy = segment;
}
}
}
boolean orderByPlaceholder = orderByExprPos > -1;
// preFrom is empty — signals template mode to CQueryBuilderRawSql (no "select" prefix handling).
// For the dynamic order-by prefix/value: only set when a ${orderBy}/${andOrderBy} placeholder was
// actually found - there is no static default order-by value at that placeholder (the placeholder
// is purely a dynamic injection point), so orderBySql is left null.
String orderByPrefix = orderByPlaceholder ? (orderByExprAnd ? "," : "order by") : null;
return new Sql(sql, "", preWhere, whereExprAnd, preHaving, havingExprAnd, orderByPrefix, null, false,
preOrderBy, postOrderBy, orderByPlaceholder);
}
private String removeWhitespace(String sql) {
if (sql == null) {
return "";
@@ -32,6 +32,12 @@ public final class DRawSqlService implements SpiRawSqlService {
return new DRawSqlBuilder(s, new SpiRawSql.ColumnMapping());
}
@Override
public RawSqlBuilder withPlaceholders(String sql) {
SpiRawSql.Sql s = DRawSqlParser.parseAsTemplate(sql);
return new DRawSqlBuilder(s, new SpiRawSql.ColumnMapping());
}
@Override
public SqlRow sqlRow(ResultSet resultSet, String dbTrueValue, boolean binaryOptimizedUUID) throws SQLException {
ResultSetMetaData meta = resultSet.getMetaData();
@@ -37,24 +37,18 @@ public interface SpiRawSql extends RawSql {
private static final long serialVersionUID = 1L;
private final boolean parsed;
private final String unparsedSql;
private final String preFrom;
private final String preWhere;
private final boolean andWhereExpr;
private final String preHaving;
private final boolean andHavingExpr;
private final String orderByPrefix;
private final String orderBy;
private final boolean distinct;
private final String preOrderBy;
private final String postOrderBy;
private final boolean orderByPlaceholder;
/**
* Construct for unparsed SQL.
@@ -70,13 +64,26 @@ public interface SpiRawSql extends RawSql {
this.orderByPrefix = null;
this.orderBy = null;
this.distinct = false;
this.preOrderBy = null;
this.postOrderBy = null;
this.orderByPlaceholder = false;
}
/**
* Construct for parsed SQL.
* Construct for parsed SQL (normal keyword-parsed mode - no ${orderBy}/${andOrderBy} placeholder support).
*/
Sql(String unparsedSql, String preFrom, String preWhere, boolean andWhereExpr,
String preHaving, boolean andHavingExpr, String orderByPrefix, String orderBy, boolean distinct) {
this(unparsedSql, preFrom, preWhere, andWhereExpr, preHaving, andHavingExpr, orderByPrefix, orderBy, distinct,
null, null, false);
}
/**
* Construct for parsed SQL, including template mode's ${orderBy}/${andOrderBy} placeholder support.
*/
Sql(String unparsedSql, String preFrom, String preWhere, boolean andWhereExpr,
String preHaving, boolean andHavingExpr, String orderByPrefix, String orderBy, boolean distinct,
String preOrderBy, String postOrderBy, boolean orderByPlaceholder) {
this.unparsedSql = unparsedSql;
this.parsed = true;
@@ -88,6 +95,9 @@ public interface SpiRawSql extends RawSql {
this.orderByPrefix = orderByPrefix;
this.orderBy = orderBy;
this.distinct = distinct;
this.preOrderBy = preOrderBy;
this.postOrderBy = postOrderBy;
this.orderByPlaceholder = orderByPlaceholder;
}
@Override
@@ -172,6 +182,32 @@ public interface SpiRawSql extends RawSql {
return orderBy;
}
/**
* Return the static SQL to emit immediately before the dynamic order-by injection point
* (template / withPlaceholders() mode only, e.g. static SQL between a ${having} and ${orderBy}
* placeholder).
*/
public String getPreOrderBy() {
return preOrderBy;
}
/**
* Return the static SQL to emit after the dynamic order-by injection point
* (template / withPlaceholders() mode only - typically empty since ORDER BY is usually last).
*/
public String getPostOrderBy() {
return postOrderBy;
}
/**
* Return true if a ${orderBy}/${andOrderBy} placeholder was found (template / withPlaceholders()
* mode only). When false, any dynamic order by set on the query is ignored rather than risk
* producing invalid SQL by injecting it at an undefined position.
*/
public boolean isOrderByPlaceholder() {
return orderByPlaceholder;
}
}
/**
@@ -48,7 +48,7 @@ public final class TableModState implements QueryCacheEntryValidate, ServerCache
boolean isValid(Set<String> tables, Instant sinceTime) {
for (String tableName : tables) {
final var modTime = tableModStamp.get(tableName);
if (modTime != null && !modTime.isBefore(sinceTime)) {
if (modTime != null && modTime.compareTo(sinceTime) > 0) {
if (log.isLoggable(TRACE)) {
log.log(TRACE, "Invalidate on table:{0}", tableName);
}
@@ -257,8 +257,8 @@ public class TransactionManager implements SpiTransactionManager {
}
@Override
public final Connection queryPlanConnection() throws SQLException {
return dataSourceSupplier.connection(null);
public final Connection queryPlanConnection(Object tenantId) throws SQLException {
return dataSourceSupplier.connection(tenantId);
}
@Override
@@ -16,7 +16,6 @@ import io.ebeaninternal.api.DbOffline;
import io.ebeaninternal.api.GeoTypeProvider;
import io.ebeaninternal.server.core.ServiceUtil;
import io.ebeaninternal.server.core.bootup.BootupClasses;
import io.ebeaninternal.server.deploy.meta.DeployBeanProperty;
import io.ebeaninternal.server.deploy.meta.DeployProperty;
import jakarta.persistence.AttributeConverter;
@@ -412,11 +411,7 @@ public final class DefaultTypeManager implements TypeManager {
if (MutationDetection.DEFAULT == prop.mutationDetection()) {
prop.setMutationDetection(jsonManager.mutationDetection());
}
Class<?> type = prop.ownerType();
if (prop instanceof DeployBeanProperty) {
type = ((DeployBeanProperty) prop).getField().getDeclaringClass();
}
var req = new ScalarJsonRequest(jsonManager, dbType, docType, type, prop.mutationDetection(), prop.name());
var req = new ScalarJsonRequest(jsonManager, dbType, docType, prop.ownerType(), prop.mutationDetection(), prop.name());
return jsonMapper.createType(req);
}
@@ -2,6 +2,8 @@ package io.ebeaninternal.server.rawsql;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.junit.jupiter.api.Assertions.assertEquals;
public class DRawSqlServiceTest {
@@ -16,4 +18,110 @@ public class DRawSqlServiceTest {
assertEquals("myschema.mytable.mycol", dRawSqlService.combine("myschema", "mytable", "mycol"));
assertEquals("myschema.mycol", dRawSqlService.combine("myschema", null, "mycol"));
}
@Test
void withPlaceholders_where() {
String sql = "with cte as (select a, b from t ${where} group by a) select a, b from cte order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.isParsed()).isTrue();
assertThat(result.getPreFrom()).isEmpty();
assertThat(result.getPreWhere()).isEqualTo("with cte as (select a, b from t");
assertThat(result.getPreHaving()).isEqualTo("group by a) select a, b from cte order by a");
assertThat(result.isAndWhereExpr()).isFalse();
}
@Test
void withPlaceholders_andWhere() {
String sql = "with cte as (select a from t where x=1 ${andWhere} group by a) select a from cte";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("with cte as (select a from t where x=1");
assertThat(result.isAndWhereExpr()).isTrue();
}
@Test
void withPlaceholders_requiresPlaceholder() {
assertThatThrownBy(() -> DRawSqlParser.parseAsTemplate("select a from t"))
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("${where}");
}
@Test
void withPlaceholders_havingOnly_noWherePlaceholder() {
String sql = "select a, sum(b) as total from t group by a ${having} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t group by a");
assertThat(result.getPreHaving()).isNull();
assertThat(result.isAndHavingExpr()).isFalse();
// trailing static SQL after the placeholder is preserved and emitted after any dynamic having.
// There is no ${orderBy}/${andOrderBy} placeholder so no dynamic order-by injection point exists -
// the static text is carried as preOrderBy and orderBy remains null/unused (getOrderByPrefix()
// falls back to its "order by" default but that value is never used - the gating in
// CQueryBuilderRawSql.orderBy() means no dynamic order by is ever appended in this case).
assertThat(result.getOrderBy()).isNull();
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_andHavingOnly_noWherePlaceholder() {
String sql = "select a, sum(b) as total from t group by a having total > 0 ${andHaving} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t group by a having total > 0");
assertThat(result.getPreHaving()).isNull();
assertThat(result.isAndHavingExpr()).isTrue();
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_whereAndHaving_bothPresent() {
String sql = "select a, sum(b) as total from t ${where} group by a ${having} order by a";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t");
assertThat(result.getPreHaving()).isEqualTo("group by a");
// no data loss - trailing "order by a" preserved and emitted after the dynamic having clause
assertThat(result.getPreOrderBy()).isEqualTo("order by a");
assertThat(result.isOrderByPlaceholder()).isFalse();
}
@Test
void withPlaceholders_orderBy() {
String sql = "select a, b from t ${where} ${orderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, b from t");
assertThat(result.getPreHaving()).isEmpty();
assertThat(result.getOrderByPrefix()).isEqualTo("order by");
assertThat(result.getOrderBy()).isNull();
assertThat(result.isOrderByPlaceholder()).isTrue();
}
@Test
void withPlaceholders_andOrderBy() {
String sql = "select a, b from t ${where} order by a ${andOrderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, b from t");
assertThat(result.getPreHaving()).isEqualTo("order by a");
assertThat(result.getOrderByPrefix()).isEqualTo(",");
assertThat(result.getOrderBy()).isNull();
assertThat(result.isOrderByPlaceholder()).isTrue();
}
@Test
void withPlaceholders_whereHavingAndOrderBy_allThreePresent() {
String sql = "select a, sum(b) as total from t ${where} group by a ${having} ${orderBy}";
SpiRawSql.Sql result = DRawSqlParser.parseAsTemplate(sql);
assertThat(result.getPreWhere()).isEqualTo("select a, sum(b) as total from t");
assertThat(result.getPreHaving()).isEqualTo("group by a");
assertThat(result.getPreOrderBy()).isEmpty();
assertThat(result.getOrderByPrefix()).isEqualTo("order by");
assertThat(result.isOrderByPlaceholder()).isTrue();
}
}
@@ -15,8 +15,9 @@ class TableModStateTest {
private final TableModState tableModState = new TableModState();
@Test
void isValid() {
void isValid() throws InterruptedException {
Instant before = Instant.now();
Thread.sleep(5);
tableModState.touch(setOf("one", "two", "three"));
+4 -4
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean ddl generation</name>
@@ -28,14 +28,14 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core-type</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -65,7 +65,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-all</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
@@ -157,6 +157,14 @@ public interface DbMigration {
*/
void setHeader(String header);
/**
* Set to true to add {@code -- @formatter:off} / {@code -- @formatter:on} guards to generated DDL scripts.
* <p>
* This prevents IDE formatters (e.g. IntelliJ "Reformat Code on commit") from modifying the SQL.
* Can also be enabled via the system property {@code ddl.migration.formatterGuards=true}.
*/
void setAddFormatterGuards(boolean formatterGuards);
/**
* Set the prefix for the version. Set this to "V" for use with Flyway.
*/
@@ -86,6 +86,7 @@ public class DefaultDbMigration implements DbMigration {
private int lockTimeoutSeconds;
protected boolean includeBuiltInPartitioning = true;
protected boolean includeIndex;
private boolean formatterGuards;
/**
* Create for offline migration generation.
@@ -186,6 +187,11 @@ public class DefaultDbMigration implements DbMigration {
this.header = header;
}
@Override
public void setAddFormatterGuards(boolean formatterGuards) {
this.formatterGuards = formatterGuards;
}
/**
* Set the specific platform to generate DDL for.
* <p>
@@ -653,7 +659,15 @@ public class DefaultDbMigration implements DbMigration {
}
private PlatformDdlWriter createDdlWriter(DatabasePlatform platform) {
return new PlatformDdlWriter(platform, databaseBuilder, lockTimeoutSeconds);
return new PlatformDdlWriter(platform, databaseBuilder, lockTimeoutSeconds, formatterGuards());
}
private boolean formatterGuards() {
String val = System.getProperty("ddl.migration.formatterGuards");
if (val != null) {
return Boolean.parseBoolean(val);
}
return formatterGuards;
}
/**
@@ -29,11 +29,13 @@ public class PlatformDdlWriter {
private final DatabaseBuilder.Settings config;
private final PlatformDdl platformDdl;
private final int lockTimeoutSeconds;
private final boolean formatterGuards;
public PlatformDdlWriter(DatabasePlatform platform, DatabaseBuilder.Settings config, int lockTimeoutSeconds) {
public PlatformDdlWriter(DatabasePlatform platform, DatabaseBuilder.Settings config, int lockTimeoutSeconds, boolean formatterGuards) {
this.platformDdl = PlatformDdlBuilder.create(platform);
this.config = config;
this.lockTimeoutSeconds = lockTimeoutSeconds;
this.formatterGuards = formatterGuards;
}
/**
@@ -87,11 +89,17 @@ public class PlatformDdlWriter {
* Write the 'Apply' DDL buffers to the writer.
*/
protected void writeApplyDdl(Writer writer, DdlWrite ddl) throws IOException {
if (formatterGuards) {
writer.append("-- @formatter:off\n");
}
String header = config.getDdlHeader();
if (header != null && !header.isEmpty()) {
writer.append(header).append('\n');
}
ddl.writeApply(writer);
if (formatterGuards) {
writer.append("-- @formatter:on\n");
}
}
/**
+2 -2
View File
@@ -3,7 +3,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@@ -15,7 +15,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core-type</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
+4 -4
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean net postgis types</name>
@@ -19,14 +19,14 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<!-- provided scope -->
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -54,7 +54,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<artifactId>ebean-opentelemetry</artifactId>
@@ -28,7 +28,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -71,21 +71,21 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-querybean</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>querybean-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
+4 -4
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean pgvector types</name>
@@ -19,14 +19,14 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<!-- provided scope -->
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -54,7 +54,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
+4 -4
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean postgis types</name>
@@ -19,14 +19,14 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-platform-postgres</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
</dependency>
<!-- provided scope -->
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -62,7 +62,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
+5 -5
View File
@@ -4,7 +4,7 @@
<parent>
<artifactId>ebean-parent</artifactId>
<groupId>io.ebean</groupId>
<version>18.1.0</version>
<version>18.2.0</version>
</parent>
<name>ebean querybean</name>
@@ -17,7 +17,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-core</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>
@@ -59,14 +59,14 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-ddl-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.ebean</groupId>
<artifactId>ebean-test</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>test</scope>
</dependency>
@@ -80,7 +80,7 @@
<dependency>
<groupId>io.ebean</groupId>
<artifactId>querybean-generator</artifactId>
<version>18.1.0</version>
<version>18.2.0</version>
<scope>provided</scope>
</dependency>

Some files were not shown because too many files have changed in this diff Show More