ENH: Add Query findEach() with batch consumer

This is a variation of findEach() that makes it easy to have a batch consumer processing a large result in batches.  For example, process in batches of 50 beans.

Note that the last batch consumed/processed will often have less than the batch size.
This commit is contained in:
Rob Bygrave
2021-03-10 00:18:35 +13:00
parent 98aee9ca62
commit 2bb5a85bd6
13 changed files with 252 additions and 101 deletions
@@ -307,6 +307,13 @@ public interface ExpressionList<T> {
*/
void findEach(Consumer<T> consumer);
/**
* Execute findEach with a batch consumer.
*
* @see Query#findEach(int, Consumer)
*/
void findEach(int batch, Consumer<List<T>> consumer);
/**
* Execute the query processing the beans one at a time with the ability to
* stop processing before reading all the beans.
@@ -159,6 +159,13 @@ public interface ExtendedServer {
*/
<T> void findEach(Query<T> query, Consumer<T> consumer, Transaction transaction);
/**
* Execute findEach with batch consumer.
*
* @see Query#findEach(int, Consumer)
*/
<T> void findEach(Query<T> query, int batch, Consumer<List<T>> consumer, Transaction t);
/**
* Execute the query visiting the each bean one at a time.
* <p>
+18 -3
View File
@@ -810,7 +810,7 @@ public interface Query<T> {
* </p>
* <p>
* This method is functionally equivalent to findIterate() but instead of using an
* iterator uses the Consumer interface which is better suited to use with Java8 closures.
* iterator uses the Consumer interface which is better suited to use with closures.
* </p>
* <pre>{@code
*
@@ -829,6 +829,21 @@ public interface Query<T> {
*/
void findEach(Consumer<T> consumer);
/**
* Execute findEach streaming query batching the results for consuming.
* <p>
* This query execution will stream the results and is suited to consuming
* large numbers of results from the database.
* <p>
* Typically we use this batch consumer when we want to do further processing on
* the beans and want to do that processing in batch form, for example - 100 at
* a time.
*
* @param batch The number of beans processed in the batch
* @param consumer Process the batch of beans
*/
void findEach(int batch, Consumer<List<T>> consumer);
/**
* Execute the query using callbacks to a visitor to process the resulting
* beans one at a time.
@@ -839,12 +854,12 @@ public interface Query<T> {
* </p>
* <p>
* This method is functionally equivalent to findIterate() but instead of using an
* iterator uses the Predicate (SAM) interface which is better suited to use with Java8 closures.
* iterator uses the Predicate interface which is better suited to use with closures.
* </p>
* <pre>{@code
*
* DB.find(Customer.class)
* .fetch("contacts", FetchConfig.ofQuery(2))
* .fetchQuery("contacts")
* .where().eq("status", Status.NEW)
* .order().asc("id")
* .setMaxRows(2000)