diff --git a/CHANGELOG.md b/CHANGELOG.md index 6fd49dd..383c07d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ Only accretive/fixative changes will be made from now on. +* 1.3.next in progress + * Address [#236](https://github.com/seancorfield/next-jdbc/issues/236) by showing examples of `run!` over `plan`. + * 1.3.847 -- 2022-11-04 * Fix [#232](https://github.com/seancorfield/next-jdbc/issues/232) by using `as-cols` in `insert-multi!` SQL builder. Thanks to @changsu-farmmorning for spotting that bug! * Fix [#229](https://github.com/seancorfield/next-jdbc/issues/229) by adding `next.jdbc.connect/uri->db-spec` which converts a URI string to a db-spec hash map; in addition, if `DriverManager/getConnection` fails, it assumes it was passed a URI instead of a JDBC URL, and retries after calling that function and then recreating the JDBC URL (which should have the effect of moving the embedded user/password credentials into the properties structure instead of the URL). diff --git a/doc/getting-started.md b/doc/getting-started.md index d234dd5..ba0280a 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -239,6 +239,18 @@ user=> (into #{} #{"apple" "banana" "cucumber"} ``` +If you want to process the rows purely for side-effects, without a result, you +can use `run!`: + +```clojure +user=> (run! #(println (:product %)) + (jdbc/plan ds ["select * from invoice where customer_id = ?" 100])) +apple +banana +cucumber +nil +``` + Any operation that can perform key-based lookup can be used here without creating hash maps from the rows: `get`, `contains?`, `find` (returns a `MapEntry` of whatever key you requested and the corresponding column value), or direct keyword access as shown above. Any operation that would require a Clojure hash map, such as `assoc` or anything that invokes `seq` (`keys`, `vals`), will cause the full row to be expanded into a hash map, such as produced by `execute!` or `execute-one!`, which implements `Datafiable` and `Navigable` and supports lazy navigation via foreign keys, explained in [`datafy`, `nav`, and the `:schema` option](/doc/datafy-nav-and-schema.md). This means that `select-keys` can be used to create regular Clojure hash map from (a subset of) columns in the row, without realizing the row, and it will not implement `Datafiable` or `Navigable`. diff --git a/doc/tips-and-tricks.md b/doc/tips-and-tricks.md index 126a623..f236d35 100644 --- a/doc/tips-and-tricks.md +++ b/doc/tips-and-tricks.md @@ -116,6 +116,15 @@ Examples: Most of this documentation describes using `plan` specifically for reducing and notes that you can avoid the overhead of realizing rows from the `ResultSet` into Clojure data structures if your reducing function uses only functions that get column values by name. If you perform any function on the row that would require an actual hash map or a sequence, the row will be realized into a full Clojure hash map via the builder function passed in the options (or via `next.jdbc.result-set/as-maps` by default). One of the benefits of reducing over `plan` is that you can stream very large result sets, very efficiently, without having the entire result set in memory (assuming your reducing function doesn't build a data structure that is too large!). See the tips below on **Streaming Result Sets**. +If you want to process a `plan` result purely for side-effects, without producing a result, +you can use `run!` instead of `reduce`: + +```clojure +(run! process-row (jdbc/plan ds ...)) +``` + +`run!` is based on `reduce` and `process-row` here takes just one argument -- +the row -- rather than the usual reducing function that takes two The result of `plan` is also foldable in the [clojure.core.reducers](https://clojure.org/reference/reducers) sense. While you could use `execute!` to produce a vector of fully-realized rows as hash maps and then fold that vector (Clojure's vectors support fork-join parallel reduce-combine), that wouldn't be possible for very large result sets. If you fold the result of `plan`, the result set will be partitioned and processed using fork-join parallel reduce-combine. Unlike reducing over `plan`, each row **is** realized into a Clojure data structure and each batch is forked for reduction as soon as that many rows have been realized. By default, `fold`'s batch size is 512 but you can specify a different value in the 4-arity call. Once the entire result set has been read, the last (partial) batch is forked for reduction. The combining operations are forked and interleaved with the reducing operations, so the order (of forked tasks) is batch-1, batch-2, combine-1-2, batch-3, combine-1&2-3, batch-4, combine-1&2&3-4, etc. The amount of parallelization you get will depend on many factors including the number of processors, the speed of your reducing function, the speed of your combining function, and the speed with which result sets can actually be streamed from your database.