luciano/next-jdbc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fixes #129 by adding builder-adapter and with-column-value

Browse source
This commit is contained in:
Sean Corfield 2020-07-07 14:19:12 -07:00
parent d443c28817
commit 8a8a0b2122
5 changed files with 127 additions and 58 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -4,6 +4,7 @@ Only accretive/fixative changes will be made from now on.
Changes made on **develop** since the 1.1.547 release:
* Fix #130 by implementing `clojure.lang.ILookup` on the three builder adapters.
* Fix #129 by adding `with-column-value` to `RowBuilder` and a more generic `builder-adapter`.
* Fix #128 by adding a test for the "not found" arity of lookup on mapified result sets.
* Correct MySQL batch statement rewrite tip: it's `:rewriteBatchedStatements true` (plural). Also surface the batch statement tips in the **Tips & Tricks** page.
* Clarify how combining is interleaving with reducing in **Reducing and Folding with `plan`**.

28
doc/result-set-builders.md
View file

@ -34,11 +34,20 @@ An example builder that converts `snake_case` database table/column names to `ke
(result-set/as-modified-maps rs (assoc opts :qualifier-fn kebab :label-fn kebab))))
```
And finally there are adapters for the existing builders that let you override the default way that columns are read from result sets:
And finally there are two styles of adapters for the existing builders that let you override the default way that columns are read from result sets.
The first style takes a `column-reader` function, which is called with the `ResultSet`, the `ResultSetMetaData`, and the column index, and is expected to read the raw column value from the result set and return it. The result is then passed through `read-column-by-index` (from `ReadableColumn`, which may be implemented directly via protocol extension or via metadata on the result of the `column-reader` function):
* `as-maps-adapter` -- adapts an existing map builder function with a new column reader,
* `as-arrays-adapter` -- adapts an existing array builder function with a new column reader.
The default `column-reader` function behavior would be:
```clojure
(defn default-column-reader
[^ResultSet rs ^ResultSetMetaData rsmeta ^Integer i]
(.getObject rs i))
```
An example column reader is provided -- `clob-column-reader` -- that still uses `.getObject` but will expand `java.sql.Clob` values into string (using the `clob->string` helper function):
```clojure
@ -47,6 +56,22 @@ An example column reader is provided -- `clob-column-reader` -- that still uses
result-set/clob-column-reader)}
```
As of 1.1.next, the second style of adapter relies on `with-column-value` from `RowBuilder` (see below) and allows you to take complete control of the column reading process. This style takes a `column-by-index-fn` function, which is called with the builder itself, the `ResultSet`, and the column index, and is expected to read the raw column value from the result set and perform any and all processing on it, before returning it. The result is added directly to the current row with no further processing.
* `builder-adapter` -- adapts any existing builder function with a new column reading function.
The default `column-by-index-fn` function behavior would be:
```clojure
(defn default-column-by-index-fn
[builder ^ResultSet rs ^Integer i]
(result-set/read-column-by-index (.getObject rs i) (:rsmeta builder) i))
```
Because the builder itself is passed in, the vector of processed column names is available as `(:cols builder)` (in addition to the `ResultSetMetaData` as `(:rsmeta builder)`). This allows you to take different actions based on the metadata or the column name, as well as bypassing the `read-column-by-index` call if you wish.
The older `as-*-adapter` functions are now implemented in terms of this `builder-adapter` because `with-column-value` abstracts away _how_ the new column's value is added to the row being built.
## RowBuilder Protocol
This protocol defines four functions and is used whenever `next.jdbc` needs to materialize a row from a `ResultSet` as a Clojure data structure:
@ -54,6 +79,7 @@ This protocol defines four functions and is used whenever `next.jdbc` needs to m
* `(->row builder)` -- produces a new row (a `(transient {})` by default),
* `(column-count builder)` -- returns the number of columns in each row,
* `(with-column builder row i)` -- given the row so far, fetches column `i` from the current row of the `ResultSet`, converts it to a Clojure value, and adds it to the row (for `as-maps` this is a call to `.getObject`, a call to `read-column-by-index` -- see the `ReadableColumn` protocol below, and a call to `assoc!`),
* `(with-column-value builder row col v)` -- given the row so far, the column name, and the column value, add the column name/value to the row in the appropriate way: this is a low-level utility, intended to be used in builders (or adapters) that want to control more of the value handling process,
* `(row! builder row)` -- completes the row (a `(persistent! row)` call by default).
`execute!` and `execute-one!` call these functions for each row they need to build. `plan` _may_ call these functions if the reducing function causes a row to be materialized.

24
src/next/jdbc/optional.clj
View file

@ -14,12 +14,17 @@
(->row [this] (transient {}))
(column-count [this] (count cols))
(with-column [this row i]
;; short-circuit on null to avoid column reading logic
(let [v (.getObject rs ^Integer i)]
(if (nil? v)
row
(assoc! row
(nth cols (dec i))
(rs/read-column-by-index v rsmeta i)))))
(rs/with-column-value this row (nth cols (dec i))
(rs/read-column-by-index v rsmeta i)))))
(with-column-value [this row col v]
;; ensure that even if this is adapted, we omit null columns
(if (nil? v)
row
(assoc! row col v)))
(row! [this row] (persistent! row))
rs/ResultSetBuilder
(->rs [this] (transient []))
@ -94,7 +99,7 @@
function, return a new builder function that uses that column reading
function instead of `.getObject` so you can override the default behavior.
This adapter omits SQL NULL values.
This adapter omits SQL NULL values, even if the underlying builder does not.
The default column-reader behavior would be equivalent to:
@ -115,12 +120,17 @@
(->row [this] (rs/->row mrsb))
(column-count [this] (rs/column-count mrsb))
(with-column [this row i]
;; short-circuit on null to avoid column reading logic
(let [v (column-reader rs (:rsmeta mrsb) i)]
(if (nil? v)
row
(assoc! row
(nth (:cols mrsb) (dec i))
(rs/read-column-by-index v (:rsmeta mrsb) i)))))
(rs/with-column-value mrsb row (nth (:cols mrsb) (dec i))
(rs/read-column-by-index v (:rsmeta mrsb) i)))))
(with-column-value [this row col v]
;; ensure that even if this is adapted, we omit null columns
(if (nil? v)
row
(rs/with-column-value mrsb row col v)))
(row! [this row] (rs/row! mrsb row))
rs/ResultSetBuilder
(->rs [this] (rs/->rs mrsb))

118
src/next/jdbc/result_set.clj
View file

@ -96,11 +96,12 @@
(get-unqualified-modified-column-names rsmeta
(assoc opts :label-fn lower-case)))
(defprotocol ReadableColumn
(defprotocol ReadableColumn :extend-via-metadata true
"Protocol for reading objects from the `java.sql.ResultSet`. Default
implementations (for `Object` and `nil`) return the argument, and the
`Boolean` implementation ensures a canonicalized `true`/`false` value,
but it can be extended to provide custom behavior for special types."
but it can be extended to provide custom behavior for special types.
Extension via metadata is supported."
(read-column-by-label [val label]
"Function for transforming values after reading them via a column label.")
(read-column-by-index [val rsmeta idx]
@ -130,6 +131,9 @@
(with-column [_ row i]
"Called with the row and the index of the column to be added;
this is expected to read the column value from the `ResultSet`!")
(with-column-value [_ row col v]
"Called with the row, the column name, and the value to be added;
this is a low-level function, typically used by `with-column`.")
(row! [_ row]
"Called once per row to finalize each row once it is complete."))
@ -145,14 +149,54 @@
(rs! [_ rs]
"Called to finalize the result set once it is complete."))
(defn builder-adapter
"Given any builder function (e.g., `as-lower-maps`) and a column reading
function, return a new builder function that uses that column reading
function instead of `.getObject` and `read-column-by-index` so you can
override the default behavior.
The default column-by-index-fn behavior would be equivalent to:
(defn default-column-by-index-fn
[builder ^ResultSet rs ^Integer i]
(read-column-by-index (.getObject rs i) (:rsmeta builder) i))
Your column-by-index-fn can use the result set metadata `(:rsmeta builder)`
and/or the (processed) column name `(nth (:cols builder) (dec i))` to
determine whether to call `.getObject` or some other method to read the
column's value, and can choose whether or not to use the `ReadableColumn`
protocol-based value processor (and could add metadata to the value to
satify that protocol on a per-instance basis)."
[builder-fn column-by-index-fn]
(fn [rs opts]
(let [builder (builder-fn rs opts)]
(reify
RowBuilder
(->row [this] (->row builder))
(column-count [this] (column-count builder))
(with-column [this row i]
(with-column-value this row (nth (:cols builder) (dec i))
(column-by-index-fn builder rs i)))
(with-column-value [this row col v]
(with-column-value builder row col v))
(row! [this row] (row! builder row))
ResultSetBuilder
(->rs [this] (->rs builder))
(with-row [this mrs row] (with-row builder mrs row))
(rs! [this mrs] (rs! builder mrs))
clojure.lang.ILookup
(valAt [this k] (get builder k))
(valAt [this k not-found] (get builder k not-found))))))
(defrecord MapResultSetBuilder [^ResultSet rs rsmeta cols]
RowBuilder
(->row [this] (transient {}))
(column-count [this] (count cols))
(with-column [this row i]
(assoc! row
(nth cols (dec i))
(read-column-by-index (.getObject rs ^Integer i) rsmeta i)))
(with-column-value this row (nth cols (dec i))
(read-column-by-index (.getObject rs ^Integer i) rsmeta i)))
(with-column-value [this row col v]
(assoc! row col v))
(row! [this row] (persistent! row))
ResultSetBuilder
(->rs [this] (transient []))
@ -226,28 +270,16 @@
Your column-reader can use the result set metadata to determine whether
to call `.getObject` or some other method to read the column's value.
`read-column-by-index` is still called on the result of that read."
`read-column-by-index` is still called on the result of that read.
Note: this is different behavior to `builder-adapter`'s `column-by-index-fn`."
[builder-fn column-reader]
(fn [rs opts]
(let [mrsb (builder-fn rs opts)]
(reify
RowBuilder
(->row [this] (->row mrsb))
(column-count [this] (column-count mrsb))
(with-column [this row i]
(assoc! row
(nth (:cols mrsb) (dec i))
(read-column-by-index (column-reader rs (:rsmeta mrsb) i)
(:rsmeta mrsb)
i)))
(row! [this row] (row! mrsb row))
ResultSetBuilder
(->rs [this] (->rs mrsb))
(with-row [this mrs row] (with-row mrsb mrs row))
(rs! [this mrs] (rs! mrsb mrs))
clojure.lang.ILookup
(valAt [this k] (get mrsb k))
(valAt [this k not-found] (get mrsb k not-found))))))
(builder-adapter builder-fn
(fn [builder rs i]
(let [^ResultSetMetaData rsmeta (:rsmeta builder)]
(read-column-by-index (column-reader rs rsmeta i)
rsmeta
i)))))
(defn clob->string
"Given a CLOB column value, read it as a string."
@ -269,7 +301,10 @@
(->row [this] (transient []))
(column-count [this] (count cols))
(with-column [this row i]
(conj! row (read-column-by-index (.getObject rs ^Integer i) rsmeta i)))
(with-column-value this row nil
(read-column-by-index (.getObject rs ^Integer i) rsmeta i)))
(with-column-value [this row _ v]
(conj! row v))
(row! [this row] (persistent! row))
ResultSetBuilder
(->rs [this] (transient [cols]))
@ -346,27 +381,16 @@
Your column-reader can use the result set metadata to determine whether
to call `.getObject` or some other method to read the column's value.
`read-column-by-index` is still called on the result of that read."
`read-column-by-index` is still called on the result of that read.
Note: this is different behavior to `builder-adapter`'s `column-by-index-fn`."
[builder-fn column-reader]
(fn [rs opts]
(let [arsb (builder-fn rs opts)]
(reify
RowBuilder
(->row [this] (->row arsb))
(column-count [this] (column-count arsb))
(with-column [this row i]
(conj! row
(read-column-by-index (column-reader rs (:rsmeta arsb) i)
(:rsmeta arsb)
i)))
(row! [this row] (row! arsb row))
ResultSetBuilder
(->rs [this] (->rs arsb))
(with-row [this mrs row] (with-row arsb mrs row))
(rs! [this mrs] (rs! arsb mrs))
clojure.lang.ILookup
(valAt [this k] (get arsb k))
(valAt [this k not-found] (get arsb k not-found))))))
(builder-adapter builder-fn
(fn [builder rs i]
(let [^ResultSetMetaData rsmeta (:rsmeta builder)]
(read-column-by-index (column-reader rs rsmeta i)
rsmeta
i)))))
(declare navize-row)

14
test/next/jdbc/result_set_test.clj
View file

@ -362,7 +362,7 @@
(defrecord Fruit [id name appearance cost grade])
(defn fruit-builder [^ResultSet rs _]
(defn fruit-builder [^ResultSet rs ^ResultSetMetaData rsmeta]
(reify
rs/RowBuilder
(->row [_] (->Fruit (.getObject rs "id")
@ -370,13 +370,21 @@
(.getObject rs "appearance")
(.getObject rs "cost")
(.getObject rs "grade")))
(with-column [_ row i] row)
(column-count [_] 0) ; no need to iterate over columns
(with-column [_ row i] row)
(with-column-value [_ row col v] row)
(row! [_ row] row)
rs/ResultSetBuilder
(->rs [_] (transient []))
(with-row [_ rs row] (conj! rs row))
(rs! [_ rs] (persistent! rs))))
(rs! [_ rs] (persistent! rs))
clojure.lang.ILookup ; only supports :cols and :rsmeta
(valAt [this k] (get this k nil))
(valAt [this k not-found]
(case k
:cols [:id :name :appearance :cost :grade]
:rsmeta rsmeta
not-found))))
(deftest custom-map-builder
(let [row (p/-execute-one (ds)