Fix #30 by adding modified builders

Adds `:label-fn` and `:qualifier-fn` options, and `as-modified-*` 
builder variants.
This commit is contained in:
Sean Corfield 2019-06-04 18:01:19 -07:00
parent fa31d7ef37
commit b64fbf35ff
7 changed files with 163 additions and 42 deletions

View file

@ -7,8 +7,9 @@ Only accretive/fixative changes will be made from now on.
The following changes have been committed to the **master** branch and will be in the next release: The following changes have been committed to the **master** branch and will be in the next release:
* Fix #24 by adding return type hints to `next.jdbc` functions. * Fix #24 by adding return type hints to `next.jdbc` functions.
* Fix #22 by adding `next.jdbc.optional` with four map builders that omit `NULL` columns from the row hash maps. * Fix #22 by adding `next.jdbc.optional` with six map builders that omit `NULL` columns from the row hash maps.
* Documentation improvements (#27, #28, and #29), including changing "connectable" to "transactable" for the `transact` function and the `with-transaction` macro (for consistency with the name of the underlying protocol). * Documentation improvements (#27, #28, and #29), including changing "connectable" to "transactable" for the `transact` function and the `with-transaction` macro (for consistency with the name of the underlying protocol).
* Fix #30 by adding `modified` variants of column name functions and builders. The `lower` variants have been rewritten in terms of these new `modified` variants. This adds `:label-fn` and `:qualifier-fn` options that mirror `:column-fn` and `:table-fn` for row builders.
## Stable Builds ## Stable Builds

View file

@ -37,6 +37,8 @@ The "friendly" SQL functions all accept the following options:
Any function that might realize a row or a result set will accept: Any function that might realize a row or a result set will accept:
* `:builder-fn` -- a function that implements the `RowBuilder` and `ResultSetBuilder` protocols; strictly speaking, `plan` and `execute-one!` only need `RowBuilder` to be implemented (and `plan` only needs that if it actually has to realize a row) but most generation functions will implement both for ease of use. * `:builder-fn` -- a function that implements the `RowBuilder` and `ResultSetBuilder` protocols; strictly speaking, `plan` and `execute-one!` only need `RowBuilder` to be implemented (and `plan` only needs that if it actually has to realize a row) but most generation functions will implement both for ease of use.
* `:label-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option must be present and should specify a string-to-string transformation that will be applied to the column label for each returned column name.
* `:qualifier-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option should specify a string-to-string transformation that will be applied to the table name for each returned column name. It can be omitted for the `as-unqualified-modified-*` variants.
## Prepared Statements ## Prepared Statements

View file

@ -7,7 +7,7 @@ The default builder for rows and result sets creates qualified keywords that mat
* `as-maps` -- table-qualified keywords as-is, the default, e.g., `:ADDRESS/ID`, `:myTable/firstName`, * `as-maps` -- table-qualified keywords as-is, the default, e.g., `:ADDRESS/ID`, `:myTable/firstName`,
* `as-unqualified-maps` -- simple keywords as-is, e.g., `:ID`, `:firstName`, * `as-unqualified-maps` -- simple keywords as-is, e.g., `:ID`, `:firstName`,
* `as-lower-maps` -- table-qualified lower-case keywords, e.g., `:address/id`, `:mytable/firstname`, * `as-lower-maps` -- table-qualified lower-case keywords, e.g., `:address/id`, `:mytable/firstname`,
* `as-unqualified-lower-maps` -- simple lower-case keywords as-is, e.g., `:id`, `:firstname`, * `as-unqualified-lower-maps` -- simple lower-case keywords, e.g., `:id`, `:firstname`,
* `as-arrays` -- table-qualified keywords as-is (vector of column names, followed by vectors of row values), * `as-arrays` -- table-qualified keywords as-is (vector of column names, followed by vectors of row values),
* `as-unqualified-arrays` -- simple keywords as-is, * `as-unqualified-arrays` -- simple keywords as-is,
* `as-lower-arrays` -- table-qualified lower-case keywords, * `as-lower-arrays` -- table-qualified lower-case keywords,
@ -15,6 +15,13 @@ The default builder for rows and result sets creates qualified keywords that mat
The reason behind the default is to a) be a simple transform, b) produce qualified keys in keeping with Clojure's direction (with `clojure.spec` etc), and c) not mess with the data. `as-arrays` is (slightly) faster than `as-maps` since it produces less data (vectors of values instead of vectors of hash maps), but the `lower` options will be slightly slower since they include (conditional) logic to convert strings to lower-case. The `unqualified` options may be slightly faster than their qualified equivalents but make no attempt to keep column names unique if your SQL joins across multiple tables. The reason behind the default is to a) be a simple transform, b) produce qualified keys in keeping with Clojure's direction (with `clojure.spec` etc), and c) not mess with the data. `as-arrays` is (slightly) faster than `as-maps` since it produces less data (vectors of values instead of vectors of hash maps), but the `lower` options will be slightly slower since they include (conditional) logic to convert strings to lower-case. The `unqualified` options may be slightly faster than their qualified equivalents but make no attempt to keep column names unique if your SQL joins across multiple tables.
In addition, the following generic builders can take `:label-fn` and `:qualifier-fn` options to control how the label and qualified are processed. The `lower` variants above are implemented in terms of these, passing `clojure.string/lower-case` for both of those options.
* `as-modified-maps` -- table-qualified keywords,
* `as-unqualified-modified-maps` -- simple keywords,
* `as-modified-arrays` -- table-qualified keywords,
* `as-unqualified-modified-arrays` -- simple keywords.
## RowBuilder Protocol ## RowBuilder Protocol
This protocol defines four functions and is used whenever `next.jdbc` needs to materialize a row from a `ResultSet` as a Clojure data structure: This protocol defines four functions and is used whenever `next.jdbc` needs to materialize a row from a `ResultSet` as a Clojure data structure:
@ -46,7 +53,7 @@ The options hash map passed to the builder function will contain a `:next.jdbc/s
## `next.jdbc.optional` ## `next.jdbc.optional`
This namespace contains variants of the four `as-maps`-style builders above that omit keys from the row hash maps if the corresponding column is `NULL`. This is in keeping with Clojure's views of "optionality" -- that optional elements should simply be omitted -- and is provided as an "opt-in" style of rows and result sets. This namespace contains variants of the six `as-maps`-style builders above that omit keys from the row hash maps if the corresponding column is `NULL`. This is in keeping with Clojure's views of "optionality" -- that optional elements should simply be omitted -- and is provided as an "opt-in" style of rows and result sets.
# ReadableColumn # ReadableColumn

View file

@ -3,7 +3,8 @@
(ns next.jdbc.optional (ns next.jdbc.optional
"Builders that treat NULL SQL values as 'optional' and omit the "Builders that treat NULL SQL values as 'optional' and omit the
corresponding keys from the Clojure hash maps for the rows." corresponding keys from the Clojure hash maps for the rows."
(:require [next.jdbc.result-set :as rs]) (:require [clojure.string :as str]
[next.jdbc.result-set :as rs])
(:import (java.sql ResultSet))) (:import (java.sql ResultSet)))
(set! *warn-on-reflection* true) (set! *warn-on-reflection* true)
@ -43,20 +44,43 @@
cols (rs/get-unqualified-column-names rsmeta opts)] cols (rs/get-unqualified-column-names rsmeta opts)]
(->MapResultSetOptionalBuilder rs rsmeta cols))) (->MapResultSetOptionalBuilder rs rsmeta cols)))
(defn as-modified-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with modified keys and nil
columns omitted.
Requires both the `:qualifier-fn` and `:label-fn` options."
[^ResultSet rs opts]
(assert (:qualifier-fn opts) ":qualifier-fn is required")
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (rs/get-modified-column-names rsmeta opts)]
(->MapResultSetOptionalBuilder rs rsmeta cols)))
(defn as-unqualified-modified-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with simple, modified keys
and nil columns omitted.
Requires the `:label-fn` option."
[^ResultSet rs opts]
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (rs/get-unqualified-modified-column-names rsmeta opts)]
(->MapResultSetOptionalBuilder rs rsmeta cols)))
(defn as-lower-maps (defn as-lower-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with lower-case keys and nil that produces bare vectors of hash map rows, with lower-case keys and nil
columns omitted." columns omitted."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-modified-maps rs (assoc opts
cols (rs/get-lower-column-names rsmeta opts)] :qualifier-fn str/lower-case
(->MapResultSetOptionalBuilder rs rsmeta cols))) :label-fn str/lower-case)))
(defn as-unqualified-lower-maps (defn as-unqualified-lower-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with simple, lower-case keys that produces bare vectors of hash map rows, with simple, lower-case keys
and nil columns omitted." and nil columns omitted."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-unqualified-modified-maps rs (assoc opts :label-fn str/lower-case)))
cols (rs/get-unqualified-lower-column-names rsmeta opts)]
(->MapResultSetOptionalBuilder rs rsmeta cols)))

View file

@ -35,22 +35,44 @@
(mapv (fn [^Integer i] (keyword (.getColumnLabel rsmeta i))) (mapv (fn [^Integer i] (keyword (.getColumnLabel rsmeta i)))
(range 1 (inc (.getColumnCount rsmeta))))) (range 1 (inc (.getColumnCount rsmeta)))))
(defn get-modified-column-names
"Given `ResultSetMetaData`, return a vector of modified column names, each
qualified by the table from which it came.
Requires both the `:qualifier-fn` and `:label-fn` options."
[^ResultSetMetaData rsmeta opts]
(assert (:qualifier-fn opts) ":qualifier-fn is required")
(assert (:label-fn opts) ":label-fn is required")
(mapv (fn [^Integer i] (keyword (some-> (.getTableName rsmeta i)
(not-empty)
((:qualifier-fn opts)))
(-> (.getColumnLabel rsmeta i)
((:label-fn opts)))))
(range 1 (inc (.getColumnCount rsmeta)))))
(defn get-unqualified-modified-column-names
"Given `ResultSetMetaData`, return a vector of unqualified modified column
names.
Requires the `:label-fn` option."
[^ResultSetMetaData rsmeta opts]
(assert (:label-fn opts) ":label-fn is required")
(mapv (fn [^Integer i] (keyword ((:label-fn opts) (.getColumnLabel rsmeta i))))
(range 1 (inc (.getColumnCount rsmeta)))))
(defn get-lower-column-names (defn get-lower-column-names
"Given `ResultSetMetaData`, return a vector of lower-case column names, each "Given `ResultSetMetaData`, return a vector of lower-case column names, each
qualified by the table from which it came." qualified by the table from which it came."
[^ResultSetMetaData rsmeta opts] [rsmeta opts]
(mapv (fn [^Integer i] (keyword (some-> (.getTableName rsmeta i) (get-modified-column-names rsmeta (assoc opts
(not-empty) :qualifier-fn str/lower-case
(str/lower-case)) :label-fn str/lower-case)))
(-> (.getColumnLabel rsmeta i)
(str/lower-case))))
(range 1 (inc (.getColumnCount rsmeta)))))
(defn get-unqualified-lower-column-names (defn get-unqualified-lower-column-names
"Given `ResultSetMetaData`, return a vector of unqualified column names." "Given `ResultSetMetaData`, return a vector of unqualified column names."
[^ResultSetMetaData rsmeta opts] [rsmeta opts]
(mapv (fn [^Integer i] (keyword (str/lower-case (.getColumnLabel rsmeta i)))) (get-unqualified-modified-column-names rsmeta
(range 1 (inc (.getColumnCount rsmeta))))) (assoc opts :label-fn str/lower-case)))
(defprotocol ReadableColumn (defprotocol ReadableColumn
"Protocol for reading objects from the `java.sql.ResultSet`. Default "Protocol for reading objects from the `java.sql.ResultSet`. Default
@ -134,21 +156,42 @@
cols (get-unqualified-column-names rsmeta opts)] cols (get-unqualified-column-names rsmeta opts)]
(->MapResultSetBuilder rs rsmeta cols))) (->MapResultSetBuilder rs rsmeta cols)))
(defn as-modified-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with modified keys.
Requires both the `:qualifier-fn` and `:label-fn` options."
[^ResultSet rs opts]
(assert (:qualifier-fn opts) ":qualifier-fn is required")
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (get-modified-column-names rsmeta opts)]
(->MapResultSetBuilder rs rsmeta cols)))
(defn as-unqualified-modified-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with simple, modified keys.
Requires the `:label-fn` option."
[^ResultSet rs opts]
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (get-unqualified-modified-column-names rsmeta opts)]
(->MapResultSetBuilder rs rsmeta cols)))
(defn as-lower-maps (defn as-lower-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with lower-case keys." that produces bare vectors of hash map rows, with lower-case keys."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-modified-maps rs (assoc opts
cols (get-lower-column-names rsmeta opts)] :qualifier-fn str/lower-case
(->MapResultSetBuilder rs rsmeta cols))) :label-fn str/lower-case)))
(defn as-unqualified-lower-maps (defn as-unqualified-lower-maps
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces bare vectors of hash map rows, with simple, lower-case keys." that produces bare vectors of hash map rows, with simple, lower-case keys."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-unqualified-modified-maps rs (assoc opts :label-fn str/lower-case)))
cols (get-unqualified-lower-column-names rsmeta opts)]
(->MapResultSetBuilder rs rsmeta cols)))
(defrecord ArrayResultSetBuilder [^ResultSet rs rsmeta cols] (defrecord ArrayResultSetBuilder [^ResultSet rs rsmeta cols]
RowBuilder RowBuilder
@ -180,23 +223,46 @@
cols (get-unqualified-column-names rsmeta opts)] cols (get-unqualified-column-names rsmeta opts)]
(->ArrayResultSetBuilder rs rsmeta cols))) (->ArrayResultSetBuilder rs rsmeta cols)))
(defn as-modified-arrays
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces a vector of modified column names followed by vectors of
row values.
Requires both the `:qualifier-fn` and `:label-fn` options."
[^ResultSet rs opts]
(assert (:qualifier-fn opts) ":qualifier-fn is required")
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (get-modified-column-names rsmeta opts)]
(->ArrayResultSetBuilder rs rsmeta cols)))
(defn as-unqualified-modified-arrays
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces a vector of simple, modified column names followed by
vectors of row values.
Requires the `:label-fn` option."
[^ResultSet rs opts]
(assert (:label-fn opts) ":label-fn is required")
(let [rsmeta (.getMetaData rs)
cols (get-unqualified-modified-column-names rsmeta opts)]
(->ArrayResultSetBuilder rs rsmeta cols)))
(defn as-lower-arrays (defn as-lower-arrays
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces a vector of lower-case column names followed by vectors of that produces a vector of lower-case column names followed by vectors of
row values." row values."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-modified-arrays rs (assoc opts
cols (get-lower-column-names rsmeta opts)] :qualifier-fn str/lower-case
(->ArrayResultSetBuilder rs rsmeta cols))) :label-fn str/lower-case)))
(defn as-unqualified-lower-arrays (defn as-unqualified-lower-arrays
"Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder` "Given a `ResultSet` and options, return a `RowBuilder` / `ResultSetBuilder`
that produces a vector of simple, lower-case column names followed by that produces a vector of simple, lower-case column names followed by
vectors of row values." vectors of row values."
[^ResultSet rs opts] [rs opts]
(let [rsmeta (.getMetaData rs) (as-unqualified-modified-arrays rs (assoc opts :label-fn str/lower-case)))
cols (get-unqualified-lower-column-names rsmeta opts)]
(->ArrayResultSetBuilder rs rsmeta cols)))
(declare navize-row) (declare navize-row)

View file

@ -37,10 +37,20 @@
(is (not (contains? row :fruit/appearance))) (is (not (contains? row :fruit/appearance)))
(is (= 3 (:fruit/id row))) (is (= 3 (:fruit/id row)))
(is (= "Peach" (:fruit/name row))))) (is (= "Peach" (:fruit/name row)))))
(testing "lower-case row builder" (testing "unqualified lower-case row builder"
(let [row (p/-execute-one (ds) (let [row (p/-execute-one (ds)
["select * from fruit where id = ?" 4] ["select * from fruit where id = ?" 4]
{:builder-fn opt/as-unqualified-lower-maps})] {:builder-fn opt/as-unqualified-lower-maps})]
(is (map? row)) (is (map? row))
(is (= 4 (:id row))) (is (= 4 (:id row)))
(is (= "Orange" (:name row)))))) (is (= "Orange" (:name row)))))
(testing "custom row builder"
(let [row (p/-execute-one (ds)
["select * from fruit where id = ?" 3]
{:builder-fn opt/as-modified-maps
:label-fn str/lower-case
:qualifier-fn identity})]
(is (map? row))
(is (not (contains? row :FRUIT/appearance)))
(is (= 3 (:FRUIT/id row)))
(is (= "Peach" (:FRUIT/name row))))))

View file

@ -95,13 +95,24 @@
(is (nil? (:fruit/appearance row))) (is (nil? (:fruit/appearance row)))
(is (= 3 (:fruit/id row))) (is (= 3 (:fruit/id row)))
(is (= "Peach" (:fruit/name row))))) (is (= "Peach" (:fruit/name row)))))
(testing "lower-case row builder" (testing "unqualified lower-case row builder"
(let [row (p/-execute-one (ds) (let [row (p/-execute-one (ds)
["select * from fruit where id = ?" 4] ["select * from fruit where id = ?" 4]
{:builder-fn rs/as-unqualified-lower-maps})] {:builder-fn rs/as-unqualified-lower-maps})]
(is (map? row)) (is (map? row))
(is (= 4 (:id row))) (is (= 4 (:id row)))
(is (= "Orange" (:name row)))))) (is (= "Orange" (:name row)))))
(testing "custom row builder"
(let [row (p/-execute-one (ds)
["select * from fruit where id = ?" 3]
{:builder-fn rs/as-modified-maps
:label-fn str/lower-case
:qualifier-fn identity})]
(is (map? row))
(is (contains? row :FRUIT/appearance))
(is (nil? (:FRUIT/appearance row)))
(is (= 3 (:FRUIT/id row)))
(is (= "Peach" (:FRUIT/name row))))))
(deftest test-mapify (deftest test-mapify
(testing "no row builder is used" (testing "no row builder is used"