Fixes #122 by adding next.jdbc/with-options

CHANGELOG.md

@@ -6,6 +6,7 @@ Changes made on develop since 1.0.462:
 * Add tests for `"jtds"` database driver (against MS SQL Server), making it officially supported.
 * Switch from OpenTable Embedded PostgreSQL to Zonky's version, so that testing can move forward from PostgreSQL 10.11 to 12.2.0.
 * Fix potential reflection warnings caused by `next.jdbc.prepare/statement` being incorrectly type-hinted.
+* Address #122 by adding `next.jdbc.with-options` that lets you wrap up a connectable along with default options that should be applied to all operations on that connectable.
 * Address #119 by clarifying realization actions in the docstrings for `row-number`, `column-names`, and `metadata`.
 * Address #115 by adding equivalent of `db-do-commands` in the `clojure.java.jdbc` migration guide.
 * Add log4j2 as a test dependency so that I have better control over logging (which makes debugging easier!).

doc/getting-started.md

@@ -108,10 +108,10 @@ user=> (jdbc/execute-one! ds ["
 insert into address(name,email)
   values('Someone Else','some@elsewhere.com')
 "] {:return-keys true :builder-fn rs/as-unqualified-lower-maps})
-{:id 2}
+{:id 3}
-user=> (jdbc/execute-one! ds ["select * from address where id = ?" 2]
+user=> (jdbc/execute-one! ds ["select * from address where id = ?" 3]
                           {:builder-fn rs/as-unqualified-lower-maps})
-{:id 2, :name "Someone Else", :email "some@elsewhere.com"}
+{:id 3, :name "Someone Else", :email "some@elsewhere.com"}
 user=>
 ```
 
@@ -121,6 +121,23 @@ Relying on the default result set builder -- and table-qualified column names --
 * If your SQL query joins tables in a way that produces duplicate column names, and you use unqualified column names, then those duplicated column names will conflict and you will get only one of them in your result -- use aliases in SQL (`as`) to make the column names distinct,
 * If your SQL query joins a table to itself under different aliases, the _qualified_ column names will conflict because they are based on the underlying table name provided by the JDBC driver rather the alias you used in your query -- again, use aliases in SQL to make those column names distinct.
 
+If you want to pass the same set of options into several operations, you can use `next.jdbc/with-options` to wrap your datasource (or connection) in a way that will pass "default options". Here's the example above rewritten with that:
+
+```clojure
+user=> (require '[next.jdbc.result-set :as rs])
+nil
+user=> (def ds-opts (jdbc/with-options ds {:builder-fn rs/as-unqualified-lower-maps}))
+#'user/ds-opts
+user=> (jdbc/execute-one! ds-opts ["
+insert into address(name,email)
+  values('Someone Else','some@elsewhere.com')
+"] {:return-keys true})
+{:id 4}
+user=> (jdbc/execute-one! ds-opts ["select * from address where id = ?" 4])
+{:id 4, :name "Someone Else", :email "some@elsewhere.com"}
+user=>
+```
+
 ### `plan` & Reducing Result Sets
 
 While the `execute!` and `execute-one!` functions are fine for retrieving result sets as data, most of the time you want to process that data efficiently without necessarily converting the entire result set into a Clojure data structure, so `next.jdbc` provides a SQL execution function that works with `reduce` and with transducers to consume the result set without the intermediate overhead of creating Clojure data structures for every row.
@@ -269,6 +286,21 @@ If `with-transaction` is given a datasource, it will create and close the connec
 
 You can read more about [working with transactions](/doc/transactions.md) further on in the documentation.
 
+> Note: Because `get-datasource` and `get-connection` return plain JDBC objects (`javax.sql.DataSource` and `java.sql.Connection` respectively), `next.jdbc/with-options` cannot flow options across those calls, so if you are explicitly managing connections or transactions as above, you would need to have local bindings for the wrapped versions:
+
+```clojure
+(with-open [con (jdbc/get-connection ds)]
+  (let [con-opts (jdbc/with-options con some-options)]
+    (jdbc/execute! con-opts ...) ; committed
+    ;; either use unwrapped version here or (:connectable con-opts)
+    (jdbc/with-transaction [tx con] ; will commit or rollback this group:
+      (let [tx-opts (jdbc/with-options tx some-options)]
+        (jdbc/execute! tx-opts ...)
+        (jdbc/execute! tx-opts ...)
+        (into [] (map :column) (jdbc/plan tx-opts ...))))
+    (jdbc/execute! con-opts ...))) ; committed
+```
+
 ### Prepared Statement Caveat
 
 Not all databases support using a `PreparedStatement` for every type of SQL operation. You might have to create a `java.sql.Statement` instead, directly from a `java.sql.Connection` and use that, without parameters, in `plan`, `execute!`, or `execute-one!`. See the following example:

doc/migration-from-clojure-java-jdbc.md

@@ -20,11 +20,11 @@ If you used `:as-arrays? true`, you will most likely want to use a `:builder-fn`
 
 ### Option Handling
 
-Because `clojure.java.jdbc` focuses on a hash map for the `db-spec` that is passed around, it can hold options that act as defaults for all operations on it. In addition, all operations in `clojure.java.jdbc` can accept a hash map of options and can pass those options down the call chain. In `next.jdbc`, `get-datasource`, `get-connection`, and `prepare` all produce Java objects that cannot have any extra options attached. On one hand, that means that you cannot provide "default options", and on the other hand it means you need to be a bit more careful to ensure that you pass the appropriate options to the appropriate function, since they cannot be passed through the call chain via the `db-spec`.
+Because `clojure.java.jdbc` focuses on a hash map for the `db-spec` that is passed around, it can hold options that act as defaults for all operations on it. In addition, all operations in `clojure.java.jdbc` can accept a hash map of options and can pass those options down the call chain. In `next.jdbc`, `get-datasource`, `get-connection`, and `prepare` all produce Java objects that cannot have any extra options attached. On one hand, that means that it is harder to provide "default options", and on the other hand it means you need to be a bit more careful to ensure that you pass the appropriate options to the appropriate function, since they cannot be passed through the call chain via the `db-spec`. That's where `next.jdbc/with-options` can come in handy to wrap a connectable (generally a datasource or a connection) but be careful where you are managing connections and/or transactions directly, as mentioned in the [Getting Started](/doc/getting-started.md) guide.
 
 In [All The Options](all-the-options.md), the appropriate options are shown for each function, as well as which options _will_ get passed down the call chain, e.g., if a function can open a connection, it will accept options for `get-connection`; if a function can build a result set, it will accept `:builder-fn`. However, `get-datasource`, `get-connection`, and `prepare` cannot propagate options any further because they produce Java objects as their results -- in particular, `prepare` can't accept `:builder-fn` because it doesn't build result sets: only `plan`, `execute-one!`, and `execute!` can use `:builder-fn`.
 
-In particular, this means that you can't globally override the default options (as you could with `clojure.java.jdbc` by adding your preferred defaults to the db-spec itself). If the default options do not suit your usage and you really don't want to override them in every call, it is recommended that you provide a wrapper namespace that implements the subset of the dozen API functions (from `next.jdbc` and `next.jdbc.sql`) that you want to use, overriding their `opts` argument with your defaults.
+In particular, this means that you can't globally override the default options (as you could with `clojure.java.jdbc` by adding your preferred defaults to the db-spec itself). If the default options do not suit your usage and you really don't want to override them in every call, it is recommended that you try to use `next.jdbc/with-options` first, and if that still doesn't satisfy you, write a wrapper namespace that implements the subset of the dozen API functions (from `next.jdbc` and `next.jdbc.sql`) that you want to use, overriding their `opts` argument with your defaults.
 
 ## Primary API
 
@@ -37,7 +37,8 @@ In particular, this means that you can't globally override the default options (
 * `execute!` -- has no direct equivalent in `clojure.java.jdbc` (but it can replace most uses of both `query` and `db-do-commands`),
 * `execute-one!` -- has no equivalent in `clojure.java.jdbc` (but it can replace most uses of `query` that currently use `:result-set-fn first`),
 * `transact` -- similar to `clojure.java.jdbc/db-transaction*`,
-* `with-transaction` -- similar to `clojure.java.jdbc/with-db-transaction`.
+* `with-transaction` -- similar to `clojure.java.jdbc/with-db-transaction`,
+* `with-options` -- provides a way to specify "default options" over a group of operations, by wrapping the connectable (datasource or connection).
 
 If you were using a bare `db-spec` hash map with `:dbtype`/`:dbname`, or a JDBC URI string everywhere, that should mostly work with `next.jdbc` since most functions accept a "connectable", but it would be better to create a datasource first, and then pass that around. Note that `clojure.java.jdbc` allowed the `jdbc:` prefix in a JDBC URI to be omitted but `next.jdbc` _requires that prefix!_
 

src/next/jdbc.clj

@@ -58,6 +58,7 @@
   In addition, wherever a `PreparedStatement` is created, you may specify:
   * `:return-keys` -- either `true` or a vector of key names to return."
   (:require [next.jdbc.connection]
+            [next.jdbc.default-options :as opts]
             [next.jdbc.prepare]
             [next.jdbc.protocols :as p]
             [next.jdbc.result-set]
@@ -265,3 +266,10 @@
   [[sym transactable opts] & body]
   (let [con (vary-meta sym assoc :tag 'java.sql.Connection)]
    `(transact ~transactable (^{:once true} fn* [~con] ~@body) ~(or opts {}))))
+
+(defn with-options
+  "Given a connectable/transactable object and a set of (default) options
+  that should be used on all operations on that object, return a new
+  wrapper object that can be used in its place."
+  [connectable opts]
+  (opts/->DefaultOptions connectable opts))

src/next/jdbc/connection.clj

@@ -4,7 +4,8 @@
   "Standard implementations of `get-datasource` and `get-connection`.
 
   Also provides `dbtypes` as a map of all known database types, and
-  the `->pool` function for creating pooled datasource objects."
+  the `->pool` and `component` functions for creating pooled datasource
+  objects."
   (:require [clojure.java.data :as j]
             [next.jdbc.protocols :as p])
   (:import (java.sql Connection DriverManager)

src/next/jdbc/default_options.clj

@@ -0,0 +1,42 @@
+;; copyright (c) 2020 Sean Corfield, all rights reserved
+
+(ns ^:no-doc next.jdbc.default-options
+  "Implementation of SQL transaction logic."
+  (:require [next.jdbc.protocols :as p]))
+
+(defrecord DefaultOptions [connectable options])
+
+(extend-protocol p/Sourceable
+  DefaultOptions
+  (get-datasource [this]
+                  (p/get-datasource (:connectable this))))
+
+(extend-protocol p/Connectable
+  DefaultOptions
+  (get-connection [this opts]
+                  (p/get-connection (:connectable this)
+                                    (merge (:options this) opts))))
+
+(extend-protocol p/Executable
+  DefaultOptions
+  (-execute [this sql-params opts]
+            (p/-execute (:connectable this) sql-params
+                        (merge (:options this) opts)))
+  (-execute-one [this sql-params opts]
+                (p/-execute-one (:connectable this) sql-params
+                                (merge (:options this) opts)))
+  (-execute-all [this sql-params opts]
+                (p/-execute-all (:connectable this) sql-params
+                                (merge (:options this) opts))))
+
+(extend-protocol p/Preparable
+  DefaultOptions
+  (prepare [this sql-params opts]
+           (p/prepare (:connectable this) sql-params
+                      (merge (:options this) opts))))
+
+(extend-protocol p/Transactable
+  DefaultOptions
+  (-transact [this body-fn opts]
+             (p/-transact (:connectable this) body-fn
+                          (merge (:options this) opts))))

test/next/jdbc/default_options_test.clj

@@ -0,0 +1,9 @@
+;; copyright (c) 2020 Sean Corfield, all rights reserved
+
+(ns next.jdbc.default-options-test
+  "Stub test namespace for default options. Nothing can really be tested
+  at this level tho'..."
+  (:require [clojure.test :refer [deftest is testing]]
+            [next.jdbc.default-options :refer :all]))
+
+(set! *warn-on-reflection* true)

test/next/jdbc_test.clj

@@ -21,120 +21,120 @@
 (specs/instrument)
 
 (deftest basic-tests
-  (testing "plan"
+  ;; use ds-opts instead of (ds) anywhere you want default options applied:
-    (is (= "Apple"
+  (let [ds-opts (jdbc/with-options (ds) (default-options))]
-           (reduce (fn [_ row] (reduced (:name row)))
+    (testing "plan"
-                   nil
+      (is (= "Apple"
-                   (jdbc/plan
+             (reduce (fn [_ row] (reduced (:name row)))
-                    (ds)
+                     nil
-                    ["select * from fruit where appearance = ?" "red"])))))
+                     (jdbc/plan
-  (testing "execute-one!"
+                      ds-opts
-    (is (nil? (jdbc/execute-one!
+                      ["select * from fruit where appearance = ?" "red"])))))
-               (ds)
+    (testing "execute-one!"
-               ["select * from fruit where appearance = ?" "neon-green"])))
+      (is (nil? (jdbc/execute-one!
-    (is (= "Apple" ((column :FRUIT/NAME)
+                 (ds)
-                    (jdbc/execute-one!
+                 ["select * from fruit where appearance = ?" "neon-green"])))
-                     (ds)
+      (is (= "Apple" ((column :FRUIT/NAME)
-                     ["select * from fruit where appearance = ?" "red"]
+                      (jdbc/execute-one!
-                     (default-options))))))
+                       ds-opts
-  (testing "execute!"
+                       ["select * from fruit where appearance = ?" "red"])))))
-    (let [rs (jdbc/execute!
+    (testing "execute!"
-              (ds)
+      (let [rs (jdbc/execute!
-              ["select * from fruit where appearance = ?" "neon-green"])]
+                ds-opts
-      (is (vector? rs))
+                ["select * from fruit where appearance = ?" "neon-green"])]
-      (is (= [] rs)))
+        (is (vector? rs))
-    (let [rs (jdbc/execute!
+        (is (= [] rs)))
-              (ds)
+      (let [rs (jdbc/execute!
-              ["select * from fruit where appearance = ?" "red"]
+                ds-opts
-              (default-options))]
+                ["select * from fruit where appearance = ?" "red"])]
-      (is (= 1 (count rs)))
+        (is (= 1 (count rs)))
-      (is (= 1 ((column :FRUIT/ID) (first rs)))))
+        (is (= 1 ((column :FRUIT/ID) (first rs)))))
-    (let [rs (jdbc/execute!
+      (let [rs (jdbc/execute!
-              (ds)
+                ds-opts
-              ["select * from fruit order by id"]
+                ["select * from fruit order by id"]
-              (assoc (default-options) :builder-fn rs/as-maps))]
+                {:builder-fn rs/as-maps})]
-      (is (every? map? rs))
+        (is (every? map? rs))
-      (is (every? meta rs))
+        (is (every? meta rs))
-      (is (= 4 (count rs)))
+        (is (= 4 (count rs)))
-      (is (= 1 ((column :FRUIT/ID) (first rs))))
+        (is (= 1 ((column :FRUIT/ID) (first rs))))
-      (is (= 4 ((column :FRUIT/ID) (last rs)))))
+        (is (= 4 ((column :FRUIT/ID) (last rs)))))
-    (let [rs (jdbc/execute!
+      (let [rs (jdbc/execute!
-              (ds)
+                ds-opts
-              ["select * from fruit order by id"]
+                ["select * from fruit order by id"]
-              (assoc (default-options) :builder-fn rs/as-arrays))]
+                {:builder-fn rs/as-arrays})]
-      (is (every? vector? rs))
+        (is (every? vector? rs))
-      (is (= 5 (count rs)))
+        (is (= 5 (count rs)))
-      (is (every? #(= 5 (count %)) rs))
+        (is (every? #(= 5 (count %)) rs))
-      ;; columns come first
+        ;; columns come first
-      (is (every? qualified-keyword? (first rs)))
+        (is (every? qualified-keyword? (first rs)))
-      ;; :FRUIT/ID should be first column
+        ;; :FRUIT/ID should be first column
-      (is (= (column :FRUIT/ID) (ffirst rs)))
+        (is (= (column :FRUIT/ID) (ffirst rs)))
-      ;; and all its corresponding values should be ints
+        ;; and all its corresponding values should be ints
-      (is (every? int? (map first (rest rs))))
+        (is (every? int? (map first (rest rs))))
-      (is (every? string? (map second (rest rs))))))
+        (is (every? string? (map second (rest rs))))))
-  (testing "execute! with adapter"
+    (testing "execute! with adapter"
-    (let [rs (jdbc/execute! ; test again, with adapter and lower columns
+      (let [rs (jdbc/execute! ; test again, with adapter and lower columns
-              (ds)
+                ds-opts
-              ["select * from fruit order by id"]
+                ["select * from fruit order by id"]
-              (assoc (default-options)
+                {:builder-fn (rs/as-arrays-adapter
-                     :builder-fn (rs/as-arrays-adapter
+                              rs/as-lower-arrays
-                                  rs/as-lower-arrays
+                              (fn [^ResultSet rs _ ^Integer i]
-                                  (fn [^ResultSet rs _ ^Integer i]
+                                (.getObject rs i)))})]
-                                    (.getObject rs i)))))]
+        (is (every? vector? rs))
-      (is (every? vector? rs))
+        (is (= 5 (count rs)))
-      (is (= 5 (count rs)))
+        (is (every? #(= 5 (count %)) rs))
-      (is (every? #(= 5 (count %)) rs))
+        ;; columns come first
-      ;; columns come first
+        (is (every? qualified-keyword? (first rs)))
-      (is (every? qualified-keyword? (first rs)))
+        ;; :fruit/id should be first column
-      ;; :fruit/id should be first column
+        (is (= :fruit/id (ffirst rs)))
-      (is (= :fruit/id (ffirst rs)))
+        ;; and all its corresponding values should be ints
-      ;; and all its corresponding values should be ints
+        (is (every? int? (map first (rest rs))))
-      (is (every? int? (map first (rest rs))))
+        (is (every? string? (map second (rest rs))))))
-      (is (every? string? (map second (rest rs))))))
+    (testing "execute! with unqualified"
-  (testing "execute! with unqualified"
+      (let [rs (jdbc/execute!
-    (let [rs (jdbc/execute!
+                (ds)
-              (ds)
+                ["select * from fruit order by id"]
-              ["select * from fruit order by id"]
+                {:builder-fn rs/as-unqualified-maps})]
-              {:builder-fn rs/as-unqualified-maps})]
+        (is (every? map? rs))
-      (is (every? map? rs))
+        (is (every? meta rs))
-      (is (every? meta rs))
+        (is (= 4 (count rs)))
-      (is (= 4 (count rs)))
+        (is (= 1 ((column :ID) (first rs))))
-      (is (= 1 ((column :ID) (first rs))))
+        (is (= 4 ((column :ID) (last rs)))))
-      (is (= 4 ((column :ID) (last rs)))))
+      (let [rs (jdbc/execute!
-    (let [rs (jdbc/execute!
+                ds-opts
-              (ds)
+                ["select * from fruit order by id"]
-              ["select * from fruit order by id"]
+                {:builder-fn rs/as-unqualified-arrays})]
-              {:builder-fn rs/as-unqualified-arrays})]
+        (is (every? vector? rs))
-      (is (every? vector? rs))
+        (is (= 5 (count rs)))
-      (is (= 5 (count rs)))
+        (is (every? #(= 5 (count %)) rs))
-      (is (every? #(= 5 (count %)) rs))
+        ;; columns come first
-      ;; columns come first
+        (is (every? simple-keyword? (first rs)))
-      (is (every? simple-keyword? (first rs)))
+        ;; :ID should be first column
-      ;; :ID should be first column
+        (is (= (column :ID) (ffirst rs)))
-      (is (= (column :ID) (ffirst rs)))
+        ;; and all its corresponding values should be ints
-      ;; and all its corresponding values should be ints
+        (is (every? int? (map first (rest rs))))
-      (is (every? int? (map first (rest rs))))
+        (is (every? string? (map second (rest rs))))))
-      (is (every? string? (map second (rest rs))))))
+    (testing "execute! with :max-rows / :maxRows"
-  (testing "execute! with :max-rows / :maxRows"
+      (let [rs (jdbc/execute!
-    (let [rs (jdbc/execute!
+                ds-opts
-              (ds)
+                ["select * from fruit order by id"]
-              ["select * from fruit order by id"]
+                {:max-rows 2})]
-              (assoc (default-options) :max-rows 2))]
+        (is (every? map? rs))
-      (is (every? map? rs))
+        (is (every? meta rs))
-      (is (every? meta rs))
+        (is (= 2 (count rs)))
-      (is (= 2 (count rs)))
+        (is (= 1 ((column :FRUIT/ID) (first rs))))
-      (is (= 1 ((column :FRUIT/ID) (first rs))))
+        (is (= 2 ((column :FRUIT/ID) (last rs)))))
-      (is (= 2 ((column :FRUIT/ID) (last rs)))))
+      (let [rs (jdbc/execute!
-    (let [rs (jdbc/execute!
+                ds-opts
-              (ds)
+                ["select * from fruit order by id"]
-              ["select * from fruit order by id"]
+                {:statement {:maxRows 2}})]
-              (assoc (default-options) :statement {:maxRows 2}))]
+        (is (every? map? rs))
-      (is (every? map? rs))
+        (is (every? meta rs))
-      (is (every? meta rs))
+        (is (= 2 (count rs)))
-      (is (= 2 (count rs)))
+        (is (= 1 ((column :FRUIT/ID) (first rs))))
-      (is (= 1 ((column :FRUIT/ID) (first rs))))
+        (is (= 2 ((column :FRUIT/ID) (last rs)))))))
-      (is (= 2 ((column :FRUIT/ID) (last rs))))))
   (testing "prepare"
+    ;; default options do not flow over get-connection
     (let [rs (with-open [con (jdbc/get-connection (ds))
                          ps  (jdbc/prepare
                               con
@@ -146,6 +146,7 @@
       (is (= 4 (count rs)))
       (is (= 1 ((column :FRUIT/ID) (first rs))))
       (is (= 4 ((column :FRUIT/ID) (last rs)))))
+    ;; default options do not flow over get-connection
     (let [rs (with-open [con (jdbc/get-connection (ds))
                          ps  (jdbc/prepare
                               con
@@ -157,6 +158,7 @@
       (is (= 1 (count rs)))
       (is (= 4 ((column :FRUIT/ID) (first rs))))))
   (testing "statement"
+    ;; default options do not flow over get-connection
     (let [rs (with-open [con (jdbc/get-connection (ds))]
                (jdbc/execute! (prep/statement con (default-options))
                               ["select * from fruit order by id"]))]
@@ -165,6 +167,7 @@
       (is (= 4 (count rs)))
       (is (= 1 ((column :FRUIT/ID) (first rs))))
       (is (= 4 ((column :FRUIT/ID) (last rs)))))
+    ;; default options do not flow over get-connection
     (let [rs (with-open [con (jdbc/get-connection (ds))]
                (jdbc/execute! (prep/statement con (default-options))
                               ["select * from fruit where id = 4"]))]