luciano/next-jdbc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fixes #82 and adds next.jdbc.prepare/statement

Browse source
This commit is contained in:
Sean Corfield 2019-12-20 15:45:22 -08:00
parent c626339681
commit a2ba8ff780
12 changed files with 133 additions and 37 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CHANGELOG.md
View file

@ -6,7 +6,9 @@ Only accretive/fixative changes will be made from now on.
The following changes have been committed to the **master** branch since the 1.0.12 release:
* None.
* Fix #82 by adding `clojure.java.data`-based support for setting arbitrary properties on `Connection` and `PreparedStatement` objects, post-creation. Note: this uses the Java reflection API under the hood.
* Adds `next.jdbc.prepare/statement` to create `Statement` objects with all the options available to `prepare` except `:return-keys`.
* Update `org.clojure/java.data` to 0.1.5 (for property setting).
## Stable Builds

2
deps.edn
View file

@ -1,6 +1,6 @@
{:paths ["src"]
:deps {org.clojure/clojure {:mvn/version "1.10.1"}
org.clojure/java.data {:mvn/version "0.1.4"}}
org.clojure/java.data {:mvn/version "0.1.5"}}
:aliases
{:test {:extra-paths ["test"]
:extra-deps {org.clojure/test.check {:mvn/version "0.10.0"}

13
doc/all-the-options.md
View file

@ -26,6 +26,7 @@ Any path that calls `get-connection` will accept the following options:
* `:auto-commit` -- a `Boolean` that determines whether operations on this connection should be automatically committed (the default, `true`) or not; note that setting `:auto-commit false` is commonly required when you want to stream result set data from a query (along with fetch size etc -- see below),
* `:read-only` -- a `Boolean` that determines whether the operations on this connection should be read-only or not (the default, `false`).
* `:connection` -- a hash map of camelCase properties to set on the `Connection` object after it is created; these correspond to `.set*` methods on the `Connection` class and are set via the Java reflection API (using `org.clojure/java.data`). If `:autoCommit` or `:readOnly` are provided, they will take precedence over the fast, specific options above.
If you need additional options set on a connection, you can either use Java interop to set them directly, or provide them as part of the "db spec" hash map passed to `get-datasource` (although then they will apply to _all_ connections obtained from that datasource).
@ -44,22 +45,26 @@ Any function that might realize a row or a result set will accept:
* `:label-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option must be present and should specify a string-to-string transformation that will be applied to the column label for each returned column name.
* `:qualifier-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option should specify a string-to-string transformation that will be applied to the table name for each returned column name for which the table name is known. It can be omitted for the `as-unqualified-modified-*` variants.
## Prepared Statements
## Statements & Prepared Statements
Any function that creates a `PreparedStatement` will accept the following options:
Any function that creates a `Statement` or a `PreparedStatement` will accept the following options (see below for additional options for `PreparedStatement`):
* `:concurrency` -- a keyword that specifies the concurrency level: `:read-only`, `:updatable`,
* `:cursors` -- a keyword that specifies whether cursors should be closed or held over a commit: `:close`, `:hold`,
* `:fetch-size` -- an integer that guides the JDBC driver in terms of how many rows to fetch at once; it is common to set `:fetch-size` to zero or a negative value in order to trigger streaming of result sets -- some JDBC drivers require additional options to be set on the connection _as well_,
* `:max-rows` -- an integer that tells the JDBC driver to limit result sets to this many rows,
* `:result-type` -- a keyword that affects how the `ResultSet` can be traversed: `:forward-only`, `:scroll-insensitive`, `:scroll-sensitive`,
* `:return-keys` -- a truthy value asks that the JDBC driver to return any generated keys created by the operation; it can be `true` or it can be a vector of keywords identifying column names that should be returned,
* `:timeout` -- an integer that specifies the timeout allowed for SQL operations.
* `:timeout` -- an integer that specifies the (query) timeout allowed for SQL operations.
* `:statement` -- a hash map of camelCase properties to set on the `Statement` or `PreparedStatement` object after it is created; these correspond to `.set*` methods on the `Statement` class (which `PreparedStatement` inherits) and are set via the Java reflection API (using `org.clojure/java.data`). If `:fetchSize`, `:maxRows`, or `:queryTimeout` are provided, they will take precedence over the fast, specific options above.
If you specify either `:concurrency` or `:result-type`, you must specify _both_ of them. If you specify `:cursors`, you must also specify `:result-type` _and_ `:concurrency`.
> Note: For MS SQL Server to return table names (for qualified column names), you must specify `:result-type` with one of the scroll values (and so you must also specify `:concurrency`).
Any function that creates a `PreparedStatement` will additionally accept the following options:
* `:return-keys` -- a truthy value asks that the JDBC driver to return any generated keys created by the operation; it can be `true` or it can be a vector of keywords identifying column names that should be returned.
Not all databases or drivers support all of these options, or all values for any given option. If `:return-keys` is a vector of column names and that is not supported, `next.jdbc` will attempt a generic "return generated keys" option instead. If that is not supported, `next.jdbc` will fall back to a regular SQL operation. If other options are not supported, you may get a `SQLException`.
In addition, `next.jdbc.prepare/execute-batch!` accepts an options hash map that can contain the following:

8
doc/getting-started.md
View file

@ -185,13 +185,15 @@ You can read more about [working with transactions](/doc/transactions.md) furthe
### Prepared Statement Caveat
Not all databases support using a `PreparedStatement` for every type of SQL operation. You might have to create a `java.sql.Statement` yourself, directly from a `java.sql.Connection` and use that, without parameters, in `plan`, `execute!`, or `execute-one!`. See the following example:
Not all databases support using a `PreparedStatement` for every type of SQL operation. You might have to create a `java.sql.Statement` instead, directly from a `java.sql.Connection` and use that, without parameters, in `plan`, `execute!`, or `execute-one!`. See the following example:
```clojure
(require '[next.jdbc.prepare :as prep])
(with-open [con (jdbc/get-connection ds)]
(jdbc/execute! (.createStatement con) ["...just a SQL string..."])
(jdbc/execute! (prep/statement con) ["...just a SQL string..."])
(jdbc/execute! con ["...some SQL..." "and" "parameters"]) ; uses PreparedStatement
(into [] (map :column) (jdbc/plan (.createStatement con) ["..."])))
(into [] (map :column) (jdbc/plan (prep/statement con) ["..."])))
```
## Connection Pooling

6
doc/prepared-statements.md
View file

@ -2,16 +2,18 @@
Under the hood, whenever you ask `next.jdbc` to execute some SQL it creates a `java.sql.PreparedStatement`, adds in the parameters you provide, and then calls `.execute` on it. Then it attempts to get a `ResultSet` from that and either return it or process it. If you asked for generated keys to be returned, that `ResultSet` will contain those generated keys if your database supports it, otherwise it will be whatever the `.execute` function produces. If no `ResultSet` is available at all, `next.jdbc` will ask for the count of updated rows and return that as if it were a result set.
> Note: Some databases do not support all SQL operations via `PreparedStatement`, in which case you may need to create a `java.sql.Statement` and pass that into `plan`, `execute!`, or `execute-one!`, along with the SQL you wish to execute. Note that such statement execution may not have parameters. See the [Prepared Statement Caveat in Getting Started](/doc/getting-started.md#prepared-statement-caveat) for an example.
> Note: Some databases do not support all SQL operations via `PreparedStatement`, in which case you may need to create a `java.sql.Statement` instead and pass that into `plan`, `execute!`, or `execute-one!`, along with the SQL you wish to execute. Note that such statement execution may not have parameters. See the [Prepared Statement Caveat in Getting Started](/doc/getting-started.md#prepared-statement-caveat) for an example.
If you have a SQL operation that you intend to run multiple times on the same `java.sql.Connection`, it may be worth creating the prepared statement yourself and reusing it. `next.jdbc/prepare` accepts a connection and a vector of SQL and optional parameters and returns a `java.sql.PreparedStatement` which can be passed to `plan`, `execute!`, or `execute-one!` as the first argument. It is your responsibility to close the prepared statement after it has been used.
If you need to pass an option map to `plan`, `execute!`, or `execute-one!` when passing a prepared statement, you must pass `nil` or `[]` as the second argument:
If you need to pass an option map to `plan`, `execute!`, or `execute-one!` when passing a statement or prepared statement, you must pass `nil` or `[]` as the second argument:
```clojure
(with-open [con (jdbc/get-connection ds)]
(with-open [ps (jdbc/prepare con ["..." ...])]
(jdbc/execute-one! ps nil {...})))
(with-open [stmt (jdbc/statement con)]
(jdbc/execute-one! stmt nil {...})))
```
You can provide the parameters in the `prepare` call or you can provide them via a call to `set-parameters` (discussed in more detail below).

17
src/next/jdbc.clj
View file

@ -38,14 +38,25 @@
column names followed by vectors of column values for each row, and
lower-case variants of each.
The following options are supported wherever a `PreparedStatement` is created:
The following options are supported wherever a `Connection` is created:
* `:auto-commit` -- either `true` or `false`,
* `:read-only` -- either `true` or `false`,
* `:connection` -- a hash map of camelCase properties to set, via reflection,
on the `Connection` object after it is created.
The following options are supported wherever a `Statement` or
`PreparedStatement` is created:
* `:concurrency` -- `:read-only`, `:updatable`,
* `:cursors` -- `:close`, `:hold`
* `:fetch-size` -- the fetch size value,
* `:max-rows` -- the maximum number of rows to return,
* `:result-type` -- `:forward-only`, `:scroll-insensitive`, `:scroll-sensitive`,
* `:return-keys` -- either `true` or a vector of key names to return,
* `:timeout` -- the query timeout."
* `:timeout` -- the query timeout,
* `:statement` -- a hash map of camelCase properties to set, via reflection,
on the `Statement` or `PreparedStatement` object after it is created.
In addition, wherever a `PreparedStatement` is created, you may specify:
* `:return-keys` -- either `true` or a vector of key names to return."
(:require [next.jdbc.connection]
[next.jdbc.prepare]
[next.jdbc.protocols :as p]

16
src/next/jdbc/connection.clj
View file

@ -5,7 +5,7 @@
Also provides `dbtypes` as a map of all known database types, and
the `->pool` function for creating pooled datasource objects."
(:require [clojure.java.data :refer [to-java]]
(:require [clojure.java.data :as j]
[next.jdbc.protocols :as p])
(:import (java.sql Connection DriverManager)
(javax.sql DataSource)
@ -221,9 +221,9 @@
`java.io.Closeable` (HikariCP does, c3p0 does not)."
[clazz db-spec]
(if (:jdbcUrl db-spec)
(to-java clazz db-spec)
(j/to-java clazz db-spec)
(let [[url etc] (spec->url+etc db-spec)]
(to-java clazz (assoc etc :jdbcUrl url)))))
(j/to-java clazz (assoc etc :jdbcUrl url)))))
(defn- string->url+etc
"Given a JDBC URL, return it with an empty set of options with no parsing."
@ -248,17 +248,23 @@
"Given a `DataSource` and a map of options, get a connection and update it
as specified by the options.
The options supported are:
These options are supported:
* `:auto-commit` -- whether the connection should be set to auto-commit or not;
without this option, the defaut is `true` -- connections will auto-commit,
* `:read-only` -- whether the connection should be set to read-only mode."
* `:read-only` -- whether the connection should be set to read-only mode,
* `:connection` -- a hash map of camelCase properties to set on the connection,
via reflection, e.g., :autoCommit, :readOnly, :schema..."
^Connection
[^DataSource datasource opts]
(let [^Connection connection (.getConnection datasource)]
;; fast, specific option handling:
(when (contains? opts :auto-commit)
(.setAutoCommit connection (boolean (:auto-commit opts))))
(when (contains? opts :read-only)
(.setReadOnly connection (boolean (:read-only opts))))
;; slow, general-purpose option handling:
(when-let [props (:connection opts)]
(j/set-properties connection props))
connection))
(extend-protocol p/Sourceable

48
src/next/jdbc/prepare.clj
View file

@ -12,7 +12,8 @@
Defines the `SettableParameter` protocol for converting Clojure values
to database-specific values."
(:require [next.jdbc.protocols :as p])
(:require [clojure.java.data :as j]
[next.jdbc.protocols :as p])
(:import (java.sql Connection
PreparedStatement
ResultSet
@ -82,7 +83,8 @@
^java.sql.PreparedStatement
[^Connection con ^String sql params
{:keys [return-keys result-type concurrency cursors
fetch-size max-rows timeout]}]
fetch-size max-rows timeout]
:as opts}]
(let [^PreparedStatement ps
(cond
return-keys
@ -124,12 +126,16 @@
"may not be specified independently.")))
:else
(.prepareStatement con sql))]
;; fast, specific option handling:
(when fetch-size
(.setFetchSize ps fetch-size))
(when max-rows
(.setMaxRows ps max-rows))
(when timeout
(.setQueryTimeout ps timeout))
;; slow, general-purpose option handling:
(when-let [props (:statement opts)]
(j/set-properties ps props))
(set-parameters ps params)))
(extend-protocol p/Preparable
@ -137,6 +143,44 @@
(prepare [this sql-params opts]
(create this (first sql-params) (rest sql-params) opts)))
(defn statement
"Given a `Connection` and some options, return a `Statement`."
^java.sql.Statement
([con] (statement con {}))
([^Connection con
{:keys [result-type concurrency cursors
fetch-size max-rows timeout]
:as opts}]
(let [^Statement stmt
(cond
(and result-type concurrency)
(if cursors
(.createStatement con
(get result-set-type result-type result-type)
(get result-set-concurrency concurrency concurrency)
(get result-set-holdability cursors cursors))
(.createStatement con
(get result-set-type result-type result-type)
(get result-set-concurrency concurrency concurrency)))
(or result-type concurrency cursors)
(throw (IllegalArgumentException.
(str ":concurrency, :cursors, and :result-type "
"may not be specified independently.")))
:else
(.createStatement con))]
;; fast, specific option handling:
(when fetch-size
(.setFetchSize stmt fetch-size))
(when max-rows
(.setMaxRows stmt max-rows))
(when timeout
(.setQueryTimeout stmt timeout))
;; slow, general-purpose option handling:
(when-let [props (:statement opts)]
(j/set-properties stmt props))
stmt)))
(defn execute-batch!
"Given a `PreparedStatement` and a vector containing parameter groups,
i.e., a vector of vector of parameters, use `.addBatch` to add each group

2
src/next/jdbc/result_set.clj
View file

@ -652,7 +652,7 @@
[{:next.jdbc/update-count (.getUpdateCount this)}]))
java.sql.Statement
;; we can't tell if this PreparedStatement will return generated
;; we can't tell if this Statement will return generated
;; keys so we pass a truthy value to at least attempt it if we
;; do not get a ResultSet back from the execute call
(-execute [this sql-params opts]

14
src/next/jdbc/specs.clj
View file

@ -21,7 +21,7 @@
[next.jdbc.connection :as connection]
[next.jdbc.prepare :as prepare]
[next.jdbc.sql :as sql])
(:import (java.sql Connection PreparedStatement)
(:import (java.sql Connection PreparedStatement Statement)
(javax.sql DataSource)))
(set! *warn-on-reflection* true)
@ -48,6 +48,7 @@
(s/def ::connection #(instance? Connection %))
(s/def ::datasource #(instance? DataSource %))
(s/def ::prepared-statement #(instance? PreparedStatement %))
(s/def ::statement #(instance? Statement %))
(s/def ::db-spec (s/or :db-spec ::db-spec-map
:jdbc-url ::jdbc-url-map
@ -90,20 +91,24 @@
:sql-params ::sql-params
:opts (s/? ::opts-map)))
(s/fdef jdbc/statement
:args (s/cat :connection ::connection
:opts (s/? ::opts-map)))
(s/fdef jdbc/plan
:args (s/alt :prepared (s/cat :stmt ::prepared-statement)
:args (s/alt :prepared (s/cat :stmt ::statement)
:sql (s/cat :connectable ::connectable
:sql-params (s/nilable ::sql-params)
:opts (s/? ::opts-map))))
(s/fdef jdbc/execute!
:args (s/alt :prepared (s/cat :stmt ::prepared-statement)
:args (s/alt :prepared (s/cat :stmt ::statement)
:sql (s/cat :connectable ::connectable
:sql-params (s/nilable ::sql-params)
:opts (s/? ::opts-map))))
(s/fdef jdbc/execute-one!
:args (s/alt :prepared (s/cat :stmt ::prepared-statement)
:args (s/alt :prepared (s/cat :stmt ::statement)
:sql (s/cat :connectable ::connectable
:sql-params (s/nilable ::sql-params)
:opts (s/? ::opts-map))))
@ -201,6 +206,7 @@
`connection/->pool
`prepare/execute-batch!
`prepare/set-parameters
`prepare/statement
`sql/insert!
`sql/insert-multi!
`sql/query

1
test/next/jdbc/test_fixtures.clj
View file

@ -4,6 +4,7 @@
"Multi-database testing fixtures."
(:require [clojure.string :as str]
[next.jdbc :as jdbc]
[next.jdbc.prepare :as prep]
[next.jdbc.sql :as sql])
(:import (com.opentable.db.postgres.embedded EmbeddedPostgres)
(javax.sql DataSource)))

39
test/next/jdbc_test.clj
View file

@ -71,7 +71,8 @@
(is (= (column :FRUIT/ID) (ffirst rs)))
;; and all its corresponding values should be ints
(is (every? int? (map first (rest rs))))
(is (every? string? (map second (rest rs)))))
(is (every? string? (map second (rest rs))))))
(testing "execute! with adapter"
(let [rs (jdbc/execute! ; test again, with adapter and lower columns
(ds)
["select * from fruit order by id"]
@ -89,7 +90,8 @@
(is (= :fruit/id (ffirst rs)))
;; and all its corresponding values should be ints
(is (every? int? (map first (rest rs))))
(is (every? string? (map second (rest rs)))))
(is (every? string? (map second (rest rs))))))
(testing "execute! with unqualified"
(let [rs (jdbc/execute!
(ds)
["select * from fruit order by id"]
@ -113,6 +115,25 @@
;; and all its corresponding values should be ints
(is (every? int? (map first (rest rs))))
(is (every? string? (map second (rest rs))))))
(testing "execute! with :max-rows / :maxRows"
(let [rs (jdbc/execute!
(ds)
["select * from fruit order by id"]
(assoc (default-options) :max-rows 2))]
(is (every? map? rs))
(is (every? meta rs))
(is (= 2 (count rs)))
(is (= 1 ((column :FRUIT/ID) (first rs))))
(is (= 2 ((column :FRUIT/ID) (last rs)))))
(let [rs (jdbc/execute!
(ds)
["select * from fruit order by id"]
(assoc (default-options) :statement {:maxRows 2}))]
(is (every? map? rs))
(is (every? meta rs))
(is (= 2 (count rs)))
(is (= 1 ((column :FRUIT/ID) (first rs))))
(is (= 2 ((column :FRUIT/ID) (last rs))))))
(testing "prepare"
(let [rs (with-open [con (jdbc/get-connection (ds))
ps (jdbc/prepare
@ -137,24 +158,20 @@
(is (= 4 ((column :FRUIT/ID) (first rs))))))
(testing "statement"
(let [rs (with-open [con (jdbc/get-connection (ds))]
(jdbc/execute! (.createStatement con)
(jdbc/execute! (prep/statement con (default-options))
["select * from fruit order by id"]))]
(is (every? map? rs))
(is (every? meta rs))
(is (= 4 (count rs)))
;; SQL Server only returns table name if result-type/concurrency
;; provided, which we can only for a PreparedStatement
(is (= 1 ((if (mssql?) :ID (column :FRUIT/ID)) (first rs))))
(is (= 4 ((if (mssql?) :ID (column :FRUIT/ID)) (last rs)))))
(is (= 1 ((column :FRUIT/ID) (first rs))))
(is (= 4 ((column :FRUIT/ID) (last rs)))))
(let [rs (with-open [con (jdbc/get-connection (ds))]
(jdbc/execute! (.createStatement con)
(jdbc/execute! (prep/statement con (default-options))
["select * from fruit where id = 4"]))]
(is (every? map? rs))
(is (every? meta rs))
(is (= 1 (count rs)))
;; SQL Server only returns table name if result-type/concurrency
;; provided, which we can only for a PreparedStatement
(is (= 4 ((if (mssql?) :ID (column :FRUIT/ID)) (first rs))))))
(is (= 4 ((column :FRUIT/ID) (first rs))))))
(testing "transact"
(is (= [{:next.jdbc/update-count 1}]
(jdbc/transact (ds)