diff --git a/CHANGELOG.md b/CHANGELOG.md index f20f73f..0173820 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,9 @@ Only accretive/fixative changes will be made from now on. The following changes have been committed to the **master** branch since the 1.0.12 release: -* None. +* Fix #82 by adding `clojure.java.data`-based support for setting arbitrary properties on `Connection` and `PreparedStatement` objects, post-creation. Note: this uses the Java reflection API under the hood. +* Adds `next.jdbc.prepare/statement` to create `Statement` objects with all the options available to `prepare` except `:return-keys`. +* Update `org.clojure/java.data` to 0.1.5 (for property setting). ## Stable Builds diff --git a/deps.edn b/deps.edn index 5b68110..9993e25 100644 --- a/deps.edn +++ b/deps.edn @@ -1,6 +1,6 @@ {:paths ["src"] :deps {org.clojure/clojure {:mvn/version "1.10.1"} - org.clojure/java.data {:mvn/version "0.1.4"}} + org.clojure/java.data {:mvn/version "0.1.5"}} :aliases {:test {:extra-paths ["test"] :extra-deps {org.clojure/test.check {:mvn/version "0.10.0"} diff --git a/doc/all-the-options.md b/doc/all-the-options.md index 98bca98..f1035b8 100644 --- a/doc/all-the-options.md +++ b/doc/all-the-options.md @@ -26,6 +26,7 @@ Any path that calls `get-connection` will accept the following options: * `:auto-commit` -- a `Boolean` that determines whether operations on this connection should be automatically committed (the default, `true`) or not; note that setting `:auto-commit false` is commonly required when you want to stream result set data from a query (along with fetch size etc -- see below), * `:read-only` -- a `Boolean` that determines whether the operations on this connection should be read-only or not (the default, `false`). +* `:connection` -- a hash map of camelCase properties to set on the `Connection` object after it is created; these correspond to `.set*` methods on the `Connection` class and are set via the Java reflection API (using `org.clojure/java.data`). If `:autoCommit` or `:readOnly` are provided, they will take precedence over the fast, specific options above. If you need additional options set on a connection, you can either use Java interop to set them directly, or provide them as part of the "db spec" hash map passed to `get-datasource` (although then they will apply to _all_ connections obtained from that datasource). @@ -44,22 +45,26 @@ Any function that might realize a row or a result set will accept: * `:label-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option must be present and should specify a string-to-string transformation that will be applied to the column label for each returned column name. * `:qualifier-fn` -- if `:builder-fn` is specified as one of `next.jdbc.result-set`'s `as-modified-*` builders, this option should specify a string-to-string transformation that will be applied to the table name for each returned column name for which the table name is known. It can be omitted for the `as-unqualified-modified-*` variants. -## Prepared Statements +## Statements & Prepared Statements -Any function that creates a `PreparedStatement` will accept the following options: +Any function that creates a `Statement` or a `PreparedStatement` will accept the following options (see below for additional options for `PreparedStatement`): * `:concurrency` -- a keyword that specifies the concurrency level: `:read-only`, `:updatable`, * `:cursors` -- a keyword that specifies whether cursors should be closed or held over a commit: `:close`, `:hold`, * `:fetch-size` -- an integer that guides the JDBC driver in terms of how many rows to fetch at once; it is common to set `:fetch-size` to zero or a negative value in order to trigger streaming of result sets -- some JDBC drivers require additional options to be set on the connection _as well_, * `:max-rows` -- an integer that tells the JDBC driver to limit result sets to this many rows, * `:result-type` -- a keyword that affects how the `ResultSet` can be traversed: `:forward-only`, `:scroll-insensitive`, `:scroll-sensitive`, -* `:return-keys` -- a truthy value asks that the JDBC driver to return any generated keys created by the operation; it can be `true` or it can be a vector of keywords identifying column names that should be returned, -* `:timeout` -- an integer that specifies the timeout allowed for SQL operations. +* `:timeout` -- an integer that specifies the (query) timeout allowed for SQL operations. +* `:statement` -- a hash map of camelCase properties to set on the `Statement` or `PreparedStatement` object after it is created; these correspond to `.set*` methods on the `Statement` class (which `PreparedStatement` inherits) and are set via the Java reflection API (using `org.clojure/java.data`). If `:fetchSize`, `:maxRows`, or `:queryTimeout` are provided, they will take precedence over the fast, specific options above. If you specify either `:concurrency` or `:result-type`, you must specify _both_ of them. If you specify `:cursors`, you must also specify `:result-type` _and_ `:concurrency`. > Note: For MS SQL Server to return table names (for qualified column names), you must specify `:result-type` with one of the scroll values (and so you must also specify `:concurrency`). +Any function that creates a `PreparedStatement` will additionally accept the following options: + +* `:return-keys` -- a truthy value asks that the JDBC driver to return any generated keys created by the operation; it can be `true` or it can be a vector of keywords identifying column names that should be returned. + Not all databases or drivers support all of these options, or all values for any given option. If `:return-keys` is a vector of column names and that is not supported, `next.jdbc` will attempt a generic "return generated keys" option instead. If that is not supported, `next.jdbc` will fall back to a regular SQL operation. If other options are not supported, you may get a `SQLException`. In addition, `next.jdbc.prepare/execute-batch!` accepts an options hash map that can contain the following: diff --git a/doc/getting-started.md b/doc/getting-started.md index 52f0bad..7a3e49f 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -185,13 +185,15 @@ You can read more about [working with transactions](/doc/transactions.md) furthe ### Prepared Statement Caveat -Not all databases support using a `PreparedStatement` for every type of SQL operation. You might have to create a `java.sql.Statement` yourself, directly from a `java.sql.Connection` and use that, without parameters, in `plan`, `execute!`, or `execute-one!`. See the following example: +Not all databases support using a `PreparedStatement` for every type of SQL operation. You might have to create a `java.sql.Statement` instead, directly from a `java.sql.Connection` and use that, without parameters, in `plan`, `execute!`, or `execute-one!`. See the following example: ```clojure +(require '[next.jdbc.prepare :as prep]) + (with-open [con (jdbc/get-connection ds)] - (jdbc/execute! (.createStatement con) ["...just a SQL string..."]) + (jdbc/execute! (prep/statement con) ["...just a SQL string..."]) (jdbc/execute! con ["...some SQL..." "and" "parameters"]) ; uses PreparedStatement - (into [] (map :column) (jdbc/plan (.createStatement con) ["..."]))) + (into [] (map :column) (jdbc/plan (prep/statement con) ["..."]))) ``` ## Connection Pooling diff --git a/doc/prepared-statements.md b/doc/prepared-statements.md index b042d33..f9992b5 100644 --- a/doc/prepared-statements.md +++ b/doc/prepared-statements.md @@ -2,16 +2,18 @@ Under the hood, whenever you ask `next.jdbc` to execute some SQL it creates a `java.sql.PreparedStatement`, adds in the parameters you provide, and then calls `.execute` on it. Then it attempts to get a `ResultSet` from that and either return it or process it. If you asked for generated keys to be returned, that `ResultSet` will contain those generated keys if your database supports it, otherwise it will be whatever the `.execute` function produces. If no `ResultSet` is available at all, `next.jdbc` will ask for the count of updated rows and return that as if it were a result set. -> Note: Some databases do not support all SQL operations via `PreparedStatement`, in which case you may need to create a `java.sql.Statement` and pass that into `plan`, `execute!`, or `execute-one!`, along with the SQL you wish to execute. Note that such statement execution may not have parameters. See the [Prepared Statement Caveat in Getting Started](/doc/getting-started.md#prepared-statement-caveat) for an example. +> Note: Some databases do not support all SQL operations via `PreparedStatement`, in which case you may need to create a `java.sql.Statement` instead and pass that into `plan`, `execute!`, or `execute-one!`, along with the SQL you wish to execute. Note that such statement execution may not have parameters. See the [Prepared Statement Caveat in Getting Started](/doc/getting-started.md#prepared-statement-caveat) for an example. If you have a SQL operation that you intend to run multiple times on the same `java.sql.Connection`, it may be worth creating the prepared statement yourself and reusing it. `next.jdbc/prepare` accepts a connection and a vector of SQL and optional parameters and returns a `java.sql.PreparedStatement` which can be passed to `plan`, `execute!`, or `execute-one!` as the first argument. It is your responsibility to close the prepared statement after it has been used. -If you need to pass an option map to `plan`, `execute!`, or `execute-one!` when passing a prepared statement, you must pass `nil` or `[]` as the second argument: +If you need to pass an option map to `plan`, `execute!`, or `execute-one!` when passing a statement or prepared statement, you must pass `nil` or `[]` as the second argument: ```clojure (with-open [con (jdbc/get-connection ds)] (with-open [ps (jdbc/prepare con ["..." ...])] (jdbc/execute-one! ps nil {...}))) + (with-open [stmt (jdbc/statement con)] + (jdbc/execute-one! stmt nil {...}))) ``` You can provide the parameters in the `prepare` call or you can provide them via a call to `set-parameters` (discussed in more detail below). diff --git a/src/next/jdbc.clj b/src/next/jdbc.clj index aad83b8..152a5eb 100644 --- a/src/next/jdbc.clj +++ b/src/next/jdbc.clj @@ -38,14 +38,25 @@ column names followed by vectors of column values for each row, and lower-case variants of each. - The following options are supported wherever a `PreparedStatement` is created: + The following options are supported wherever a `Connection` is created: + * `:auto-commit` -- either `true` or `false`, + * `:read-only` -- either `true` or `false`, + * `:connection` -- a hash map of camelCase properties to set, via reflection, + on the `Connection` object after it is created. + + The following options are supported wherever a `Statement` or + `PreparedStatement` is created: * `:concurrency` -- `:read-only`, `:updatable`, * `:cursors` -- `:close`, `:hold` * `:fetch-size` -- the fetch size value, * `:max-rows` -- the maximum number of rows to return, * `:result-type` -- `:forward-only`, `:scroll-insensitive`, `:scroll-sensitive`, - * `:return-keys` -- either `true` or a vector of key names to return, - * `:timeout` -- the query timeout." + * `:timeout` -- the query timeout, + * `:statement` -- a hash map of camelCase properties to set, via reflection, + on the `Statement` or `PreparedStatement` object after it is created. + + In addition, wherever a `PreparedStatement` is created, you may specify: + * `:return-keys` -- either `true` or a vector of key names to return." (:require [next.jdbc.connection] [next.jdbc.prepare] [next.jdbc.protocols :as p] diff --git a/src/next/jdbc/connection.clj b/src/next/jdbc/connection.clj index b452248..bcdb335 100644 --- a/src/next/jdbc/connection.clj +++ b/src/next/jdbc/connection.clj @@ -5,7 +5,7 @@ Also provides `dbtypes` as a map of all known database types, and the `->pool` function for creating pooled datasource objects." - (:require [clojure.java.data :refer [to-java]] + (:require [clojure.java.data :as j] [next.jdbc.protocols :as p]) (:import (java.sql Connection DriverManager) (javax.sql DataSource) @@ -221,9 +221,9 @@ `java.io.Closeable` (HikariCP does, c3p0 does not)." [clazz db-spec] (if (:jdbcUrl db-spec) - (to-java clazz db-spec) + (j/to-java clazz db-spec) (let [[url etc] (spec->url+etc db-spec)] - (to-java clazz (assoc etc :jdbcUrl url))))) + (j/to-java clazz (assoc etc :jdbcUrl url))))) (defn- string->url+etc "Given a JDBC URL, return it with an empty set of options with no parsing." @@ -248,17 +248,23 @@ "Given a `DataSource` and a map of options, get a connection and update it as specified by the options. - The options supported are: + These options are supported: * `:auto-commit` -- whether the connection should be set to auto-commit or not; without this option, the defaut is `true` -- connections will auto-commit, - * `:read-only` -- whether the connection should be set to read-only mode." + * `:read-only` -- whether the connection should be set to read-only mode, + * `:connection` -- a hash map of camelCase properties to set on the connection, + via reflection, e.g., :autoCommit, :readOnly, :schema..." ^Connection [^DataSource datasource opts] (let [^Connection connection (.getConnection datasource)] + ;; fast, specific option handling: (when (contains? opts :auto-commit) (.setAutoCommit connection (boolean (:auto-commit opts)))) (when (contains? opts :read-only) (.setReadOnly connection (boolean (:read-only opts)))) + ;; slow, general-purpose option handling: + (when-let [props (:connection opts)] + (j/set-properties connection props)) connection)) (extend-protocol p/Sourceable diff --git a/src/next/jdbc/prepare.clj b/src/next/jdbc/prepare.clj index d81ce99..65be567 100644 --- a/src/next/jdbc/prepare.clj +++ b/src/next/jdbc/prepare.clj @@ -12,7 +12,8 @@ Defines the `SettableParameter` protocol for converting Clojure values to database-specific values." - (:require [next.jdbc.protocols :as p]) + (:require [clojure.java.data :as j] + [next.jdbc.protocols :as p]) (:import (java.sql Connection PreparedStatement ResultSet @@ -82,7 +83,8 @@ ^java.sql.PreparedStatement [^Connection con ^String sql params {:keys [return-keys result-type concurrency cursors - fetch-size max-rows timeout]}] + fetch-size max-rows timeout] + :as opts}] (let [^PreparedStatement ps (cond return-keys @@ -124,12 +126,16 @@ "may not be specified independently."))) :else (.prepareStatement con sql))] + ;; fast, specific option handling: (when fetch-size (.setFetchSize ps fetch-size)) (when max-rows (.setMaxRows ps max-rows)) (when timeout (.setQueryTimeout ps timeout)) + ;; slow, general-purpose option handling: + (when-let [props (:statement opts)] + (j/set-properties ps props)) (set-parameters ps params))) (extend-protocol p/Preparable @@ -137,6 +143,44 @@ (prepare [this sql-params opts] (create this (first sql-params) (rest sql-params) opts))) +(defn statement + "Given a `Connection` and some options, return a `Statement`." + ^java.sql.Statement + ([con] (statement con {})) + ([^Connection con + {:keys [result-type concurrency cursors + fetch-size max-rows timeout] + :as opts}] + (let [^Statement stmt + (cond + (and result-type concurrency) + (if cursors + (.createStatement con + (get result-set-type result-type result-type) + (get result-set-concurrency concurrency concurrency) + (get result-set-holdability cursors cursors)) + (.createStatement con + (get result-set-type result-type result-type) + (get result-set-concurrency concurrency concurrency))) + + (or result-type concurrency cursors) + (throw (IllegalArgumentException. + (str ":concurrency, :cursors, and :result-type " + "may not be specified independently."))) + :else + (.createStatement con))] + ;; fast, specific option handling: + (when fetch-size + (.setFetchSize stmt fetch-size)) + (when max-rows + (.setMaxRows stmt max-rows)) + (when timeout + (.setQueryTimeout stmt timeout)) + ;; slow, general-purpose option handling: + (when-let [props (:statement opts)] + (j/set-properties stmt props)) + stmt))) + (defn execute-batch! "Given a `PreparedStatement` and a vector containing parameter groups, i.e., a vector of vector of parameters, use `.addBatch` to add each group diff --git a/src/next/jdbc/result_set.clj b/src/next/jdbc/result_set.clj index 9025397..98c458e 100644 --- a/src/next/jdbc/result_set.clj +++ b/src/next/jdbc/result_set.clj @@ -652,7 +652,7 @@ [{:next.jdbc/update-count (.getUpdateCount this)}])) java.sql.Statement - ;; we can't tell if this PreparedStatement will return generated + ;; we can't tell if this Statement will return generated ;; keys so we pass a truthy value to at least attempt it if we ;; do not get a ResultSet back from the execute call (-execute [this sql-params opts] diff --git a/src/next/jdbc/specs.clj b/src/next/jdbc/specs.clj index c153564..d0e458c 100644 --- a/src/next/jdbc/specs.clj +++ b/src/next/jdbc/specs.clj @@ -21,7 +21,7 @@ [next.jdbc.connection :as connection] [next.jdbc.prepare :as prepare] [next.jdbc.sql :as sql]) - (:import (java.sql Connection PreparedStatement) + (:import (java.sql Connection PreparedStatement Statement) (javax.sql DataSource))) (set! *warn-on-reflection* true) @@ -48,6 +48,7 @@ (s/def ::connection #(instance? Connection %)) (s/def ::datasource #(instance? DataSource %)) (s/def ::prepared-statement #(instance? PreparedStatement %)) +(s/def ::statement #(instance? Statement %)) (s/def ::db-spec (s/or :db-spec ::db-spec-map :jdbc-url ::jdbc-url-map @@ -90,20 +91,24 @@ :sql-params ::sql-params :opts (s/? ::opts-map))) +(s/fdef jdbc/statement + :args (s/cat :connection ::connection + :opts (s/? ::opts-map))) + (s/fdef jdbc/plan - :args (s/alt :prepared (s/cat :stmt ::prepared-statement) + :args (s/alt :prepared (s/cat :stmt ::statement) :sql (s/cat :connectable ::connectable :sql-params (s/nilable ::sql-params) :opts (s/? ::opts-map)))) (s/fdef jdbc/execute! - :args (s/alt :prepared (s/cat :stmt ::prepared-statement) + :args (s/alt :prepared (s/cat :stmt ::statement) :sql (s/cat :connectable ::connectable :sql-params (s/nilable ::sql-params) :opts (s/? ::opts-map)))) (s/fdef jdbc/execute-one! - :args (s/alt :prepared (s/cat :stmt ::prepared-statement) + :args (s/alt :prepared (s/cat :stmt ::statement) :sql (s/cat :connectable ::connectable :sql-params (s/nilable ::sql-params) :opts (s/? ::opts-map)))) @@ -201,6 +206,7 @@ `connection/->pool `prepare/execute-batch! `prepare/set-parameters + `prepare/statement `sql/insert! `sql/insert-multi! `sql/query diff --git a/test/next/jdbc/test_fixtures.clj b/test/next/jdbc/test_fixtures.clj index 34da179..db341eb 100644 --- a/test/next/jdbc/test_fixtures.clj +++ b/test/next/jdbc/test_fixtures.clj @@ -4,6 +4,7 @@ "Multi-database testing fixtures." (:require [clojure.string :as str] [next.jdbc :as jdbc] + [next.jdbc.prepare :as prep] [next.jdbc.sql :as sql]) (:import (com.opentable.db.postgres.embedded EmbeddedPostgres) (javax.sql DataSource))) diff --git a/test/next/jdbc_test.clj b/test/next/jdbc_test.clj index ba3887f..b87678a 100644 --- a/test/next/jdbc_test.clj +++ b/test/next/jdbc_test.clj @@ -71,7 +71,8 @@ (is (= (column :FRUIT/ID) (ffirst rs))) ;; and all its corresponding values should be ints (is (every? int? (map first (rest rs)))) - (is (every? string? (map second (rest rs))))) + (is (every? string? (map second (rest rs)))))) + (testing "execute! with adapter" (let [rs (jdbc/execute! ; test again, with adapter and lower columns (ds) ["select * from fruit order by id"] @@ -89,7 +90,8 @@ (is (= :fruit/id (ffirst rs))) ;; and all its corresponding values should be ints (is (every? int? (map first (rest rs)))) - (is (every? string? (map second (rest rs))))) + (is (every? string? (map second (rest rs)))))) + (testing "execute! with unqualified" (let [rs (jdbc/execute! (ds) ["select * from fruit order by id"] @@ -113,6 +115,25 @@ ;; and all its corresponding values should be ints (is (every? int? (map first (rest rs)))) (is (every? string? (map second (rest rs)))))) + (testing "execute! with :max-rows / :maxRows" + (let [rs (jdbc/execute! + (ds) + ["select * from fruit order by id"] + (assoc (default-options) :max-rows 2))] + (is (every? map? rs)) + (is (every? meta rs)) + (is (= 2 (count rs))) + (is (= 1 ((column :FRUIT/ID) (first rs)))) + (is (= 2 ((column :FRUIT/ID) (last rs))))) + (let [rs (jdbc/execute! + (ds) + ["select * from fruit order by id"] + (assoc (default-options) :statement {:maxRows 2}))] + (is (every? map? rs)) + (is (every? meta rs)) + (is (= 2 (count rs))) + (is (= 1 ((column :FRUIT/ID) (first rs)))) + (is (= 2 ((column :FRUIT/ID) (last rs)))))) (testing "prepare" (let [rs (with-open [con (jdbc/get-connection (ds)) ps (jdbc/prepare @@ -137,24 +158,20 @@ (is (= 4 ((column :FRUIT/ID) (first rs)))))) (testing "statement" (let [rs (with-open [con (jdbc/get-connection (ds))] - (jdbc/execute! (.createStatement con) + (jdbc/execute! (prep/statement con (default-options)) ["select * from fruit order by id"]))] (is (every? map? rs)) (is (every? meta rs)) (is (= 4 (count rs))) - ;; SQL Server only returns table name if result-type/concurrency - ;; provided, which we can only for a PreparedStatement - (is (= 1 ((if (mssql?) :ID (column :FRUIT/ID)) (first rs)))) - (is (= 4 ((if (mssql?) :ID (column :FRUIT/ID)) (last rs))))) + (is (= 1 ((column :FRUIT/ID) (first rs)))) + (is (= 4 ((column :FRUIT/ID) (last rs))))) (let [rs (with-open [con (jdbc/get-connection (ds))] - (jdbc/execute! (.createStatement con) + (jdbc/execute! (prep/statement con (default-options)) ["select * from fruit where id = 4"]))] (is (every? map? rs)) (is (every? meta rs)) (is (= 1 (count rs))) - ;; SQL Server only returns table name if result-type/concurrency - ;; provided, which we can only for a PreparedStatement - (is (= 4 ((if (mssql?) :ID (column :FRUIT/ID)) (first rs)))))) + (is (= 4 ((column :FRUIT/ID) (first rs)))))) (testing "transact" (is (= [{:next.jdbc/update-count 1}] (jdbc/transact (ds)