diff --git a/CHANGELOG.md b/CHANGELOG.md index c3ef004..e815865 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ Changes made on master since 1.0.476: ## Stable Builds +* 2020-06-24 -- 1.0.478 + * Address #123 by adding `next.jdbc.types` namespace, full of auto-generated `as-xxx` functions, one for each of the `java.sql.Types` values. + * 2020-06-22 -- 1.0.476 * Extend default options behavior to `next.jdbc.sql` functions. diff --git a/README.md b/README.md index 7d25434..d8ae26d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ The next generation of `clojure.java.jdbc`: a new low-level Clojure wrapper for The latest versions on Clojars and on cljdoc: -[![Clojars Project](https://clojars.org/seancorfield/next.jdbc/latest-version.svg)](https://clojars.org/seancorfield/next.jdbc) [![cljdoc badge](https://cljdoc.org/badge/seancorfield/next.jdbc?1.0.476)](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT) +[![Clojars Project](https://clojars.org/seancorfield/next.jdbc/latest-version.svg)](https://clojars.org/seancorfield/next.jdbc) [![cljdoc badge](https://cljdoc.org/badge/seancorfield/next.jdbc?1.0.478)](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT) The documentation on [cljdoc.org](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT) is for the current version of `next.jdbc`: @@ -14,7 +14,7 @@ The documentation on [cljdoc.org](https://cljdoc.org/d/seancorfield/next.jdbc/CU * [Migrating from `clojure.java.jdbc`](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/doc/migration-from-clojure-java-jdbc) * Feedback via [issues](https://github.com/seancorfield/next-jdbc/issues) or in the [`#sql` channel on the Clojurians Slack](https://clojurians.slack.com/messages/C1Q164V29/details/) or the [`#sql` stream on the Clojurians Zulip](https://clojurians.zulipchat.com/#narrow/stream/152063-sql). -The documentation on GitHub is for **develop** since the 1.0.476 release -- [see the CHANGELOG](https://github.com/seancorfield/next-jdbc/blob/develop/CHANGELOG.md) and then read the [corresponding updated documentation](https://github.com/seancorfield/next-jdbc/tree/develop/doc) on GitHub if you want. +The documentation on GitHub is for **develop** since the 1.0.478 release -- [see the CHANGELOG](https://github.com/seancorfield/next-jdbc/blob/develop/CHANGELOG.md) and then read the [corresponding updated documentation](https://github.com/seancorfield/next-jdbc/tree/develop/doc) on GitHub if you want. This project follows the version scheme MAJOR.MINOR.COMMITS where MAJOR and MINOR provide some relative indication of the size of the change, but do not follow semantic versioning. In general, all changes endeavor to be non-breaking (by moving to new names rather than by breaking existing names). COMMITS is an ever-increasing counter of commits since the beginning of this repository. diff --git a/doc/getting-started.md b/doc/getting-started.md index 71e51e9..5d39e28 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -9,12 +9,12 @@ It is designed to work with Clojure 1.10 or later, supports `datafy`/`nav`, and You can add `next.jdbc` to your project with either: ```clojure -seancorfield/next.jdbc {:mvn/version "1.0.476"} +seancorfield/next.jdbc {:mvn/version "1.0.478"} ``` for `deps.edn` or: ```clojure -[seancorfield/next.jdbc "1.0.476"] +[seancorfield/next.jdbc "1.0.478"] ``` for `project.clj` or `build.boot`. @@ -29,7 +29,7 @@ For the examples in this documentation, we will use a local H2 database on disk, ```clojure ;; deps.edn {:deps {org.clojure/clojure {:mvn/version "1.10.1"} - seancorfield/next.jdbc {:mvn/version "1.0.476"} + seancorfield/next.jdbc {:mvn/version "1.0.478"} com.h2database/h2 {:mvn/version "1.4.199"}}} ``` @@ -398,12 +398,14 @@ If you are using [Component](https://github.com/stuartsierra/component), a conne By default, `next.jdbc` relies on the JDBC driver to handle all data type conversions when reading from a result set (to produce Clojure values from SQL values) or setting parameters (to produce SQL values from Clojure values). Sometimes that means that you will get back a database-specific Java object that would need to be manually converted to a Clojure data structure, or that certain database column types require you to manually construct the appropriate database-specific Java object to pass into a SQL operation. You can usually automate those conversions using either the [`ReadableColumn` protocol](/doc/result-set-builders.md#readablecolumn) (for converting database-specific types to Clojure values) or the [`SettableParameter` protocol](/doc/prepared-statements.md#prepared-statement-parameters) (for converting Clojure values to database-specific types). -In particular, PostgreSQL does not seem to perform a conversion from `java.util.Date` to a SQL data type automatically. You must `require` the [`next.jdbc.date-time` namespace](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time) to enable that conversion. +In particular, PostgreSQL does not seem to perform a conversion from `java.util.Date` to a SQL data type automatically. You can `require` the [`next.jdbc.date-time` namespace](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time) to enable that conversion. If you are working with Java Time, some JDBC drivers will automatically convert `java.time.Instant` (and `java.time.LocalDate` and `java.time.LocalDateTime`) to a SQL data type automatically, but others will not. Requiring `next.jdbc.date-time` will enable those automatic conversions for all databases. > Note: `next.jdbc.date-time` also provides functions you can call to enable automatic conversion of SQL date/timestamp types to Clojure data types when reading result sets. If you need specific conversions beyond that to happen automatically, consider extending the `ReadableColumn` protocol, mentioned above. +The `next.jdbc.types` namespace provides over three dozen convenience functions for "type hinting" values so that the JDBC driver might automatically handle some conversions that the default parameter setting function does not. Each function is named for the corresponding SQL type, prefixed by `as-`: `as-bigint`, `as-other`, `as-real`, etc. An example of where this helps is when dealing with PostgreSQL enumerated types: the default behavior, when passed a string that should correspond to an enumerated type, is to throw an exception that `column "..." is of type ... but expression is of type character varying`. You can wrap such strings with `(as-other "...")` which tells PostgreSQL to treat this as `java.sql.Types/OTHER` when setting the parameter. + ## Processing Database Metadata JDBC provides several features that let you introspect the database to obtain lists of tables, views, and so on. `next.jdbc` does not provide any specific functions for this but you can easily get this metadata from a `java.sql.Connection` and turn it into Clojure data as follows: diff --git a/doc/prepared-statements.md b/doc/prepared-statements.md index 28e8169..a3e7259 100644 --- a/doc/prepared-statements.md +++ b/doc/prepared-statements.md @@ -54,6 +54,8 @@ You can also extend this protocol via metadata so you can do it on a per-object (with-meta obj {'next.jdbc.prepare/set-parameter (fn [v ps i]...)}) ``` +The `next.jdbc.types` namespace provides functions to wrap values with per-object implementations of `set-parameter` for every standard `java.sql.Types` value. Each is named `as-xxx` corresponding to `java.sql.Types/XXX`. + The converse, converting database-specific types to Clojure values is handled by the `ReadableColumn` protocol, discussed in the previous section ([Result Set Builders](/doc/result-set-builders.md#readablecolumn)). As noted above, `next.jdbc.prepare/set-parameters` is available for you to call on any existing `PreparedStatement` to set or update the parameters that will be used when the statement is executed: diff --git a/doc/tips-and-tricks.md b/doc/tips-and-tricks.md index 4a44a4d..70635b4 100644 --- a/doc/tips-and-tricks.md +++ b/doc/tips-and-tricks.md @@ -109,6 +109,27 @@ If you have a query where you want to select where a column is `IN` a sequence o What does this mean for your use of `next.jdbc`? In `plan`, `execute!`, and `execute-one!`, you can use `col = ANY(?)` in the SQL string and a single primitive array parameter, such as `(int-array [1 2 3 4])`. That means that in `next.jdbc.sql`'s functions that take a where clause (`find-by-keys`, `update!`, and `delete!`) you can specify `["col = ANY(?)" (int-array data)]` for what would be a `col IN (?,?,?,,,?)` where clause for other databases and require multiple values. +PostgreSQL has a SQL extension for defining enumerated types and the default `set-parameter` implementation will not work for those. You can use `next.jdbc.types/as-other` to wrap string values in a way that the JDBC driver will convert them to enumerated type values: + +```sql +CREATE TYPE language AS ENUM('en','fr','de'); + +CREATE TABLE person ( + ... + speaks language NOT NULL, + ... +); +``` + +```clojure +(require '[next.jdbc.sql :as sql] + '[next.jdbc.types :refer [as-other]]) + +(sql/insert! ds :person {:speaks (as-other "fr")}) +``` + +That call produces a vector `["fr"]` with metadata that implements `set-parameter` such that `.setObject()` is called with `java.sql.Types/OTHER` which allows PostgreSQL to "convert" the string `"fr"` to the corresponding `language` enumerated type value. + ### Streaming Result Sets You can get PostgreSQL to stream very large result sets (when you are reducing over `plan`) by setting the following options: @@ -150,7 +171,7 @@ create table example( ;; => #:example{:tags ["tag1" "tag2"]} ``` -Note: PostgreSQL JDBC driver supports only 7 primitive array types, but not such as `UUID[]` - +> Note: PostgreSQL JDBC driver supports only 7 primitive array types, but not array types like `UUID[]` - [PostgreSQL™ Extensions to the JDBC API](https://jdbc.postgresql.org/documentation/head/arrays.html). ### Working with Date and Time diff --git a/pom.xml b/pom.xml index df665fe..5c99f17 100644 --- a/pom.xml +++ b/pom.xml @@ -3,7 +3,7 @@ 4.0.0 seancorfield next.jdbc - 1.0.476 + 1.0.478 next.jdbc The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases. https://github.com/seancorfield/next-jdbc @@ -22,7 +22,7 @@ https://github.com/seancorfield/next-jdbc scm:git:git://github.com/seancorfield/next-jdbc.git scm:git:ssh://git@github.com/seancorfield/next-jdbc.git - v1.0.476 + v1.0.478 diff --git a/src/next/jdbc/prepare.clj b/src/next/jdbc/prepare.clj index 7ad757b..2192b4d 100644 --- a/src/next/jdbc/prepare.clj +++ b/src/next/jdbc/prepare.clj @@ -15,7 +15,11 @@ See also https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.date-time for implementations of `SettableParameter` that provide automatic - conversion of Java Time objects to SQL data types." + conversion of Java Time objects to SQL data types. + + See also https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT/api/next.jdbc.types + for `as-xxx` functions that provide per-instance implementations of + `SettableParameter` for each of the standard `java.sql.Types` values." (:require [clojure.java.data :as j] [next.jdbc.protocols :as p]) (:import (java.sql Connection diff --git a/src/next/jdbc/types.clj b/src/next/jdbc/types.clj new file mode 100644 index 0000000..8dcb1e8 --- /dev/null +++ b/src/next/jdbc/types.clj @@ -0,0 +1,39 @@ +;; copyright (c) 2018-2020 Sean Corfield, all rights reserved + +(ns next.jdbc.types + "Provides convenience functions for wrapping values you pass into SQL + operations that have per-instance implementations of `SettableParameter` + so that `.setObject()` is called with the appropriate `java.sql.Types` value." + (:require [clojure.string :as str] + [next.jdbc.prepare :as prep]) + (:import (java.lang.reflect Field Modifier) + (java.sql PreparedStatement))) + +(set! *warn-on-reflection* true) + +(defmacro ^:private all-types + [] + (let [names + (into [] + (comp (filter #(Modifier/isStatic (.getModifiers ^Field %))) + (map #(.getName ^Field %))) + (.getDeclaredFields java.sql.Types))] + `(do + ~@(for [n names] + (let [as-n (symbol (str "as-" + (-> n + (str/lower-case) + (str/replace "_" "-"))))] + `(defn ~as-n + ~(str "Wrap a Clojure value in a vector with metadata to implement `set-parameter` + so that `.setObject()` is called with the `java.sql.Types/" n "` SQL type.") + [~'obj] + (with-meta [~'obj] + {'next.jdbc.prepare/set-parameter + (fn [[v#] ^PreparedStatement s# ^long i#] + (.setObject s# i# v# ~(symbol "java.sql.Types" n)))}))))))) + +(all-types) + +(comment + (macroexpand '(all-types))) diff --git a/test/next/jdbc/sql_test.clj b/test/next/jdbc/sql_test.clj index f383926..c10473d 100644 --- a/test/next/jdbc/sql_test.clj +++ b/test/next/jdbc/sql_test.clj @@ -8,7 +8,8 @@ [next.jdbc.sql :as sql] [next.jdbc.test-fixtures :refer [with-test-db ds column default-options - derby? jtds? maria? mssql? mysql? postgres? sqlite?]])) + derby? jtds? maria? mssql? mysql? postgres? sqlite?]] + [next.jdbc.types :refer [as-other as-real as-varchar]])) (set! *warn-on-reflection* true) @@ -79,8 +80,9 @@ :else :FRUIT/ID)] (testing "single insert/delete" (is (== 5 (new-key (sql/insert! (ds) :fruit - {:name "Kiwi" :appearance "green & fuzzy" - :cost 100 :grade 99.9})))) + {:name (as-varchar "Kiwi") + :appearance "green & fuzzy" + :cost 100 :grade (as-real 99.9)})))) (is (= 5 (count (sql/query (ds) ["select * from fruit"])))) (is (= {:next.jdbc/update-count 1} (sql/delete! (ds) :fruit {:id 5}))) @@ -156,3 +158,8 @@ (when (postgres?) (let [data (sql/find-by-keys (ds) :fruit ["id = any(?)" (int-array [1 2 3 4])])] (is (= 4 (count data)))))) + +(deftest enum-pg + (when (postgres?) + (let [r (sql/insert! (ds) :lang_test {:lang (as-other "fr")})] + (is (= {:lang_test/lang "fr"} r))))) diff --git a/test/next/jdbc/test_fixtures.clj b/test/next/jdbc/test_fixtures.clj index e41a2ac..3d65724 100644 --- a/test/next/jdbc/test_fixtures.clj +++ b/test/next/jdbc/test_fixtures.clj @@ -140,7 +140,19 @@ (catch Throwable _))) (try (do-commands con [(str "DROP TABLE " fruit)]) - (catch Throwable _)) + (catch Exception _)) + (when (postgres?) + (try + (do-commands con ["DROP TABLE LANG_TEST"]) + (catch Exception _)) + (try + (do-commands con ["DROP TYPE LANGUAGE"]) + (catch Exception _)) + (do-commands con ["CREATE TYPE LANGUAGE AS ENUM('en','fr','de')"]) + (do-commands con [" +CREATE TABLE LANG_TEST ( + LANG LANGUAGE NOT NULL +)"])) (do-commands con [(str " CREATE TABLE " fruit " ( ID INTEGER " auto-inc-pk ", diff --git a/test/next/jdbc/types_test.clj b/test/next/jdbc/types_test.clj new file mode 100644 index 0000000..d3706ae --- /dev/null +++ b/test/next/jdbc/types_test.clj @@ -0,0 +1,14 @@ +;; copyright (c) 2020 Sean Corfield, all rights reserved + +(ns next.jdbc.types-test + "Some tests for the type-assist functions." + (:require [clojure.test :refer [deftest is testing]] + [next.jdbc.types :refer [as-varchar]])) + +(set! *warn-on-reflection* true) + +(deftest as-varchar-test + (let [v (as-varchar "Hello")] + (is (= ["Hello"] v)) + (is (contains? (meta v) 'next.jdbc.prepare/set-parameter)) + (is (fn? (get (meta v) 'next.jdbc.prepare/set-parameter)))))