From de0afef8139e92c9dec8406981d88eefc1c83343 Mon Sep 17 00:00:00 2001 From: Sean Corfield Date: Fri, 24 May 2019 16:54:10 -0700 Subject: [PATCH] Prepare for Beta 1 Primarily a documentation update pass, since the code changes were already made. --- CHANGELOG.md | 21 +++++++++++++++------ CONTRIBUTING.md | 2 ++ README.md | 18 +++++++++--------- doc/all-the-options.md | 2 +- doc/getting-started.md | 12 +++++------- doc/prepared-statements.md | 2 +- doc/result-set-builders.md | 2 +- pom.xml | 2 +- 8 files changed, 35 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 442e567..ace9d7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Change Log +Only accretive/fixative changes will be made from now on (Beta 1). + +## Beta Builds (Stable) + +* 2019-05-24 -- 1.0.0-beta1: + * Set up CircleCI testing (just local DBs for now). + * Address #21 by adding `next.jdbc.specs` and documenting basic usage. + * Fix #19 by caching loaded database driver classes. + * Address #16 by renaming `reducible!` to `plan` (**BREAKING CHANGE!**). + * Address #3 by deciding to maintain this library outside Clojure Contrib. + +## Alpha Builds + * 2019-05-04 -- 1.0.0-alpha13 -- Fix #18 by removing more keys from properties when creating connections. * 2019-04-26 -- 1.0.0-alpha12 -- Fix #17 by renaming `:next.jdbc/sql-string` to `:next.jdbc/sql-params` (**BREAKING CHANGE!**) and pass whole vector. * 2019-04-24 -- 1.0.0-alpha11 -- Rename `:gen-fn` to `:builder-fn` (**BREAKING CHANGE!**); Fix #13 by adding documentation for `datafy`/`nav`/`:schema`; Fix #15 by automatically adding `:next.jdbc/sql-string` (as of 1.0.0-alpha12: `:next.jdbc/sql-params`) into the options hash map, so custom builders can depend on the SQL string. @@ -8,10 +21,6 @@ ## Unreleased Changes -The following changes have been committed to the **master** branch and will be in the next release (Beta 1): +The following changes have been committed to the **master** branch and will be in the next release: -* Set up CircleCI testing (just local DBs for now). -* Address #21 by adding `next.jdbc.specs` and documenting basic usage. -* Fix #19 by caching loaded database driver classes. -* Address #16 by renaming `reducible!` to `plan` (**BREAKING CHANGE!**). -* Address #3 by deciding to maintain this library outside Clojure Contrib. +* None at this time. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1af7116..ba85e21 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,3 +4,5 @@ Feel free to [open Issues](https://github.com/seancorfield/next-jdbc/issues) wit and suggestions -- I'm happy to have discussions about any aspect of the library! I welcome Pull Requests for source code changes *that are accompanied by tests*, and for documentation changes. For any substantial change, please open an issue for discussion first, or find me in the `#sql` stream on Clojurians Zulip or the `#sql` channel on Clojurians Slack to chat about it. + +In particular, as of 1.0.0 Beta 1, no breaking changes will be entertained. All future changes must either be purely accretive or purely fixative. diff --git a/README.md b/README.md index 9abc4e0..b547f56 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ The latest versions on Clojars and on cljdoc: [![Clojars Project](https://clojars.org/seancorfield/next.jdbc/latest-version.svg)](https://clojars.org/seancorfield/next.jdbc) [![cljdoc badge](https://cljdoc.org/badge/seancorfield/next.jdbc)](https://cljdoc.org/d/seancorfield/next.jdbc/CURRENT) -This documentation is for the upcoming 1.0.0-beta1 release. +This documentation is for the 1.0.0-beta1 release. * [Getting Started](/doc/getting-started.md) * [Migrating from `clojure.java.jdbc`](/doc/migration-from-clojure-java-jdbc.md) @@ -18,17 +18,17 @@ This documentation is for the upcoming 1.0.0-beta1 release. Why another JDBC library? Why a different API from `clojure.java.jdbc`? -* Performance: there's a surprising amount of overhead in how `ResultSet` objects are converted to sequences of hash maps – which can be really noticeable for large result sets – so I want a better way to handle that. There's also quite a bit of overhead and complexity in all the conditional logic and parsing that is associated with `db-spec`-as-hash-map. +* Performance: there's a surprising amount of overhead in how `ResultSet` objects are converted to sequences of hash maps in `clojure.java.jdbc` – which can be really noticeable for large result sets – so I wanted a better way to handle that. There's also quite a bit of overhead and complexity in all the conditional logic and parsing that is associated with `db-spec`-as-hash-map. * A more modern API, based on using qualified keywords and transducers etc: `:qualifier` and `reducible-query` in recent `clojure.java.jdbc` versions were steps toward that but there's a lot of "legacy" API in the library and I want to present a more focused, more streamlined API so folks naturally use the `IReduceInit` / transducer approach from day one and benefit from qualified keywords. * Simplicity: `clojure.java.jdbc` uses a variety of ways to execute SQL which can lead to inconsistencies and surprises – `query`, `execute!`, and `db-do-commands` are all different ways to execute different types of SQL statement so you have to remember which is which and you often have to watch out for restrictions in the underlying JDBC API. -Those are my three primary drivers. In addition, the `db-spec`-as-hash-map approach in `clojure.java.jdbc` has caused a lot of frustration and confusion in the past, especially with the wide range of conflicting options that are supported. `next.jdbc` is heavily protocol-based so it's easier to mix'n'match how you use it with direct Java JDBC code (and the protocol-based approach contributes to the improved performance overall). There's a much clearer path of `db-spec` -> `DataSource` -> `Connection` now, which should steer people toward more connection reuse and better performing apps. +Those were my three primary drivers. In addition, the `db-spec`-as-hash-map approach in `clojure.java.jdbc` has caused a lot of frustration and confusion in the past, especially with the wide range of conflicting options that are supported. `next.jdbc` is heavily protocol-based so it's easier to mix'n'match how you use it with direct Java JDBC code (and the protocol-based approach contributes to the improved performance overall). There's a much clearer path of `db-spec` -> `DataSource` -> `Connection` now, which should steer people toward more connection reuse and better performing apps. I also wanted `datafy`/`nav` support baked right in (it was added to `clojure.java.jdbc` back in December 2018 as an undocumented, experimental API in a separate namespace). It is the default behavior for `execute!` and `execute!`. The protocol-based function `next.jdbc.result-set/datafiable-row` can be used with `plan` if you need to add `datafy`/`nav` support to rows you are creating in your reduction. -As `next.jdbc` moves from alpha to beta, the last breaking change has been made (renaming `reducible!` to `plan`) and the API should be considered stable. Only accretive and fixative changes will be made from now on. +As `next.jdbc` moved from alpha to beta, the last breaking change was made (renaming `reducible!` to `plan`) and the API should now be considered stable. Only accretive and fixative changes will be made from now on. -[Alpha builds have been available on Clojars](https://clojars.org/seancorfield/next.jdbc) for over a month (as of 2019-05-22). The first beta build will be available shortly. The "syntactic sugar" SQL functions (`insert!`, `query`, `update!`, and `delete!`) go beyond what I wanted to include in the core API so they are in `next.jdbc.sql`. I know that their equivalents in `clojure.java.jdbc` are heavily used (based on the number of questions and JIRA issues I get). +After a month of alpha builds being available for testing, the first [beta build is available on Clojars](https://clojars.org/seancorfield/next.jdbc) (as of 2019-05-24). In addition to the small, core API in `next.jdbc`, there are "syntactic sugar" SQL functions (`insert!`, `query`, `update!`, and `delete!`) available in `next.jdbc.sql` that are similar to the main API in `clojure.java.jdbc`. See [Migrating from `clojure.java.jdbc`](/doc/migration-from-clojure-java-jdbc.md) for more detail about the differences. ## Usage @@ -55,12 +55,12 @@ Since `next.jdbc` uses raw Java JDBC types, you can use `with-open` directly to ### Usage scenarios -There are three intended usage scenarios that may drive the API to change: -* Execute a SQL statement to obtain a single, fully-realized, `Datafiable` hash map that represents either the first row from a `ResultSet`, the first generated keys result (again, from a `ResultSet`), or the first result where neither of those are available (`next.jdbc` yields `{:next.jdbc/update-count N}` when it can only return an update count). This usage is currently supported by `execute-one!`. +There are three intended usage scenarios that have driven the design of the API: +* Execute a SQL statement and process it in a single eager operation, which may allow for the results to be streamed from the database (how to persuade JDBC to do that is database-specific!), and which cleans up resources before returning the result -- even if the reduction is short-circuited via `reduced`. This usage is supported by `plan`. This is likely to be the fastest approach and should be the first option you consider for SQL queries. +* Execute a SQL statement to obtain a single, fully-realized, `Datafiable` hash map that represents either the first row from a `ResultSet`, the first generated keys result (again, from a `ResultSet`), or the first result where neither of those are available (`next.jdbc` yields `{:next.jdbc/update-count N}` when it can only return an update count). This usage is supported by `execute-one!`. This is probably your best choice for most non-query operations. * Execute a SQL statement to obtain a fully-realized, `Datafiable` result set -- a vector of hash maps. This usage is supported by `execute!`. You can also produce a vector of column names/row values (`next.jdbc.result-set/as-arrays`). -* Execute a SQL statement and process it in a single eager operation, which may allow for the results to be streamed from the database (how to persuade JDBC to do that is database-specific!), and which cleans up resources before returning the result -- even if the reduction is short-circuited via `reduced`. This usage is supported by `plan`. -In addition, convenience functions -- "syntactic sugar" -- are provided to insert rows, run queries, update rows, and delete rows, using the same names as in `clojure.java.jdbc`. These are in `next.jdbc.sql` since they involve SQL creation -- they may move into a separate "sibling" library, since they are not part of the intended core API. +In addition, convenience functions -- "syntactic sugar" -- are provided to insert rows, run queries, update rows, and delete rows, using the same names as in `clojure.java.jdbc`. These are in `next.jdbc.sql` since they involve SQL creation -- they are not considered part of the core API. ## More Detailed Documentation diff --git a/doc/all-the-options.md b/doc/all-the-options.md index 809524e..616213e 100644 --- a/doc/all-the-options.md +++ b/doc/all-the-options.md @@ -44,7 +44,7 @@ Any function that creates a `PreparedStatement` will accept the following option * `:concurrency` -- a keyword that specifies the concurrency level: `:read-only`, `:updatable`, * `:cursors` -- a keyword that specifies whether cursors should be closed or held over a commit: `:close`, `:hold`, -* `:fetch-size` -- an integer that guides the JDBC driver in terms of how many rows to fetch at once; it is common to set `:fetch-size` to a negative value in order to trigger streaming of result sets -- some JDBC drivers require additional options to be set on the connection _as well_, +* `:fetch-size` -- an integer that guides the JDBC driver in terms of how many rows to fetch at once; it is common to set `:fetch-size` to zero or a negative value in order to trigger streaming of result sets -- some JDBC drivers require additional options to be set on the connection _as well_, * `:max-rows` -- an integer that tells the JDBC driver to limit result sets to this many rows, * `:result-type` -- a keyword that affects how the `ResultSet` can be traversed: `:forward-only`, `:scroll-insensitive`, `:scroll-sensitive`, * `:return-keys` -- a truthy value asks that the JDBC driver to return any generated keys created by the operation; it can be `true` or it can be a vector of keywords identifying column names that should be returned, diff --git a/doc/getting-started.md b/doc/getting-started.md index 516508c..4adb022 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -2,23 +2,23 @@ The `next.jdbc` library provides a simpler, faster alternative to the [`clojure.java.jdbc`](https://github.com/clojure/java.jdbc) Contrib library and is the next step in the evolution of that library. -It is designed to work with Clojure 1.10 or later, supports `datafy`/`nav`, and by default produces hash maps with automatically qualified keywords, indicating source tables and column names. +It is designed to work with Clojure 1.10 or later, supports `datafy`/`nav`, and by default produces hash maps with automatically qualified keywords, indicating source tables and column names (labels). ## Installation You can add `next.jdbc` to your project with either: ```clojure -{seancorfield/next.jdbc {:mvn/version "1.0.0-alpha13"}} +{seancorfield/next.jdbc {:mvn/version "1.0.0-beta1"}} ``` for `deps.edn` or: ```clojure -[seancorfield/next.jdbc "1.0.0-alpha13"] +[seancorfield/next.jdbc "1.0.0-beta1"] ``` for `project.clj` or `build.boot`. -In addition, you will need to add dependencies for the JDBC drivers you wish to use for whatever databases you are using. You can see the drivers and versions that `next.jdbc` is tested against in [the project's `deps.edn` file](https://github.com/seancorfield/next-jdbc/blob/master/deps.edn#L6-L16), but many other JDBC drivers for other databases should also work (e.g., Oracle, Red Shift). +In addition, you will need to add dependencies for the JDBC drivers you wish to use for whatever databases you are using. You can see the drivers and versions that `next.jdbc` is tested against in [the project's `deps.edn` file](https://github.com/seancorfield/next-jdbc/blob/master/deps.edn#L6-L14), but many other JDBC drivers for other databases should also work (e.g., Oracle, Red Shift). ## An Example REPL Session @@ -29,7 +29,7 @@ For the examples in this documentation, we will use a local H2 database on disk, ```clojure ;; deps.edn {:deps {org.clojure/clojure {:mvn/version "1.10.0"} - seancorfield/next.jdbc {:mvn/version "1.0.0-alpha13"} + seancorfield/next.jdbc {:mvn/version "1.0.0-beta1"} com.h2database/h2 {:mvn/version "1.4.197"}}} ``` @@ -138,8 +138,6 @@ If `with-transaction` is given a datasource, it will create and close the connec ## Support from Specs -_Coming in Beta 1!_ - As you are developing with `next.jdbc`, it can be useful to have assistance from `clojure.spec` in checking calls to `next.jdbc`'s functions, to provide explicit argument checking and/or better error messages for some common mistakes, e.g., trying to pass a plain SQL string where a vector (containing a SQL string, and no parameters) is expected. You can enable argument checking for functions in both `next.jdbc` and `next.jdbc.sql` by requiring the `next.jdbc.specs` namespace and instrumenting the functions. A convenience function is provided: diff --git a/doc/prepared-statements.md b/doc/prepared-statements.md index b9aa503..d69f699 100644 --- a/doc/prepared-statements.md +++ b/doc/prepared-statements.md @@ -1,6 +1,6 @@ # Prepared Statements -Under the hood, whenever you ask `next.jdbc` to execute some SQL it creates a `java.sql.PreparedStatement`, adds in the parameters you provide, and then calls `.execute` on it. Then it attempts to get a `ResultSet` from that and either return it or process it. If you asked for generated keys to be returned, that `ResultSet` will contain the those generated keys if your database supports it, otherwise it will be whatever the `.execute` function produces. If no `ResultSet` is available at all, `next.jdbc` will ask for the count of updated rows and return that as if it were a result set. +Under the hood, whenever you ask `next.jdbc` to execute some SQL it creates a `java.sql.PreparedStatement`, adds in the parameters you provide, and then calls `.execute` on it. Then it attempts to get a `ResultSet` from that and either return it or process it. If you asked for generated keys to be returned, that `ResultSet` will contain those generated keys if your database supports it, otherwise it will be whatever the `.execute` function produces. If no `ResultSet` is available at all, `next.jdbc` will ask for the count of updated rows and return that as if it were a result set. If you have a SQL operation that you intend to run multiple times on the same `java.sql.Connection`, it may be worth creating the prepared statement yourself and reusing it. `next.jdbc/prepare` accepts a connection and a vector of SQL and optional parameters and returns a `java.sql.PreparedStatement` which can be passed to `plan`, `execute!`, or `execute-one!` as the first argument. It is your responsibility to close the prepared statement after it has been used. diff --git a/doc/result-set-builders.md b/doc/result-set-builders.md index 4f852bf..7e0c066 100644 --- a/doc/result-set-builders.md +++ b/doc/result-set-builders.md @@ -40,7 +40,7 @@ Only `execute!` expects this protocol to be implemented. `execute-one!` and `pla The `as-*` functions described above are all implemented in terms of these protocols. They are passed the `ResultSet` object and the options hash map (as passed into various `next.jdbc` functions). They return an implementation of the protocols that is then used to build rows and the result set. Note that the `ResultSet` passed in is _mutable_ and is advanced from row to row by the SQL execution function, so each time `->row` is called, the underlying `ResultSet` object points at each new row in turn. By contrast, `->rs` (which is only called by `execute!`) is invoked _before_ the `ResultSet` is advanced to the first row. -The options hash map for any `next.jdbc` function can contain a `:builder-fn` key and the value is used as the row/result set builder function. The tests for `next.jdbc.result-set` include a [record-based builder function](https://github.com/seancorfield/next-jdbc/blob/master/test/next/jdbc/result_set_test.clj#L148-L164) as an example of how you can extend this to satisfy your needs. +The options hash map for any `next.jdbc` function can contain a `:builder-fn` key and the value is used as the row/result set builder function. The tests for `next.jdbc.result-set` include a [record-based builder function](https://github.com/seancorfield/next-jdbc/blob/master/test/next/jdbc/result_set_test.clj#L143-L161) as an example of how you can extend this to satisfy your needs. The options hash map passed to the builder function will contain a `:next.jdbc/sql-params` key, whose value is the SQL + parameters vector passed into the top-level `next.jdbc` functions (`plan`, `execute!`, and `execute-one!`). diff --git a/pom.xml b/pom.xml index 3c750c1..d058814 100644 --- a/pom.xml +++ b/pom.xml @@ -3,7 +3,7 @@ 4.0.0 seancorfield next.jdbc - 1.0.0-alpha13 + 1.0.0-beta1 next.jdbc The next generation of clojure.java.jdbc: a new low-level Clojure wrapper for JDBC-based access to databases.