From 801061afa880d1b7e26e401dec04c7bee2566c60 Mon Sep 17 00:00:00 2001 From: Sean Corfield Date: Wed, 24 Apr 2019 15:51:30 -0700 Subject: [PATCH] Document the datafy/nav/:schema stuff --- README.md | 1 + doc/all-the-options.md | 2 +- doc/cljdoc.edn | 1 + doc/datafy-nav-and-schema.md | 48 +++++++++++++++++++++++++ doc/migration-from-clojure-java-jdbc.md | 2 +- 5 files changed, 52 insertions(+), 2 deletions(-) create mode 100644 doc/datafy-nav-and-schema.md diff --git a/README.md b/README.md index 1ba5dba..4fa6bd5 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,7 @@ In addition, convenience functions -- "syntactic sugar" -- are provided to inser * [Prepared Statements](/doc/prepared-statements.md) * [Transactions](/doc/transactions.md) * [All The Options](/doc/all-the-options.md) +* [`datafy`, `nav`, and `:schema`](/doc/datafy-nav-and-schema.md) * [Migration from `clojure.java.jdbc`](/doc/migration-from-clojure-java-jdbc.md) ## License diff --git a/doc/all-the-options.md b/doc/all-the-options.md index 1e5746e..83323d2 100644 --- a/doc/all-the-options.md +++ b/doc/all-the-options.md @@ -60,4 +60,4 @@ The `transact` function and `with-transaction` macro accept the following option * `:read-only` -- a `Boolean` that indicates whether the transaction should be read-only or not (the default), * `:rollback-only` -- a `Boolean` that indicates whether the transaction should commit on success (the default) or rollback. -[<: Transactions](/doc/transactions.md) | [Migration from `clojure.java.jdbc` :>](/doc/migration-from-clojure-java-jdbc.md) +[<: Transactions](/doc/transactions.md) | [`datafy`, `nav`, and `:schema` :>](/doc/datafy-nav-and-schema.md) diff --git a/doc/cljdoc.edn b/doc/cljdoc.edn index c10462a..18bada3 100644 --- a/doc/cljdoc.edn +++ b/doc/cljdoc.edn @@ -6,4 +6,5 @@ ["Prepared Statements" {:file "doc/prepared-statements.md"}] ["Transactions" {:file "doc/transactions.md"}]] ["All The Options" {:file "doc/all-the-options.md"}] + ["datafy, nav, and :schema" {:file "doc/datafy-nav-schema.md"}] ["Migration from clojure.java.jdbc" {:file "doc/migration-from-clojure-java-jdbc.md"}]]} diff --git a/doc/datafy-nav-and-schema.md b/doc/datafy-nav-and-schema.md new file mode 100644 index 0000000..db8b4d1 --- /dev/null +++ b/doc/datafy-nav-and-schema.md @@ -0,0 +1,48 @@ +# `datafy`, `nav`, and the `:schema` option + +Clojure 1.10 introduced a new namespace, [`clojure.datafy`](http://clojure.github.io/clojure/clojure.datafy-api.html), and two new protocols (`Datafiable` and `Navigable`) that allow for generalized, lazy navigation around data structures. Cognitect also released [REBL](http://rebl.cognitect.com/) -- a graphical, interactive tool for browsing Clojure data structures, based on the new `datafy` and `nav` functions. + +Shortly after, I added experimental support to `clojure.java.jdbc` for `datafy` and `nav` that supported lazy navigation through result sets into foreign key relationships and connected rows and tables. `next.jdbc` bakes that support into result sets produced by `execute!` and `execute-one!`. + +## The `datafy`/`nav` Lifecycle + +Here's how the process works: + +* `execute!` and `execute-one!` produce result sets containing rows that are `Datafiable`, +* Tools like REBL can call `datafy` on result sets to render them as "pure data" (which they already are, but this makes them also `Navigable`), +* Tools like REBL allow users to "drill down" into elements of rows in the "pure data" result set, using `nav`, +* If a column in a row represents a foreign key into another table, calling `nav` will fetch the related row(s), +* Those can in turn be `datafy`'d and `nav`'d to continue drilling down through connected data in the database. + +## Identifying Foreign Keys + +By default, `next.jdbc` assumes that a column named `id` or `_id` is a foreign key into a table called `` with a primary key called `id`. As an example, if you have a table `address` which has columns `id` (the primary key), `name`, `email`, etc, and a table `contact` which has various columns including `addressid`, then if you retrieve a result set based on `contact`, call `datafy` on it and then "drill down" into the columns, when `(nav row :contact/addressid v)` is called (where `v` is the value of that column in that row) `next.jdbc`'s implementation of `nav` will fetch a single row from the `address` table, identified by `id` matching `v`. + +You can override this default behavior for any column in any table by providing a `:schema` option that is a hash map whose keys are column names (usually the table-qualified keywords that `next.jdbc` produces by default) and whose values are tuples containing the name of the table to which that column is a foreign key and the name of the key column within that table. These tuples can optionally include a third value which indicates the cardinality of the foreign key relationship: `:one` or `:many`. The default is `:one` and indicates a one-to-one or many-to-one relationship -- `nav`igation will produce a single row. `:many` indicates a one-to-many or many-to-many relationship -- `nav`igation will produce a result set. + +The default behavior in the example above is equivalent to this `:schema` value: + +```clojure +{:contact/addressid [:address :id :one]} ; :one is the default and could be omitted +``` + +If you had a table to track the current valid/bouncing status of email addresses, where `email` is the primary key, you could provide automatic navigation into that using: + +```clojure +{:contact/addressid [:address :id :one] + :address/email [:deliverability :email]} +``` + +If you use foreign key constraints in your database, you could probably generate this `:schema` data structure automatically from the metadata in your database. + +## Behind The Scenes + +Making rows datafiable is implemented by adding metadata to each row with a key of `clojure.core.protocols/datafy` and a function as the value. That function closes over the connectable and options passed in to the `execute!` or `execute-one!` call that produced the result set containing those rows. + +When called (`datafy` on a row), it adds metadata to the row with a key of `clojure.core.protocols/nav` and another function as the value. That function also closes over the connectable and options passed in. + +When that is called (`nav` on a row, column name, and column value), if a `:schema` entry exists for that column or it matches the default convention described above, then it will fetch row(s) using `next.jdbc`'s `Executable` functions `-execute-one` or `-execute-all`, passing in the connectable and options closed over. + +The protocol `next.jdbc.result-set/DatafiableRow` has a default implementation of `datafiable-row` for `clojure.lang.IObj` that just adds the metadata to support `datafy`. There is also an implementation baked into the result set handling behind `reducible!` so that you can call `datafiable-row` directly during reduction and get a fully-realized row that can be `datafy`'d (and then `nav`igated). + +[<: All The Options](/doc/all-the-options.md) | [Migration from `clojure.java.jdbc` :>](/doc/migration-from-clojure-java-jdbc.md) diff --git a/doc/migration-from-clojure-java-jdbc.md b/doc/migration-from-clojure-java-jdbc.md index 3b6e7ac..348ef97 100644 --- a/doc/migration-from-clojure-java-jdbc.md +++ b/doc/migration-from-clojure-java-jdbc.md @@ -56,4 +56,4 @@ These are mostly drawn from Issue #5 although most of the bullets in that issue * `with-transaction` can take a `:rollback-only` option, but there is no way to change a transaction to rollback _dynamically_; throw an exception instead (all transactions roll back on an exception) * The extension points for setting parameters and reading columns are now `SettableParameter` and `ReadableColumn` protocols. -[<: All The Options](/doc/all-the-options.md) +[<: `datafy`, `nav`, and `:schema`](/doc/datafy-nav-and-schema.md)