honeysql/doc/extending-honeysql.md
lread 35c5aee584 Test code blocks in docs with test-doc-blocks
Resolves #290

**Build**

New commands:
- `gen-doc-tests` - only regenerates tests if stale,
   use `clean` command to force regen
- `run-doc-tests` - calls gen-doc-tests then runs tests,
  accepts the same parameters as run-tests.
  Can specify `:platform`
    - `:cljs` - run tests under ClojureScript
    - otherwise Clojure where we can specify one of: `:1.9`
    `:1.10` `:master`

I'm not sure if my use of the `:platform` parameter jives with
your `:aliases` parameter used for `run-tests`.
Can adjust if you like.

Example usages:
```shell
clojure -T:build gen-doc-tests

clojure -T:build run-doc-tests :platform :cljs

clojure -T:build run-doc-tests

clojure -T:build run-doc-tests :platform :1.10
```

The `ci` command has been updated to generate and run doc tests for same
platforms as unit tests.

**Articles**

In addition to `README.md`, now testing doc blocks in all articles
under `doc` dir excepting `doc/operator-reference.md` which does not
have any runnable code blocks.

**Skipped**

Any code block that is intentionally not runnable has been marked to be
skipped via: `<!-- :test-doc-blocks/skip -->`.

**Consistency**

I noticed that some code blocks use REPL syntax:
```Clojure
user=> (+ 1 2 3)
6
```
and others use editor syntax:
```Clojure
(+ 1 2 3)
;;=> 6
```
some places also omit the comment for editor style:
```Clojure
(+ 1 2 3)
=> 6
```
All of this is just fine with test-doc-blocks.
I left the inconsistency as is, but can make a pass for consistency upon
request.

**HoneySQL state**

I noticed a code block that set the sql dialect was affecting other
tests. I simply restored the dialect to the default at the end of the
code block.

**Un-tweaked output**

Some code blocks had string output hand-tweaked for readability.
These have been adjusted to instead use `sql/format`'s `:pretty` option.
In some cases the output is not as readable as the hand-tweaked version.
I humbly suggest that perhaps `:pretty` output could perhaps be
improved (instead of having test-doc-blocks somehow adapt).

**Corrections**

There were very few code blocks that required fixing due to incorrect
output/code.  Please review the diffs carefully to make sure all is as
expected.

**refer-clojure :excludes**

Not currently supported for test-doc-blocks, not a real issue for
Clojure, we'll see warnings under Clojure, but that's probably ok.

But I might actually need it for ClojureScript.
I was finding that `for` did not get overridden by our helper
`:refer` in CloureScript.

Will add proper support to test-doc-blocks but in the short-term,
will use `h/for`.

**ns requires adjustments**

Any specific case of `(ns my-ns (require [my-require :as a]))` is now
the REPL friendly `(require '[my-require :as a])`

Any missing required `requires` were added.

The HoneySQL docs seem to encourage the use of referred vars for
helpers. Although this has the con of overlaps with Clojure core vars,
it is also convenient for Clojure when using `:refer :all`.

**ClojureScript :refer**

ClojureScript does not support `:refer :all` and each var must be
specified in place of `:all`.

I have adjusted examples accordingly to work with both Clojure and
ClojureScript.
2021-08-27 18:39:07 -04:00

4.3 KiB

Extending HoneySQL

Out of the box, HoneySQL supports most standard ANSI SQL clauses and expressions but where it doesn't support something you need you can add new clauses, new operators, and new "functions" (or "special syntax").

There are three extension points in honey.sql that let you register formatters or behavior corresponding to clauses, operators, and functions.

Built in clauses include: :select, :from, :where and many more. Built in operators include: :=, :+, :mod. Built in functions (special syntax) include: :array, :case, :cast, :inline, :raw and many more.

Registering a New Clause Formatter

honey.sql/register-clause! accepts a keyword (or a symbol) that should be treated as a new clause in a SQL statement, a "formatter", and a keyword (or a symbol) that identifies an existing clause that this new one should be ordered before.

The formatter can either be a function of two arguments or a previously registered clause (so that you can easily reuse formatters).

The formatter function will be called with:

  • The clause name (always as a keyword),
  • The sequence of arguments provided.

The third argument to register-clause! allows you to insert your new clause formatter so that clauses are formatted in the correct order for your SQL dialect. For example, :select comes before :from which comes before :where. You can call clause-order to see what the current ordering of clauses is.

Note: if you call register-clause! more than once for the same clause, the last call "wins". This allows you to correct an incorrect clause order insertion by simply calling register-clause! again with a different third argument.

Registering a New Operator

honey.sql/register-op! accepts a keyword (or a symbol) that should be treated as a new infix operator.

By default, operators are treated as strictly binary -- accepting just two arguments -- and an exception will be thrown if they are provided less than two or more than two arguments. You can optionally specify that an operator can take any number of arguments with :variadic true:

(require '[honey.sql :as sql])

(sql/register-op! :<=> :variadic true)
;; and then use the new operator:
(sql/format {:select [:*], :from [:table], :where [:<=> 13 :x 42]})
;; will produce:
;;=> ["SELECT * FROM table WHERE ? <=> x <=> ?" 13 42]

If you are building expressions programmatically, you may want your new operator to ignore "empty" expressions, i.e., where your expression-building code might produce nil. The built-in operators :and and :or ignore such nil expressions. You can specify :ignore-nil true to achieve that:

(sql/register-op! :<=> :variadic true :ignore-nil true)
;; and then use the new operator:
(sql/format {:select [:*], :from [:table], :where [:<=> nil :x 42]})
;; will produce:
;;=> ["SELECT * FROM table WHERE x <=> ?" 42]

Registering a New Function (Special Syntax)

honey.sql/register-fn! accepts a keyword (or a symbol) that should be treated as new syntax (as a function call), and a "formatter". The formatter can either be a function of two arguments or a previously registered "function" (so that you can easily reuse formatters).

The formatter function will be called with:

  • The function name (always as a keyword),
  • The sequence of arguments provided.

For example:

(sql/register-fn! :foo (fn [f args] ..))

(sql/format {:select [:*], :from [:table], :where [:foo 1 2 3]})

Your formatter function will be called with :foo and (1 2 3). It should return a vector containing a SQL string followed by any parameters:

(sql/register-fn! :foo (fn [f args] ["FOO(?)" (first args)]))

(sql/format {:select [:*], :from [:table], :where [:foo 1 2 3]})
;; produces:
;;=> ["SELECT * FROM table WHERE FOO(?)" 1]

In practice, it is likely that your formatter would call sql/sql-kw on the function name to produce a SQL representation of it and would call sql/format-expr on each argument:

(defn- foo-formatter [f [x]]
  (let [[sql & params] (sql/format-expr x)]
    (into [(str (sql/sql-kw f) "(" sql ")")] params)))

(sql/register-fn! :foo foo-formatter)

(sql/format {:select [:*], :from [:table], :where [:foo [:+ :a 1]]})
;; produces:
;;=> ["SELECT * FROM table WHERE FOO(a + ?)" 1]