Test code blocks in docs with test-doc-blocks

Resolves #290

**Build**

New commands:
- `gen-doc-tests` - only regenerates tests if stale,
   use `clean` command to force regen
- `run-doc-tests` - calls gen-doc-tests then runs tests,
  accepts the same parameters as run-tests.
  Can specify `:platform`
    - `:cljs` - run tests under ClojureScript
    - otherwise Clojure where we can specify one of: `:1.9`
    `:1.10` `:master`

I'm not sure if my use of the `:platform` parameter jives with
your `:aliases` parameter used for `run-tests`.
Can adjust if you like.

Example usages:
```shell
clojure -T:build gen-doc-tests

clojure -T:build run-doc-tests :platform :cljs

clojure -T:build run-doc-tests

clojure -T:build run-doc-tests :platform :1.10
```

The `ci` command has been updated to generate and run doc tests for same
platforms as unit tests.

**Articles**

In addition to `README.md`, now testing doc blocks in all articles
under `doc` dir excepting `doc/operator-reference.md` which does not
have any runnable code blocks.

**Skipped**

Any code block that is intentionally not runnable has been marked to be
skipped via: `<!-- :test-doc-blocks/skip -->`.

**Consistency**

I noticed that some code blocks use REPL syntax:
```Clojure
user=> (+ 1 2 3)
6
```
and others use editor syntax:
```Clojure
(+ 1 2 3)
;;=> 6
```
some places also omit the comment for editor style:
```Clojure
(+ 1 2 3)
=> 6
```
All of this is just fine with test-doc-blocks.
I left the inconsistency as is, but can make a pass for consistency upon
request.

**HoneySQL state**

I noticed a code block that set the sql dialect was affecting other
tests. I simply restored the dialect to the default at the end of the
code block.

**Un-tweaked output**

Some code blocks had string output hand-tweaked for readability.
These have been adjusted to instead use `sql/format`'s `:pretty` option.
In some cases the output is not as readable as the hand-tweaked version.
I humbly suggest that perhaps `:pretty` output could perhaps be
improved (instead of having test-doc-blocks somehow adapt).

**Corrections**

There were very few code blocks that required fixing due to incorrect
output/code.  Please review the diffs carefully to make sure all is as
expected.

**refer-clojure :excludes**

Not currently supported for test-doc-blocks, not a real issue for
Clojure, we'll see warnings under Clojure, but that's probably ok.

But I might actually need it for ClojureScript.
I was finding that `for` did not get overridden by our helper
`:refer` in CloureScript.

Will add proper support to test-doc-blocks but in the short-term,
will use `h/for`.

**ns requires adjustments**

Any specific case of `(ns my-ns (require [my-require :as a]))` is now
the REPL friendly `(require '[my-require :as a])`

Any missing required `requires` were added.

The HoneySQL docs seem to encourage the use of referred vars for
helpers. Although this has the con of overlaps with Clojure core vars,
it is also convenient for Clojure when using `:refer :all`.

**ClojureScript :refer**

ClojureScript does not support `:refer :all` and each var must be
specified in place of `:all`.

I have adjusted examples accordingly to work with both Clojure and
ClojureScript.
This commit is contained in:
lread 2021-08-27 18:16:18 -04:00
parent 584cd1c711
commit 35c5aee584
10 changed files with 331 additions and 175 deletions

View file

@ -24,8 +24,8 @@ for copying and pasting directly into your SQL tool of choice!
## Note on code samples
All sample code in this README is automatically run as a unit test using
[seancorfield/readme](https://github.com/seancorfield/readme).
Sample code in this documentation is verified via
[lread/test-doc-blocks](https://github.com/lread/test-doc-blocks).
Some of these samples show pretty-printed SQL: HoneySQL 2.x supports `:pretty true` which inserts newlines between clauses in the generated SQL strings.
@ -37,6 +37,8 @@ HoneySQL 1.x will continue to get critical security fixes but otherwise should b
## Usage
From Clojure:
<!-- {:test-doc-blocks/reader-cond :clj} -->
```clojure
(refer-clojure :exclude '[filter for group-by into partition-by set update])
(require '[honey.sql :as sql]
@ -51,6 +53,19 @@ HoneySQL 1.x will continue to get critical security fixes but otherwise should b
'[clojure.core :as c])
```
From ClojureScript, we don't have `:refer :all`. If we want to use `:refer`, we have no choice but to be specific:
<!-- {:test-doc-blocks/reader-cond :cljs} -->
```Clojure
(require '[honey.sql :as sql]
'[honey.sql.helpers :refer [select select-distinct from
join left-join right-join
where for group-by having union
order-by limit offset values columns
insert-into set composite
delete delete-from truncate] :as h]
'[clojure.core :as c])
```
Everything is built on top of maps representing SQL queries:
```clojure
@ -78,7 +93,8 @@ HoneySQL is a relatively "pure" library, it does not manage your JDBC connection
or run queries for you, it simply generates SQL strings. You can then pass them
to a JDBC library, such as [`next.jdbc`](https://github.com/seancorfield/next-jdbc):
```clj
<!-- :test-doc-blocks/skip -->
```clojure
(jdbc/execute! conn (sql/format sqlmap))
```
@ -543,9 +559,9 @@ These can be combined to allow more fine-grained control over SQL generation:
```
```clojure
call-qualify-map
=> '{:where [:and [:= :a [:param :baz]] [:= :b [:inline 42]]]
:from (:foo)
:select [[[:foo :bar]] [[:raw "@var := foo.bar"]]]}
=> {:where [:and [:= :a [:param :baz]] [:= :b [:inline 42]]]
:from (:foo)
:select [[[:foo :bar]] [[:raw "@var := foo.bar"]]]}
```
```clojure
(sql/format call-qualify-map {:params {:baz "BAZ"}})
@ -639,7 +655,7 @@ If `<table(s)>` and `<qualifier>` are both omitted, you may also omit the `[`..`
(-> (select :foo.a)
(from :foo)
(where [:= :foo.a "baz"])
(for :update)
(h/for :update)
(sql/format))
=> ["SELECT foo.a FROM foo WHERE foo.a = ? FOR UPDATE" "baz"]
```
@ -730,7 +746,9 @@ OFFSET ?
```
```clojure
;; Printable and readable
(= big-complicated-map (read-string (pr-str big-complicated-map)))
(require '[clojure.edn :as edn])
(= big-complicated-map (edn/read-string (pr-str big-complicated-map)))
=> true
```

View file

@ -9,9 +9,11 @@
For more information, run:
clojure -A:deps -T:build help/doc"
(:require [clojure.tools.build.api :as b]
(:require [babashka.fs :as fs]
[clojure.tools.build.api :as b]
[clojure.tools.deps.alpha :as t]
[deps-deploy.deps-deploy :as dd]))
[deps-deploy.deps-deploy :as dd]
[lread.test-doc-blocks :as tdb]))
(def lib 'com.github.seancorfield/honeysql)
(def version (format "2.0.%s" (b/git-count-revs nil)))
@ -51,7 +53,22 @@
(when-not (zero? exit)
(throw (ex-info (str "Task failed for: " aliases) {})))))
(defn readme "Run the README tests." [opts] (run-task [:readme]) opts)
(defn gen-doc-tests "Generate tests from doc code blocks" [opts]
(let [docs ["README.md"
"doc/clause-reference.md"
"doc/differences-from-1-x.md"
"doc/extending-honeysql.md"
"doc/general-reference.md"
"doc/getting-started.md"
"doc/postgresql.md"
"doc/special-syntax.md"]
updated-docs (fs/modified-since "target/test-doc-blocks" (conj docs "build.clj" "deps.edn"))]
(if (seq updated-docs)
(do
(println "gen-doc-tests: Regenerating: found newer:" (mapv str updated-docs))
(tdb/gen-tests {:docs docs}))
(println "gen-doc-tests: Tests already generated")))
opts)
(defn eastwood "Run Eastwood." [opts] (run-task [:eastwood]) opts)
@ -67,9 +84,30 @@
(run-task (into [:test] aliases))
opts)
(defn run-doc-tests
"Run generated doc tests.
Optionally specify :platform:
:1.9 -- test against Clojure 1.9 (the default)
:1.10 -- test against Clojure 1.10.3
:master -- test against Clojure 1.11 master snapshot
:cljs -- test against ClojureScript"
[{:keys [platform] :or {platform :1.9} :as opts}]
(gen-doc-tests opts)
(let [aliases (case platform
:cljs [platform :test-doc :test-doc-cljs]
[platform :runner :test-doc :test-doc-clj])]
(run-task aliases))
opts)
(defn ci "Run the CI pipeline of tests (and build the JAR)." [opts]
(-> opts
(readme)
(clean)
(as-> opts
(reduce (fn [opts platform]
(run-doc-tests (assoc opts :platform platform)))
opts
[:cljs :1.9 :1.10 :master]))
(eastwood)
(as-> opts
(reduce (fn [opts alias]

View file

@ -3,7 +3,9 @@
:deps {org.clojure/clojure {:mvn/version "1.9.0"}}
:aliases
{;; for help: clojure -A:deps -T:build help/doc
:build {:deps {io.github.clojure/tools.build {:git/tag "v0.1.9" :git/sha "6736c83"}
:build {:deps {babashka/fs {:mvn/version "0.0.5"}
com.github.lread/test-doc-blocks {:mvn/version "1.0.137-alpha"}
io.github.clojure/tools.build {:git/tag "v0.1.9" :git/sha "6736c83"}
io.github.slipset/deps-deploy {:sha "b4359c5d67ca002d9ed0c4b41b710d7e5a82e3bf"}}
:ns-default build}
@ -13,18 +15,25 @@
:master {:override-deps {org.clojure/clojure {:mvn/version "1.11.1-master-SNAPSHOT"}}}
;; running tests/checks of various kinds:
:test ; can also run clojure -X:test
{:extra-paths ["test"]
:extra-deps {io.github.cognitect-labs/test-runner
{:git/tag "v0.4.0" :git/sha "334f2e2"}}
:exec-fn cognitect.test-runner.api/test}
;; various "runners" for tests/CI:
:runner
{:main-opts ["-m" "cognitect.test-runner"]}
:test {:extra-paths ["test"]}
:runner ; can also run clojure -X:test
{:extra-deps {io.github.cognitect-labs/test-runner
{:git/tag "v0.4.0" :git/sha "334f2e2"}}
:main-opts ["-m" "cognitect.test-runner"]
:exec-fn cognitect.test-runner.api/test}
;; various "runners" for tests/CI:
:cljs {:extra-deps {olical/cljs-test-runner {:mvn/version "3.8.0"}}
:main-opts ["-m" "cljs-test-runner.main"]}
:readme {:extra-deps {seancorfield/readme {:mvn/version "1.0.16"}}
:main-opts ["-m" "seancorfield.readme"]}
:test-doc {:extra-paths ["target/test-doc-blocks/test"]}
:test-doc-clj {:main-opts ["-m" "cognitect.test-runner"
"-d" "target/test-doc-blocks/test"]}
:test-doc-cljs {:main-opts ["-m" "cljs-test-runner.main"
"-c" "{:warnings,{:single-segment-namespace,false}}"
"-d" "target/test-doc-blocks/test"]}
:eastwood {:extra-deps {jonase/eastwood {:mvn/version "0.5.1"}}
:main-opts ["-m" "eastwood.lint" "{:source-paths,[\"src\"]}"]}}}

View file

@ -14,6 +14,13 @@ dialects that HoneySQL supports.
DDL clauses are listed first, followed by SQL clauses.
The examples herein assume:
```clojure
(require '[honey.sql :as sql]
'[honey.sql.helpers :refer [select from join-by left-join join
where order-by over partition-by window]])
```
# DDL Clauses
HoneySQL supports the following DDL clauses as a data DSL.
@ -76,7 +83,7 @@ user=> (sql/format {:alter-table :fruit
["ALTER TABLE fruit ADD INDEX look(appearance)"]
user=> (sql/format {:alter-table :fruit
:add-index [:unique nil :color :appearance]})
["ALTER TABLE fruit ADD UNIQUE(color,appearance)"]
["ALTER TABLE fruit ADD UNIQUE(color, appearance)"]
user=> (sql/format {:alter-table :fruit :drop-index :look})
["ALTER TABLE fruit DROP INDEX look"]
```
@ -108,12 +115,7 @@ user=> (sql/format {:create-table :fruit
[[:id :int [:not nil]]
[:name [:varchar 32] [:not nil]]
[:cost :float :null]]})
;; reformatted for clarity:
["CREATE TABLE fruit (
id INT NOT NULL,
name VARCHAR(32) NOT NULL,
cost FLOAT NULL
)"]
["CREATE TABLE fruit (id INT NOT NULL, name VARCHAR(32) NOT NULL, cost FLOAT NULL)"]
```
The `:with-columns` clause is formatted as if `{:inline true}`
@ -544,25 +546,30 @@ user=> (sql/format {:select [:t.ref :pp.code]
[:using :id]]
:join [[:logtransaction :log]
[:= :t.id :log.id]]]
:where [:= "settled" :pp.status]})
;; newlines inserted for readability:
["SELECT t.ref, pp.code FROM transaction AS t
LEFT JOIN paypal_tx AS pp USING (id)
INNER JOIN logtransaction AS log ON t.id = log.id
WHERE ? = pp.status" "settled"]
;; or using helpers:
:where [:= "settled" :pp.status]}
{:pretty true})
["
SELECT t.ref, pp.code
FROM transaction AS t
LEFT JOIN paypal_tx AS pp USING (id) INNER JOIN logtransaction AS log ON t.id = log.id
WHERE ? = pp.status
" "settled"]
;; or the equivalent using helpers:
user=> (sql/format (-> (select :t.ref :pp.code)
(from [:transaction :t])
(join-by (left-join [:paypal-tx :pp]
[:using :id])
(join [:logtransaction :log]
[:= :t.id :log.id]))
(where := "settled" :pp.status)))
;; newlines inserted for readability:
["SELECT t.ref, pp.code FROM transaction AS t
LEFT JOIN paypal_tx AS pp USING (id)
INNER JOIN logtransaction AS log ON t.id = log.id
WHERE ? = pp.status" "settled"]
(where := "settled" :pp.status))
{:pretty true})
["
SELECT t.ref, pp.code
FROM transaction AS t
LEFT JOIN paypal_tx AS pp USING (id) INNER JOIN logtransaction AS log ON t.id = log.id
WHERE ? = pp.status
" "settled"]
```
Without `:join-by`, a `:join` would normally be generated before a `:left-join`.
@ -667,25 +674,25 @@ user=> (sql/format {:select [:id
:w
:MaxSalary]]]]
:from [:employee]
:window [:w {:partition-by [:department]}]})
;; newlines inserted for readability:
["SELECT id,
AVG(salary) OVER (PARTITION BY department ORDER BY designation ASC) AS Average,
MAX(salary) OVER w AS MaxSalary
FROM employee
WINDOW w AS (PARTITION BY department)"]
:window [:w {:partition-by [:department]}]}
{:pretty true})
["
SELECT id, AVG(salary) OVER (PARTITION BY department ORDER BY designation ASC) AS Average, MAX(salary) OVER w AS MaxSalary
FROM employee
WINDOW w AS (PARTITION BY department)
"]
;; easier to write with helpers (and easier to read!):
user=> (sql/format (-> (select :id
(over [[:avg :salary] (-> (partition-by :department) (order-by :designation)) :Average]
[[:max :salary] :w :MaxSalary]))
(from :employee)
(window :w (partition-by :department))))
;; newlines inserted for readability:
["SELECT id,
AVG(salary) OVER (PARTITION BY department ORDER BY designation ASC) AS Average,
MAX(salary) OVER w AS MaxSalary
FROM employee
WINDOW w AS (PARTITION BY department)"]
(window :w (partition-by :department)))
{:pretty true})
["
SELECT id, AVG(salary) OVER (PARTITION BY department ORDER BY designation ASC) AS Average, MAX(salary) OVER w AS MaxSalary
FROM employee
WINDOW w AS (PARTITION BY department)
"]
```
The window function in the `:over` expression may be `{}` or `nil`:

View file

@ -19,17 +19,30 @@ In addition, HoneySQL 2.x contains different namespaces so you can have both ver
### HoneySQL 1.x
In `deps.edn`:
<!-- :test-doc-blocks/skip -->
```clojure
;; in deps.edn:
honeysql {:mvn/version "1.0.461"}
;; or, more correctly:
honeysql/honeysql {:mvn/version "1.0.461"}
```
;; in use:
Required as:
<!-- :test-doc-blocks/skip -->
```clojure
(ns my.project
(:require [honeysql.core :as sql]))
```
...
Or if in the REPL:
<!-- :test-doc-blocks/skip -->
```clojure
(require '[honeysql.core :as sq])
```
In use:
<!-- :test-doc-blocks/skip -->
```clojure
(sql/format {:select [:*] :from [:table] :where [:= :id 1]})
;;=> ["SELECT * FROM table WHERE id = ?" 1]
(sql/format {:select [:*] :from [:table] :where [:= :id 1]} :quoting :mysql)
@ -47,15 +60,26 @@ Supported Clojure versions: 1.7 and later.
### HoneySQL 2.x
In `deps.edn`:
<!-- :test-doc-blocks/skip -->
```clojure
;; in deps.edn:
com.github.seancorfield/honeysql {:mvn/version "2.0.783"}
```
;; in use:
Required as:
<!-- :test-doc-blocks/skip -->
```clojure
(ns my.project
(:require [honey.sql :as sql]))
```
...
Or if in the REPL:
```clojure
(require '[honey.sql :as sql])
```
In use:
```clojure
(sql/format {:select [:*] :from [:table] :where [:= :id 1]})
;;=> ["SELECT * FROM table WHERE id = ?" 1]
(sql/format {:select [:*] :from [:table] :where [:= :id 1]} {:dialect :mysql})
@ -158,7 +182,7 @@ it should have been a function, and in 2.x it is:
```clojure
;; 1.x: EXISTS should never have been implemented as SQL syntax: it's an operator!
;; (sq/format {:exists {:select [:a] :from [:foo]}})
;;=> ["EXISTS (SELECT a FROM foo)"]
;; -> ["EXISTS (SELECT a FROM foo)"]
;; 2.x: select function call with an alias:
user=> (sql/format {:select [[[:exists {:select [:a] :from [:foo]}] :x]]})

View file

@ -50,6 +50,8 @@ two arguments. You can optionally specify that an operator
can take any number of arguments with `:variadic true`:
```clojure
(require '[honey.sql :as sql])
(sql/register-op! :<=> :variadic true)
;; and then use the new operator:
(sql/format {:select [:*], :from [:table], :where [:<=> 13 :x 42]})
@ -86,6 +88,7 @@ The formatter function will be called with:
For example:
<!-- :test-doc-blocks/skip -->
```clojure
(sql/register-fn! :foo (fn [f args] ..))

View file

@ -12,6 +12,8 @@ because `:quoted true` was specified, the literal name of an unqualified,
single-segment keyword or symbol is used as-is and quoted:
```clojure
(require '[honey.sql :as sql])
(sql/format {:select :foo-bar} {:quoted true})
;;=> ["SELECT \"foo-bar\""]
(sql/format {:select :foo-bar} {:dialect :mysql})

View file

@ -8,12 +8,14 @@ data to a SQL statement (string) and any parameters it needs.
For the Clojure CLI, add the following dependency to your `deps.edn` file:
<!-- :test-doc-blocks/skip -->
```clojure
com.github.seancorfield/honeysql {:mvn/version "2.0.783"}
```
For Leiningen, add the following dependency to your `project.clj` file:
<!-- :test-doc-blocks/skip -->
```clojure
[com.github.seancorfield/honeysql "2.0.783"]
```
@ -41,11 +43,9 @@ SQL string as the first element followed by any parameter
values identified in the SQL expressions:
```clojure
(ns my.example
(:require [honey.sql :as sql]))
(require '[honey.sql :as sql])
(sql/format {:select [:*], :from [:table], :where [:= :id 1]})
;; produces:
;;=> ["SELECT * FROM table WHERE id = ?" 1]
```
@ -65,7 +65,6 @@ that represents a SQL entity and its alias (where aliases are allowed):
```clojure
(sql/format {:select [:t.id [:name :item]], :from [[:table :t]], :where [:= :id 1]})
;; produces:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
```
@ -80,9 +79,10 @@ avoid evaluation:
```clojure
(sql/format '{select [t.id [name item]], from [[table t]], where [= id 1]})
;; or you can use (..) instead of [..] when quoted:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
;; or you can use (..) instead of [..] when quoted to produce the same result:
(sql/format '{select (t.id (name item)), from ((table t)), where (= id 1)})
;; also produces:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
```
@ -91,10 +91,10 @@ keywords (or symbols) and the namespace portion will treated as
the table name, i.e., `:foo/bar` instead of `:foo.bar`:
```clojure
;; notice the following both produce the same result:
(sql/format {:select [:t/id [:name :item]], :from [[:table :t]], :where [:= :id 1]})
;; and
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
(sql/format '{select [t/id [name item]], from [[table t]], where [= id 1]})
;; both produce:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
```
@ -116,6 +116,7 @@ described in the [Special Syntax](special-syntax.md) section.
Some examples:
<!-- :test-doc-blocks/skip -->
```clojure
[:= :a 42] ;=> "a = ?" with a parameter of 42
[:+ 42 :a :b] ;=> "? + a + b" with a parameter of 42
@ -133,6 +134,7 @@ Another form of special syntax that is treated as function calls
is keywords or symbols that begin with `%`. Such keywords (or symbols)
are split at `.` and turned into function calls:
<!-- :test-doc-blocks/skip -->
```clojure
%now ;=> NOW()
%count.* ;=> COUNT(*)
@ -143,6 +145,7 @@ are split at `.` and turned into function calls:
If you need to reference a table or alias for a column, you can use
qualified names in a function invocation:
<!-- :test-doc-blocks/skip -->
```clojure
%max.foo/bar ;=> MAX(foo.bar)
```
@ -179,6 +182,7 @@ that are not keywords or symbols are lifted out as positional parameters.
They are replaced by `?` in the generated SQL string and added to the
parameter list in order:
<!-- :test-doc-blocks/skip -->
```clojure
[:between :size 10 20] ;=> "size BETWEEN ? AND ?" with parameters 10 and 20
```
@ -195,11 +199,11 @@ call as the `:params` key of the options hash map.
(sql/format {:select [:*] :from [:table]
:where [:= :a :?x]}
{:params {:x 42}})
["SELECT * FROM table WHERE a = ?" 42]
;;=> ["SELECT * FROM table WHERE a = ?" 42]
(sql/format {:select [:*] :from [:table]
:where [:= :a [:param :x]]}
{:params {:x 42}})
["SELECT * FROM table WHERE a = ?" 42]
;;=> ["SELECT * FROM table WHERE a = ?" 42]
```
## Functional Helpers
@ -210,15 +214,13 @@ SQL queries with raw Clojure data structures, a
is also available. These functions are generally variadic and threadable:
```clojure
(ns my.example
(:require [honey.sql :as sql]
[honey.sql.helpers :refer [select from where]]))
(require '[honey.sql :as sql]
'[honey.sql.helpers :refer [select from where]])
(-> (select :t/id [:name :item])
(from [:table :t])
(where [:= :id 1])
(sql/format))
;; produces:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
```
@ -238,7 +240,6 @@ can make it easier to build queries programmatically:
(where [:= :id 1])
(select [:name :item])
(sql/format))
;; produces:
;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1]
```
@ -252,7 +253,6 @@ you need to explicitly remove the prior value:
(dissoc :select)
(select [:name :item])
(sql/format))
;; produces:
;;=> ["SELECT name AS item FROM table AS t WHERE id = ?" 1]
```
@ -314,6 +314,8 @@ dialect in a `format` call, they will be quoted. If you don't
specify a dialect in the `format` call, you can specify
`:quoted true` to have SQL entities quoted.
<!-- Reminder to doc author:
Reset dialect to default so other blocks are not affected for test-doc-blocks -->
```clojure
(sql/format '{select (id) from (table)} {:quoted true})
;;=> ["SELECT \"id\" FROM \"table\""]
@ -323,6 +325,11 @@ specify a dialect in the `format` call, you can specify
;;=> nil
(sql/format '{select (id) from (table)} {:quoted true})
;;=> ["SELECT [id] FROM [table]"]
;; and to the default of :ansi
(sql/set-dialect! :ansi)
;;=> nil
(sql/format '{select (id) from (table)} {:quoted true})
;;=> ["SELECT \"id\" FROM \"table\""]
```
Out of the box, as part of the extended ANSI SQL support,

View file

@ -18,6 +18,22 @@ HoneySQL not to do that. There are two possible approaches:
1. Use named parameters (e.g., `[:param :myval]`) instead of having the values directly in the DSL structure and then pass `{:params {:myval some-json}}` as part of the options in the call to `format`, or
2. Use `[:lift ..]` wrapped around any structured values which tells HoneySQL not to interpret the vector or hash map value as a DSL: `[:lift some-json]`.
The code example herein assume:
```clojure
(require '[honey.sql :as sql]
'[honey.sql.helpers :refer [select from where
update set
insert-into values
create-table with-columns create-view create-extension
add-column alter-table add-index
modify-column rename-column rename-table
drop-table drop-column drop-index drop-extension
upsert returning on-conflict on-constraint
do-update-set do-nothing]])
```
Clojure users can opt for the shorter `(require '[honey.sql :as sql] '[honey.sql.helpers :refer :all])` but this syntax is not available to ClojureScript users.
## Upsert
Upserting data is relatively easy in PostgreSQL
@ -34,11 +50,16 @@ user=> (-> (insert-into :distributors)
(upsert (-> (on-conflict :did)
(do-update-set :dname)))
(returning :*)
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?), (?, ?)
ON CONFLICT (did) DO UPDATE SET dname = EXCLUDED.dname RETURNING *"
5 "Gizmo Transglobal" 6 "Associated Computing, Inc"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?), (?, ?)
ON CONFLICT (did)
DO UPDATE SET dname = EXCLUDED.dname
RETURNING *
"
5 "Gizmo Transglobal"
6 "Associated Computing, Inc"]
```
However, the nested `upsert` helper is no longer needed
@ -51,11 +72,16 @@ user=> (-> (insert-into :distributors)
(on-conflict :did)
(do-update-set :dname)
(returning :*)
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?), (?, ?)
ON CONFLICT (did) DO UPDATE SET dname = EXCLUDED.dname RETURNING *"
5 "Gizmo Transglobal" 6 "Associated Computing, Inc"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?), (?, ?)
ON CONFLICT (did)
DO UPDATE SET dname = EXCLUDED.dname
RETURNING *
"
5 "Gizmo Transglobal"
6 "Associated Computing, Inc"]
```
Similarly, the `do-nothing` helper behaves just the same
@ -66,11 +92,14 @@ user=> (-> (insert-into :distributors)
(values [{:did 7 :dname "Redline GmbH"}])
(upsert (-> (on-conflict :did)
do-nothing))
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?)
ON CONFLICT (did) DO NOTHING"
7 "Redline GmbH"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?)
ON CONFLICT (did)
DO NOTHING
"
7 "Redline GmbH"]
```
As above, the nested `upsert` helper is no longer needed:
@ -80,11 +109,14 @@ user=> (-> (insert-into :distributors)
(values [{:did 7 :dname "Redline GmbH"}])
(on-conflict :did)
do-nothing
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?)
ON CONFLICT (did) DO NOTHING"
7 "Redline GmbH"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?)
ON CONFLICT (did)
DO NOTHING
"
7 "Redline GmbH"]
```
`ON CONSTRAINT` is handled slightly differently to the nilenso library,
@ -96,22 +128,29 @@ user=> (-> (insert-into :distributors)
;; can specify as a nested clause...
(on-conflict (on-constraint :distributors_pkey))
do-nothing
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?)
ON CONFLICT ON CONSTRAINT distributors_pkey DO NOTHING"
9 "Antwerp Design"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?)
ON CONFLICT ON CONSTRAINT distributors_pkey
DO NOTHING
"
9 "Antwerp Design"]
user=> (-> (insert-into :distributors)
(values [{:did 9 :dname "Antwerp Design"}])
;; ...or as two separate clauses
on-conflict
(on-constraint :distributors_pkey)
do-nothing
sql/format)
;; newlines inserted for readability:
["INSERT INTO distributors (did, dname) VALUES (?, ?)
ON CONFLICT ON CONSTRAINT distributors_pkey DO NOTHING"
9 "Antwerp Design"]
(sql/format {:pretty true}))
["
INSERT INTO distributors
(did, dname) VALUES (?, ?)
ON CONFLICT
ON CONSTRAINT distributors_pkey
DO NOTHING
"
9 "Antwerp Design"]
```
As above, the `upsert` helper has been omitted here.
@ -124,12 +163,14 @@ user=> (-> (insert-into :user)
(values [{:phone "5555555" :name "John"}])
(on-conflict :phone (where [:<> :phone nil]))
(do-update-set :phone :name (where [:= :user.active false]))
sql/format)
;; newlines inserted for readability:
["INSERT INTO user (phone, name) VALUES (?, ?)
ON CONFLICT (phone) WHERE phone IS NOT NULL
DO UPDATE SET phone = EXCLUDED.phone, name = EXCLUDED.name
WHERE user.active = FALSE" "5555555" "John"]
(sql/format {:pretty true}))
["
INSERT INTO user
(phone, name) VALUES (?, ?)
ON CONFLICT (phone) WHERE phone IS NOT NULL
DO UPDATE SET phone = EXCLUDED.phone, name = EXCLUDED.name WHERE user.active = FALSE
"
"5555555" "John"]
;; using the DSL directly:
user=> (sql/format
{:insert-into :user
@ -137,16 +178,20 @@ user=> (sql/format
:on-conflict [:phone
{:where [:<> :phone nil]}]
:do-update-set {:fields [:phone :name]
:where [:= :user.active false]}})
;; newlines inserted for readability:
["INSERT INTO user (phone, name) VALUES (?, ?)
ON CONFLICT (phone) WHERE phone IS NOT NULL
DO UPDATE SET phone = EXCLUDED.phone, name = EXCLUDED.name
WHERE user.active = FALSE" "5555555" "John"]
:where [:= :user.active false]}}
{:pretty true})
["
INSERT INTO user
(phone, name) VALUES (?, ?)
ON CONFLICT (phone) WHERE phone IS NOT NULL
DO UPDATE SET phone = EXCLUDED.phone, name = EXCLUDED.name WHERE user.active = FALSE
"
"5555555" "John"]
```
By comparison, this is the DSL structure that nilenso would have required:
<!-- :test-doc-blocks/skip -->
```clojure
;; NOT VALID FOR HONEYSQL!
{:insert-into :user
@ -230,12 +275,12 @@ user=> (-> (create-table :distributors)
;; "serial" is inlined as 'SERIAL':
[:default [:nextval "serial"]]]
[:name [:varchar 40] [:not nil]]])
sql/format)
(sql/format {:pretty true}))
;; newlines inserted for readability:
["CREATE TABLE distributors (
did INTEGER PRIMARY KEY DEFAULT NEXTVAL('SERIAL'),
name VARCHAR(40) NOT NULL
)"]
["
CREATE TABLE distributors
(did INTEGER PRIMARY KEY DEFAULT NEXTVAL('SERIAL'), name VARCHAR(40) NOT NULL)
"]
;; PostgreSQL CHECK constraint is supported:
user=> (-> (create-table :products)
(with-columns [[:product_no :integer]
@ -243,20 +288,16 @@ user=> (-> (create-table :products)
[:price :numeric [:check [:> :price 0]]]
[:discounted_price :numeric]
[[:check [:and [:> :discounted_price 0] [:> :price :discounted_price]]]]])
sql/format)
;; newlines inserted for readability:
["CREATE TABLE products (
product_no INTEGER,
name TEXT,
price NUMERIC CHECK(PRICE > 0),
discounted_price NUMERIC,
CHECK((discounted_price > 0) AND (price > discounted_price))
)"]
(sql/format {:pretty true}))
["
CREATE TABLE products
(product_no INTEGER, name TEXT, price NUMERIC CHECK(PRICE > 0), discounted_price NUMERIC, CHECK((discounted_price > 0) AND (price > discounted_price)))
"]
;; conditional creation:
user=> (-> (create-table :products :if-not-exists)
...
(with-columns [[:name :text]])
sql/format)
["CREATE TABLE IF NOT EXISTS products (...)"]
["CREATE TABLE IF NOT EXISTS products (name TEXT)"]
;; drop table:
user=> (sql/format (drop-table :cities))
["DROP TABLE cities"]
@ -339,10 +380,7 @@ user=> (-> (alter-table :fruit)
user=> (sql/format (alter-table :fruit
(add-column :skin [:varchar 16] nil)
(add-index :unique :fruit-name :name)))
;; newlines inserted for readability:
["ALTER TABLE fruit
ADD COLUMN skin VARCHAR(16) NULL,
ADD UNIQUE fruit_name(name)"]
["ALTER TABLE fruit ADD COLUMN skin VARCHAR(16) NULL, ADD UNIQUE fruit_name(name)"]
```
## Filter / Within Group

View file

@ -13,6 +13,8 @@ a sequence, and produces `ARRAY[?, ?, ..]` for the elements
of that sequence (as SQL parameters):
```clojure
(require '[honey.sql :as sql])
(sql/format-expr [:array (range 5)])
;;=> ["ARRAY[?, ?, ?, ?, ?]" 0 1 2 3 4]
```
@ -36,8 +38,7 @@ may be `:else` (or `'else`) to produce `ELSE`, otherwise
```clojure
(sql/format-expr [:case [:< :a 10] "small" [:> :a 100] "big" :else "medium"])
;;=> ["CASE WHEN a < ? THEN ? WHEN a > ? THEN ? ELSE ? END"
;; 10 "small" 100 "big" "medium"]
;; => ["CASE WHEN a < ? THEN ? WHEN a > ? THEN ? ELSE ? END" 10 "small" 100 "big" "medium"]
```
## cast
@ -76,6 +77,7 @@ SQL entity. This is intended for use in contexts that would
otherwise produce a sequence of SQL keywords, such as when
constructing DDL statements.
<!-- :test-doc-blocks/skip -->
```clojure
[:tablespace :quux]
;;=> TABLESPACE QUUX
@ -89,9 +91,9 @@ Intended to be used with regular expression patterns to
specify the escape characters (if any).
```clojure
(format {:select :* :from :foo
:where [:similar-to :foo [:escape "bar" [:inline "*"]]]})
;;=> ["SELECT * FROM foo WHERE foo SIMILAR TO ? ESCAPE '*'" "bar"]))))
(sql/format {:select :* :from :foo
:where [:similar-to :foo [:escape "bar" [:inline "*"]]]})
;;=> ["SELECT * FROM foo WHERE foo SIMILAR TO ? ESCAPE '*'" "bar"]
```
## filter, within-group
@ -104,34 +106,38 @@ Filter generally expects an aggregate expression and a `WHERE` clause.
Within group generally expects an aggregate expression and an `ORDER BY` clause.
```clojure
(format {:select [:a :b [[:filter :%count.* {:where [:< :x 100]}] :c]
[[:within-group [:percentile_disc [:inline 0.25]]
{:order-by [:a]}] :inter_max]
[[:within-group [:percentile_cont [:inline 0.25]]
{:order-by [:a]}] :abs_max]]
:from :aa})
;; newlines added for readability:
;;=> ["SELECT a, b, COUNT(*) FILTER (WHERE x < ?) AS c,
;;=> PERCENTILE_DISC(0.25) WITHIN GROUP (ORDER BY a ASC) AS inter_max,
;;=> PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY a ASC) AS abs_max
;;=> FROM aa" 100]
(sql/format {:select [:a :b [[:filter :%count.* {:where [:< :x 100]}] :c]
[[:within-group [:percentile_disc [:inline 0.25]]
{:order-by [:a]}] :inter_max]
[[:within-group [:percentile_cont [:inline 0.25]]
{:order-by [:a]}] :abs_max]]
:from :aa}
{:pretty true})
;;=> ["
SELECT a, b, COUNT(*) FILTER (WHERE x < ?) AS c, PERCENTILE_DISC(0.25) WITHIN GROUP (ORDER BY a ASC) AS inter_max, PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY a ASC) AS abs_max
FROM aa
"
100]
```
There are helpers for both `filter` and `within-group`. Be careful with `filter`
since it shadows `clojure.core/filter`:
```clojure
(format (-> (select :a :b [(filter :%count.* (where :< :x 100)) :c]
[(within-group [:percentile_disc [:inline 0.25]]
(order-by :a)) :inter_max]
[(within-group [:percentile_cont [:inline 0.25]]
(order-by :a)) :abs_max])
(from :aa)))
;; newlines added for readability:
;;=> ["SELECT a, b, COUNT(*) FILTER (WHERE x < ?) AS c,
;;=> PERCENTILE_DISC(0.25) WITHIN GROUP (ORDER BY a ASC) AS inter_max,
;;=> PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY a ASC) AS abs_max
;;=> FROM aa" 100]
(require '[honey.sql.helpers :refer [select filter within-group from order-by where]])
(sql/format (-> (select :a :b [(filter :%count.* (where :< :x 100)) :c]
[(within-group [:percentile_disc [:inline 0.25]]
(order-by :a)) :inter_max]
[(within-group [:percentile_cont [:inline 0.25]]
(order-by :a)) :abs_max])
(from :aa))
{:pretty true})
;;=> ["
SELECT a, b, COUNT(*) FILTER (WHERE x < ?) AS c, PERCENTILE_DISC(0.25) WITHIN GROUP (ORDER BY a ASC) AS inter_max, PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY a ASC) AS abs_max
FROM aa
"
100]
```
## inline
@ -214,15 +220,15 @@ by an ordering specifier, which can be an expression or a pair of expression
and direction (`:asc` or `:desc`):
```clojure
(format {:select [[[:array_agg [:order-by :a [:b :desc]]]]] :from :table})
(sql/format {:select [[[:array_agg [:order-by :a [:b :desc]]]]] :from :table})
;;=> ["SELECT ARRAY_AGG(a ORDER BY b DESC) FROM table"]
(format (-> (select [[:array_agg [:order-by :a [:b :desc]]]])
(sql/format (-> (select [[:array_agg [:order-by :a [:b :desc]]]])
(from :table)))
;;=> ["SELECT ARRAY_AGG(a ORDER BY b DESC) FROM table"]
(format {:select [[[:string_agg :a [:order-by [:inline ","] :a]]]] :from :table})
(sql/format {:select [[[:string_agg :a [:order-by [:inline ","] :a]]]] :from :table})
;;=> ["SELECT STRING_AGG(a, ',' ORDER BY a ASC) FROM table"]
(format (-> (select [[:string_agg :a [:order-by [:inline ","] :a]]])
(from :table)))
(sql/format (-> (select [[:string_agg :a [:order-by [:inline ","] :a]]])
(from :table)))
;;=> ["SELECT STRING_AGG(a, ',' ORDER BY a ASC) FROM table"]
```
@ -285,7 +291,7 @@ parameters from them:
(sql/format {:select [:a [[:raw ["@var := " [:inline "foo"]]]]]})
;;=> ["SELECT a, @var := 'foo'"]
(sql/format {:select [:a [[:raw ["@var := " ["foo"]]]]]})
;;=> ["SELECT a, @var := ?" "foo"]
;;=> ["SELECT a, @var := (?)" "foo"]
```
`:raw` is also supported as a SQL clause for the same reason.
@ -304,6 +310,7 @@ specifications).
If no arguments are provided, these render as just SQL
keywords (uppercase):
<!-- :test-doc-blocks/skip -->
```clojure
[:foreign-key] ;=> FOREIGN KEY
[:primary-key] ;=> PRIMARY KEY
@ -311,6 +318,7 @@ keywords (uppercase):
Otherwise, these render as regular function calls:
<!-- :test-doc-blocks/skip -->
```clojure
[:foreign-key :a] ;=> FOREIGN KEY(a)
[:primary-key :x :y] ;=> PRIMARY KEY(x, y)
@ -326,6 +334,7 @@ argument. If two or more arguments are provided, this
renders as a SQL keyword followed by the first argument,
followed by the rest as a regular argument list:
<!-- :test-doc-blocks/skip -->
```clojure
[:default] ;=> DEFAULT
[:default 42] ;=> DEFAULT 42
@ -339,6 +348,7 @@ followed by the rest as a regular argument list:
These behave like the group above except that if the
first argument is `nil`, it is omitted:
<!-- :test-doc-blocks/skip -->
```clojure
[:index :foo :bar :quux] ;=> INDEX foo(bar, quux)
[:index nil :bar :quux] ;=> INDEX(bar, quux)