From 4efa9a38effc10b11d65e9f7373b9e4e3da7242c Mon Sep 17 00:00:00 2001 From: Sean Corfield Date: Sat, 21 Sep 2024 18:12:10 -0700 Subject: [PATCH] address #539 by improving getting started Signed-off-by: Sean Corfield --- CHANGELOG.md | 1 + README.md | 17 ++++++++-------- doc/getting-started.md | 36 +++++++++++++++++++-------------- src/honey/sql/helpers.cljc | 2 +- src/honey/sql/pg_ops.cljc | 12 +++++------ src/honey/sql/protocols.cljc | 2 +- test/honey/bigquery_test.cljc | 2 +- test/honey/cache_test.clj | 2 +- test/honey/ops_test.cljc | 2 +- test/honey/sql/pg_ops_test.cljc | 2 +- test/honey/union_test.cljc | 2 +- 11 files changed, 44 insertions(+), 36 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fdfe1f9..d9cf04d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,7 @@ # Changes * 2.6.next in progress + * Getting Started updated based on feedback from Los Angeles Clojure meetup walkthrough [#539](https://github.com/seancorfield/honeysql/issues/539). * Update Clojure version to 1.12.0. * 2.6.1161 -- 2024-08-29 diff --git a/README.md b/README.md index eec21b9..64c24a1 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ This project follows the version scheme MAJOR.MINOR.COMMITS where MAJOR and MINO HoneySQL 2.x requires Clojure 1.9 or later. -Compared to 1.x, HoneySQL 2.x provides a streamlined codebase and a simpler method for extending the DSL. It also supports SQL dialects out-of-the-box and will be extended to support vendor-specific language features over time (unlike 1.x). +Compared to the [legacy 1.x version](#1.x), HoneySQL 2.x provides a streamlined codebase and a simpler method for extending the DSL. It also supports SQL dialects out-of-the-box and will be extended to support vendor-specific language features over time (unlike 1.x). > Note: you can use 1.x and 2.x side-by-side as they use different group IDs and different namespaces. This allows for a piecemeal migration. See this [summary of differences between 1.x and 2.x](doc/differences-from-1-x.md) if you are migrating from 1.x! @@ -35,12 +35,6 @@ Sample code in this documentation is verified via Some of these samples show pretty-printed SQL: HoneySQL 2.x supports `:pretty true` which inserts newlines between clauses in the generated SQL strings. -### HoneySQL 1.x (legacy) - -[![Clojars](https://img.shields.io/badge/clojars-honeysql_1.0.461-lightblue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAABjFBMVEUAAAAdCh0qDikdChwAAAAnDSY0EjM2FjUnDiYnDSYnDSYpDigyEDEEAQRGNUb///////8mDSYAAAAAAAAAAAAFAgUqEyoAAAAAAAAAAAAFAgUAAABXU1c2FjVMx+dQx+f///////9Nx+b////4/f6y4vRPt+RQtOT///9Qt+P///8oDSey4vRQr9/////3/P5hzelNx+dNx+dNx+f///8AAAAuDy0zETIAAAAoDScAAAAAAAARBREAAAAvDy40ETMwEC9gSF+Ne42ilKKuoK6Rg5B5ZXlaP1o4Gzf///9nTWZ4YncyEDF/bn/8/Pz9/P339/c1FTUlDCRRM1AbCRtlS2QyEDEuDy1gRWAxEDAzETIwEC/g4OAvDy40EjOaiZorDiq9sbzNyM3UzdQyEDE0ETMzETKflZ/UzdQ5Fzmu4fNYyuhNx+dPt+RLu9xQyOhBbo81GTuW2vCo4PJNx+c4MFE5N1lHiLFEhKQyEDGDboMzETI5Fjh5bXje2d57aHrIw8jc2NyWhJUrDioxe9o4AAAAPnRSTlMAkf+IAQj9+e7n6e31RtqAD/QAAAED+A0ZEQ8DwvkLBsmcR4aG8+cdAD6C8/MC94eP+qoTrgH+/wj1HA8eEvpXOCUAAAABYktHRA8YugDZAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3wcHFjou4Z/shwAAAUpJREFUOMul0/VTwzAUB/AAwyW4y3B3h8EDNuTh7u6UDHcd8I+TbHSjWdrjju/1h77kc+3Lu5aQvyakF/r6B5wu1+DQMEBomLRtG0EpozYDCEccA4iIjIqOiY0bB5iYxHgZ4FQCpYneKmmal0aQPMOXZnUAvJhLkbpInf8NFtKCTrGImK6DJcTlDGl/BXGV6oCsrSNIYAM3aQDwl2xJYBtBB5lZAuyYgWzY3YMcNcjN2wc4EGMEFTg8+hlyfgEenygAj71Q9FBExH0wKC4p1bRTJlJWXqEAVNM05ovbXfkPAHBmAUQPAGaAsXMBLiwA8z3h0gRcsWsObuAWLJu8Awb3ZoB5T8EvS/CgBo9Y5Z8TPwXBJwlUI9Ia/yRrEZ8lID71Olrf0MiamkkL4kurDEjba+C/e2sninR0wrsH8eMTvrqIWbodjh7jyjdtCY3Aniz4jwAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAxNS0wNy0wN1QyMjo1ODo0NiswMjowMCgWtSoAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMTUtMDctMDdUMjI6NTg6NDYrMDI6MDBZSw2WAAAAAElFTkSuQmCC)](https://clojars.org/honeysql/honeysql) [![cljdoc badge](https://cljdoc.org/badge/honeysql/honeysql?1.0.461)](https://cljdoc.org/d/honeysql/honeysql/CURRENT) - -HoneySQL 1.x will continue to get critical security fixes but otherwise should be considered "legacy" at this point. - ## Usage This section includes a number of usage examples but does not dive deep into the @@ -1015,8 +1009,15 @@ You can also register SQL clauses, specifying the keyword, the formatting functi If you find yourself registering an operator, a function (syntax), or a new clause, consider submitting a [pull request to HoneySQL](https://github.com/seancorfield/honeysql/pulls) so others can use it, too. If it is dialect-specific, let me know in the pull request. + +## HoneySQL 1.x (legacy) + +[![Clojars](https://img.shields.io/badge/clojars-honeysql_1.0.461-lightblue.svg?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAABjFBMVEUAAAAdCh0qDikdChwAAAAnDSY0EjM2FjUnDiYnDSYnDSYpDigyEDEEAQRGNUb///////8mDSYAAAAAAAAAAAAFAgUqEyoAAAAAAAAAAAAFAgUAAABXU1c2FjVMx+dQx+f///////9Nx+b////4/f6y4vRPt+RQtOT///9Qt+P///8oDSey4vRQr9/////3/P5hzelNx+dNx+dNx+f///8AAAAuDy0zETIAAAAoDScAAAAAAAARBREAAAAvDy40ETMwEC9gSF+Ne42ilKKuoK6Rg5B5ZXlaP1o4Gzf///9nTWZ4YncyEDF/bn/8/Pz9/P339/c1FTUlDCRRM1AbCRtlS2QyEDEuDy1gRWAxEDAzETIwEC/g4OAvDy40EjOaiZorDiq9sbzNyM3UzdQyEDE0ETMzETKflZ/UzdQ5Fzmu4fNYyuhNx+dPt+RLu9xQyOhBbo81GTuW2vCo4PJNx+c4MFE5N1lHiLFEhKQyEDGDboMzETI5Fjh5bXje2d57aHrIw8jc2NyWhJUrDioxe9o4AAAAPnRSTlMAkf+IAQj9+e7n6e31RtqAD/QAAAED+A0ZEQ8DwvkLBsmcR4aG8+cdAD6C8/MC94eP+qoTrgH+/wj1HA8eEvpXOCUAAAABYktHRA8YugDZAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3wcHFjou4Z/shwAAAUpJREFUOMul0/VTwzAUB/AAwyW4y3B3h8EDNuTh7u6UDHcd8I+TbHSjWdrjju/1h77kc+3Lu5aQvyakF/r6B5wu1+DQMEBomLRtG0EpozYDCEccA4iIjIqOiY0bB5iYxHgZ4FQCpYneKmmal0aQPMOXZnUAvJhLkbpInf8NFtKCTrGImK6DJcTlDGl/BXGV6oCsrSNIYAM3aQDwl2xJYBtBB5lZAuyYgWzY3YMcNcjN2wc4EGMEFTg8+hlyfgEenygAj71Q9FBExH0wKC4p1bRTJlJWXqEAVNM05ovbXfkPAHBmAUQPAGaAsXMBLiwA8z3h0gRcsWsObuAWLJu8Awb3ZoB5T8EvS/CgBo9Y5Z8TPwXBJwlUI9Ia/yRrEZ8lID71Olrf0MiamkkL4kurDEjba+C/e2sninR0wrsH8eMTvrqIWbodjh7jyjdtCY3Aniz4jwAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAxNS0wNy0wN1QyMjo1ODo0NiswMjowMCgWtSoAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMTUtMDctMDdUMjI6NTg6NDYrMDI6MDBZSw2WAAAAAElFTkSuQmCC)](https://clojars.org/honeysql/honeysql) [![cljdoc badge](https://cljdoc.org/badge/honeysql/honeysql?1.0.461)](https://cljdoc.org/d/honeysql/honeysql/CURRENT) + +HoneySQL 1.x will continue to get critical security fixes but otherwise should be considered "legacy" at this point. + ## License -Copyright (c) 2020-2022 Sean Corfield. HoneySQL 1.x was copyright (c) 2012-2020 Justin Kramer and Sean Corfield. +Copyright (c) 2020-2024 Sean Corfield. HoneySQL 1.x was copyright (c) 2012-2020 Justin Kramer and Sean Corfield. Distributed under the Eclipse Public License, the same as Clojure. diff --git a/doc/getting-started.md b/doc/getting-started.md index 56b535b..9d156ce 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -32,7 +32,7 @@ required -- using [John Shaffer](https://github.com/john-shaffer)'s awesome SQL statements are represented as hash maps, with keys that represent clauses in SQL. SQL expressions are generally -represented as sequences, where the first element identifies +represented as vectors, where the first element identifies the function or operator and the remaining elements are the arguments or operands. @@ -54,11 +54,13 @@ or symbols, are treated as positional parameters and replaced by `?` in the SQL string and lifted out into the vector that is returned from `format`. -Most clauses expect a sequence as their value, containing +Most clauses expect a vector as their value, containing either a list of SQL entities or the representation of a SQL expression. Some clauses accept a single SQL entity. A few -accept a more specialized form (such as `:set` accepting a -hash map of SQL entities and SQL expressions). +accept a more specialized form (such as `:set` within an `:update` clause +accepting a hash map of SQL entities and SQL expressions). + +> Note: clauses can have a list as their value, but literal vectors and keywords are easier to type without quoting. A SQL entity can be a simple keyword (or symbol) or a pair that represents a SQL entity and its alias (where aliases are allowed): @@ -86,6 +88,8 @@ avoid evaluation: ;;=> ["SELECT t.id, name AS item FROM table AS t WHERE id = ?" 1] ``` +> Note: these quoted forms may be appealing to users familiar with Datalog-family query languages, and they can be easier to type (and read) in some cases since you do not need to add `:` (shift-`;` on most keyboards) to the start of each SQL entity. The quoted forms do not work well in the [HoneySQL web app](https://john.shaffe.rs/honeysql/) so it's better to stick with vectors and keywords when using that. + If you wish, you can specify SQL entities as namespace-qualified keywords (or symbols) and the namespace portion will treated as the table name, i.e., `:foo/bar` instead of `:foo.bar`: @@ -101,8 +105,8 @@ the table name, i.e., `:foo/bar` instead of `:foo.bar`: ## SQL Expressions In addition to using hash maps to describe SQL clauses, -HoneySQL uses sequences to describe SQL expressions. Any -sequence that begins with a keyword (or symbol) is considered +HoneySQL uses vectors to describe SQL expressions. Any +vector that begins with a keyword (or symbol) is considered to be a kind of function invocation. Certain "functions" are considered to be "special syntax" and have custom rendering. Some "functions" are considered to be operators. In general, @@ -151,22 +155,22 @@ Some examples: [:now] ;=> "NOW()" [:count :*] ;=> "COUNT(*)" [:or [:<> :name nil] [:= :status-id 0]] ;=> "(name IS NOT NULL) OR (status_id = ?)" -;; with a parameter of 0 -- the nil value is inlined as NULL +;; the nil value is inlined as NULL but 0 is provided as a parameter ``` `:inline` is an example of "special syntax" and it renders its -(single) argument as part of the SQL string generated by `format`. +arguments as part of the SQL string generated by `format`. Another form of special syntax that is treated as function calls -is keywords or symbols that begin with `%`. Such keywords (or symbols) +is keywords or symbols that begin with `%`. Such keywords (or quoted symbols) are split at `.` and turned into function calls: ```clojure -%now ;=> NOW() -%count.* ;=> COUNT(*) -%max.foo ;=> MAX(foo) -%f.a.b ;=> F(a,b) +:%now ;=> NOW() +:%count.* ;=> COUNT(*) +:%max.foo ;=> MAX(foo) +:%f.a.b ;=> F(a,b) ``` If you need to reference a table or alias for a column, you can use @@ -200,6 +204,8 @@ expression requires an extra level of nesting: ;;=> ["SELECT x, y AS d, Z(e), Z(f) AS g"] (sql/format {:select [:x [:y :d] [:%z.e] [:%z.f :g]]}) ;;=> ["SELECT x, y AS d, Z(e), Z(f) AS g"] +(sql/format {:select [:x [:y :d] :%z.e [:%z.f :g]]}) +;;=> ["SELECT x, y AS d, Z(e), Z(f) AS g"] ``` ## SQL Parameters @@ -255,7 +261,7 @@ Or with `:numbered true`: ## Functional Helpers -In addition to the hash map (and sequences) approach of building +In addition to the hash map (and vectors) approach of building SQL queries with raw Clojure data structures, a [namespace full of helper functions](https://cljdoc.org/d/com.github.seancorfield/honeysql/CURRENT/api/honey.sql.helpers) is also available. These functions are generally variadic and threadable: @@ -348,7 +354,7 @@ The most visible difference between dialects is how SQL entities should be quoted (if the `:quoted true` option is provided to `format`). Most databases use `"` for quoting (the `:ansi` and `:oracle` dialects). The `:sqlserver` dialect uses `[`..`]` and the `:mysql` dialect uses -```..```. In addition, the `:oracle` dialect disables `AS` in aliases. +```` .. ````. In addition, the `:oracle` dialect disables `AS` in aliases. > Note: by default, quoting is **off** which produces cleaner-looking SQL and assumes you control all the symbols/keywords used as table, column, and function names -- the "SQL entities". If you are building any SQL or DDL where the table, column, or function names could be provided by an external source, **you should specify `:quoted true` to ensure all SQL entities are safely quoted**. As of 2.3.928, if you do _not_ specify `:quoted` as an option, HoneySQL will automatically quote any SQL entities that seem unusual, i.e., that contain any characters that are not alphanumeric or underscore. Purely alphanumeric entities will not be quoted (no entities were quoted by default prior to 2.3.928). You can prevent that auto-quoting by explicitly passing `:quoted false` into the `format` call but, from a security point of view, you should think very carefully before you do that: quoting entity names helps protect you from injection attacks! As of 2.4.947, you can change the default setting of `:quoted` from `nil` to `true` (or `false`) via the `set-options!` function. diff --git a/src/honey/sql/helpers.cljc b/src/honey/sql/helpers.cljc index 6c597a8..5654813 100644 --- a/src/honey/sql/helpers.cljc +++ b/src/honey/sql/helpers.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2020-2023 sean corfield, all rights reserved +;; copyright (c) 2020-2024 sean corfield, all rights reserved (ns honey.sql.helpers "Helper functions for the built-in clauses in honey.sql. diff --git a/src/honey/sql/pg_ops.cljc b/src/honey/sql/pg_ops.cljc index 62021ec..48b6691 100644 --- a/src/honey/sql/pg_ops.cljc +++ b/src/honey/sql/pg_ops.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2022 sean corfield, all rights reserved +;; copyright (c) 2022-2024 sean corfield, all rights reserved (ns honey.sql.pg-ops "Register all the PostgreSQL JSON/JSONB operators @@ -29,12 +29,12 @@ (def -> "The -> operator for accessing nested JSON(B) values as JSON(B). - Ex.: + Ex.: ```clojure (sql/format {:select [[[:->> [:-> :my_column \"kids\" [:inline 0]] \"name\"]]]}) ; => [\"SELECT (my_column -> ? -> 0) ->> ?\" \"kids\" \"name\"] ``` - + Notice we need to wrap the keys/indices with :inline if we don't want them to become parameters." :->) (def ->> "The ->> operator - like -> but returns the value as text instead of a JSON object." :->>) @@ -47,7 +47,7 @@ (def ?& "The ?& operator - do all of the strings in the text array exist as top-level keys or array elements?" :?&) (def || "The || operator - concatenates two jsonb values (arrays or objects; anything else treated as 1-element array)." :||) (def - - "The - operator: + "The - operator: - text value: deletes a key (and its value) from a JSON object, or matching string value(s) from a JSON array - text[] array value: as above, but for all the provided keys - int value: deletes the array element with specified index (negative integers count from the end)" @@ -56,9 +56,9 @@ (def at? "The @? operator - does JSON path return any item for the specified JSON value?" (keyword "@?")) (def atat "The @@ operator: - - returns the result of a JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. + - returns the result of a JSON path predicate check for the specified JSON value. Only the first item of the result is taken into account. If the result is not Boolean, then NULL is returned. - - checks if a text search vector (or a text value implicitly converted to a text search vector) matches a text search query. Returns a Boolean." + - checks if a text search vector (or a text value implicitly converted to a text search vector) matches a text search query. Returns a Boolean." (keyword "@@")) (def tilde "The case-sensitive regex match operator." (keyword "~")) diff --git a/src/honey/sql/protocols.cljc b/src/honey/sql/protocols.cljc index 4cf6081..13c9ca0 100644 --- a/src/honey/sql/protocols.cljc +++ b/src/honey/sql/protocols.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2022 sean corfield, all rights reserved +;; copyright (c) 2022-2024 sean corfield, all rights reserved (ns honey.sql.protocols "InlineValue -- a protocol that defines how to inline diff --git a/test/honey/bigquery_test.cljc b/test/honey/bigquery_test.cljc index 831c3d9..5dc3a16 100644 --- a/test/honey/bigquery_test.cljc +++ b/test/honey/bigquery_test.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2022 sean corfield, all rights reserved +;; copyright (c) 2022-2024 sean corfield, all rights reserved (ns honey.bigquery-test (:refer-clojure :exclude [format]) diff --git a/test/honey/cache_test.clj b/test/honey/cache_test.clj index 3830e7f..15f361f 100644 --- a/test/honey/cache_test.clj +++ b/test/honey/cache_test.clj @@ -1,4 +1,4 @@ -;; copyright (c) 2022 sean corfield, all rights reserved +;; copyright (c) 2022-2024 sean corfield, all rights reserved (ns honey.cache-test (:refer-clojure :exclude [format group-by]) diff --git a/test/honey/ops_test.cljc b/test/honey/ops_test.cljc index 24d9e8d..6c84c1f 100644 --- a/test/honey/ops_test.cljc +++ b/test/honey/ops_test.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2023 sean corfield, all rights reserved +;; copyright (c) 2023-2024 sean corfield, all rights reserved (ns honey.ops-test (:refer-clojure :exclude [format]) diff --git a/test/honey/sql/pg_ops_test.cljc b/test/honey/sql/pg_ops_test.cljc index a310b79..8199cd7 100644 --- a/test/honey/sql/pg_ops_test.cljc +++ b/test/honey/sql/pg_ops_test.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2022 sean corfield, all rights reserved +;; copyright (c) 2022-2024 sean corfield, all rights reserved (ns honey.sql.pg-ops-test (:require [clojure.test :refer [deftest is testing]] diff --git a/test/honey/union_test.cljc b/test/honey/union_test.cljc index 695e9ec..3d1df2c 100644 --- a/test/honey/union_test.cljc +++ b/test/honey/union_test.cljc @@ -1,4 +1,4 @@ -;; copyright (c) 2023 sean corfield, all rights reserved +;; copyright (c) 2023-2024 sean corfield, all rights reserved (ns honey.union-test (:refer-clojure :exclude [format])