From f0c835347868680fd7a9949100d639ce790bd853 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Wed, 20 Sep 2017 20:20:55 -0400 Subject: [PATCH 01/13] Add file for documentation about recursive path. --- Using-Specter-Recursively.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 Using-Specter-Recursively.md diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md new file mode 100644 index 0000000..e69de29 From 168beaf1ae026315b2ab2146904618a154913447 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Wed, 20 Sep 2017 20:25:45 -0400 Subject: [PATCH 02/13] Write initial intro. --- Using-Specter-Recursively.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index e69de29..3b2bb61 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -0,0 +1 @@ +Specter is useful for navigating through nested data structures, and then returning (selecting) or transforming what it finds. Specter can also work its magic **recursively**. Many of Specter's most important and powerful use cases in your codebase will require you to use Specter's recursive features. However, just as recursion can be difficult to grok at first, using Specter recursively can be challenging. This guide is designed to help you feel at home using Specter recursively. From 94e0df85b205411f5c4d4bcddaa4f1d548e2311d Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Wed, 20 Sep 2017 20:28:31 -0400 Subject: [PATCH 03/13] Add sections. --- Using-Specter-Recursively.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 3b2bb61..69be46a 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -1 +1,5 @@ Specter is useful for navigating through nested data structures, and then returning (selecting) or transforming what it finds. Specter can also work its magic **recursively**. Many of Specter's most important and powerful use cases in your codebase will require you to use Specter's recursive features. However, just as recursion can be difficult to grok at first, using Specter recursively can be challenging. This guide is designed to help you feel at home using Specter recursively. + +# A Review of Recursion +# Using Specter Recursively +# Applications From 1e206091c10df2435abb3cfaa2df58598a676ee7 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Thu, 21 Sep 2017 08:32:12 -0400 Subject: [PATCH 04/13] Begin explanation. --- Using-Specter-Recursively.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 69be46a..4727d18 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -1,5 +1,37 @@ Specter is useful for navigating through nested data structures, and then returning (selecting) or transforming what it finds. Specter can also work its magic **recursively**. Many of Specter's most important and powerful use cases in your codebase will require you to use Specter's recursive features. However, just as recursion can be difficult to grok at first, using Specter recursively can be challenging. This guide is designed to help you feel at home using Specter recursively. # A Review of Recursion + +Most simple functions do not call themselves. For example: + +```clojure +(defn square [x] (* x x)) +``` + +Instead of calling `square`, itself, `square` calls another function, `*` or multiply. This, of course, serves the purpose of squaring the input. You don't always need recursion. But sometimes you need a function to call itself - to recur. To take another example from mathematics, you need recursion to implement factorials. Factorials are equal to the product of every positive number equal to or less than the number. For example, the factorial of 5 ("5!") is `5 * 4 * 3 * 2 * 1 = 120`. + +Here is a simple implementation of factorials in Clojure: + +```clojure +=> (defn factorial [x] + (if (< x 2) + 1 + (* x (factorial (dec x))))) +#'playground.specter/factorial +=> (factorial 5) +120 +``` + +Here, the function definition *calls itself* with the symbol `factorial`. You still supply input - the value x - but producing the function's return value requires calling the function itself. + +So there are two elements to basic recursion: a name or symbol for the recursion point or function, and input arguments. + # Using Specter Recursively + +Using Specter recursively uses these two elements: a name (`self-sym` or self-symbol), and arguments (`params` or parameters). But it adds a third element: the `path` that the + +This is actually similar to the function body. + +requires **three elements**: a name (self-sym), arguments + # Applications From e33a84aaf7e108a3d241bea76a8b735283a68dc1 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Thu, 21 Sep 2017 13:28:54 -0400 Subject: [PATCH 05/13] Better explanation of elements of recursion. --- Using-Specter-Recursively.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 4727d18..893c717 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -14,7 +14,7 @@ Here is a simple implementation of factorials in Clojure: ```clojure => (defn factorial [x] - (if (< x 2) + (if (< x 2) 1 (* x (factorial (dec x))))) #'playground.specter/factorial @@ -24,7 +24,10 @@ Here is a simple implementation of factorials in Clojure: Here, the function definition *calls itself* with the symbol `factorial`. You still supply input - the value x - but producing the function's return value requires calling the function itself. -So there are two elements to basic recursion: a name or symbol for the recursion point or function, and input arguments. +We can see that there are up to three elements to basic recursion: + * a name or symbol for the recursion point or function (defn function-name) + * optional input arguments + * the body of the recursion - what you **do** recursively - and, at some point, calling the function by its name # Using Specter Recursively From 25efdf1741a840f1139aa2aa093aed8f43e2c726 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Thu, 21 Sep 2017 13:29:34 -0400 Subject: [PATCH 06/13] Add TOC. --- Using-Specter-Recursively.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 893c717..bd1f849 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -1,5 +1,14 @@ Specter is useful for navigating through nested data structures, and then returning (selecting) or transforming what it finds. Specter can also work its magic **recursively**. Many of Specter's most important and powerful use cases in your codebase will require you to use Specter's recursive features. However, just as recursion can be difficult to grok at first, using Specter recursively can be challenging. This guide is designed to help you feel at home using Specter recursively. + +**Table of Contents** + +- [A Review of Recursion](#a-review-of-recursion) +- [Using Specter Recursively](#using-specter-recursively) +- [Applications](#applications) + + + # A Review of Recursion Most simple functions do not call themselves. For example: From fd8b0068cd5b05650c411823612002df85cde93d Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Thu, 21 Sep 2017 13:33:47 -0400 Subject: [PATCH 07/13] Better explanation of using Specter recursively. Also, QUESTION. --- Using-Specter-Recursively.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index bd1f849..5f06459 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -40,10 +40,12 @@ We can see that there are up to three elements to basic recursion: # Using Specter Recursively -Using Specter recursively uses these two elements: a name (`self-sym` or self-symbol), and arguments (`params` or parameters). But it adds a third element: the `path` that the +The core way that Specter enables recursion is through the parameterized macro, `recursive-path`. `recursive-path` takes the following form: -This is actually similar to the function body. +`(recursive-path params self-sym path)` -requires **three elements**: a name (self-sym), arguments +You can see that there are three arguments, which match onto our list above. (They are, however, in a different order.) The `self-sym` is the name; the `params` are the optional input arguments; and the `path` is the body of the recursion. This path can be recursive, referencing itself by the `self-sym`. + +(NOTE: QUESTION: Is there a reason why the self-sym is the second argument?) # Applications From b54b5e631e38afc6d86aa08d76863123d87f3320 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Thu, 21 Sep 2017 13:40:27 -0400 Subject: [PATCH 08/13] Add advice about storing recursive paths. --- Using-Specter-Recursively.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 5f06459..ffb36ea 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -40,7 +40,7 @@ We can see that there are up to three elements to basic recursion: # Using Specter Recursively -The core way that Specter enables recursion is through the parameterized macro, `recursive-path`. `recursive-path` takes the following form: +The main way that Specter enables recursion is through the parameterized macro, `recursive-path`. `recursive-path` takes the following form: `(recursive-path params self-sym path)` @@ -48,4 +48,6 @@ You can see that there are three arguments, which match onto our list above. (Th (NOTE: QUESTION: Is there a reason why the self-sym is the second argument?) +Rather than providing a recursive function, `recursive-path` provides a recursive path that can be composed with other navigators. Accordingly, it is often useful - but not necessary - to define a recursive-path using `def`, so that it can be called in multiple places, not just the place where you initially need it. + # Applications From d3b4c51dd5d8ecee0cbeb622555c84893bd6c77a Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Sat, 30 Sep 2017 12:50:42 -0400 Subject: [PATCH 09/13] Add description. --- Using-Specter-Recursively.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index ffb36ea..30c7a55 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -11,13 +11,17 @@ Specter is useful for navigating through nested data structures, and then return # A Review of Recursion +Before we review how recursion works with Specter, it might be worth a brief refresher on recursion more broadly. If you're familiar with this material, feel free to skip this section. But a brief review will allow us to separate reviewing the concept of recursion from learning how to combine that concept with Specter's functionality. + Most simple functions do not call themselves. For example: ```clojure (defn square [x] (* x x)) ``` -Instead of calling `square`, itself, `square` calls another function, `*` or multiply. This, of course, serves the purpose of squaring the input. You don't always need recursion. But sometimes you need a function to call itself - to recur. To take another example from mathematics, you need recursion to implement factorials. Factorials are equal to the product of every positive number equal to or less than the number. For example, the factorial of 5 ("5!") is `5 * 4 * 3 * 2 * 1 = 120`. +Instead of calling `square`, itself, `square` calls another function, `*` or multiply. This, of course, serves the purpose of squaring the input. You don't always need recursion. + +But sometimes you need a function to call itself - to recur. To take another example from mathematics, you need recursion to implement factorials. Factorials are equal to the product of every positive number equal to or less than the number. For example, the factorial of 5 ("5!") is `5 * 4 * 3 * 2 * 1 = 120`. Here is a simple implementation of factorials in Clojure: From b10e5cf9875894c278deeea21f2f94d2abe2ebfa Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Sat, 30 Sep 2017 13:07:33 -0400 Subject: [PATCH 10/13] Add examples. --- Using-Specter-Recursively.md | 71 ++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 30c7a55..d5a6bd2 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -6,6 +6,9 @@ Specter is useful for navigating through nested data structures, and then return - [A Review of Recursion](#a-review-of-recursion) - [Using Specter Recursively](#using-specter-recursively) - [Applications](#applications) + - [Navigate to all values in a tree](#navigate-to-all-values-in-a-tree) + - [Navigate to all of the instances of one key in a map](#navigate-to-all-of-the-instances-of-one-key-in-a-map) + - [Find the "index route" of a value within a data structure](#find-the-index-route-of-a-value-within-a-data-structure) @@ -55,3 +58,71 @@ You can see that there are three arguments, which match onto our list above. (Th Rather than providing a recursive function, `recursive-path` provides a recursive path that can be composed with other navigators. Accordingly, it is often useful - but not necessary - to define a recursive-path using `def`, so that it can be called in multiple places, not just the place where you initially need it. # Applications + +Here are some examples of using Specter recursively with `recursive-path`. + +## Navigate to all values in a tree + +```clojure +=> (def tree-walker (recursive-path [] p (if-path vector? [ALL p] STAY))) +#'playground.specter/tree-walker +;; Get all of the values nested within vectors +=> (select tree-walker [1 [2 [3 4] 5] [[6]]]) +[1 2 3 4 5 6] +;; Transform all of the values within vectors +=> (transform tree-walker inc [1 [2 [3 4] 5] [[6]]]) +[2 [3 [4 5] 6] [[7]]] +;; Increment even leaves +=> (transform [tree-walker even?] inc [1 [2 [3 4] 5] [[6]]]) +[1 [3 [3 5] 5] [[7]]] +;; Get odd leaves +=> (select [tree-walker odd?] [1 [2 [3 4] 5] [[6]]]) +;; => [1 3 5] +;; Reverse order of even leaves (order based on depth-first search) +=> (transform (subselect tree-walker even?) reverse [1 [2 [3 4] 5] [[6]]]) +[1 [6 [3 4] 5] [[2]]] +``` + +## Navigate to all of the instances of one key in a map + +```clojure +=> (def map-key-walker (recursive-path [akey] p [ALL (if-path [FIRST #(= % akey)] LAST [LAST p])])) +#'playground.specter/map-key-walker +;; Get all the vals for key :aaa, regardless of where they are in the structure +=> (select (map-key-walker :aaa) {:a {:aaa 3 :b {:c {:aaa 2} :aaa 1}}}) +[3 2 1] +;; Transform all the vals for key :aaa, regardless of where they are in the structure +=> (transform (map-key-walker :aaa) inc {:a {:aaa 3 :b {:c {:aaa 2} :aaa 1}}}) +{:a {:aaa 4, :b {:c {:aaa 3}, :aaa 2}}} +``` + +## Find the "index route" of a value within a data structure + +This example comes from [a Stack Overflow question](https://stackoverflow.com/questions/45764946/how-to-find-indexes-in-deeply-nested-data-structurevectors-and-lists-in-clojur). + +```clojure +=> (defn find-index-route [v data] + (let [walker (recursive-path [] p + (if-path sequential? + [INDEXED-VALS + (if-path [LAST (pred= v)] + FIRST + [(collect-one FIRST) LAST p])])) + ret (select-first walker data)] + (if (or (vector? ret) (nil? ret)) ret [ret]))) +#'playground.specter/find-index-route +=> (find-index-route :my-key '(1 2 :my-key)) +[2] +=> (find-index-route :my-key '(1 2 "a" :my-key "b")) +[3] +=> (find-index-route :my-key '(1 2 [:my-key] "c")) +[2 0] +=> (find-index-route :my-key '(1 2 [3 [:my-key]])) +[2 1 0] +=> (find-index-route :my-key '(1 2 [3 [[] :my-key]])) +[2 1 1] +=> (find-index-route :my-key '(1 2 [3 [4 5 6 (:my-key)]])) +[2 1 3 0] +=> (find-index-route :my-key '(1 2 [3 [[]]])) +nil +``` From 6f2ed10907d97c3ccd897abf1ea9e267f66965f1 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Sun, 1 Oct 2017 12:50:50 -0400 Subject: [PATCH 11/13] Remove question. --- Using-Specter-Recursively.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index d5a6bd2..b0f3d25 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -53,8 +53,6 @@ The main way that Specter enables recursion is through the parameterized macro, You can see that there are three arguments, which match onto our list above. (They are, however, in a different order.) The `self-sym` is the name; the `params` are the optional input arguments; and the `path` is the body of the recursion. This path can be recursive, referencing itself by the `self-sym`. -(NOTE: QUESTION: Is there a reason why the self-sym is the second argument?) - Rather than providing a recursive function, `recursive-path` provides a recursive path that can be composed with other navigators. Accordingly, it is often useful - but not necessary - to define a recursive-path using `def`, so that it can be called in multiple places, not just the place where you initially need it. # Applications From 24e86fdab6a4e493c625aac449be167af93b2bcb Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Sun, 1 Oct 2017 13:36:56 -0400 Subject: [PATCH 12/13] Rename TREE-VALUES. --- Using-Specter-Recursively.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index b0f3d25..55039a8 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -62,22 +62,22 @@ Here are some examples of using Specter recursively with `recursive-path`. ## Navigate to all values in a tree ```clojure -=> (def tree-walker (recursive-path [] p (if-path vector? [ALL p] STAY))) -#'playground.specter/tree-walker +=> (def TREE-VALUES (recursive-path [] p (if-path vector? [ALL p] STAY))) +#'playground.specter/TREE-VALUES ;; Get all of the values nested within vectors -=> (select tree-walker [1 [2 [3 4] 5] [[6]]]) +=> (select TREE-VALUES [1 [2 [3 4] 5] [[6]]]) [1 2 3 4 5 6] ;; Transform all of the values within vectors -=> (transform tree-walker inc [1 [2 [3 4] 5] [[6]]]) +=> (transform TREE-VALUES inc [1 [2 [3 4] 5] [[6]]]) [2 [3 [4 5] 6] [[7]]] ;; Increment even leaves -=> (transform [tree-walker even?] inc [1 [2 [3 4] 5] [[6]]]) +=> (transform [TREE-VALUES even?] inc [1 [2 [3 4] 5] [[6]]]) [1 [3 [3 5] 5] [[7]]] ;; Get odd leaves -=> (select [tree-walker odd?] [1 [2 [3 4] 5] [[6]]]) +=> (select [TREE-VALUES odd?] [1 [2 [3 4] 5] [[6]]]) ;; => [1 3 5] ;; Reverse order of even leaves (order based on depth-first search) -=> (transform (subselect tree-walker even?) reverse [1 [2 [3 4] 5] [[6]]]) +=> (transform (subselect TREE-VALUES even?) reverse [1 [2 [3 4] 5] [[6]]]) [1 [6 [3 4] 5] [[2]]] ``` From 48c066a28b507003731feb1ab5cc096f53a7b9b1 Mon Sep 17 00:00:00 2001 From: Michael Fogleman Date: Mon, 2 Oct 2017 07:20:43 -0400 Subject: [PATCH 13/13] Add more thorough explanation of TREE-VALUES. Via http://nathanmarz.com/blog/clojures-missing-piece.html --- Using-Specter-Recursively.md | 72 +++++++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 18 deletions(-) diff --git a/Using-Specter-Recursively.md b/Using-Specter-Recursively.md index 55039a8..cfc6875 100644 --- a/Using-Specter-Recursively.md +++ b/Using-Specter-Recursively.md @@ -5,8 +5,8 @@ Specter is useful for navigating through nested data structures, and then return - [A Review of Recursion](#a-review-of-recursion) - [Using Specter Recursively](#using-specter-recursively) +- [A Basic Example](#a-basic-example) - [Applications](#applications) - - [Navigate to all values in a tree](#navigate-to-all-values-in-a-tree) - [Navigate to all of the instances of one key in a map](#navigate-to-all-of-the-instances-of-one-key-in-a-map) - [Find the "index route" of a value within a data structure](#find-the-index-route-of-a-value-within-a-data-structure) @@ -55,32 +55,68 @@ You can see that there are three arguments, which match onto our list above. (Th Rather than providing a recursive function, `recursive-path` provides a recursive path that can be composed with other navigators. Accordingly, it is often useful - but not necessary - to define a recursive-path using `def`, so that it can be called in multiple places, not just the place where you initially need it. -# Applications +# A Basic Example -Here are some examples of using Specter recursively with `recursive-path`. - -## Navigate to all values in a tree +Suppose you had a tree represented using vectors: + +```clojure +=> (def tree [1 [2 [[3]] 4] [[5] 6] [7] 8 [[9]]]) +#'playground.specter/tree +``` + +You can define a navigator to all the leaves of the tree like this: + +```clojure +=> (def TREE-VALUES + (recursive-path [] p + (if-path vector? + [ALL p] + STAY))) +``` + +`recursive-path` allows your path definition to refer to itself, in this case using `p`. This path definition leverages the `if-path` navigator which uses the predicate to determine which path to continue on. If currently navigated to a vector, it recurses navigation at all elements of the vector. Otherwise, it uses `STAY` to stop traversing and finish navigation at the current point. The effect of all this is a depth-first traversal to each leaf node. + +Now we can get all of the values nested within vectors: ```clojure -=> (def TREE-VALUES (recursive-path [] p (if-path vector? [ALL p] STAY))) -#'playground.specter/TREE-VALUES -;; Get all of the values nested within vectors => (select TREE-VALUES [1 [2 [3 4] 5] [[6]]]) [1 2 3 4 5 6] -;; Transform all of the values within vectors +``` + +Or transform all of those values: + +```clojure => (transform TREE-VALUES inc [1 [2 [3 4] 5] [[6]]]) [2 [3 [4 5] 6] [[7]]] -;; Increment even leaves -=> (transform [TREE-VALUES even?] inc [1 [2 [3 4] 5] [[6]]]) -[1 [3 [3 5] 5] [[7]]] -;; Get odd leaves -=> (select [TREE-VALUES odd?] [1 [2 [3 4] 5] [[6]]]) -;; => [1 3 5] -;; Reverse order of even leaves (order based on depth-first search) -=> (transform (subselect TREE-VALUES even?) reverse [1 [2 [3 4] 5] [[6]]]) -[1 [6 [3 4] 5] [[2]]] ``` +You can also compose `TREE-VALUES` with other navigators to do all sorts of manipulations. For example, you can increment even leaves: + +```clojure +=> (transform [TREE-VALUES even?] inc tree) +[1 [3 [[3]] 5] [[5] 7] [7] 9 [[9]]] +``` + +Or get the odd leaves: + +```clojure +=> (select [TREE-VALUES odd?] tree) +[1 3 5 7 9] +``` + +Or reverse the order of the even leaves (where the order is based on a depth-first search): + +```clojure +=> (transform (subselect TREE-VALUES even?) reverse tree) +[1 [8 [[3]] 6] [[5] 4] [7] 2 [[9]]] +``` + +That you can define how to get to the values you care about once and easily reuse that logic for both querying and transformation is invaluable. And, as always, the performance is near-optimal for both querying and transformation. + +# Applications + +Here are some other examples of using Specter recursively with `recursive-path`. + ## Navigate to all of the instances of one key in a map ```clojure