Merge branch 'master' into gh-pages

.gitignore

@@ -9,4 +9,5 @@ pom.xml.asc
 .lein-repl-history
 .lein-plugins/
 .lein-failures
+.cljs_node_repl
 out/

.travis.yml

@@ -1,3 +1,8 @@
 language: clojure
+lein: 2.7.1
+cache:
+  directories:
+  - $HOME/.m2
 script:
   - lein test
+  - lein doo node test-build once

CHANGES.md

@@ -1,4 +1,30 @@
-## 0.13.3-SNAPSHOT
+## 1.0.3-SNAPSHOT
+
+## 1.0.2
+
+* Added `pred=`, `pred<`, `pred>`, `pred<=`, `pred>=` for filtering using common comparisons
+* Add `map-key` navigator
+* Add `set-elem` navigator
+* Add `ALL-WITH-META` navigator
+* `walker` and `codewalker` can now be used with `NONE` to remove elements
+* Improve `walker` performance by 70% by replacing clojure.walk implementation with custom recursive path
+* Extend `ALL` to work on records (navigate to key/value pairs)
+* Add ability to declare a function for end index of `srange-dynamic` that takes in the result of the start index fn. Use `end-fn` macro to declare this function (takes in 2 args of [collection, start-index]). Functions defined with normal mechanisms (e.g. `fn`) will still only take in the collection as an argument.
+* Workaround for ClojureScript bug that emits warnings for vars named the same as a private var in cljs.core (in this case `NONE`, added as private var to cljs.core with 1.9.562)
+* For ALL transforms on maps, interpret transformed key/value pair of size < 2 as removal
+* Bug fix: Fix incorrect inline compilation when a dynamic function invocation is nested in a data structure within a parameter to a navigator builder
+
+## 1.0.1
+
+* `subselect`/`filterer` can remove entries in source by transforming to a smaller sequence
+* Add `satisfies-protpath?`
+* Inline cache vars are marked private so as not to interfere with tooling
+* Improve performance of `ALL` transform on lists by 20%
+* Bug fix: Using `pred` no longer inserts unnecessary `coerce-nav` call at callsite
+* Bug fix: Dynamic navs in argument position to another nav now properly expanded and compiled
+* Bug fix: Dynamic parameters nested inside data structures as arguments are now compiled correctly by inline compiler
+
+## 1.0.0
 
 * Transform to `com.rpl.specter/NONE` to remove elements from data structures. Works with `keypath` (for both sequences and maps), `must`, `nthpath`, `ALL`, `MAP-VALS`, `FIRST`, and `LAST`
 * Add `nthpath` navigator

DEVELOPER.md

@@ -7,8 +7,6 @@ lein cleantest
 # Running ClojureScript tests
 
 ```
-rm -rf .cljs_node_repl
 lein javac
-rlwrap java -cp `lein classpath` clojure.main repl.clj
+lein doo node test-build once
-(require 'com.rpl.specter.cljs-test-runner)
 ```

README.md

@@ -80,7 +80,7 @@ The latest release version of Specter is hosted on [Clojars](https://clojars.org
 
 # Learn Specter
 
-- Introductory blog post: [Functional-navigational programming in Clojure(Script) with Specter](http://nathanmarz.com/blog/functional-navigational-programming-in-clojurescript-with-sp.html)
+- Introductory blog post: [Clojure's missing piece](http://nathanmarz.com/blog/clojures-missing-piece.html)
 - Presentation about Specter: [Specter: Powerful and Simple Data Structure Manipulation](https://www.youtube.com/watch?v=VTCy_DkAJGk)
   - Note that this presentation was given before Specter's inline compilation/caching system was developed. You no longer need to do anything special to get near-optimal performance.
 - List of navigators with examples: [This wiki page](https://github.com/nathanmarz/specter/wiki/List-of-Navigators) provides a more comprehensive overview than the API docs about the behavior of specific navigators and includes many examples.
@@ -90,8 +90,7 @@ The latest release version of Specter is hosted on [Clojars](https://clojars.org
 
 Specter's API is contained in these files:
 
-- [macros.clj](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/macros.clj): This contains the core `select/transform/etc.` operations as well as macros for defining new navigators.
+- [specter.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter.cljc): This contains the built-in navigators and the definition of the core operations.
-- [specter.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter.cljc): This contains the built-in navigators and functional versions of `select/transform/etc.`
 - [transients.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/transients.cljc): This contains navigators for transient collections.
 - [zipper.cljc](https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/zipper.cljc): This integrates zipper-based navigation into Specter.
 
@@ -193,7 +192,7 @@ user> (transform [(srange 4 11) (filterer even?)]
 Append [:c :d] to every subsequence that has at least two even numbers:
 ```clojure
 user> (setval [ALL
-               (selected? (filterer even?) (view count) #(>= % 2))
+               (selected? (filterer even?) (view count) (pred>= 2))
                END]
               [:c :d]
               [[1 2 3 4 5 6] [7 0 -1] [8 8] []])
@@ -293,6 +292,15 @@ Here's how to reverse the positions of all even numbers in a tree (with order ba
 ;; => [1 10 [3 [[8]] 5] [6 [7 4] 9 [[2]]]]
 ```
 
+# ClojureScript
+
+Specter supports ClojureScript! However, some of the differences between Clojure and ClojureScript affect how you use Specter in ClojureScript, in particular with the namespace declarations. In Clojure, you might `(use 'com.rpl.specter)` or say `(:require [com.rpl.specter :refer :all])` in your namespace declaration. But in ClojureScript, these options [aren't allowed](https://groups.google.com/d/msg/clojurescript/SzYK08Oduxo/MxLUjg50gQwJ). Instead, consider using one of these options:
+
+```clojure
+(:require [com.rpl.specter :as s])
+(:require [com.rpl.specter :as s :refer-macros [select transform]]) ;; add in the Specter macros that you need
+```
+
 # Future work
 - Integrate Specter with other kinds of data structures, such as graphs. Desired navigations include: reduction in topological order, navigate to outgoing/incoming nodes, to a subgraph (with metadata indicating how to attach external edges on transformation), to node attributes, to node values, to specific nodes.
 - Make it possible to parallelize selects/transforms

VERSION

@@ -1 +1 @@
-1.0.0
+1.0.3-SNAPSHOT

project.clj

@@ -9,7 +9,8 @@
   :test-paths ["test", "target/test-classes"]
   :auto-clean false
   :dependencies [[riddley "0.1.12"]]
-  :plugins [[lein-codox "0.9.5"]]
+  :plugins [[lein-codox "0.9.5"]
+            [lein-doo "0.1.7"]]
   :codox {:source-paths ["target/classes" "src/clj"]
           :namespaces [com.rpl.specter
                        com.rpl.specter.zipper
@@ -20,8 +21,15 @@
              #".*"             "https://github.com/nathanmarz/specter/tree/{version}/src/clj/{classpath}#L{line}"}}
 
 
+  :cljsbuild {:builds [{:id "test-build"
+                        :source-paths ["src/clj" "target/classes" "test"]
+                        :compiler {:output-to "out/testable.js"
+                                   :main 'com.rpl.specter.cljs-test-runner
+                                   :target :nodejs
+                                   :optimizations :none}}]}
+
   :profiles {:dev {:dependencies
-                   [[org.clojure/test.check "0.7.0"]
+                   [[org.clojure/test.check "0.9.0"]
                     [org.clojure/clojure "1.8.0"]
                     [org.clojure/clojurescript "1.9.229"]]}
 

scripts/benchmarks.clj

@@ -1,8 +1,8 @@
 (ns com.rpl.specter.benchmarks
   (:use [com.rpl.specter]
-        [com.rpl.specter.transients]
+        [com.rpl.specter.transients])
-        [com.rpl.specter.impl :only [benchmark]])
+  (:require [clojure.walk :as walk]
-  (:require [clojure.walk :as walk]))
+            [com.rpl.specter.impl :as i]))
 
 
 ;; run via `lein repl` with `(load-file "scripts/benchmarks.clj")`
@@ -55,7 +55,7 @@
        (println "\n********************************\n"))))
 
 (defn specter-dynamic-nested-get [data a b c]
-  (select-any [(keypath a) (keypath b) (keypath c)] data))
+  (select-any (keypath a b c) data))
 
 
 (defn get-k [k] (fn [m next] (next (get m k))))
@@ -85,7 +85,6 @@
     (select-any [(keypath :a) (keypath :b) (keypath :c)] data)))
 
 
-
 (let [data {:a {:b {:c 1}}}]
   (run-benchmark "set value in nested map" 2500000
     (assoc-in data [:a :b :c] 1)
@@ -135,6 +134,13 @@
           ret)))))
 
 
+(let [data '(1 2 3 4 5)]
+  (run-benchmark "transform values of a list" 500000
+    (transform ALL inc data)
+    (doall (sequence (map inc) data))
+    (reverse (into '() (map inc) data))
+    ))
+
 (let [data {:a 1 :b 2 :c 3 :d 4}]
   (run-benchmark "transform values of a small map" 500000
     (into {} (for [[k v] data] [k (inc v)]))
@@ -145,8 +151,10 @@
     (transform MAP-VALS inc data)
     (zipmap (keys data) (map inc (vals data)))
     (into {} (map (fn [e] [(key e) (inc (val e))]) data))
+    (into {} (map (fn [e] [(key e) (inc (val e))])) data)
     (map-vals-map-iterable data inc)
-    (map-vals-map-iterable-transient data inc)))
+    (map-vals-map-iterable-transient data inc)
+    ))
 
 
 (let [data (->> (for [i (range 1000)] [i i]) (into {}))]
@@ -160,6 +168,7 @@
     (transform MAP-VALS inc data)
     (zipmap (keys data) (map inc (vals data)))
     (into {} (map (fn [e] [(key e) (inc (val e))]) data))
+    (into {} (map (fn [e] [(key e) (inc (val e))])) data)
     (map-vals-map-iterable data inc)
     (map-vals-map-iterable-transient data inc)))
 
@@ -353,3 +362,15 @@
     (reduce-kv (fn [m k v] (assoc m (keyword (str *ns*) (name k)) v)) {} data)
     (setval [MAP-KEYS NAMESPACE] (str *ns*) data)
     ))
+
+(defnav walker-old [afn]
+  (select* [this structure next-fn]
+    (i/walk-select afn next-fn structure))
+  (transform* [this structure next-fn]
+    (i/walk-until afn next-fn structure)))
+
+(let [data {:a [1 2 {:c '(3 4) :d {:e [1 2 3] 7 8 9 10}}]}]
+  (run-benchmark "walker vs. clojure.walk version" 150000
+    (transform (walker number?) inc data)
+    (transform (walker-old number?) inc data)
+    ))

scripts/cljs-repl.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+rlwrap java -cp `lein classpath` clojure.main scripts/repl.clj
+

src/clj/com/rpl/specter.cljc

@@ -10,10 +10,15 @@
                defdynamicnav
                dynamicnav
                richnav
-               defrichnav]]
+               defrichnav
+               recursive-path]]
 
             [com.rpl.specter.util-macros :refer
               [doseqres]]))
+  ;; workaround for cljs bug that emits warnings for vars named the same as a
+  ;; private var in cljs.core (in this case `NONE`, added as private var to
+  ;; cljs.core with 1.9.562)
+  #?(:cljs (:refer-clojure :exclude [NONE]))
 
   (:use [com.rpl.specter.protocols :only [ImplicitNav RichNavigator]]
     #?(:clj [com.rpl.specter.util-macros :only [doseqres]]))
@@ -32,10 +37,15 @@
 (defn wrap-dynamic-nav [f]
   (fn [& args]
     (let [ret (apply f args)]
-      (if (and (sequential? ret) (static-path? ret))
+      (cond (and (sequential? ret) (static-path? ret))
-        (i/comp-paths* ret)
+            (i/comp-paths* ret)
-        ret
+
-        ))))
+            (and (sequential? ret) (= 1 (count ret)))
+            (first ret)
+
+            :else
+            ret
+            ))))
 
 #?(:clj
    (do
@@ -69,7 +79,7 @@
                 curr-params# [~@curr-params]]
             (if (every? (complement i/dynamic-param?) curr-params#)
               (apply builder# curr-params#)
-              (com.rpl.specter.impl/->DynamicFunction builder# curr-params#)))))
+              (com.rpl.specter.impl/->DynamicFunction builder# curr-params# nil)))))
 
      (defmacro late-bound-nav [bindings & impls]
        (late-bound-operation bindings `nav impls))
@@ -186,7 +196,7 @@
                   [e]
 
                   (sequential? e)
-                  (ic-possible-params e)))
+                  (concat (if (vector? e) [e]) (ic-possible-params e))))
 
 
           path)))
@@ -238,7 +248,7 @@
 
              cache-sym (vary-meta
                         (gensym "pathcache")
-                        merge {:cljs.analyzer/no-resolve true :no-doc true})
+                        merge {:cljs.analyzer/no-resolve true :no-doc true :private true})
 
              info-sym (gensym "info")
 
@@ -425,6 +435,9 @@
                   (let [inav# ~retrieve]
                     (i/exec-transform* inav# ~@rargs))))))))
 
+     (defmacro satisfies-protpath? [protpath o]
+       `(satisfies? ~(protpath-sym protpath) ~o))
+
      (defn extend-protocolpath* [protpath-prot extensions]
        (let [m (-> protpath-prot :sigs keys first)
              params (-> protpath-prot :sigs first last :arglists first)]
@@ -439,7 +452,12 @@
              embed (vec (for [[t p] extensions] [t `(quote ~p)]))]
          `(extend-protocolpath*
            ~(protpath-sym protpath)
-           ~embed)))))
+           ~embed)))
+
+    (defmacro end-fn [& args]
+      `(n/->SrangeEndFunction (fn ~@args)))
+
+    ))
 
 
 
@@ -639,6 +657,19 @@
   (transform* [this structure next-fn]
     (n/all-transform structure next-fn)))
 
+(defnav
+  ^{:doc "Same as ALL, except maintains metadata on the structure."}
+  ALL-WITH-META
+  []
+  (select* [this structure next-fn]
+    (n/all-select structure next-fn))
+  (transform* [this structure next-fn]
+    (let [m (meta structure)
+          res (n/all-transform structure next-fn)]
+      (if (some? res)
+        (with-meta res m)
+        ))))
+
 (defnav
   ^{:doc "Navigate to each value of the map. This is more efficient than
           navigating via [ALL LAST]"}
@@ -679,14 +710,17 @@
   (n/PosNavigator n/get-first n/update-first))
 
 (defnav
-  ^{:doc "Uses start-fn and end-fn to determine the bounds of the subsequence
+  ^{:doc "Uses start-index-fn and end-index-fn to determine the bounds of the subsequence
-          to select when navigating. Each function takes in the structure as input."}
+          to select when navigating. `start-index-fn` takes in the structure as input. `end-index-fn`
+          can be one of two forms. If a regular function (e.g. defined with `fn`), it takes in only the structure as input. If a function defined using special `end-fn` macro, it takes in the structure and the result of `start-index-fn`."}
   srange-dynamic
-  [start-fn end-fn]
+  [start-index-fn end-index-fn]
   (select* [this structure next-fn]
-    (n/srange-select structure (start-fn structure) (end-fn structure) next-fn))
+    (let [s (start-index-fn structure)]
+      (n/srange-select structure s (n/invoke-end-fn end-index-fn structure s) next-fn)))
   (transform* [this structure next-fn]
-    (n/srange-transform structure (start-fn structure) (end-fn structure) next-fn)))
+    (let [s (start-index-fn structure)]
+      (n/srange-transform structure s (n/invoke-end-fn end-index-fn structure s) next-fn))))
 
 
 (defnav
@@ -814,32 +848,18 @@
       (merge (reduce dissoc structure m-keys)
              newmap))))
 
-(defnav
-  ^{:doc "Using clojure.walk, navigate the data structure until reaching
-          a value for which `afn` returns truthy."}
-  walker
-  [afn]
-  (select* [this structure next-fn]
-    (n/walk-select afn next-fn structure))
-  (transform* [this structure next-fn]
-    (n/walk-until afn next-fn structure)))
-
-(defnav
-  ^{:doc "Like `walker` but maintains metadata of any forms traversed."}
-  codewalker
-  [afn]
-  (select* [this structure next-fn]
-    (n/walk-select afn next-fn structure))
-  (transform* [this structure next-fn]
-    (i/codewalk-until afn next-fn structure)))
-
 (defdynamicnav subselect
   "Navigates to a sequence that contains the results of (select ...),
   but is a view to the original structure that can be transformed.
 
   Requires that the input navigators will walk the structure's
   children in the same order when executed on \"select\" and then
-  \"transform\"."
+  \"transform\".
+
+  If transformed sequence is smaller than input sequence, missing entries
+  will be filled in with NONE, triggering removal if supported by that navigator.
+
+  Value collection (e.g. collect, collect-one) may not be used in the subpath."
   [& path]
   (late-bound-nav [late (late-path path)]
     (select* [this structure next-fn]
@@ -849,11 +869,57 @@
             transformed (next-fn select-result)
             values-to-insert (i/mutable-cell transformed)]
         (compiled-transform late
-                            (fn [_] (let [next-val (first (i/get-cell values-to-insert))]
+                            (fn [_] (let [vs (i/get-cell values-to-insert)]
-                                      (i/update-cell! values-to-insert rest)
+                                      (if vs
-                                      next-val))
+                                        (do (i/update-cell! values-to-insert next)
+                                            (first vs))
+                                        NONE
+                                        )))
                             structure)))))
 
+(defrichnav
+  ^{:doc "Navigates to the given key in the map (not to the value). Navigates only if the
+          key currently exists in the map. Can transform to NONE to remove the key/value
+          pair from the map."}
+  map-key
+  [key]
+  (select* [this vals structure next-fn]
+    (if (contains? structure key)
+      (next-fn vals key)
+      NONE
+      ))
+  (transform* [this vals structure next-fn]
+    (if (contains? structure key)
+      (let [newkey (next-fn vals key)
+            dissoced (dissoc structure key)]
+        (if (identical? NONE newkey)
+          dissoced
+          (assoc dissoced newkey (get structure key))
+          ))
+      structure
+      )))
+
+(defrichnav
+  ^{:doc "Navigates to the given element in the set only if it exists in the set.
+          Can transform to NONE to remove the element from the set."}
+  set-elem
+  [elem]
+  (select* [this vals structure next-fn]
+    (if (contains? structure elem)
+      (next-fn vals elem)
+      NONE
+      ))
+  (transform* [this vals structure next-fn]
+    (if (contains? structure elem)
+      (let [newelem (next-fn vals elem)
+            removed (disj structure elem)]
+        (if (identical? NONE newelem)
+          removed
+          (conj removed newelem)
+          ))
+      structure
+      )))
+
 (def ^{:doc "Navigate to the specified keys one after another. If navigate to NONE,
              that element is removed from the map or vector."}
   keypath
@@ -944,7 +1010,11 @@
 (defdynamicnav filterer
   "Navigates to a view of the current sequence that only contains elements that
   match the given path. An element matches the selector path if calling select
-  on that element with the path yields anything other than an empty sequence."
+  on that element with the path yields anything other than an empty sequence.
+
+  For transformation: `NONE` entries in the result sequence cause corresponding entries in
+  input to be removed. A result sequence smaller than the input sequence is equivalent to
+  padding the result sequence with `NONE` at the end until the same size as the input."
   [& path]
   (subselect ALL (selected? path)))
 
@@ -973,10 +1043,18 @@
 
 (def
   ^{:doc "Keeps the element only if it matches the supplied predicate. This is the
-          late-bound parameterized version of using a function directly in a path."}
+          late-bound parameterized version of using a function directly in a path."
+    :direct-nav true}
   pred
   i/pred*)
 
+
+(defn ^:direct-nav pred= [v] (pred #(= % v)))
+(defn ^:direct-nav pred< [v] (pred #(< % v)))
+(defn ^:direct-nav pred> [v] (pred #(> % v)))
+(defn ^:direct-nav pred<= [v] (pred #(<= % v)))
+(defn ^:direct-nav pred>= [v] (pred #(>= % v)))
+
 (extend-type nil
   ImplicitNav
   (implicit-nav [this] STAY))
@@ -1212,3 +1290,21 @@
    to implement post-order traversal."
   [& path]
   (multi-path path STAY))
+
+(def
+  ^{:doc "Navigate the data structure until reaching
+          a value for which `afn` returns truthy. Has
+          same semantics as clojure.walk."}
+  walker
+  (recursive-path [afn] p
+    (cond-path (pred afn) STAY
+               coll? [ALL p]
+               )))
+
+(def
+  ^{:doc "Like `walker` but maintains metadata of any forms traversed."}
+  codewalker
+  (recursive-path [afn] p
+    (cond-path (pred afn) STAY
+               coll? [ALL-WITH-META p]
+               )))

src/clj/com/rpl/specter/impl.cljc

@@ -2,7 +2,10 @@
   #?(:cljs (:require-macros
             [com.rpl.specter.util-macros
              :refer [doseqres mk-comp-navs mk-late-fn mk-late-fn-records]]))
-
+  ;; workaround for cljs bug that emits warnings for vars named the same as a
+  ;; private var in cljs.core (in this case `NONE`, added as private var to
+  ;; cljs.core with 1.9.562)
+  #?(:cljs (:refer-clojure :exclude [NONE]))
   (:use [com.rpl.specter.protocols :only
           [select* transform* collect-val RichNavigator]]
         #?(:clj [com.rpl.specter.util-macros :only [doseqres mk-comp-navs]]))
@@ -435,7 +438,7 @@
   [path])
 
 (defrecord DynamicFunction
-  [op params])
+  [op params code])
 
 (defn dynamic-param? [o]
   (contains? #{DynamicPath DynamicVal DynamicFunction} (type o)))
@@ -572,6 +575,25 @@
         (with-meta ret (meta structure))
         ret))))
 
+(defn walk-select [pred continue-fn structure]
+  (let [ret (mutable-cell NONE)
+        walker (fn this [structure]
+                 (if (pred structure)
+                   (let [r (continue-fn structure)]
+                     (if-not (identical? r NONE)
+                       (set-cell! ret r))
+                     r)
+
+                   (walk/walk this identity structure)))]
+
+    (walker structure)
+    (get-cell ret)))
+
+(defn walk-until [pred on-match-fn structure]
+  (if (pred structure)
+    (on-match-fn structure)
+    (walk/walk (partial walk-until pred on-match-fn) identity structure)))
+
 
 #?(:clj
    (do
@@ -645,20 +667,23 @@
   (-> o meta :direct-nav))
 
 (defn all-static? [params]
-  (every? (complement dynamic-param?) params))
+  (identical? NONE (walk-select dynamic-param? identity params)))
 
 (defn late-resolved-fn [afn]
   (fn [& args]
     (if (all-static? args)
       (apply afn args)
-      (->DynamicFunction afn args)
+      (->DynamicFunction afn args nil)
       )))
 
+(defn preserve-map [afn o]
+  (if (or (list? o) (seq? o))
+    (map afn o)
+    (into (empty o) (map afn o))))
+
 (defn- magic-precompilation* [o]
   (cond (sequential? o)
-        (if (or (list? o) (seq? o))
+        (preserve-map magic-precompilation* o)
-          (map magic-precompilation* o)
-          (into (empty o) (map magic-precompilation* o)))
 
         (instance? VarUse o)
         (if (dynamic-var? (:var o))
@@ -684,7 +709,7 @@
           (if (or (-> op meta :dynamicnav)
                   (all-static? (conj params op)))
             (magic-precompilation* (apply op params))
-            (->DynamicFunction op params)))
+            (->DynamicFunction op params (:code o))))
 
         :else
         ;; this handles dynamicval as well
@@ -694,22 +719,21 @@
   ([o] (static-combine o true))
   ([o nav-pos?]
    (cond (sequential? o)
-         (do
+         (if nav-pos?
-           (if-not nav-pos?
-             ;; should never happen...
-             (throw-illegal "Cannot statically combine sequential when not in nav pos"))
            (let [res (continuous-subseqs-transform*
                        rich-nav?
                        (doall (map static-combine (flatten o)))
                        (fn [s] [(comp-paths* s)]))]
              (if (= 1 (count res))
                (first res)
-               res)))
+               res))
+           (preserve-map #(static-combine % false) o))
 
          (instance? DynamicFunction o)
          (->DynamicFunction
           (static-combine (:op o) false)
-          (doall (map #(static-combine % false) (:params o))))
+          (doall (map #(static-combine % false) (:params o)))
+          (:code o))
 
          (instance? DynamicPath o)
          (->DynamicPath (static-combine (:path o)))
@@ -785,6 +809,10 @@
 
 (declare resolve-nav-code)
 
+(defn dynamic->code [o]
+  ;; works because both DynamicVal and DynamicFunction have :code field
+  (walk-until dynamic-param? :code o))
+
 (defn resolve-arg-code [o possible-params]
   (cond (instance? DynamicFunction o)
         (let [op (resolve-arg-code (:op o) possible-params)
@@ -800,7 +828,14 @@
         (resolve-nav-code o possible-params)
 
         :else
-        (static-val-code o)))
+        ;; handle dynamic params nested inside data structures
+        ;; e.g. (terminal-val [v])
+        (if (identical? NONE (walk-select dynamic-param? identity o))
+          (static-val-code o)
+          ;; done this way so it's compatible with cljs as well (since this dynamic val will be
+          ;; a possible param)
+          (resolve-arg-code (->DynamicVal (dynamic->code o)) possible-params)
+          )))
 
 (defn resolve-nav-code [o possible-params]
   (cond

src/clj/com/rpl/specter/navs.cljc

@@ -8,7 +8,6 @@
   (:use #?(:clj [com.rpl.specter.macros :only [defnav defrichnav]])
         #?(:clj [com.rpl.specter.util-macros :only [doseqres]]))
   (:require [com.rpl.specter.impl :as i]
-            [clojure.walk :as walk]
             #?(:clj [clojure.core.reducers :as r])))
 
 
@@ -22,27 +21,6 @@
   [compiled-path vals structure]
   (not (not-selected?* compiled-path vals structure)))
 
-(defn walk-select [pred continue-fn structure]
-  (let [ret (i/mutable-cell i/NONE)
-        walker (fn this [structure]
-                 (if (pred structure)
-                   (let [r (continue-fn structure)]
-                     (if-not (identical? r i/NONE)
-                       (i/set-cell! ret r))
-                     r)
-
-                   (walk/walk this identity structure)))]
-
-    (walker structure)
-    (i/get-cell ret)))
-
-
-(defn key-select [akey structure next-fn]
-  (next-fn (get structure akey)))
-
-(defn key-transform [akey structure next-fn]
-  (assoc structure akey (next-fn (get structure akey))))
-
 
 (defn all-select [structure next-fn]
   (doseqres i/NONE [e structure]
@@ -61,11 +39,14 @@
 (defprotocol AllTransformProtocol
   (all-transform [structure next-fn]))
 
+(defn void-transformed-kv-pair? [newkv]
+  (or (identical? newkv i/NONE) (< (count newkv) 2)))
+
 (defn- non-transient-map-all-transform [structure next-fn empty-map]
   (reduce-kv
     (fn [m k v]
       (let [newkv (next-fn [k v])]
-        (if (identical? newkv i/NONE)
+        (if (void-transformed-kv-pair? newkv)
           m
           (assoc m (nth newkv 0) (nth newkv 1)))))
 
@@ -77,12 +58,14 @@
 
 
 (defn- all-transform-list [structure next-fn]
-  ;; this is done to maintain order, otherwise lists get reversed
+  (doall (sequence (comp (map next-fn) (filter not-NONE?)) structure)))
-  (->> structure
+
-       (into '()
+(defn- all-transform-record [structure next-fn]
-             (comp (map next-fn) (filter not-NONE?)))
+  (reduce
-       reverse
+    (fn [res kv] (conj res (next-fn kv)))
-       ))
+    structure
+    structure
+    ))
 
 (extend-protocol AllTransformProtocol
   nil
@@ -127,7 +110,7 @@
              (let [k (.next k-it)
                    v (.next v-it)
                    newkv (next-fn [k v])]
-               (if (identical? newkv i/NONE)
+               (if (void-transformed-kv-pair? newkv)
                 (do
                   (i/update-cell! none-cell inc)
                   (recur (+ i 2) j))
@@ -153,6 +136,10 @@
   (all-transform [structure next-fn]
     (non-transient-map-all-transform structure next-fn (empty structure)))
 
+  #?(:clj clojure.lang.IRecord)
+  #?(:clj
+  (all-transform [structure next-fn]
+    (all-transform-record structure next-fn)))
 
   #?(:clj clojure.lang.PersistentHashMap :cljs cljs.core/PersistentHashMap)
   (all-transform [structure next-fn]
@@ -160,7 +147,7 @@
       (reduce-kv
         (fn [m k v]
           (let [newkv (next-fn [k v])]
-            (if (identical? newkv i/NONE)
+            (if (void-transformed-kv-pair? newkv)
               m
               (assoc! m (nth newkv 0) (nth newkv 1)))))
 
@@ -183,7 +170,7 @@
                (reduce-kv
                  (fn [m k v]
                    (let [newkv (next-fn [k v])]
-                     (if (identical? newkv i/NONE)
+                     (if (void-transformed-kv-pair? newkv)
                       m
                       (assoc m (nth newkv 0) (nth newkv 1)))))
 
@@ -201,12 +188,28 @@
   #?(:cljs default)
   #?(:cljs
      (all-transform [structure next-fn]
-       (let [empty-structure (empty structure)]
+       (if (record? structure)
-         (if (and (list? empty-structure) (not (queue? empty-structure)))
+         ;; this case is solely for cljs since extending to IRecord doesn't work for cljs
-           (all-transform-list structure next-fn)
+         (all-transform-record structure next-fn)
-           (into empty-structure
+         (let [empty-structure (empty structure)]
-                 (comp (map next-fn) (filter not-NONE?))
+           (cond
-                 structure))))))
+             (and (list? empty-structure) (not (queue? empty-structure)))
+             (all-transform-list structure next-fn)
+
+             (map? structure)
+             (reduce-kv
+               (fn [m k v]
+                 (let [newkv (next-fn [k v])]
+                   (if (void-transformed-kv-pair? newkv)
+                    m
+                    (assoc m (nth newkv 0) (nth newkv 1)))))
+                    empty-structure
+                    structure)
+
+             :else
+             (into empty-structure
+                   (comp (map next-fn) (filter not-NONE?))
+                   structure)))))))
 
 
 
@@ -600,12 +603,6 @@
     (empty? s)))
 
 
-(defn walk-until [pred on-match-fn structure]
-  (if (pred structure)
-    (on-match-fn structure)
-    (walk/walk (partial walk-until pred on-match-fn) identity structure)))
-
-
 (defn- do-keypath-transform [vals structure key next-fn]
   (let [newv (next-fn vals (get structure key))]
     (if (identical? newv i/NONE)
@@ -662,3 +659,12 @@
              []
              [v])
            ))))))
+
+(defrecord SrangeEndFunction [end-fn])
+
+;; done this way to maintain backwards compatibility
+(defn invoke-end-fn [end-fn structure start]
+  (if (instance? SrangeEndFunction end-fn)
+    ((:end-fn end-fn) structure start)
+    (end-fn structure)
+    ))

test/com/rpl/specter/cljs_test_helpers.clj

@@ -6,8 +6,8 @@
         vars (vec (map first parts))
         genned (reduce
                 (fn [curr [v code]]
-                  `(cljs.test.check.generators/bind ~code (fn [~v] ~curr)))
+                  `(clojure.test.check.generators/bind ~code (fn [~v] ~curr)))
-                `(cljs.test.check.generators/return ~vars)
+                `(clojure.test.check.generators/return ~vars)
                 (reverse parts))]
-    `(cljs.test.check.properties/for-all [~vars ~genned]
+    `(clojure.test.check.properties/for-all [~vars ~genned]
                    ~@body)))

test/com/rpl/specter/cljs_test_runner.cljs

@@ -1,8 +1,7 @@
 (ns com.rpl.specter.cljs-test-runner
-  (:require [cljs.test :as test :refer-macros [run-tests]]
+  (:require [doo.runner :refer-macros [doo-tests]]
             [com.rpl.specter.core-test]
             [com.rpl.specter.zipper-test]))
 
-
+(doo-tests 'com.rpl.specter.core-test
-(run-tests 'com.rpl.specter.core-test)
+           'com.rpl.specter.zipper-test)
-(run-tests 'com.rpl.specter.zipper-test)

test/com/rpl/specter/core_test.cljc

@@ -1,7 +1,7 @@
 (ns com.rpl.specter.core-test
   #?(:cljs (:require-macros
             [cljs.test :refer [is deftest]]
-            [cljs.test.check.cljs-test :refer [defspec]]
+            [clojure.test.check.clojure-test :refer [defspec]]
             [com.rpl.specter.cljs-test-helpers :refer [for-all+]]
             [com.rpl.specter.test-helpers :refer [ic-test]]
             [com.rpl.specter
@@ -10,7 +10,7 @@
                       select-first transform setval replace-in
                       select-any selected-any? collected? traverse
                       multi-transform path dynamicnav recursive-path
-                      defdynamicnav traverse-all]]))
+                      defdynamicnav traverse-all satisfies-protpath? end-fn]]))
   (:use
     #?(:clj [clojure.test :only [deftest is]])
     #?(:clj [clojure.test.check.clojure-test :only [defspec]])
@@ -21,15 +21,15 @@
                     select-first transform setval replace-in
                     select-any selected-any? collected? traverse
                     multi-transform path dynamicnav recursive-path
-                    defdynamicnav traverse-all]]))
+                    defdynamicnav traverse-all satisfies-protpath? end-fn]]))
 
 
 
   (:require #?(:clj [clojure.test.check.generators :as gen])
             #?(:clj [clojure.test.check.properties :as prop])
-            #?(:cljs [cljs.test.check :as tc])
+            #?(:cljs [clojure.test.check :as tc])
-            #?(:cljs [cljs.test.check.generators :as gen])
+            #?(:cljs [clojure.test.check.generators :as gen])
-            #?(:cljs [cljs.test.check.properties :as prop :include-macros true])
+            #?(:cljs [clojure.test.check.properties :as prop :include-macros true])
             [com.rpl.specter :as s]
             [com.rpl.specter.transients :as t]
             [clojure.set :as set]))
@@ -1422,6 +1422,7 @@
 
 (deftest string-navigation-test
   (is (= "ad" (setval (s/srange 1 3) "" "abcd")))
+  (is (= "abcxd" (setval [(s/srange 1 3) s/END] "x" "abcd")))
   (is (= "bc" (select-any (s/srange 1 3) "abcd")))
   (is (= "ab" (setval s/END "b" "a")))
   (is (= "ba" (setval s/BEGINNING "b" "a")))
@@ -1507,3 +1508,81 @@
   (is (= "b" (setval s/FIRST s/NONE "ab")))
   (is (= "a" (setval s/LAST s/NONE "ab")))
   )
+
+(deftest nested-dynamic-arg-test
+  (let [foo (fn [v] (multi-transform (s/terminal-val [v]) nil))]
+    (is (= [1] (foo 1)))
+    (is (= [10] (foo 10)))
+    ))
+
+(deftest filterer-remove-test
+  (is (= [1 :a 3 5] (setval (s/filterer even?) [:a] [1 2 3 4 5])))
+  )
+
+(deftest helper-preds-test
+  (let [data [1 2 2 3 4 0]]
+    (is (= [2 2] (select [s/ALL (s/pred= 2)] data)))
+    (is (= [1 2 2 0] (select [s/ALL (s/pred< 3)] data)))
+    (is (= [1 2 2 3 0] (select [s/ALL (s/pred<= 3)] data)))
+    (is (= [4] (select [s/ALL (s/pred> 3)] data)))
+    (is (= [3 4] (select [s/ALL (s/pred>= 3)] data)))
+    ))
+
+(deftest map-key-test
+  (is (= {:c 3} (setval (s/map-key :a) :b {:c 3})))
+  (is (= {:b 2} (setval (s/map-key :a) :b {:a 2})))
+  (is (= {:b 2} (setval (s/map-key :a) :b {:a 2 :b 1})))
+  (is (= {:b 2} (setval (s/map-key :a) s/NONE {:a 1 :b 2})))
+  )
+
+(deftest set-elem-test
+  (is (= #{:b :d} (setval (s/set-elem :a) :x #{:b :d})))
+  (is (= #{:x :a} (setval (s/set-elem :b) :x #{:b :a})))
+  (is (= #{:a} (setval (s/set-elem :b) :a #{:b :a})))
+  (is (= #{:b} (setval (s/set-elem :a) s/NONE #{:a :b})))
+  )
+
+;; this function necessary to trigger the bug from happening
+(defn inc2 [v] (inc v))
+(deftest dynamic-function-arg-test
+  (is (= {[2] 4} (let [a 1] (transform (s/keypath [(inc2 a)]) inc {[2] 3}))))
+  )
+
+(defrecord FooW [a b])
+
+(deftest walker-test
+  (is (= [1 2 3 4 5 6] (select (s/walker number?) [{1 2 :b '(3 :c 4)} 5 #{6 :d}])))
+  (is (= [{:b '(:c)} #{:d}] (setval (s/walker number?) s/NONE [{:q 3 10 :l 1 2 :b '(3 :c 4)} 5 #{6 :d}])))
+  (is (= [{:q 4 11 :l 2 3 :b '(4 :c 5)} 6 #{7 :d}]
+         (transform (s/walker number?) inc [{:q 3 10 :l 1 2 :b '(3 :c 4)} 5 #{6 :d}])))
+  (let [f (->FooW 1 2)]
+    (is (= [[:a 1] [:b 2]] (select (s/walker (complement record?)) f)))
+    (is (= (assoc f :a! 1 :b! 2) (setval [(s/walker (complement record?)) s/FIRST s/NAME s/END] "!" f)))
+    (is (= (assoc f :b 1 :c 2) (transform [(s/walker (complement record?)) s/FIRST] (fn [k] (if (= :a k) :b :c)) f)))
+    ))
+
+(def MIDDLE
+  (s/comp-paths
+    (s/srange-dynamic
+      (fn [aseq] (long (/ (count aseq) 2)))
+      (end-fn [aseq s] (if (empty? aseq) 0 (inc s))))
+    s/FIRST
+    ))
+
+(deftest srange-dynamic-test
+  (is (= 2 (select-any MIDDLE [1 2 3])))
+  (is (identical? s/NONE (select-any MIDDLE [])))
+  (is (= 1 (select-any MIDDLE [1])))
+  (is (= 2 (select-any MIDDLE [1 2])))
+  (is (= [1 3 3] (transform MIDDLE inc [1 2 3])))
+  )
+
+#?(:clj
+  (do
+    (defprotocolpath FooPP)
+    (extend-protocolpath FooPP String s/STAY)
+
+    (deftest satisfies-protpath-test
+      (is (satisfies-protpath? FooPP "a"))
+      (is (not (satisfies-protpath? FooPP 1)))
+      )))

test/com/rpl/specter/zipper_test.cljc

@@ -1,7 +1,7 @@
 (ns com.rpl.specter.zipper-test
   #?(:cljs (:require-macros
             [cljs.test :refer [is deftest]]
-            [cljs.test.check.cljs-test :refer [defspec]]
+            [clojure.test.check.clojure-test :refer [defspec]]
             [com.rpl.specter.cljs-test-helpers :refer [for-all+]]
             [com.rpl.specter
               :refer [declarepath providepath select select-one select-one!
@@ -17,9 +17,9 @@
 
   (:require #?(:clj [clojure.test.check.generators :as gen])
             #?(:clj [clojure.test.check.properties :as prop])
-            #?(:cljs [cljs.test.check :as tc])
+            #?(:cljs [clojure.test.check :as tc])
-            #?(:cljs [cljs.test.check.generators :as gen])
+            #?(:cljs [clojure.test.check.generators :as gen])
-            #?(:cljs [cljs.test.check.properties :as prop :include-macros true])
+            #?(:cljs [clojure.test.check.properties :as prop :include-macros true])
             [com.rpl.specter :as s]
             [com.rpl.specter.zipper :as z]))