:gen -> :gem-wrap in middleware

* as preparation for support of interceptors

doc/performance.md

@@ -15,13 +15,6 @@ There are many great routing libraries for Clojure(Script), but not many are opt
 * Always be measuring
 * Don't trust the (micro-)benchmarks
 
-### Performance guides
-
-Some things related to performance:
-
-* avoid wildcard-routes - it's an order of magnitude slower to match than non-wildcard routes
-* it's ok to mix non-wildcard and wildcard routes in a same routing tree as long as you don't disable the [conflict resolution](basics/route_conflicts.md) => if no conflicting routes are found, a `:mixed-router` can be created, which collects all non-wildcard routes into a separate fast subrouter.
-
 ### Does routing performance matter?
 
 Well, it depends. Some tested routing libs seem to spend more time resolving the routes than it takes to encode & decode a 1k JSON payload. For busy sites, this actually matters.
@@ -40,12 +33,12 @@ The routing sample taken from [bide](https://github.com/funcool/bide) README, ru
      ["/auth/recovery/token/:token" :auth/recovery]
      ["/workspace/:project/:page" :workspace/page]]))
 
-;; Execution time mean : 3.488297 µs -> 286M ops/sec
+;; Execution time mean : 3.2 µs -> 312M ops/sec
 (cc/quick-bench
   (dotimes [_ 1000]
     (r/match-by-path routes "/auth/login")))
 
-;; Execution time mean : 692.905995 µs -> 1.4M ops/sec
+;; Execution time mean : 530 µs -> 1.9M ops/sec
 (cc/quick-bench
   (dotimes [_ 1000]
     (r/match-by-path routes "/workspace/1/1")))
@@ -53,19 +46,22 @@ The routing sample taken from [bide](https://github.com/funcool/bide) README, ru
 
 ### Is that good?
 
-Based on some [quick perf tests](https://github.com/metosin/reitit/tree/master/perf-test/clj/reitit), the first lookup is two orders of magnitude faster than other tested Clojure routing libraries. The second lookup is 3-18x faster.
+Based on some [quick perf tests](https://github.com/metosin/reitit/tree/master/perf-test/clj/reitit), the first (static path) lookup is 300-500x faster and the second (wildcard path) lookup is 4-24x faster that the other tested routing libs (ataraxy, bidi, compojure and pedestal).
 
-But, most micro-benchmarks lie. For example, Pedestal is always matching the `:request-method` which means it does more work. With real life routing trees, the differences are most likely more subtle, or some other lib might be actually faster.
+But, one shoudn't trust the benchmarks. Many libraries (here: compojure, pedestal and ataraxy) always match also on the request-method so they do more work. Also, real-life routing tables might look different and different libs might behave differently.
 
-### So why test?
+But, the perf should be good.
+
+### Value of perf tests?
 
 Real value of perf tests is to get a internal baseline to optimize against. Also, to ensure that new features don't regress the performance.
 
 It might be interesting to look out of the box and compare the fast Clojure routing libs to routers in other languages, like the [routers in Go](https://github.com/julienschmidt/go-http-routing-benchmark).
 
-### Roadmap
+### Performance guides
 
-Currently, the non-wildcard routes are already really fast to match, but wildcard routes use only a naive linear scan. Plan is to add a optimized [Trie](https://en.wikipedia.org/wiki/Trie)-based router. See
+Few things that have an effect on performance:
-[httprouter](https://github.com/julienschmidt/httprouter#how-does-it-work) and [Pedestal](https://github.com/pedestal/pedestal/pull/330) for details.
 
-PRs welcome.
+* Wildcard-routes are an order of magnitude slower than static routes
+* It's ok to mix non-wildcard and wildcard routes in a same routing tree as long as you don't disable the [conflict resolution](basics/route_conflicts.md) => if no conflicting routes are found, a `:mixed-router` can be created, which internally has a fast static path router and a separate wildcard-router. So, the static paths are still fast.
+* Move computation from request processing time into creation time, using by compiling [middleware](ring/compiling_middleware.md) & [route data](advanced/configuring_routers.md).

doc/ring/compiling_middleware.md

@@ -2,15 +2,15 @@
 
 The [dynamic extensions](dynamic_extensions.md) is a easy way to extend the system. To enable fast lookups into route data, we can compile them into any shape (records, functions etc.) we want, enabling fast access at request-time.
 
-Still, we can do better. As we know the exact route that middleware/interceptor is linked to, we can pass the (compiled) route information into the middleware/interceptor at creation-time. It can do local reasoning: extract and transform relevant data just for it and pass it into the actual request-handler via a closure - yielding much faster runtime processing. Middleware/interceptor can also decide not to mount itself. Why mount a `wrap-enforce-roles` middleware for a route if there are no roles required for it?
+Still, we can do much better. As we know the exact route that middleware/interceptor is linked to, we can pass the (compiled) route information into the middleware/interceptor at creation-time. It can do local reasoning: extract and transform relevant data just for it and pass it into the actual request-handler via a closure - yielding much faster runtime processing. It can also decide not to mount itself by returning `nil`. Why mount a `wrap-enforce-roles` middleware for a route if there are no roles required for it?
 
-To enable this we use [middleware records](data_driven_middleware.md) `:gen` hook instead of the normal `:wrap`. `:gen` expects a function of `route-meta router-opts => wrap`. Middleware can also return `nil`, which effective unmounts the middleware.
+To enable this we use [middleware records](data_driven_middleware.md) `:gen-wrap` key instead of the normal `:wrap`. `:gen-wrap` expects a function of `route-meta router-opts => ?wrap`.
 
-To demonstrate the two approaches, below are response coercion middleware written as normal ring middleware function and as middleware record with `:gen`. These are the actual codes are from [`reitit.ring.coercion`](https://github.com/metosin/reitit/blob/master/src/reitit/ring/coercion.cljc):
+To demonstrate the two approaches, below are response coercion middleware written as normal ring middleware function and as middleware record with `:gen-wrap`. Actual codes can be found in [`reitit.ring.coercion`](https://github.com/metosin/reitit/blob/master/src/reitit/ring/coercion.cljc):
 
 ## Naive
 
-* Extracts the compiled route information on every request.
+* Reads the compiled route information on every request.
 
 ```clj
 (defn wrap-coerce-response
@@ -49,13 +49,15 @@ To demonstrate the two approaches, below are response coercion middleware writte
 * Mounts only if `:coercion` and `:responses` are defined for the route
 
 ```clj
+(require '[reitit.ring.middleware :as middleware])
+
 (def gen-wrap-coerce-response
   "Generator for pluggable response coercion middleware.
   Expects a :coercion of type `reitit.coercion.protocol/Coercion`
   and :responses from route meta, otherwise does not mount."
   (middleware/create
     {:name ::coerce-response
-     :gen (fn [{:keys [responses coercion opts]} _]
+     :gen-wrap (fn [{:keys [responses coercion opts]} _]
                 (if (and coercion responses)
                   (let [coercers (response-coercers coercion responses opts)]
                     (fn [handler]
@@ -66,4 +68,4 @@ To demonstrate the two approaches, below are response coercion middleware writte
                          (handler request #(respond (coerce-response coercers request %)) raise)))))))}))
 ```
 
-The `:gen` -version has 50% less code, is easier to reason about and is twice as faster on basic perf tests.
+The latter has 50% less code, is easier to reason about and is much faster.

doc/ring/data_driven_middleware.md

@@ -1,38 +1,98 @@
 # Data-driven Middleware
 
-Reitit supports first-class data-driven middleware via `reitit.ring.middleware/Middleware` records, created with `reitit.ring.middleware/create` function. The following keys have special purpose:
+Ring [defines middleware](https://github.com/ring-clojure/ring/wiki/Concepts#middleware) as a function of type `handler & opts => request => response`. It's easy to undrstand and enables great performance, but makes the middleware-chain opaque, making things like documentation and debugging hard.
+
+Reitit does things bit differently:
+
+1. middleware is defined as a vector (of middleware) enabling the chain to be malipulated before turned into the optimized runtime chain.
+2. middleware can be defined as first-class data entries
+
+### Middleware as data
+
+Everything that is defined inside the `:middleware` vector in the route data is coerced into `reitit.ring.middleware/Middleware` Records with the help of `reitit.ring.middleware/IntoMiddleware` Protocol. By default, it transforms functions, maps and `Middleware` records. For the actual
+
+Records can have arbitrary keys, but the default keys have a special purpose:
 
 | key         | description |
-| -----------|-------------|
+| ------------|-------------|
-| `:name`    | Name of the middleware as qualified keyword (optional,recommended for libs)
+| `:name`     | Name of the middleware as a qualified keyword (optional)
-| `:wrap`    | The actual middleware function of `handler args? => request => response`
+| `:wrap`     | The actual middleware function of `handler & args => request => response`
-| `:gen`     | Middleware compile function, see [compiling middleware](compiling_middleware.md).
+| `:gen-wrap` | Middleware function generation function, see [compiling middleware](compiling_middleware.md).
 
-When routes are compiled, all middleware are expanded (and optionally compiled) into `Middleware` Records and stored in compilation results for later use (api-docs etc). For actual request processing, they are unwrapped into normal middleware functions and composed together producing zero runtime performance penalty. Middleware expansion is backed by `reitit.middleware/IntoMiddleware` protocol, enabling plain clojure(script) maps to be used.
+Middleware Records are accessible in their raw form in the compiled route results, thus available for inventories, creating api-docs etc.
 
-A Record:
+For the actual request processing, the Records are unwrapped into normal functions, yielding zero runtime penalty.
+
+### Creating Middleware
+
+The following produce identical middleware runtime function.
+
+#### Function
 
 ```clj
-(require '[reitit.middleware :as middleware])
+(defn wrap [handler id]
+  (fn [request]
+    (handler (update request ::acc (fnil conj []) id))))
+```
+
+### Record
+
+```clj
+(require '[reitit.ring.middleware :as middleware])
 
 (def wrap2
   (middleware/create
     {:name ::wrap2
-     :description "a nice little mw, takes 1 arg."
+     :description "Middleware that does things."
      :wrap wrap}))
 ```
 
-As plain map:
+#### Map
 
 ```clj
-;; plain map
 (def wrap3
   {:name ::wrap3
-   :description "a nice little mw, :api as arg"
+   :description "Middleware that does things."
-   :wrap (fn [handler]
+   :wrap wrap})
-           (wrap handler :api))})
 ```
 
-### TODO
+### Using Middleware
 
-more!
+```clj
+(require '[reitit.ring :as ring])
+
+(defn handler [{:keys [::acc]}]
+  {:status 200, :body (conj acc :handler)})
+
+(def app
+  (ring/ring-handler
+    (ring/router
+      ["/api" {:middleware [[wrap 1] [wrap2 2]]}
+       ["/ping" {:get {:middleware [[wrap3 3]]
+                       :handler handler}}]])))
+```
+
+All the middleware are called correctly:
+
+```clj
+(app {:request-method :get, :uri "/api/ping"})
+; {:status 200, :body [1 2 3 :handler]}
+```
+
+### Future
+
+Some things bubblin' under:
+
+* Hooks to manipulate the `:middleware` chain before compilation
+* Support `Keyword` expansion into Middleware, enabling external Middleware Registries (duct/integrant/macchiato -style)
+* Support Middleware dependency resolution with new keys `:requires` and `:provides`. Values are set of top-level keys of the request. e.g.
+   * `InjectUserIntoRequestMiddleware` requires `#{:session}` and provides `#{:user}`
+   * `AuthorizationMiddleware` requires `#{:user}`
+* Support partial `s/keys` route data specs with Middleware (and Router). Merged together to define sound spec for the route data and/or route data for a given route.
+   * e.g. `AuthrorizationMiddleware` has a spec defining `:roles` key (a set of keywords)
+   * Documentation for the route data
+   * Route data is validated against the spec:
+      * Complain of keywords that are not handled by anything
+      * Propose fixes for typos (Figwheel-style)
+
+Ideas welcome & see [issues](https://github.com/metosin/reitit/issues) for details.

modules/reitit-ring/src/reitit/ring/coercion.cljc

@@ -136,7 +136,7 @@
   and :parameters from route meta, otherwise does not mount."
   (middleware/create
     {:name ::coerce-parameters
-     :gen (fn [{:keys [parameters coercion]} _]
+     :gen-wrap (fn [{:keys [parameters coercion]} _]
                  (if (and coercion parameters)
                    (let [coercers (request-coercers coercion parameters)]
                      (fn [handler]
@@ -182,7 +182,7 @@
   and :responses from route meta, otherwise does not mount."
   (middleware/create
     {:name ::coerce-response
-     :gen (fn [{:keys [responses coercion opts]} _]
+     :gen-wrap (fn [{:keys [responses coercion opts]} _]
                  (if (and coercion responses)
                    (let [coercers (response-coercers coercion responses opts)]
                      (fn [handler]

modules/reitit-ring/src/reitit/ring/middleware.cljc

@@ -8,11 +8,11 @@
 (defrecord Middleware [name wrap])
 (defrecord Endpoint [meta handler middleware])
 
-(defn create [{:keys [name gen wrap] :as m}]
+(defn create [{:keys [name wrap gen-wrap] :as m}]
-  (when (and gen wrap)
+  (when (and wrap gen-wrap)
     (throw
       (ex-info
-        (str "Middleware can't both :wrap and :gen defined " m) m)))
+        (str "Middleware can't both :wrap and :gen-wrap defined " m) m)))
   (map->Middleware m))
 
 (extend-protocol IntoMiddleware
@@ -40,13 +40,13 @@
     (into-middleware (create this) meta opts))
 
   Middleware
-  (into-middleware [{:keys [wrap gen] :as this} meta opts]
+  (into-middleware [{:keys [wrap gen-wrap] :as this} meta opts]
-    (if-not gen
+    (if-not gen-wrap
       this
-      (if-let [wrap (gen meta opts)]
+      (if-let [wrap (gen-wrap meta opts)]
         (map->Middleware
           (-> this
-              (dissoc :gen)
+              (dissoc :gen-wrap)
               (assoc :wrap wrap))))))
 
   nil

perf-test/clj/reitit/perf_test.clj

@@ -71,45 +71,43 @@
 
   (suite "static route")
 
-  ;; 1800 µs
+  ;; 1600 µs
   (title "bidi")
-  (let [call #(bidi/match-route bidi-routes "/auth/login")]
+  (assert (bidi/match-route bidi-routes "/auth/login"))
-    (assert (call))
   (cc/quick-bench
     (dotimes [_ 1000]
-        (call))))
+      (bidi/match-route bidi-routes "/auth/login")))
 
   ;; 1400 µs
   (title "ataraxy")
-  (let [call #(ataraxy/matches ataraxy-routes {:uri "/auth/login"})]
+  (let [request {:uri "/auth/login"}]
-    (assert (call))
+    (assert (ataraxy/matches ataraxy-routes request))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (ataraxy/matches ataraxy-routes request))))
 
-  ;; 1200 µs
+  ;; 1000 µs
   (title "pedestal - map-tree => prefix-tree")
-  (let [call #(pedestal/find-route pedestal-router {:path-info "/auth/login" :request-method :get})]
+  (let [request {:path-info "/auth/login" :request-method :get}]
-    (assert (call))
+    (assert (pedestal/find-route pedestal-router {:path-info "/auth/login" :request-method :get}))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (pedestal/find-route pedestal-router {:path-info "/auth/login" :request-method :get}))))
 
-  ;; 1400 µs
+  ;; 1500 µs
   (title "compojure-api")
-  (let [call #(compojure-api-routes {:uri "/auth/login", :request-method :get})]
+  (let [request {:uri "/auth/login", :request-method :get}]
-    (assert (call))
+    (assert (compojure-api-routes request))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (compojure-api-routes request))))
 
-  ;; 3.5 µs (300-500x)
+  ;; 3.2 µs (300-500x)
   (title "reitit")
-  (let [call #(reitit/match-by-path reitit-routes "/auth/login")]
+  (assert (reitit/match-by-path reitit-routes "/auth/login"))
-    (assert (call))
   (cc/quick-bench
     (dotimes [_ 1000]
-        (call)))))
+      (reitit/match-by-path reitit-routes "/auth/login"))))
 
 (defn routing-test2 []
 
@@ -117,44 +115,42 @@
 
   ;; 12800 µs
   (title "bidi")
-  (let [call #(bidi/match-route bidi-routes "/workspace/1/1")]
+  (assert (bidi/match-route bidi-routes "/workspace/1/1"))
-    (assert (call))
   (cc/quick-bench
     (dotimes [_ 1000]
-        (call))))
+      (bidi/match-route bidi-routes "/workspace/1/1")))
 
   ;; 2800 µs
   (title "ataraxy")
-  (let [call #(ataraxy/matches ataraxy-routes {:uri "/workspace/1/1"})]
+  (let [request {:uri "/workspace/1/1"}]
-    (assert (call))
+    (assert (ataraxy/matches ataraxy-routes request))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (ataraxy/matches ataraxy-routes request))))
 
-  ;; 2300 µs
+  ;; 2100 µs
-  (title "pedestal - map-tree => prefix-tree")
+  (title "pedestal")
-  (let [call #(pedestal/find-route pedestal-router {:path-info "/workspace/1/1" :request-method :get})]
+  (let [request {:path-info "/workspace/1/1" :request-method :get}]
-    (assert (call))
+    (assert (pedestal/find-route pedestal-router request))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (pedestal/find-route pedestal-router request))))
 
-  ;; 3800 µs
+  ;; 3500 µs
   (title "compojure-api")
-  (let [call #(compojure-api-routes {:uri "/workspace/1/1", :request-method :get})]
+  (let [request {:uri "/workspace/1/1", :request-method :get}]
-    (assert (call))
+    (assert (compojure-api-routes request))
     (cc/quick-bench
       (dotimes [_ 1000]
-        (call))))
+        (compojure-api-routes request))))
 
   ;; 710 µs (3-18x)
-  ;; 540 µs (4-23x) -23% prefix-tree-router
+  ;; 530 µs (4-24x) -25% prefix-tree-router
   (title "reitit")
-  (let [call #(reitit/match-by-path reitit-routes "/workspace/1/1")]
+  (assert (reitit/match-by-path reitit-routes "/workspace/1/1"))
-    (assert (call))
   (cc/quick-bench
     (dotimes [_ 1000]
-        (call)))))
+      (reitit/match-by-path reitit-routes "/workspace/1/1"))))
 
 (defn reverse-routing-test []
 

test/cljc/reitit/middleware_test.cljc

@@ -10,14 +10,14 @@
 
   (testing "middleware records"
 
-    (testing ":wrap & :gen are exclusive"
+    (testing ":wrap & :gen-wrap are exclusive"
       (is (thrown-with-msg?
             ExceptionInfo
-            #"Middleware can't both :wrap and :gen defined"
+            #"Middleware can't both :wrap and :gen-wrap defined"
             (middleware/create
               {:name ::test
                :wrap identity
-               :gen (constantly identity)}))))
+               :gen-wrap (constantly identity)}))))
 
     (testing "middleware"
       (let [calls (atom 0)
@@ -74,7 +74,7 @@
 
     (testing "compiled Middleware"
       (let [calls (atom 0)
-            mw {:gen (fn [meta _]
+            mw {:gen-wrap (fn [meta _]
                             (swap! calls inc)
                             (fn [handler value]
                               (swap! calls inc)
@@ -100,8 +100,8 @@
               (is (= 2 @calls)))))
 
         (testing "nil unmounts the middleware"
-          (let [app (->app [{:gen (constantly nil)}
+          (let [app (->app [{:gen-wrap (constantly nil)}
-                            {:gen (constantly nil)}] identity)]
+                            {:gen-wrap (constantly nil)}] identity)]
             (dotimes [_ 10]
               (is (= :request (app :request))))))))))
 
@@ -144,9 +144,9 @@
       (testing "with nested middleware"
         (is (= [:api :admin :ok :admin :api] (app "/api/admin/ping"))))
 
-      (testing ":gen middleware can be unmounted at creation-time"
+      (testing ":gen-wrap middleware can be unmounted at creation-time"
-        (let [mw1 {:name ::mw1, :gen (constantly #(mw % ::mw1))}
+        (let [mw1 {:name ::mw1, :gen-wrap (constantly #(mw % ::mw1))}
-              mw2 {:name ::mw2, :gen (constantly nil)}
+              mw2 {:name ::mw2, :gen-wrap (constantly nil)}
               mw3 {:name ::mw3, :wrap #(mw % ::mw3)}
               router (middleware/router
                        ["/api" {:name ::api