Merge pull request #79 from metosin/ring-perf-and-resources

Ring perf and serve static resources
This commit is contained in:
Tommi Reiman 2018-04-25 12:49:47 +03:00 committed by GitHub
commit d00c7122e7
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
16 changed files with 407 additions and 64 deletions

View file

@ -19,9 +19,21 @@
; :path "/coffee/luwak"}
```
### `reitit-ring`
* `reitit.ring/default-handler` now works correctly with async ring
* new helper `reitit.ring/router` to compose routes outside of a router.
* `reitit.ring/create-resource-handler` function to serve static routes. See (docs)[https://metosin.github.io/reitit/ring/static.html].
* new dependencies:
```clj
[ring/ring-core "1.6.3"]
```
### `reitit-swagger`
* New module to produce swagger-docs from routing tree, including `Coercion` definitions. Works with both middleware & interceptors.
* New module to produce swagger-docs from routing tree, including `Coercion` definitions. Works with both middleware & interceptors and Schema & Spec. See [docs](https://metosin.github.io/reitit/swagger.html).
```clj
(require '[reitit.ring :as ring])
@ -40,7 +52,7 @@
["/swagger.json"
{:get {:no-doc true
:swagger {:info {:title "my-api"}}
:handler swagger/swagger-spec-handler}}]
:handler (swagger/create-swagger-handler)}}]
["/spec" {:coercion spec/coercion}
["/plus"

View file

@ -0,0 +1 @@
{"hello": "file"}

View file

@ -0,0 +1 @@
<xml><hello>file</hello></xml>

View file

@ -21,6 +21,7 @@
* [Dev Workflow](advanced/dev_workflow.md)
* [Ring](ring/README.md)
* [Ring-router](ring/ring.md)
* [Static Resources](ring/static.md)
* [Dynamic Extensions](ring/dynamic_extensions.md)
* [Data-driven Middleware](ring/data_driven_middleware.md)
* [Pluggable Coercion](ring/coercion.md)
@ -28,5 +29,5 @@
* [Compiling Middleware](ring/compiling_middleware.md)
* [Performance](performance.md)
* [Interceptors (WIP)](interceptors.md)
* [Swagger & Openapi (WIP)](openapi.md)
* [Swagger-support](swagger.md)
* [FAQ](faq.md)

View file

@ -1,6 +1,7 @@
# Ring
* [Ring-router](ring.md)
* [Static Resources](static.md)
* [Dynamic Extensions](dynamic_extensions.md)
* [Data-driven Middleware](data_driven_middleware.md)
* [Pluggable Coercion](coercion.md)

64
doc/ring/static.md Normal file
View file

@ -0,0 +1,64 @@
# Static Resources (Clojure Only)
Static resources can be served with a help of `reitit.ring/create-resource-handler`. It takes optionally an options map and returns a ring handler to serve files from Classpath. It returns `java.io.File` instances, so ring adapters can use NIO to effective Stream the files.
There are two options to serve the files.
## Internal routes
This is good option if static files can be from non-conflicting paths, e.g. `"/assets/*"`.
```clj
(require '[reitit.ring :as ring])
(ring/ring-handler
(ring/router
[["/ping" (constantly {:status 200, :body "pong"})]
["/assets/*" (ring/create-resource-handler)]])
(ring/create-default-handler))
```
To serve static files with conflicting routes, e.g. `"/*#`, one needs to disable the conflict resolution:
```clj
(require '[reitit.ring :as ring])
(ring/ring-handler
(ring/router
[["/ping" (constantly {:status 200, :body "pong"})]
["/*" (ring/create-resource-handler)]]
{:conflicts (constantly nil)})
(ring/create-default-handler))
```
## External routes
To serve files from conflicting paths, e.g. `"/*"`, one option is to mount them to default-handler branch of `ring-handler`. This way, they are only served if none of the actual routes have matched.
```clj
(ring/ring-handler
(ring/router
["/ping" (constantly {:status 200, :body "pong"})])
(ring/routes
(ring/create-resource-handler {:path "/"})
(ring/create-default-handler)))
```
## Configuration
`reitit.ring/create-resource-handler` takes optionally an options map to configure how the files are being served.
| key | description |
| -----------------|-------------|
| :parameter | optional name of the wildcard parameter, defaults to unnamed keyword `:`
| :root | optional resource root, defaults to `"public"`
| :path | optional path to mount the handler to. Works only if mounted outside of a router.
| :loader | optional class loader to resolve the resources
| :allow-symlinks? | allow symlinks that lead to paths outside the root classpath directories, defaults to `false`
### TODO
* support for things like `:cache`, `:last-modified?`, `:index-files` and `:gzip`
* support for ClojureScript
* serve from file-system

View file

@ -1,14 +1,6 @@
# Swagger & OpenAPI (WIP)
# Swagger
Goal is to support both [Swagger](https://swagger.io/) & [OpenAPI](https://www.openapis.org/) for route documentation. Documentation is extracted from existing coercion definitions `:parameters`, `:responses` and from a set of new doumentation keys.
Swagger-support draft works, but only for Clojure.
### TODO
* [metosin/schema-tools#38](https://github.com/metosin/schema-tools/issues/38): extract Schema-swagger from [ring-swagger](https://github.com/metosin/ring-swagger) into [schema-tools](https://github.com/metosin/schema-tools) to support both Clojure & ClojureScript
* separate modules for the swagger2 & openapi
* [metosin/spec-tools#105](https://github.com/metosin/spec-tools/issues/105): support Openapi
Reitit supports [Swagger](https://swagger.io/) to generate route documentation. Documentation is extracted from existing coercion definitions `:parameters`, `:responses` and from a set of new doumentation keys.
### Example
@ -25,6 +17,7 @@ Current `reitit-swagger` draft (with `reitit-ring` & data-specs):
(ring/ring-handler
(ring/router
["/api"
;; identify a swagger api
;; there can be several in a routing tree
{:swagger {:id :math}}
@ -33,7 +26,14 @@ Current `reitit-swagger` draft (with `reitit-ring` & data-specs):
["/swagger.json"
{:get {:no-doc true
:swagger {:info {:title "my-api"}}
:handler swagger/swagger-spec-handler}}]
:handler (swagger/create-swagger-handler)}}]
;; the (undocumented) swagger-ui
;; [org.webjars/swagger-ui "3.13.4"]
["/docs/*"
{:get {:no-doc true
:handler (ring/create-resource-handler
{:root "META-INF/resources/webjars/swagger-ui"})}}]
["/minus"
{:get {:summary "minus"

View file

@ -6,4 +6,5 @@
:plugins [[lein-parent "0.3.2"]]
:parent-project {:path "../../project.clj"
:inherit [:deploy-repositories :managed-dependencies]}
:dependencies [[metosin/reitit-core]])
:dependencies [[metosin/reitit-core]
[ring/ring-core]])

View file

@ -2,10 +2,13 @@
(:require [meta-merge.core :refer [meta-merge]]
[reitit.middleware :as middleware]
[reitit.core :as r]
[reitit.impl :as impl]))
[reitit.impl :as impl]
#?@(:clj [
[ring.util.mime-type :as mime-type]
[ring.util.response :as response]])))
(def http-methods #{:get :head :patch :delete :options :post :put})
(defrecord Methods [get head post put delete trace options connect patch any])
(def http-methods #{:get :head :post :put :delete :connect :options :trace :patch})
(defrecord Methods [get head post put delete connect options trace patch])
(defrecord Endpoint [data handler path method middleware])
(defn- group-keys [data]
@ -15,6 +18,22 @@
[top (assoc childs k v)]
[(assoc top k v) childs])) [{} {}] data))
(defn routes
"Create a ring handler by combining several handlers into one."
[& handlers]
(let [single-arity (apply some-fn handlers)]
(fn
([request]
(single-arity request))
([request respond raise]
(letfn [(f [handlers]
(if (seq handlers)
(let [handler (first handlers)
respond' #(if % (respond %) (f (rest handlers)))]
(handler request respond' raise))
(respond nil)))]
(f handlers))))))
(defn create-default-handler
"A default ring handler that can handle the following cases,
configured via options:
@ -48,6 +67,41 @@
(respond (error-handler request)))
(respond (not-found request)))))))
#?(:clj
(defn create-resource-handler
"A ring handler for serving classpath resources, configured via options:
| key | description |
| -----------------|-------------|
| :parameter | optional name of the wildcard parameter, defaults to unnamed keyword `:`
| :root | optional resource root, defaults to `\"public\"`
| :path | optional path to mount the handler to. Works only if mounted outside of a router.
| :loader | optional class loader to resolve the resources
| :allow-symlinks? | allow symlinks that lead to paths outside the root classpath directories, defaults to `false`"
([]
(create-resource-handler nil))
([{:keys [parameter root path loader allow-symlinks?]
:or {parameter (keyword "")
root "public"}}]
(let [options {:root root, :loader loader, :allow-symlinks? allow-symlinks?}
path-size (count path)
create (fn [handler]
(fn
([request] (handler request))
([request respond _] (respond (handler request)))))
response (fn [path]
(if-let [response (response/resource-response path options)]
(response/content-type response (mime-type/ext-mime-type path))))
handler (if path
(fn [request]
(let [uri (:uri request)]
(if (>= (count uri) path-size)
(response (subs uri path-size)))))
(fn [request]
(let [path (-> request :path-params parameter)]
(or (response path) {:status 404}))))]
(create handler)))))
(defn ring-handler
"Creates a ring-handler out of a ring-router.
Supports both 1 (sync) and 3 (async) arities.
@ -64,29 +118,24 @@
(let [method (:request-method request :any)
path-params (:path-params match)
result (:result match)
handler (or (-> result method :handler)
(-> result :any (:handler default-handler)))
handler (-> result method :handler (or default-handler))
request (-> request
(impl/fast-assoc :path-params path-params)
(impl/fast-assoc ::r/match match)
(impl/fast-assoc ::r/router router)
(cond-> (seq path-params) (impl/fast-assoc :path-params path-params)))
response (handler request)]
(if (nil? response)
(default-handler request)
response))
(impl/fast-assoc ::r/router router))]
(or (handler request) (default-handler request)))
(default-handler request)))
([request respond raise]
(if-let [match (r/match-by-path router (:uri request))]
(let [method (:request-method request :any)
path-params (:path-params match)
result (:result match)
handler (or (-> result method :handler)
(-> result :any (:handler default-handler)))
handler (-> result method :handler (or default-handler))
request (-> request
(impl/fast-assoc :path-params path-params)
(impl/fast-assoc ::r/match match)
(impl/fast-assoc ::r/router router)
(cond-> (seq path-params) (impl/fast-assoc :path-params path-params)))]
(handler request respond raise))
(impl/fast-assoc ::r/router router))]
((routes handler default-handler) request respond raise))
(default-handler request respond raise))))
{::r/router router}))))
@ -109,14 +158,21 @@
(-> (middleware/compile-result [p d] opts s)
(map->Endpoint)
(assoc :path p)
(assoc :method m)))]
(assoc :method m)))
->methods (fn [any? data]
(reduce
(fn [acc method]
(cond-> acc
any? (assoc method (->endpoint path data method nil))))
(map->Methods {})
http-methods))]
(if-not (seq childs)
(map->Methods {:any (->endpoint path top :any nil)})
(->methods true top)
(reduce-kv
(fn [acc method data]
(let [data (meta-merge top data)]
(assoc acc method (->endpoint path data method method))))
(map->Methods {:any (if (:handler top) (->endpoint path data :any nil))})
(->methods (:handler top) data)
childs))))
(defn router

View file

@ -65,24 +65,24 @@
{:name ::swagger
:spec ::spec})
(defn swagger-spec-handler
"Ring handler to emit swagger spec."
[{:keys [::r/router ::r/match :request-method]}]
(let [{:keys [id] :as swagger} (-> match :result request-method :data :swagger)
swagger (set/rename-keys swagger {:id :x-id})
accept-route #(-> % second :swagger :id (= id))
transform-endpoint (fn [[method {{:keys [coercion no-doc swagger] :as data} :data}]]
(if (and data (not no-doc))
[method
(meta-merge
(if coercion
(coercion/-get-apidocs coercion :swagger data))
(select-keys data [:tags :summary :description])
(dissoc swagger :id))]))
transform-path (fn [[p _ c]]
(if-let [endpoint (some->> c (keep transform-endpoint) (seq) (into {}))]
[p endpoint]))]
(if id
(let [paths (->> router (r/routes) (filter accept-route) (map transform-path) (into {}))]
{:status 200
:body (meta-merge swagger {:paths paths})}))))
(defn create-swagger-handler []
"Create a ring handler to emit swagger spec."
(fn [{:keys [::r/router ::r/match :request-method]}]
(let [{:keys [id] :as swagger} (-> match :result request-method :data :swagger)
swagger (set/rename-keys swagger {:id :x-id})
accept-route #(-> % second :swagger :id (= id))
transform-endpoint (fn [[method {{:keys [coercion no-doc swagger] :as data} :data}]]
(if (and data (not no-doc))
[method
(meta-merge
(if coercion
(coercion/-get-apidocs coercion :swagger data))
(select-keys data [:tags :summary :description])
(dissoc swagger :id))]))
transform-path (fn [[p _ c]]
(if-let [endpoint (some->> c (keep transform-endpoint) (seq) (into {}))]
[p endpoint]))]
(if id
(let [paths (->> router (r/routes) (filter accept-route) (map transform-path) (into {}))]
{:status 200
:body (meta-merge swagger {:paths paths})})))))

View file

@ -69,7 +69,7 @@
;; 25310 / 25126
"regex"
;; 84149 / 84867
;; 88060 / 90778
(title "reitit")
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:2048/product/foo
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:2048/twenty/bar
@ -77,5 +77,5 @@
(assert (= {:status 200, :body "Got twenty id bar"} (app {:request-method :get, :uri "/twenty/bar"}))))
(comment
(web/run app {:port 2048})
(web/run app {:port 2048, :dispatch? false, :server {:always-set-keep-alive false}})
(routing-test))

View file

@ -0,0 +1,38 @@
(ns reitit.ring-perf-test
(:require [criterium.core :as cc]
[reitit.perf-utils :refer :all]
[reitit.ring :as ring]))
;;
;; start repl with `lein perf repl`
;; perf measured with the following setup:
;;
;; Model Name: MacBook Pro
;; Model Identifier: MacBookPro11,3
;; Processor Name: Intel Core i7
;; Processor Speed: 2,5 GHz
;; Number of Processors: 1
;; Total Number of Cores: 4
;; L2 Cache (per Core): 256 KB
;; L3 Cache: 6 MB
;; Memory: 16 GB
;;
(def app
(ring/ring-handler
(ring/router
[["/auth/login" identity]
["/auth/recovery/token/:token" identity]
["/workspace/:project/:page" identity]])))
(comment
(let [request {:request-method :post, :uri "/auth/login"}]
;; 192ns (initial)
;; 163ns (always assoc path params)
;; 132ns (expand methods)
(cc/quick-bench
(app request))
;; 113ns (don't inject router)
;; 89ns (don't inject router & match)
))

View file

@ -0,0 +1,107 @@
(ns reitit.static-perf-test
(:require [reitit.perf-utils :refer :all]
[immutant.web :as web]
[reitit.ring :as ring]
[clojure.java.io :as io]
[criterium.core :as cc]
[ring.util.response]
[ring.middleware.defaults]
[ring.middleware.resource]
[ring.util.mime-type]))
;;
;; start repl with `lein perf repl`
;; perf measured with the following setup:
;;
;; Model Name: MacBook Pro
;; Model Identifier: MacBookPro113
;; Processor Name: Intel Core i7
;; Processor Speed: 2,5 GHz
;; Number of Processors: 1
;; Total Number of Cores: 4
;; L2 Cache (per Core): 256 KB
;; L3 Cache: 6 MB
;; Memory: 16 GB
;;
(def app1
(ring/ring-handler
(ring/router
[["/ping" (constantly {:status 200, :body "pong"})]
["/files/*" (ring/create-resource-handler)]])
(ring/create-default-handler)))
(def app2
(ring/ring-handler
(ring/router
["/ping" (constantly {:status 200, :body "pong"})])
(some-fn
(ring/create-resource-handler {:path "/files"})
(ring/create-default-handler))))
(def wrap-resource
(-> (constantly {:status 200, :body "pong"})
(ring.middleware.resource/wrap-resource "public")))
(def wrap-defaults
(-> (constantly {:status 200, :body "pong"})
(ring.middleware.defaults/wrap-defaults ring.middleware.defaults/site-defaults)))
(comment
(def server (web/run #'app {:port 3000, :dispatch? false, :server {:always-set-keep-alive false}}))
(routing-test))
(defn bench-resources []
;; 134µs
(cc/quick-bench
(ring.util.response/resource-response "hello.json" {:root "public"}))
;; 144µs
(cc/quick-bench
(app1 {:request-method :get, :uri "/files/hello.json"}))
;; 144µs
(cc/quick-bench
(app2 {:request-method :get, :uri "/files/hello.json"}))
;; 143µs
(cc/quick-bench
(wrap-resource {:request-method :get, :uri "/hello.json"}))
;; 163µs
(cc/quick-bench
(wrap-defaults {:request-method :get, :uri "/hello.json"})))
(defn bench-handler []
;; 140ns
(cc/quick-bench
(app1 {:request-method :get, :uri "/ping"}))
;; 134ns
(cc/quick-bench
(app2 {:request-method :get, :uri "/ping"}))
;; 108µs
(cc/quick-bench
(wrap-resource {:request-method :get, :uri "/ping"}))
;; 146µs
(cc/quick-bench
(wrap-defaults {:request-method :get, :uri "/ping"})))
(comment
(bench-resources)
(bench-handler)
(let [file (-> "logback.xml" io/resource io/file)
name (.getName file)]
;; 639ns
(cc/quick-bench
(ring.util.mime-type/ext-mime-type name))
;; 106ns
(cc/quick-bench
(reitit.ring.mime/ext-mime-type name reitit.ring.mime/default-mime-types))))

View file

@ -17,6 +17,7 @@
[metosin/reitit-swagger "0.1.1-SNAPSHOT"]
[meta-merge "1.0.0"]
[ring/ring-core "1.6.3"]
[metosin/spec-tools "0.6.2-SNAPSHOT"]
[metosin/schema-tools "0.10.2-SNAPSHOT"]]
@ -60,7 +61,8 @@
"-Dclojure.compiler.direct-linking=true"]
:test-paths ["perf-test/clj"]
:dependencies [[compojure "1.6.1"]
[org.immutant/immutant "2.1.10"]
[ring/ring-defaults "0.3.1"]
[ikitommi/immutant-web "3.0.0-alpha1"]
[io.pedestal/pedestal.route "0.5.3"]
[org.clojure/core.async "0.4.474"]
[ataraxy "0.4.0"]
@ -70,7 +72,7 @@
"-XX:+PrintCompilation"
"-XX:+UnlockDiagnosticVMOptions"
"-XX:+PrintInlining"]}}
:aliases {"all" ["with-profile" "dev"]
:aliases {"all" ["with-profile" "dev,default"]
"perf" ["with-profile" "default,dev,perf"]
"test-clj" ["all" "do" ["bat-test"] ["check"]]
"test-browser" ["doo" "chrome-headless" "test"]

View file

@ -17,7 +17,7 @@
(defn handler
([{:keys [::mw]}]
{:status 200 :body (conj mw :ok)})
([request respond raise]
([request respond _]
(respond (handler request))))
(deftest ring-router-test
@ -227,11 +227,11 @@
(app {:request-method :post, :uri "/ping"} respond raise)
(is (= 405 (:status (respond))))
(is (= ::nil (raise)))))
(testing "if handler rejects, nil in still returned."
(testing "if handler rejects"
(let [respond (promise)
raise (promise)]
(app {:request-method :get, :uri "/pong"} respond raise)
(is (= nil (respond)))
(is (= 406 (:status (respond))))
(is (= ::nil (raise))))))))))
(deftest middleware-transform-test
@ -264,3 +264,44 @@
(let [app (create {::middleware/transform #(interleave % (repeat (middleware "debug")))})]
(is (= {:status 200, :body [:olipa "debug" :kerran "debug" :avaruus "debug" :ok]}
(app request)))))))
#?(:clj
(deftest resource-handler-test
(doseq [[test app] [["inside a router"
(ring/ring-handler
(ring/router
[["/ping" (constantly {:status 200, :body "pong"})]
["/files/*" (ring/create-resource-handler)]])
(ring/create-default-handler))]
["outside of a router"
(ring/ring-handler
(ring/router
["/ping" (constantly {:status 200, :body "pong"})])
(ring/routes
(ring/create-resource-handler {:path "/files"})
(ring/create-default-handler)))]]]
(testing test
(testing "different file-types"
(let [response (app {:uri "/files/hello.json", :request-method :get})]
(is (= "application/json" (get-in response [:headers "Content-Type"])))
(is (get-in response [:headers "Last-Modified"]))
(is (= "{\"hello\": \"file\"}" (slurp (:body response)))))
(let [response (app {:uri "/files/hello.xml", :request-method :get})]
(is (= "text/xml" (get-in response [:headers "Content-Type"])))
(is (get-in response [:headers "Last-Modified"]))
(is (= "<xml><hello>file</hello></xml>\n" (slurp (:body response))))))
(testing "not found"
(let [response (app {:uri "/files/not-found", :request-method :get})]
(is (= 404 (:status response)))))
(testing "3-arity"
(let [result (atom nil)
respond (partial reset! result)
raise ::not-called]
(app {:uri "/files/hello.xml", :request-method :get} respond raise)
(is (= "text/xml" (get-in @result [:headers "Content-Type"])))
(is (get-in @result [:headers "Last-Modified"]))
(is (= "<xml><hello>file</hello></xml>\n" (slurp (:body @result))))))))))

View file

@ -16,7 +16,7 @@
["/swagger.json"
{:get {:no-doc true
:swagger {:info {:title "my-api"}}
:handler swagger/swagger-spec-handler}}]
:handler (swagger/create-swagger-handler)}}]
["/spec" {:coercion spec/coercion}
["/plus"
@ -59,7 +59,25 @@
:uri "/api/swagger.json"}))]
(is (= {:x-id ::math
:info {:title "my-api"}
:paths {"/api/schema/plus" {:get {:summary "plus"}} ;; TODO: implement!
:paths {"/api/schema/plus" {:get {:parameters [{:description ""
:format "int32"
:in "query"
:name "x"
:required true
:type "integer"}
{:description ""
:format "int32"
:in "query"
:name "y"
:required true
:type "integer"}]
:responses {200 {:description ""
:schema {:additionalProperties false
:properties {"total" {:format "int32"
:type "integer"}}
:required ["total"]
:type "object"}}}
:summary "plus"}}
"/api/spec/plus" {:get {:parameters [{:description ""
:format "int64"
:in "query"