[mod] Rename "middleware" -> "transform" (xfn)

Users caught by this change should receive a clear compile-time error.

Apologies for the nuissance!! This change is part of a final review
of names before the release of v1 final.

Motivations:

  - "xfn" is a lot shorter than "middleware", making it more
    convenient to use at signal calls, compare:

    (log! {:middleware my-fn} "msg")
    (log! {:xfn my-fn} "msg"}

  - "middleware" was originally chosen to match Timbre's terminology,
    but actually carries some misleading connotations that in hindsight
    are probably better avoided while we still have the chance to change
    this.
This commit is contained in:
Peter Taoussanis 2025-03-10 08:53:06 +01:00
parent c78eb07385
commit 7cccf672f5
14 changed files with 113 additions and 114 deletions

View file

@ -113,15 +113,14 @@ It enables you to write code that is **information-verbose by default**.
;; Set minimum level for `event!` signals for particular ns pattern ;; Set minimum level for `event!` signals for particular ns pattern
(t/set-min-level! :event "taoensso.sente.*" :warn) (t/set-min-level! :event "taoensso.sente.*" :warn)
;; Use middleware to: ;; Use transforms (xfns) to filter and/or arbitrarily modify signals
;; - Transform signals ;; by signal data/content/etc.
;; - Filter signals by arb conditions (incl. data/content)
(t/set-middleware! (t/set-xfn!
(fn [signal] (fn [signal]
(if (-> signal :data :skip-me?) (if (-> signal :data :skip-me?)
nil ; Filter signal (don't handle) nil ; Filter signal (don't handle)
(assoc signal :passed-through-middleware? true)))) (assoc signal :transformed? true))))
(t/with-signal (t/event! ::my-id {:data {:skip-me? true}})) ; => nil (t/with-signal (t/event! ::my-id {:data {:skip-me? true}})) ; => nil
(t/with-signal (t/event! ::my-id {:data {:skip-me? false}})) ; => {...} (t/with-signal (t/event! ::my-id {:data {:skip-me? false}})) ; => {...}
@ -251,7 +250,7 @@ Detailed help is available without leaving your IDE:
| :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------- | | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals | | [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options when creating signals | | [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options when creating signals |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) | | [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to transforms/handlers) |
| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal filtering and transformation | | [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal filtering and transformation |
| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handler management | | [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handler management |
| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options | | [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options |
@ -314,7 +313,7 @@ You can also easily [write your own handlers](../../wiki/4-Handlers#writing-hand
## Community ## Community
My plan for Telemere is to offer a **stable core of limited scope**, then to focus on making it as easy for the **community** to write additional stuff like handlers, middleware, and utils. My plan for Telemere is to offer a **stable core of limited scope**, then to focus on making it as easy for the **community** to write additional stuff like handlers, transforms, and utils.
See [here](../../wiki/8-Community) for community resources. See [here](../../wiki/8-Community) for community resources.

View file

@ -59,15 +59,14 @@
;; Set minimum level for `event!` signals for particular ns pattern ;; Set minimum level for `event!` signals for particular ns pattern
(t/set-min-level! :event "taoensso.sente.*" :warn) (t/set-min-level! :event "taoensso.sente.*" :warn)
;; Use middleware to: ;; Use transforms (xfns) to filter and/or arbitrarily modify signals
;; - Transform signals ;; by signal data/content/etc.
;; - Filter signals by arb conditions (incl. data/content)
(t/set-middleware! (t/set-xfn!
(fn [signal] (fn [signal]
(if (-> signal :data :skip-me?) (if (-> signal :data :skip-me?)
nil ; Filter signal (don't handle) nil ; Filter signal (don't handle)
(assoc signal :passed-through-middleware? true)))) (assoc signal :transformed? true))))
(t/with-signal (t/event! ::my-id {:data {:skip-me? true}})) ; => nil (t/with-signal (t/event! ::my-id {:data {:skip-me? true}})) ; => nil
(t/with-signal (t/event! ::my-id {:data {:skip-me? false}})) ; => {...} (t/with-signal (t/event! ::my-id {:data {:skip-me? false}})) ; => {...}
@ -333,16 +332,16 @@
(t/with-signal (t/with-signal
(t/event! ::my-id (t/event! ::my-id
{:my-middleware-data "foo" {:my-data-for-xfn "foo"
:my-handler-data "bar"})) :my-data-for-handler "bar"}))
;; %> ;; %>
;; {;; App-level kvs included inline (assoc'd to signal root) ;; {;; App-level kvs included inline (assoc'd to signal root)
;; :my-middleware-data "foo" ;; :my-data-for-xfn "foo"
;; :my-handler-data "bar" ;; :my-data-for-handler "bar"
;; :kvs ; And also collected together under ":kvs" key ;; :kvs ; And also collected together under ":kvs" key
;; {:my-middleware-data "foo" ;; {:my-data-for-xfn "foo"
;; :my-handler-data "bar"} ;; :my-data-for-handler "bar"}
;; ... } ;; ... }
;;;; Misc extra examples ;;;; Misc extra examples

View file

@ -1,5 +1,5 @@
Signals are maps with {:keys [inst id ns level data msg_ ...]}, though they Signals are maps with {:keys [inst id ns level data msg_ ...]}, though they
can be modified by signal and/or handler middleware. can be modified by call and/or handler transform (xfns).
Default signal keys: Default signal keys:
@ -33,7 +33,7 @@ Default signal keys:
<kvs> ---------- Other arb app-level ?kvs given to signal creator. Typically NOT included <kvs> ---------- Other arb app-level ?kvs given to signal creator. Typically NOT included
in handler output, so a great way to provide custom data/opts for use in handler output, so a great way to provide custom data/opts for use
(only) by custom middleware/handlers. (only) by custom transforms/handlers.
If anything is unclear, please ping me (@ptaoussanis) so that I can improve these docs! If anything is unclear, please ping me (@ptaoussanis) so that I can improve these docs!

View file

@ -25,17 +25,18 @@ All options are available for all signal creator calls:
`:coords` ------ Custom ?[line column] to override auto signal callsite info `:coords` ------ Custom ?[line column] to override auto signal callsite info
`:elidable?` --- Should signal be subject to compile-time elision? (Default: true) `:elidable?` --- Should signal be subject to compile-time elision? (Default: true)
`:trace?` ------ Should tracing be enabled for `:run` form?
`:sample-rate` - ?rate ∈ℝ[0,1] for signal sampling (0.75 => allow 75% of signals, nil => allow all) `:sample-rate` - ?rate ∈ℝ[0,1] for signal sampling (0.75 => allow 75% of signals, nil => allow all)
`:when` -------- Arb ?form; when present, form must return truthy to allow signal `:when` -------- Arb ?form; when present, form must return truthy to allow signal
`:rate-limit` -- ?spec as given to `taoensso.telemere/rate-limiter`, see its docstring for details `:rate-limit` -- ?spec as given to `taoensso.telemere/rate-limiter`, see its docstring for details
`:rate-limit-by` When present, rate limits will be enforced independently for each id (any Clojure value!) `:rate-limit-by` When present, rate limits will be enforced independently for each id (any Clojure value!)
`:middleware` -- Optional (fn [signal]) => ?modified-signal to apply when signal is created, as per `with-middleware` `:xfn` --------- Optional transform (fn [signal]) => ?modified-signal to apply when signal is created, as per `with-xfn`
`:middleware+` - Optional (fn [signal]) => ?modified-signal to apply when signal is created, as per `with-middleware+` `:xfn+` -------- Optional extra transform (fn [signal]) => ?modified-signal to apply when signal is created, as per `with-xfn+`
`:trace?` ------ Should tracing be enabled for `:run` form?
<kvs> ---------- Other arb app-level ?kvs to incl. in signal. Typically NOT included in <kvs> ---------- Other arb app-level ?kvs to incl. in signal. Typically NOT included in
handler output, so a great way to provide custom data/opts for use handler output, so a great way to provide custom data/opts for use
(only) by custom middleware/handlers. (only) by custom transforms/handlers.
If anything is unclear, please ping me (@ptaoussanis) so that I can improve these docs! If anything is unclear, please ping me (@ptaoussanis) so that I can improve these docs!

View file

@ -25,7 +25,7 @@
;; Via `sigs/def-api` ;; Via `sigs/def-api`
without-filters with-kind-filter with-ns-filter with-id-filter without-filters with-kind-filter with-ns-filter with-id-filter
with-min-level with-handler with-handler+ with-min-level with-handler with-handler+
with-ctx with-ctx+ with-middleware with-middleware+]]))) with-ctx with-ctx+ with-xfn with-xfn+]])))
(comment (comment
(remove-ns (symbol (str *ns*))) (remove-ns (symbol (str *ns*)))
@ -50,7 +50,7 @@
add-handler! remove-handler! stop-handlers! add-handler! remove-handler! stop-handlers!
^:dynamic *ctx* set-ctx! #?(:clj with-ctx) #?(:clj with-ctx+) ^:dynamic *ctx* set-ctx! #?(:clj with-ctx) #?(:clj with-ctx+)
^:dynamic *middleware* set-middleware! #?(:clj with-middleware) #?(:clj with-middleware+)) ^:dynamic *xfn* set-xfn! #?(:clj with-xfn) #?(:clj with-xfn+))
(sigs/def-api (sigs/def-api
{:sf-arity 4 {:sf-arity 4
@ -73,7 +73,7 @@
enc/chance enc/chance
enc/rate-limiter enc/rate-limiter
enc/newline enc/newline
enc/comp-middleware sigs/comp-xfn
sigs/default-handler-dispatch-opts sigs/default-handler-dispatch-opts
#?(:clj truss/keep-callsite) #?(:clj truss/keep-callsite)

View file

@ -18,7 +18,7 @@
#?(:clj #?(:clj
(enc/declare-remote (enc/declare-remote
^:dynamic taoensso.telemere/*ctx* ^:dynamic taoensso.telemere/*ctx*
^:dynamic taoensso.telemere/*middleware* ^:dynamic taoensso.telemere/*xfn*
^:dynamic taoensso.telemere/*uid-fn* ^:dynamic taoensso.telemere/*uid-fn*
^:dynamic taoensso.telemere/*otel-tracer*)) ^:dynamic taoensso.telemere/*otel-tracer*))
@ -393,7 +393,7 @@
'( [& opts-kvs] '( [& opts-kvs]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, ; Undocumented [#_elide? #_allow? #_callsite-id, ; Undocumented
elidable? coords #_inst #_uid #_middleware #_middleware+, elidable? coords #_inst #_uid #_xfn #_xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
#_ctx #_ctx+ #_parent #_root #_trace?, #_do #_let #_data #_msg #_error #_run #_& #_kvs]}]) #_ctx #_ctx+ #_parent #_root #_trace?, #_do #_let #_data #_msg #_error #_run #_& #_kvs]}])
@ -401,7 +401,7 @@
'( [& opts-kvs] '( [& opts-kvs]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, ; Undocumented [#_elide? #_allow? #_callsite-id, ; Undocumented
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error run & kvs]}]) ctx ctx+ parent root trace?, do let data msg error run & kvs]}])
@ -410,7 +410,7 @@
[level msg] [level msg]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error #_run & kvs]} ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}
msg]) msg])
@ -421,7 +421,7 @@
[id [id
{:as opts-map :keys {:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}]) ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}])
@ -430,7 +430,7 @@
[id run] [id run]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error run & kvs]} ctx ctx+ parent root trace?, do let data msg error run & kvs]}
run]) run])
@ -440,7 +440,7 @@
[level run] [level run]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error run & kvs]} ctx ctx+ parent root trace?, do let data msg error run & kvs]}
run]) run])
@ -450,7 +450,7 @@
[id error] [id error]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error #_run & kvs]} ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}
error]) error])
@ -460,7 +460,7 @@
[id run] [id run]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, catch-val, [#_elide? #_allow? #_callsite-id, catch-val,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error #_run & kvs]} ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}
run]) run])
@ -470,7 +470,7 @@
[opts-or-id] [opts-or-id]
[{:as opts-map :keys [{:as opts-map :keys
[#_elide? #_allow? #_callsite-id, [#_elide? #_allow? #_callsite-id,
elidable? coords inst uid middleware middleware+, elidable? coords inst uid xfn xfn+,
sample-rate kind ns id level when rate-limit rate-limit-by, sample-rate kind ns id level when rate-limit rate-limit-by,
ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}]) ctx ctx+ parent root trace?, do let data msg error #_run & kvs]}])
@ -618,15 +618,15 @@
`(taoensso.encore.signals/update-ctx taoensso.telemere/*ctx* ~ctx+) `(taoensso.encore.signals/update-ctx taoensso.telemere/*ctx* ~ctx+)
(get opts :ctx `taoensso.telemere/*ctx*)) (get opts :ctx `taoensso.telemere/*ctx*))
middleware-form xfn-form
(if-let [middleware+ (get opts :middleware+)] (if-let [xfn+ (get opts :xfn+)]
`(taoensso.encore/comp-middleware taoensso.telemere/*middleware* ~middleware+) `(taoensso.encore.signals/comp-xfn taoensso.telemere/*xfn* ~xfn+)
(get opts :middleware `taoensso.telemere/*middleware*)) (get opts :xfn `taoensso.telemere/*xfn*))
kvs-form kvs-form
(not-empty (not-empty
(dissoc opts (dissoc opts
:elidable? :coords :inst :uid :middleware :middleware+, :elidable? :coords :inst :uid :xfn :xfn+,
:sample-rate :ns :kind :id :level :filter :when #_:rate-limit #_:rate-limit-by, :sample-rate :ns :kind :id :level :filter :when #_:rate-limit #_:rate-limit-by,
:ctx :ctx+ :parent #_:trace?, :do :let :data :msg :error, :ctx :ctx+ :parent #_:trace?, :do :let :data :msg :error,
:run :run-form :run-val, :elide? :allow? #_:callsite-id :otel/context)) :run :run-form :run-val, :elide? :allow? #_:callsite-id :otel/context))
@ -677,14 +677,14 @@
`(enc/bound-delay `(enc/bound-delay
;; Delay (cache) shared by all handlers, incl. `:let` eval, ;; Delay (cache) shared by all handlers, incl. `:let` eval,
;; signal construction, middleware, etc. Throws caught by handler. ;; signal construction, transform (xfn), etc. Throws caught by handler.
~do-form ~do-form
(let [~@let-form ; Allow to throw, eval BEFORE data, msg, etc. (let [~@let-form ; Allow to throw, eval BEFORE data, msg, etc.
signal# ~signal-form] signal# ~signal-form]
;; Final unwrapped signal value visible to users/handler-fns, allow to throw ;; Final unwrapped signal value visible to users/handler-fns, allow to throw
(if-let [sig-middleware# ~middleware-form] (if-let [xfn# ~xfn-form]
(sig-middleware# signal#) ; Apply signal middleware, can throw (xfn# signal#) ; Apply call transform, can throw
(do signal#))))) (do signal#)))))
;; Trade-off: avoid double `run-form` expansion ;; Trade-off: avoid double `run-form` expansion

View file

@ -1,5 +1,5 @@
(ns taoensso.telemere.utils (ns taoensso.telemere.utils
"Misc utils useful for Telemere handlers, middleware, etc." "Misc utils useful for Telemere handlers, transforms, etc."
(:refer-clojure :exclude [newline]) (:refer-clojure :exclude [newline])
(:require (:require
[clojure.string :as str] [clojure.string :as str]

View file

@ -250,35 +250,35 @@
(with-sig (sig! {:level :info, :ctx+ {:baz :qux}})))] (with-sig (sig! {:level :info, :ctx+ {:baz :qux}})))]
(is (sm? sv {:ctx {:foo :bar, :baz :qux}}) "`*ctx*` can be updated via call opt"))]) (is (sm? sv {:ctx {:foo :bar, :baz :qux}}) "`*ctx*` can be updated via call opt"))])
(testing "Middleware" (testing "Transforms"
[(testing "Dynamic middleware (`*middleware*`)" [(testing "Dynamic transforms (`*xfn*`)"
[(is (sm? (tel/with-middleware nil (with-sig (sig! {:level :info }))) {:level :info }) "nil middleware ~ identity") [(is (sm? (tel/with-xfn nil (with-sig (sig! {:level :info }))) {:level :info }) "nil xfn ~ identity")
(is (sm? (tel/with-middleware identity (with-sig (sig! {:level :info }))) {:level :info }) "nil middleware ~ identity") (is (sm? (tel/with-xfn identity (with-sig (sig! {:level :info }))) {:level :info }) "nil xfn ~ identity")
(is (sm? (tel/with-middleware #(assoc % :foo 1) (with-sig (sig! {:level :info }))) {:level :info, :foo 1 })) (is (sm? (tel/with-xfn #(assoc % :foo 1) (with-sig (sig! {:level :info }))) {:level :info, :foo 1 }))
(is (sm? (tel/with-middleware #(assoc % :foo 1) (with-sig (sig! {:level :info, :middleware #(assoc % :foo 2)}))) {:level :info, :foo 2 }) "call > dynamic") (is (sm? (tel/with-xfn #(assoc % :foo 1) (with-sig (sig! {:level :info, :xfn #(assoc % :foo 2)}))) {:level :info, :foo 2 }) "call > dynamic")
(is (sm? (tel/with-middleware #(assoc % :foo 1) (with-sig (sig! {:level :info, :middleware nil}))) {:level :info, :foo :submap/nx}) "call > dynamic") (is (sm? (tel/with-xfn #(assoc % :foo 1) (with-sig (sig! {:level :info, :xfn nil}))) {:level :info, :foo :submap/nx}) "call > dynamic")
(is (= (tel/with-middleware #(do nil) (with-sig (sig! {:level :info }))) nil) "return nil => suppress") (is (= (tel/with-xfn #(do nil) (with-sig (sig! {:level :info }))) nil) "return nil => suppress")
(is (sm? (tel/with-middleware #(do nil) (with-sig (sig! {:level :info, :middleware nil}))) {:level :info}) "call > dynamic")]) (is (sm? (tel/with-xfn #(do nil) (with-sig (sig! {:level :info, :xfn nil}))) {:level :info}) "call > dynamic")])
(testing "Call middleware" (testing "Call transforms"
(let [c (enc/counter) (let [c (enc/counter)
{rv1 :value, [sv1] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))})) {rv1 :value, [sv1] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c)))}))
{rv2 :value, [sv2] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false})) {rv2 :value, [sv2] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false}))
{rv3 :value, [sv3] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))})) {rv3 :value, [sv3] :signals} (with-sigs :raw nil (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c)))}))
{rv4 :value, [sv4] :signals} (with-sigs :raw nil (sig! {:level :info, :middleware (fn [_] "signal-value")})) {rv4 :value, [sv4] :signals} (with-sigs :raw nil (sig! {:level :info, :xfn (fn [_] "signal-value")}))
{rv5 :value, [sv5] :signals} (with-sigs :raw nil (sig! {:level :info, :middleware (fn [_] nil)}))] {rv5 :value, [sv5] :signals} (with-sigs :raw nil (sig! {:level :info, :xfn (fn [_] nil)}))]
[(is (= rv1 0)) (is (sm? sv1 {:m1 1 :m2 2})) [(is (= rv1 0)) (is (sm? sv1 {:m1 1 :m2 2}))
(is (= rv2 3)) (is (nil? sv2)) (is (= rv2 3)) (is (nil? sv2))
(is (= rv3 4)) (is (sm? sv3 {:m1 5 :m2 6})) (is (= rv3 4)) (is (sm? sv3 {:m1 5 :m2 6}))
(is (= rv4 true)) (is (= sv4 "signal-value")) (is (= rv4 true)) (is (= sv4 "signal-value"))
(is (= rv5 true)) (is (nil? sv5)) (is (= rv5 true)) (is (nil? sv5))
(is (= @c 7) "3x run + 4x middleware")])) (is (= @c 7) "3x run + 4x xfn")]))
(testing "Mixed middleware" (testing "Mixed transforms"
[(let [sv [(let [sv
(binding [tel/*middleware* #(assoc % :foo true)] (binding [tel/*xfn* #(assoc % :foo true)]
(with-sig (sig! {:level :info, :middleware+ #(assoc % :bar true)})))] (with-sig (sig! {:level :info, :xfn+ #(assoc % :bar true)})))]
(is (sm? sv {:foo true, :bar true})))])]) (is (sm? sv {:foo true, :bar true})))])])
#?(:clj #?(:clj
@ -293,35 +293,35 @@
(deftest _handlers (deftest _handlers
;; Basic handler tests are in Encore ;; Basic handler tests are in Encore
[(testing "Handler middleware" [(testing "Handler transforms"
(let [c (enc/counter) (let [c (enc/counter)
sv-h1_ (atom nil) sv-h1_ (atom nil)
sv-h2_ (atom nil) sv-h2_ (atom nil)
wh1 (sigs/wrap-handler :hid1 (fn [sv] (reset! sv-h1_ sv)) nil {:async nil, :middleware (tel/comp-middleware #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))}) wh1 (sigs/wrap-handler :hid1 (fn [sv] (reset! sv-h1_ sv)) nil {:async nil, :xfn (tel/comp-xfn #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))})
wh2 (sigs/wrap-handler :hid2 (fn [sv] (reset! sv-h2_ sv)) nil {:async nil, :middleware (tel/comp-middleware #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))})] wh2 (sigs/wrap-handler :hid2 (fn [sv] (reset! sv-h2_ sv)) nil {:async nil, :xfn (tel/comp-xfn #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))})]
;; Note that call middleware output is cached and shared across all handlers ;; Note that call xfn output is cached and shared across all handlers
(binding [impl/*sig-handlers* [wh1 wh2]] (binding [impl/*sig-handlers* [wh1 wh2]]
(let [;; 1x run + 4x handler middleware + 2x call middleware = 7x (let [;; 1x run + 4x handler xfn + 2x call xfn = 7x
rv1 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))}) rv1 (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c)))})
sv1-h1 @sv-h1_ sv1-h1 @sv-h1_
sv1-h2 @sv-h2_ sv1-h2 @sv-h2_
c1 @c c1 @c
;; 1x run ;; 1x run
rv2 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false}) rv2 (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false})
sv2-h1 @sv-h1_ sv2-h1 @sv-h1_
sv2-h2 @sv-h2_ sv2-h2 @sv-h2_
c2 @c ; 8 c2 @c ; 8
;; 1x run + 4x handler middleware + 2x call middleware = 7x ;; 1x run + 4x handler xfn + 2x call xfn = 7x
rv3 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))}) rv3 (sig! {:level :info, :run (c), :xfn (tel/comp-xfn #(assoc % :m1 (c)) #(assoc % :m2 (c)))})
sv3-h1 @sv-h1_ sv3-h1 @sv-h1_
sv3-h2 @sv-h2_ sv3-h2 @sv-h2_
c3 @c ; 15 c3 @c ; 15
;; 4x handler middleware ;; 4x handler xfn
rv4 (sig! {:level :info, :middleware (fn [_] {:my-sig-val? true})}) rv4 (sig! {:level :info, :xfn (fn [_] {:my-sig-val? true})})
sv4-h1 @sv-h1_ sv4-h1 @sv-h1_
sv4-h2 @sv-h2_ sv4-h2 @sv-h2_
c4 @c] c4 @c]
@ -330,10 +330,10 @@
(is (= rv2 7)) (is (sm? sv2-h1 {:m1 1, :m2 2, :hm1 3, :hm2 4})) (is (sm? sv2-h2 {:m1 1, :m2 2, :hm1 5, :hm2 6})) (is (= rv2 7)) (is (sm? sv2-h1 {:m1 1, :m2 2, :hm1 3, :hm2 4})) (is (sm? sv2-h2 {:m1 1, :m2 2, :hm1 5, :hm2 6}))
(is (= rv3 8)) (is (sm? sv3-h1 {:m1 9, :m2 10, :hm1 11, :hm2 12})) (is (sm? sv3-h2 {:m1 9, :m2 10, :hm1 13, :hm2 14})) (is (= rv3 8)) (is (sm? sv3-h1 {:m1 9, :m2 10, :hm1 11, :hm2 12})) (is (sm? sv3-h2 {:m1 9, :m2 10, :hm1 13, :hm2 14}))
(is (= rv4 true)) (is (sm? sv4-h1 {:my-sig-val? true, :hm1 15, :hm2 16})) (is (sm? sv4-h2 {:my-sig-val? true, :hm1 17, :hm2 18})) (is (= rv4 true)) (is (sm? sv4-h1 {:my-sig-val? true, :hm1 15, :hm2 16})) (is (sm? sv4-h2 {:my-sig-val? true, :hm1 17, :hm2 18}))
(is (= c1 7) "1x run + 4x handler middleware + 2x call middleware") (is (= c1 7) "1x run + 4x handler xfn + 2x call xfn")
(is (= c2 8) "2x run + 4x handler middleware + 2x call middleware") (is (= c2 8) "2x run + 4x handler xfn + 2x call xfn")
(is (= c3 15) "3x run + 8x handler middleware + 4x call middleware") (is (= c3 15) "3x run + 8x handler xfn + 4x call xfn")
(is (= c4 19) "3x run + 12x handler middleware + 4x call middleware")])))) (is (= c4 19) "3x run + 12x handler xfn + 4x call xfn")]))))
(testing "Handler binding conveyance" (testing "Handler binding conveyance"
(let [a (atom nil) (let [a (atom nil)
@ -379,7 +379,7 @@
(test1 64 {:async {:mode :dropping, :buffer-size 64}}) (test1 64 {:async {:mode :dropping, :buffer-size 64}})
(test1 64 {:async {:mode :sliding, :buffer-size 64}})]))))))]) (test1 64 {:async {:mode :sliding, :buffer-size 64}})]))))))])
(def ^:dynamic *throwing-handler-middleware?* false) (def ^:dynamic *throwing-handler-xfn?* false)
(deftest _throwing (deftest _throwing
(let [sv_ (atom :nx) (let [sv_ (atom :nx)
@ -393,7 +393,7 @@
(tel/with-handler :hid1 (tel/with-handler :hid1
(fn [sv] (force (:data sv)) (reset! sv_ sv)) (fn [sv] (force (:data sv)) (reset! sv_ sv))
{:async nil, :error-fn (fn [x] (reset! error_ x)), :rl-error nil, {:async nil, :error-fn (fn [x] (reset! error_ x)), :rl-error nil,
:middleware (fn [sv] (if *throwing-handler-middleware?* (ex1!) sv))} :xfn (fn [sv] (if *throwing-handler-xfn?* (ex1!) sv))}
[(is (->> (sig! {:level :info, :when (ex1!)}) (throws? :ex-info "Ex1")) "`~filterable-expansion/allow` throws at call") [(is (->> (sig! {:level :info, :when (ex1!)}) (throws? :ex-info "Ex1")) "`~filterable-expansion/allow` throws at call")
(is (->> (sig! {:level :info, :inst (ex1!)}) (throws? :ex-info "Ex1")) "`~inst-form` throws at call") (is (->> (sig! {:level :info, :inst (ex1!)}) (throws? :ex-info "Ex1")) "`~inst-form` throws at call")
@ -409,15 +409,15 @@
(is (= @sv_ :nx)) (is (= @sv_ :nx))
(is (sm? @error_ {:handler-id :hid1, :error ex1}))]) (is (sm? @error_ {:handler-id :hid1, :error ex1}))])
(testing "Throwing call middleware" (testing "Throwing call transform"
(reset-state!) (reset-state!)
[(is (true? (sig! {:level :info, :middleware (fn [_] (ex1!))}))) [(is (true? (sig! {:level :info, :xfn (fn [_] (ex1!))})))
(is (= @sv_ :nx)) (is (= @sv_ :nx))
(is (sm? @error_ {:handler-id :hid1, :error ex1}))]) (is (sm? @error_ {:handler-id :hid1, :error ex1}))])
(testing "Throwing handler middleware" (testing "Throwing handler transform"
(reset-state!) (reset-state!)
(binding [*throwing-handler-middleware?* true] (binding [*throwing-handler-xfn?* true]
[(is (true? (sig! {:level :info}))) [(is (true? (sig! {:level :info})))
(is (= @sv_ :nx)) (is (= @sv_ :nx))
(is (sm? @error_ {:handler-id :hid1, :error ex1}))])) (is (sm? @error_ {:handler-id :hid1, :error ex1}))]))

View file

@ -175,10 +175,10 @@ A signal will be provided to a handler iff ALL of the following are true:
- a. Compile time: not applicable - a. Compile time: not applicable
- b. Runtime: sample rate, kind, ns, id, level, when fn, rate limit - b. Runtime: sample rate, kind, ns, id, level, when fn, rate limit
- 3. **Call middleware** `(fn [signal]) => ?modified-signal` returns non-nil - 3. **Call transform** `(fn [signal]) => ?modified-signal` returns non-nil
- 4. **Handler middleware** `(fn [signal]) => ?modified-signal` returns non-nil - 4. **Handler transform** `(fn [signal]) => ?modified-signal` returns non-nil
> Middleware provides a flexible way to modify and/or filter signals by arbitrary signal data/content conditions (return nil to skip). > Transform fns provides a flexible way to modify and/or filter signals by arbitrary signal data/content conditions (return nil to skip handling).
Quick examples of some basic filtering: Quick examples of some basic filtering:
@ -206,7 +206,7 @@ Telemere includes extensive internal help docstrings:
| :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------- | | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals | | [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options when creating signals | | [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options when creating signals |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) | | [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to transforms/handlers) |
| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal filtering and transformation | | [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal filtering and transformation |
| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handler management | | [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handler management |
| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options | | [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options |

View file

@ -16,4 +16,4 @@ This flow is visualized below:
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/signal-flow.svg" alt="Telemere signal flowchart" width="640"/> <img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/signal-flow.svg" alt="Telemere signal flowchart" width="640"/>
- `A/sync queue` semantics are specified via [handler dispatch options](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options). - `A/sync queue` semantics are specified via [handler dispatch options](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options).
- The shared **call middleware cache** is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.). - The shared **call transform** cache is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.).

View file

@ -12,10 +12,10 @@ A signal will be provided to a handler iff ALL of the following are true:
- a. Compile time: not applicable - a. Compile time: not applicable
- b. Runtime: sample rate, kind, ns, id, level, when fn, rate limit - b. Runtime: sample rate, kind, ns, id, level, when fn, rate limit
- 3. **Call middleware** `(fn [signal]) => ?modified-signal` returns non-nil - 3. **Call transform** `(fn [signal]) => ?modified-signal` returns non-nil
- 4. **Handler middleware** `(fn [signal]) => ?modified-signal` returns non-nil - 4. **Handler transform** `(fn [signal]) => ?modified-signal` returns non-nil
> Middleware provides a flexible way to modify and/or filter signals by arbitrary signal data/content conditions (return nil to skip). > Transform fns provides a flexible way to modify and/or filter signals by arbitrary signal data/content conditions (return nil to skip handling).
See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering. See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering.

View file

@ -48,11 +48,11 @@ There's two kinds of config relevant to all signal handlers:
## Dispatch opts ## Dispatch opts
Handler dispatch opts includes dispatch priority (determines order in which handlers are called), handler filtering, handler middleware, a/sync queue semantics, back-pressure opts, etc. Handler dispatch opts includes dispatch priority (determines order in which handlers are called), handler filtering, handler transform, a/sync queue semantics, back-pressure opts, etc.
See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for full info, and [`default-handler-dispatch-opts`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#default-handler-dispatch-opts) for defaults. See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for full info, and [`default-handler-dispatch-opts`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#default-handler-dispatch-opts) for defaults.
Note that handler middleware in particular is an easily overlooked but powerful feature, allowing you to arbitrarily transform and/or filter every [signal map](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) before it is given to each handler. Note that the handler transform is an easily overlooked but powerful feature, allowing you to arbitrarily modify and/or filter every [signal map](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) before it is given to each handler.
## Handler-specific opts ## Handler-specific opts
@ -120,27 +120,27 @@ Note that when writing JSON with Clojure, you *must* provide an appropriate `pr-
### Handler-specific per-signal kvs ### Handler-specific per-signal kvs
Telemere includes a handy mechanism for including arbitrary app-level data/opts in individual signals for use by custom middleware and/or handlers. Telemere includes a handy mechanism for including arbitrary app-level data/opts in individual signals for use by custom transforms and/or handlers.
Any *non-standard* (app-level) keys you include in your signal constructor opts will automatically be included in created signals, e.g.: Any *non-standard* (app-level) keys you include in your signal constructor opts will automatically be included in created signals, e.g.:
```clojure ```clojure
(t/with-signal (t/with-signal
(t/event! ::my-id (t/event! ::my-id
{:my-middleware-data "foo" {:my-data-for-xfn "foo"
:my-handler-data "bar"})) :my-data-for-handler "bar"}))
;; %> ;; %>
;; {;; App-level kvs included inline (assoc'd to signal root) ;; {;; App-level kvs included inline (assoc'd to signal root)
;; :my-middleware-data "foo" ;; :my-data-for-xfn "foo"
;; :my-handler-data "bar" ;; :my-data-for-handler "bar"
;; :kvs ; And also collected together under ":kvs" key ;; :kvs ; And also collected together under ":kvs" key
;; {:my-middleware-data "foo" ;; {:my-data-for-xfn "foo"
;; :my-handler-data "bar"} ;; :my-data-for-handler "bar"}
;; ... } ;; ... }
``` ```
These app-level data/opts are typically NOT included by default in handler output, making them a great way to convey data/opts to custom middleware/handlers. These app-level data/opts are typically NOT included by default in handler output, making them a great way to convey data/opts to custom transforms/handlers.
# Managing handlers # Managing handlers

View file

@ -80,9 +80,9 @@ Consider the [differences](https://www.youtube.com/watch?v=oyLBGkS5ICk) between
This way you can see all your ids in one place, and precise info on when ids were added/removed/changed. This way you can see all your ids in one place, and precise info on when ids were added/removed/changed.
- Use **signal call middleware** to your advantage. - Use **signal call transforms** to your advantage.
The result of call middleware is cached and *shared between all handlers* making it an efficient place to transform signals. For this reason - prefer signal middleware to handler middleware when possible/convenient. The result of call-side signal transforms is cached and *shared between all handlers* making it an efficient place to modify signals going to >1 handler.
- Signal and handler **sampling is multiplicative**. - Signal and handler **sampling is multiplicative**.
@ -94,12 +94,12 @@ Consider the [differences](https://www.youtube.com/watch?v=oyLBGkS5ICk) between
So for `n` randomly sampled signals matching some criteria, you'd have seen an estimated `Σ(1.0/sample-rate_i)` such signals _without_ sampling, etc. So for `n` randomly sampled signals matching some criteria, you'd have seen an estimated `Σ(1.0/sample-rate_i)` such signals _without_ sampling, etc.
- Middleware can return any type, but it's best to return only `nil` or a map. This ensures maximum compatibility with community middleware, handlers, and tools. - Transforms can technically return any type, but it's best to return only `nil` or a map. This ensures maximum compatibility with community transforms, handlers, and tools.
- Middleware can be used to **filter signals** by returning `nil`. - Transforms can be used to **filter signals** by returning `nil`.
- Middleware can be used to **split signals**: - Transforms can be used to **split signals**:
Your middleware can *call signal creators* like any other code. Return `nil` after to filter the source signal. Just be aware that new signals will re-enter your handler queue/s as would any other signal - and so may be subject to handling delay and normal handler queue back-pressure. Your transforms can *call signal creators* like any other code. Return `nil` after to filter the source signal. Just be aware that new signals will re-enter your handler queue/s as would any other signal - and so may be subject to handling delay and normal handler queue back-pressure.
See also the [`dispatch-signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#dispatch-signal!) util. See also the [`dispatch-signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#dispatch-signal!) util.
@ -122,7 +122,7 @@ Consider the [differences](https://www.youtube.com/watch?v=oyLBGkS5ICk) between
Note that all app-level kvs will *also* be available *together* under the signal's `:kvs` key. Note that all app-level kvs will *also* be available *together* under the signal's `:kvs` key.
App-level kvs are typically *not* included in handler output, so are a great way of providing custom data/opts for use (only) by custom middleware or handlers. App-level kvs are typically *not* included in handler output, so are a great way of providing custom data/opts for use (only) by custom transforms or handlers.
- Signal `kind` can be useful in advanced cases. - Signal `kind` can be useful in advanced cases.

View file

@ -1,4 +1,4 @@
My plan for Telemere is to offer a **stable core of limited scope**, then to focus on making it as easy for the **community** to write additional stuff like handlers, middleware, and utils. My plan for Telemere is to offer a **stable core of limited scope**, then to focus on making it as easy for the **community** to write additional stuff like handlers, transforms, and utils.
**PRs very welcome** to add links to this page! **PRs very welcome** to add links to this page!
@ -20,7 +20,7 @@ Includes videos, tutorials, demo projects, etc.
# Handlers and tools # Handlers and tools
Includes libraries or examples for handlers (see [Writing handlers](./4-Handlers#writing-handlers)), middleware, handler utils (e.g. formatters), tools for analyzing signals, etc. [PRs](../wiki#contributions-welcome) welcome for additions! Includes libraries or examples for handlers (see [Writing handlers](./4-Handlers#writing-handlers)), transforms, handler utils (e.g. formatters), tools for analyzing signals, etc. [PRs](../wiki#contributions-welcome) welcome for additions!
| Type | Description | | Type | Description |
| ---- | :------------------------------------------------------------ | | ---- | :------------------------------------------------------------ |