From ca9b27f8951d9f9b72149e3808f3be59e2686dc1 Mon Sep 17 00:00:00 2001 From: Peter Taoussanis Date: Tue, 14 May 2024 13:48:35 +0200 Subject: [PATCH] [new] Updates for latest Encore signal toolkit changes --- CHANGELOG.md | 1 + README.md | 34 +++++------ examples.cljc | 21 +++---- project.clj | 2 +- .../signal-docstrings/catch-to-error!.txt | 4 +- ...nv-config.txt => environmental-config.txt} | 7 ++- resources/signal-docstrings/error!.txt | 4 +- resources/signal-docstrings/event!.txt | 4 +- resources/signal-docstrings/log!.txt | 4 +- resources/signal-docstrings/signal!.txt | 4 +- .../signal-docstrings/signal-creators.txt | 4 +- resources/signal-docstrings/signal-flow.txt | 49 ---------------- resources/signal-docstrings/spy!.txt | 4 +- resources/signal-docstrings/trace!.txt | 4 +- shadow-cljs.edn | 2 +- src/taoensso/telemere.cljc | 32 +++-------- src/taoensso/telemere/consoles.cljc | 6 +- src/taoensso/telemere/impl.cljc | 9 +-- src/taoensso/telemere/open_telemetry.clj | 2 +- src/taoensso/telemere/postal.clj | 2 +- src/taoensso/telemere/slack.clj | 2 +- src/taoensso/telemere/timbre.cljc | 4 +- src/taoensso/telemere/utils.cljc | 4 +- test/taoensso/telemere_tests.cljc | 4 +- wiki/1-Getting-started.md | 57 ++++++++----------- wiki/2-Architecture.md | 17 +----- wiki/3-Config.md | 14 ++--- wiki/4-Handlers.md | 27 +++++---- wiki/6-FAQ.md | 12 +++- wiki/7-Tips.md | 2 +- 30 files changed, 135 insertions(+), 207 deletions(-) rename resources/signal-docstrings/{filter-env-config.txt => environmental-config.txt} (87%) delete mode 100644 resources/signal-docstrings/signal-flow.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index b882ff5..5b34b50 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,7 @@ Big thanks to those that have patiently helped report and debug issues, especial * \[new] Ongoing [API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere) and [wiki](https://github.com/taoensso/telemere/wiki) doc improvements * \[new] [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) can now specify per-handler `:max-shutdown-msecs` (beta12) +* \[new] [`with-handler`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-handler) and [`with-handler+`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-handler+) now auto shutdown handlers after use (beta12) * \[new] (Advanced) Handler fns can now include `:dispatch-opts` metadata, useful for handler authors that want to set defaults for use by [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) (beta8) * \[new] Added [Slack handler](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) (beta8) * \[new] Added [TCP](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) and [UDP](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) socket handlers (beta7) diff --git a/README.md b/README.md index 9df4d22..a5f635d 100644 --- a/README.md +++ b/README.md @@ -98,8 +98,8 @@ See for intro and usage: ;;; A quick taste of filtering... -(t/set-ns-filter! {:deny "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter -(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter +(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter +(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter (t/set-min-level! :warn) ; Set minimum level (t/set-min-level! :log :debug) ; Set minimul level for `:log` signals @@ -114,7 +114,7 @@ See [examples.cljc](https://github.com/taoensso/telemere/blob/master/examples.cl See relevant docstrings (links below) for usage info- -### Signal creators +### Creating signals | Name | Signal kind | Main arg | Optional arg | Returns | | :---------------------------------------------------------------------------------------------------------- | :---------- | :------- | :------------- | :--------------------------- | @@ -126,25 +126,19 @@ See relevant docstrings (links below) for usage info- | [`catch->error!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#catch-%3Eerror!) | `:error` | `form` | `opts`/`id` | Form value or given fallback | | [`signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#signal!) | `` | `opts` | - | Depends on opts | -### Signal filters - -| Global | Dynamic | Filters by | -| :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------- | -| [`set-kind-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-kind-filter!) | [`with-kind-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-kind-filter) | Signal kind (`:log`, `:event`, etc.) | -| [`set-ns-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-ns-filter!) | [`with-ns-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-ns-filter) | Signal namespace | -| [`set-id-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-id-filter!) | [`with-id-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-id-filter) | Signal id | -| [`set-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-min-level) | [`with-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-min-level) | Signal level (minimum can be specified by kind and/or ns) | - ### Internal help -| Var | Help with | -| :---------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- | -| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators | -| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators | -| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content | -| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | -| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | -| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | +Help is available without leaving your IDE: + +| Var | Help with | +| :---------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ | +| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals | +| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Signal options | +| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) | +| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal and handler filters | +| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handlers | +| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options | +| [`help:environmental-config`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:environmental-config) | Config via JVM properties, environment variables, or classpath resources. | ### Included handlers diff --git a/examples.cljc b/examples.cljc index da3a720..0553f40 100644 --- a/examples.cljc +++ b/examples.cljc @@ -39,8 +39,8 @@ ;;; A quick taste of filtering... -(t/set-ns-filter! {:deny "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter -(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter +(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter +(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter (t/set-min-level! :warn) ; Set minimum level (t/set-min-level! :log :debug) ; Set minimul level for `log!` signals @@ -185,17 +185,18 @@ (let [handler-fn (fn a-handler:my-handler ; Note naming convention - ;; Shutdown arity - called by Telemere exactly once when the handler - ;; is to be shut down. - ([] - ;; Can no-op, or finalize/free resources as necessary. - ) - - ;; Main arity - called by Telemere whenever the handler should handle - ;; the given signal. Never called after shutdown. + ;; Main arity, called by Telemere when handler should process given signal ([signal] ;; Do something with given signal (write to console/file/queue/db, etc.). ;; Return value is ignored. + ) + + ;; Optional stop arity for handlers that need to close/release resources or + ;; otherwise finalize themselves during system shutdown, etc. Called by + ;; Telemere when appropriate, but ONLY IF the handler's dispatch options + ;; include a truthy `:needs-stopping?` value (false by default). + ([] + ;; Close/release resources, etc. )) ;; (Advanced) optional default handler dispatch opts, diff --git a/project.clj b/project.clj index 194617d..cf9d831 100644 --- a/project.clj +++ b/project.clj @@ -8,7 +8,7 @@ :url "https://www.eclipse.org/legal/epl-v10.html"} :dependencies - [[com.taoensso/encore "3.110.0"]] + [[com.taoensso/encore "3.112.0"]] :test-paths ["test" #_"src"] diff --git a/resources/signal-docstrings/catch-to-error!.txt b/resources/signal-docstrings/catch-to-error!.txt index 3892292..12d9072 100644 --- a/resources/signal-docstrings/catch-to-error!.txt +++ b/resources/signal-docstrings/catch-to-error!.txt @@ -31,8 +31,8 @@ Tips: See also `error!`. -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/filter-env-config.txt b/resources/signal-docstrings/environmental-config.txt similarity index 87% rename from resources/signal-docstrings/filter-env-config.txt rename to resources/signal-docstrings/environmental-config.txt index 864bf53..0196eb8 100644 --- a/resources/signal-docstrings/filter-env-config.txt +++ b/resources/signal-docstrings/environmental-config.txt @@ -1,3 +1,6 @@ +Telemere supports extensive environmental config via JVM properties, +environment variables, or classpath resources. + Environmental filter config includes: Kind filter: @@ -23,10 +26,12 @@ Environmental filter config includes: Examples: `taoensso.telemere.rt-min-level.edn` -> ":info" - `TAOENSSO_TELEMERE_RT_NS_FILTER_EDN` -> "{:deny \"taoensso.*\"}" + `TAOENSSO_TELEMERE_RT_NS_FILTER_EDN` -> "{:disallow \"taoensso.*\"}" `taoensso.telemere.rt-id-filter.cljs.edn` -> "#{:my-id1 :my-id2}" `TAOENSSO_TELEMERE_RT_KIND_FILTER_CLJ_EDN` -> "nil" +For other (non-filter) environmental config, see the relevant docstrings. + Tips: - The above ids are for runtime filters (the most common). diff --git a/resources/signal-docstrings/error!.txt b/resources/signal-docstrings/error!.txt index 19e1a88..0de6c93 100644 --- a/resources/signal-docstrings/error!.txt +++ b/resources/signal-docstrings/error!.txt @@ -24,8 +24,8 @@ Tips: - `error` arg is a platform error (`java.lang.Throwable` or `js/Error`). - Can conveniently be wrapped by `throw`: (throw (error! ...)). -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/event!.txt b/resources/signal-docstrings/event!.txt index f42213e..8ae4bbc 100644 --- a/resources/signal-docstrings/event!.txt +++ b/resources/signal-docstrings/event!.txt @@ -29,8 +29,8 @@ Tips: Mnemonic: the arg that's typically larger is *always* in the rightmost position, and for `event!` that's the `level-or-opts` arg. -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/log!.txt b/resources/signal-docstrings/log!.txt index 40c64ab..830a889 100644 --- a/resources/signal-docstrings/log!.txt +++ b/resources/signal-docstrings/log!.txt @@ -29,8 +29,8 @@ Tips: - `msg` arg may be a string, or vector of strings to join with `\space`. - See also `msg-splice`, `msg-skip` utils. -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/signal!.txt b/resources/signal-docstrings/signal!.txt index b6429fd..04c6a9e 100644 --- a/resources/signal-docstrings/signal!.txt +++ b/resources/signal-docstrings/signal!.txt @@ -21,8 +21,8 @@ Tips: - Test using `with-signal`: (with-signal (signal! ...)). - Supports the same options [2] as other signals [1]. -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/signal-creators.txt b/resources/signal-docstrings/signal-creators.txt index 84d8e82..0d7c3a8 100644 --- a/resources/signal-docstrings/signal-creators.txt +++ b/resources/signal-docstrings/signal-creators.txt @@ -26,7 +26,7 @@ and return values), making them more/less convenient for certain use cases: - `log!` emphasizes messages, while `event!` emphasizes ids. - `signal!` is the generic creator, and is used by all the others. -------------------------------------------------------------------- +---------------------------------------------------------------------- [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/signal-flow.txt b/resources/signal-docstrings/signal-flow.txt deleted file mode 100644 index 3f616a8..0000000 --- a/resources/signal-docstrings/signal-flow.txt +++ /dev/null @@ -1,49 +0,0 @@ -A signal will be provided to a handler iff ALL of the following are true: - - 1. Signal (creation) is allowed by compile-time filters - 2. Signal (creation) is allowed by runtime filters - 3. Signal (handling) is allowed by handler filters - - 4. Signal middleware does not suppress the signal (return nil) - 5. Handler middleware does not suppress the signal (return nil) - -For 1-3, filtering may depend on (in order): - - Sample rate → namespace → kind → id → level → when form/fn → rate limit - -Compile-time vs runtime filtering: - - Compile-time filtering is an advanced feature that can be tricky to set - and use correctly. Most folks will want ONLY runtime filtering. - - Compile-time filtering works by eliding (completely removing the code for) - disallowed signals. This means zero performance cost for these signals, - but also means that compile-time filtering is PERMANENT once applied. - - So if you set `:info` as the compile-time minimum level, that'll REMOVE - CODE for every signal call below `:info` level. To decrease that minimum - level, you'll need to rebuild. - - Compile-time filtering can be set ONLY with environmental config - (JVM properties, environment variables, or classpath resources). - -Signal and handler sampling is multiplicative: - - Both signals and handlers can have independent sample rates, and these - MULTIPLY! - - If a signal is created with 20% sampling and a handler handles 50% - of received signals, then 10% of possible signals will be handled - (50% of 20%). - - The multiplicative rate is helpfully reflected in each signal's final - `:sample-rate` value. - -For more info: - - - Signal visual flowchart, Ref. - - On signal filters, see: `help:signal-filters` docstring - - On handler filters, see: `help:signal-handlers` docstring - -If anything is unclear, please ping me (@ptaoussanis) so that I can -improve these docs! diff --git a/resources/signal-docstrings/spy!.txt b/resources/signal-docstrings/spy!.txt index 9fb97e3..6eaa4f4 100644 --- a/resources/signal-docstrings/spy!.txt +++ b/resources/signal-docstrings/spy!.txt @@ -34,8 +34,8 @@ Tips: - Can be useful to wrap with `catch->error!`: (catch->error! ::error-id (spy! ...)). -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/resources/signal-docstrings/trace!.txt b/resources/signal-docstrings/trace!.txt index 1a9629e..eae6a45 100644 --- a/resources/signal-docstrings/trace!.txt +++ b/resources/signal-docstrings/trace!.txt @@ -38,8 +38,8 @@ Tips: refers to the general action of tracing program flow rather than to the common logging level of the same name. -------------------------------------------------------------------- +---------------------------------------------------------------------- [1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...) [2] See `help:signal-options` - {:keys [kind level id data ...]} [3] See `help:signal-content` - {:keys [kind level id data ...]} -[4] See `help:signal-flow` - (filters, handling, etc.) +[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.) diff --git a/shadow-cljs.edn b/shadow-cljs.edn index 1561174..5d70bd3 100644 --- a/shadow-cljs.edn +++ b/shadow-cljs.edn @@ -1,7 +1,7 @@ {;;:lein true :source-paths ["src" "test"] :dependencies - [[com.taoensso/encore "3.110.0"] + [[com.taoensso/encore "3.112.0"] [cider/cider-nrepl "0.47.0"] [binaryage/devtools "1.0.7"]] diff --git a/src/taoensso/telemere.cljc b/src/taoensso/telemere.cljc index 6e09f0e..f4e9191 100644 --- a/src/taoensso/telemere.cljc +++ b/src/taoensso/telemere.cljc @@ -32,11 +32,12 @@ (remove-ns 'taoensso.telemere) (:api (enc/interns-overview))) -(enc/assert-min-encore-version [3 110 0]) +(enc/assert-min-encore-version [3 112 0]) ;;;; TODO -;; - Add handlers: Logstash, Carmine, Datadog, Kafka ;; - Native OpenTelemetry traces and spans +;; - Solution and docs for lib authors +;; - Add handlers: Logstash, Carmine, Datadog, Kafka ;; - Update Tufte (signal API, config API, signal keys, etc.) ;; - Update Timbre (signal API, config API, signal keys, backport improvements) @@ -47,21 +48,7 @@ :sf-arity 4 :ct-sig-filter impl/ct-sig-filter :*rt-sig-filter* impl/*rt-sig-filter* - :*sig-handlers* impl/*sig-handlers* - :sig-filter-env-config-help - (impl/signal-docstring :filter-env-config)}) - -(comment help:filters) - -(comment - [level-aliases - - help:handlers get-handlers add-handler! remove-handler! - with-handler with-handler+ shut-down-handlers! - - help:filtering get-filters get-min-level - set-kind-filter! set-ns-filter! set-id-filter! set-min-level! - with-kind-filter with-ns-filter with-id-filter with-min-level]) + :*sig-handlers* impl/*sig-handlers*}) ;;;; Aliases @@ -90,12 +77,11 @@ ;;;; Help -(impl/defhelp help:signal-creators :signal-creators) -(impl/defhelp help:signal-options :signal-options) -(impl/defhelp help:signal-flow :signal-flow) -(impl/defhelp help:signal-content :signal-content) -(enc/defalias help:signal-filters help:filters) ; Via Encore -(enc/defalias help:signal-handlers help:handlers) ; Via Encore +(do + (impl/defhelp help:signal-creators :signal-creators) + (impl/defhelp help:signal-options :signal-options) + (impl/defhelp help:signal-content :signal-content) + (impl/defhelp help:environmental-config :environmental-config)) ;;;; Context diff --git a/src/taoensso/telemere/consoles.cljc b/src/taoensso/telemere/consoles.cljc index e895cc6..bfb6f2e 100644 --- a/src/taoensso/telemere/consoles.cljc +++ b/src/taoensso/telemere/consoles.cljc @@ -35,7 +35,7 @@ (let [error-signal? utils/error-signal?] (fn a-handler:console - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (let [^java.io.Writer stream (case stream @@ -70,7 +70,7 @@ (let [js-console-logger utils/js-console-logger] (fn a-handler:console - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (when-let [output (output-fn signal)] (let [logger (js-console-logger (get signal :level))] @@ -115,7 +115,7 @@ :raw-error? true})] (fn a-handler:console-raw - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (let [{:keys [level error]} signal logger (js-console-logger level)] diff --git a/src/taoensso/telemere/impl.cljc b/src/taoensso/telemere/impl.cljc index 2e92964..81a3098 100644 --- a/src/taoensso/telemere/impl.cljc +++ b/src/taoensso/telemere/impl.cljc @@ -106,7 +106,7 @@ (def ^:public msg-skip "For use within signal message vectors. - Special value that will be ignored (no-op) when creating message. + Special value that will be ignored (noop) when creating message. Useful for conditionally skipping parts of message content, etc.: (signal! {:msg [\"Hello\" (if msg-skip) \"world\"] <...>}) or @@ -242,13 +242,10 @@ [ns kind id level signal-value_] sigs/IFilterableSignal (allow-signal? [_ sig-filter] (sig-filter ns kind id level)) - (signal-value [_ handler-context] + (signal-value [_ handler-sample-rate] (let [sig-val @signal-value_] (or - (when-let [handler-sample-rate - (when-let [^taoensso.encore.signals.HandlerContext hc handler-context] - (.-sample-rate hc))] - + (when handler-sample-rate (when (map? sig-val) ;; Replace call sample rate with combined (call * handler) sample rate (assoc sig-val :sample-rate diff --git a/src/taoensso/telemere/open_telemetry.clj b/src/taoensso/telemere/open_telemetry.clj index b5e0bb8..e34c13f 100644 --- a/src/taoensso/telemere/open_telemetry.clj +++ b/src/taoensso/telemere/open_telemetry.clj @@ -212,7 +212,7 @@ (let [] (fn a-handler:open-telemetry-logger - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (let [{:keys [ns inst level msg_]} signal logger (.get logger-provider (or ns "default")) diff --git a/src/taoensso/telemere/postal.clj b/src/taoensso/telemere/postal.clj index e849322..9292cb4 100644 --- a/src/taoensso/telemere/postal.clj +++ b/src/taoensso/telemere/postal.clj @@ -107,7 +107,7 @@ (let [handler-fn (fn a-handler:postal - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (enc/when-let [subject (subject-fn signal) body (body-fn signal)] diff --git a/src/taoensso/telemere/slack.clj b/src/taoensso/telemere/slack.clj index 2ca78bb..a727b20 100644 --- a/src/taoensso/telemere/slack.clj +++ b/src/taoensso/telemere/slack.clj @@ -74,7 +74,7 @@ handler-fn (fn a-handler:slack - ([]) ; Shut down (no-op) + ([]) ; Shut down (noop) ([signal] (when-let [output (output-fn signal)] (slack.chat/post-message conn-opts channel-id diff --git a/src/taoensso/telemere/timbre.cljc b/src/taoensso/telemere/timbre.cljc index a9822ca..a560b0d 100644 --- a/src/taoensso/telemere/timbre.cljc +++ b/src/taoensso/telemere/timbre.cljc @@ -159,5 +159,5 @@ #?(:clj (defmacro with-context+ "Prefer `telemere/with-ctx+`." [context & body] `(tel/with-ctx+ ~context (do ~@body)))) (defn shutdown-appenders! - "Prefer `telemere/shut-down-handlers!`." - [] (tel/shut-down-handlers!)) + "Prefer `telemere/stop-handlers!`." + [] (tel/stop-handlers!)) diff --git a/src/taoensso/telemere/utils.cljc b/src/taoensso/telemere/utils.cljc index 40472b0..d78b401 100644 --- a/src/taoensso/telemere/utils.cljc +++ b/src/taoensso/telemere/utils.cljc @@ -176,7 +176,7 @@ (defn file-writer "Experimental, subject to change. Opens the specified file and returns a stateful fn of 2 arities: - [content] => Writes given content to file, or no-ops if closed. + [content] => Writes given content to file, or noops if closed. [] => Closes the writer. Useful for basic handlers that write to a file, etc. @@ -273,7 +273,7 @@ (defn tcp-socket-writer "Experimental, subject to change. Connects to specified TCP socket and returns a stateful fn of 2 arities: - [content] => Writes given content to socket, or no-ops if closed. + [content] => Writes given content to socket, or noops if closed. [] => Closes the writer. Useful for basic handlers that write to a TCP socket, etc. diff --git a/test/taoensso/telemere_tests.cljc b/test/taoensso/telemere_tests.cljc index 4793118..a11851e 100644 --- a/test/taoensso/telemere_tests.cljc +++ b/test/taoensso/telemere_tests.cljc @@ -325,8 +325,8 @@ c6 (enc/counter)] (tel/with-handler :hid1 - (fn ([_] (c5)) ([ ] (c6))) - dispatch-opts + (fn ([_] (c5)) ([] (c6))) + (assoc dispatch-opts :needs-stopping? true) (do (dotimes [_ n] (fp (fn [] (c1) (tel/event! ::ev-id1 {:run (c2), :do (c3)}) (c4)))) (fp))) diff --git a/wiki/1-Getting-started.md b/wiki/1-Getting-started.md index 0d37279..9ac553e 100644 --- a/wiki/1-Getting-started.md +++ b/wiki/1-Getting-started.md @@ -89,7 +89,7 @@ And setup your namespace imports: Telemere is configured sensibly out-the-box. See section [3-Config](./3-Config) for customization. -**Default minimum level**: `:info` (signals with lower levels will no-op). +**Default minimum level**: `:info` (signals with lower levels will noop). **Default signal handlers**: @@ -122,7 +122,7 @@ Run [`check-intakes`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/tao # Usage -## Create signals +## Creating signals Use whichever signal creator is most convenient for your needs: @@ -136,12 +136,12 @@ Use whichever signal creator is most convenient for your needs: | [`catch->error!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#catch-%3Eerror!) | `:error` | `form` | `opts`/`id` | Form value or given fallback | [`signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#signal!) | `` | `opts` | - | Depends on opts -- See [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) for more info on signal creators. -- See [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) for signal options (shared by all creators). - See relevant docstrings (links above) for usage info. +- See [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) for more about signal creators. +- See [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) for options shared by all signal creators. - See [examples.cljc](https://github.com/taoensso/telemere/blob/master/examples.cljc) for REPL-ready examples. -## Check signals +## Checking signals Use the [`with-signal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-signal) or (advanced) [`with-signals`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-signals) utils to help test/debug the signals that you're creating: @@ -160,17 +160,18 @@ Use the [`with-signal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/t Both have several options, see their docstrings (links above) for details. -## Filter signals +## Filtering A signal will be provided to a handler iff ALL of the following are true: - 1. Signal **creation** is allowed by **compile-time** filter config - 2. Signal **creation** is allowed by **runtime** filter config - 3. Signal **handling** is allowed by **handler** filter config + 1. Signal **creation** is allowed by **compile-time** "signal filters" + 2. Signal **creation** is allowed by **runtime** "signal filters" + 3. Signal **handling** is allowed by **runtime** "handler filters" 4. Signal **middleware** does not suppress the signal (return nil) 5. Handler **middleware** does not suppress the signal (return nil) -For 1-3, filtering may depend on (in order): +All filters (1-3) may depend on (in order): + Sample rate → namespace → kind → id → level → when form/fn → rate limit Quick examples of some basic filtering: @@ -183,36 +184,24 @@ Quick examples of some basic filtering: (t/with-min-level :trace ; Override global minimum level (t/with-signal (t/event! ::my-id1 :debug))) ; => {:keys [inst id ...]} -;; Deny all signals in matching namespaces -(t/set-ns-filter! {:deny "some.nosy.namespace.*"}) +;; Disallow all signals in matching namespaces +(t/set-ns-filter! {:disallow "some.nosy.namespace.*"}) ``` - Filtering is always O(1), except for rate limits which are O(n_windows). -- Signal and handler sampling is *multiplicative*: if a signal is created with *20%* sampling and a handler handles *50%* of received signals, then *10%* of possible signals will be handled. This multiplicative rate is helpfully reflected in each signal's final `:sample-rate` value. -- See [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) for internal docs on signal flow. +- See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering. - See section [2-Architecture](./2-Architecture) for a flowchart / visual aid. -Runtime signal filters can be set with: - -| Global | Dynamic | Filters by | -| :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------- | -| [`set-kind-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-kind-filter!) | [`with-kind-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-kind-filter) | Signal kind (`:log`, `:event`, etc.) | -| [`set-ns-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-ns-filter!) | [`with-ns-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-ns-filter) | Signal namespace | -| [`set-id-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-id-filter!) | [`with-id-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-id-filter) | Signal id | -| [`set-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-min-level) | [`with-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-min-level) | Signal level (minimum can be specified by kind and/or ns) | - -- See relevant docstrings (links above) for usage info. -- Compile-time filters are set with environmental config, see section [3-Config](./3-Config). - # Internal help Telemere includes extensive internal help docstrings: -| Var | Help with | -| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- | -| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators | -| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators | -| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content | -| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | -| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | -| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | +| Var | Help with | +| :---------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ | +| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals | +| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Signal options | +| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) | +| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal and handler filters | +| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handlers | +| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options | +| [`help:environmental-config`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:environmental-config) | Config via JVM properties, environment variables, or classpath resources. diff --git a/wiki/2-Architecture.md b/wiki/2-Architecture.md index cb99aea..c30528e 100644 --- a/wiki/2-Architecture.md +++ b/wiki/2-Architecture.md @@ -11,20 +11,9 @@ console/file/queue/db, etc.). So you *call* a *signal creator* to (conditionally) create a *signal* (map) which is then dispatched to registered _signal handlers_ for (conditional) handling. -This flow is described by [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow), and is visualized below: +This flow is visualized below: Telemere signal flowchart -- `A/sync queue` semantics are configured per handler when calling [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!). Default semantics are: async with a dropping buffer, and 1 handler thread. -- The shared **signal middleware cache** is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.). - -For more info see: - -| Var | Help with | -| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- | -| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators | -| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators | -| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content | -| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | -| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | -| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | +- `A/sync queue` semantics are specified via [handler dispatch options](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options). +- The shared **signal middleware cache** is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.). \ No newline at end of file diff --git a/wiki/3-Config.md b/wiki/3-Config.md index 7dfa546..346cd97 100644 --- a/wiki/3-Config.md +++ b/wiki/3-Config.md @@ -1,20 +1,20 @@ See below for config by topic- -# Signal filtering +# Filtering A signal will be provided to a handler iff ALL of the following are true: - 1. Signal **creation** is allowed by **compile-time** filter config - 2. Signal **creation** is allowed by **runtime** filter config - 3. Signal **handling** is allowed by **handler** filter config + 1. Signal **creation** is allowed by **compile-time** "signal filters" + 2. Signal **creation** is allowed by **runtime** "signal filters" + 3. Signal **handling** is allowed by **runtime** "handler filters" 4. Signal **middleware** does not suppress the signal (return nil) 5. Handler **middleware** does not suppress the signal (return nil) -For 1-3, filtering may depend on (in order): +All filters (1-3) may depend on (in order): + Sample rate → namespace → kind → id → level → when form/fn → rate limit -- See [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) for info on signal creation filters, **environmental config**, etc. -- See [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) for info on signal handler filters. +See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering. # Signal handlers diff --git a/wiki/4-Handlers.md b/wiki/4-Handlers.md index 9d52619..6ecd149 100644 --- a/wiki/4-Handlers.md +++ b/wiki/4-Handlers.md @@ -38,9 +38,9 @@ There's two kinds of config relevant to all signal handlers: ## Dispatch opts -Handler dispatch opts includes dispatch priority, handler filtering, handler middleware, queue semantics, back-pressure opts, etc. +Handler dispatch opts includes dispatch priority (determines order in which handlers are called), handler filtering, handler middleware, a/sync queue semantics, back-pressure opts, etc. -This is all specified when calling [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) - and documented there. +See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for full info, and [`default-handler-dispatch-opts`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#default-handler-dispatch-opts) for defaults. Note that handler middleware in particular is an often overlooked but powerful feature, allowing you to arbitrarily transform and/or filter every [signal map](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) before it is given to the handler. @@ -127,7 +127,7 @@ These user-level data/opts are typically NOT included by default in handler outp # Managing handlers -See [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) for info on handler management. +See [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) for info on handler management. ## Managing handlers on startup @@ -141,6 +141,10 @@ If you want to manage handlers **conditionally** based on **environmental config Use this to easily define your own arbitrary cross-platform config, and make whatever conditional handler management decisions you'd like. +## Handler stats + +By default, Telemere handlers maintain comprehensive internal info about their handling times and outcomes. See [`get-handlers-stats`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#get-handlers-stats) for more. + # Writing handlers Writing your own signal handlers for Telemere is straightforward, and a reasonable choice if you prefer customizing behaviour that way, or want to write signals to a DB/format/service for which a ready-made handler isn't available. @@ -180,17 +184,18 @@ For more complex cases, or for handlers that you want to make available for use (let [handler-fn (fn a-handler:my-handler ; Note naming convention - ;; Shutdown arity - called by Telemere exactly once when the handler - ;; is to be shut down. - ([] - ;; Can no-op, or finalize/free resources as necessary. - ) - - ;; Main arity - called by Telemere whenever the handler should handle - ;; the given signal. Never called after shutdown. + ;; Main arity, called by Telemere when handler should process given signal ([signal] ;; Do something with given signal (write to console/file/queue/db, etc.). ;; Return value is ignored. + ) + + ;; Optional stop arity for handlers that need to close/release resources or + ;; otherwise finalize themselves during system shutdown, etc. Called by + ;; Telemere when appropriate, but ONLY IF the handler's dispatch options + ;; include a truthy `:needs-stopping?` value (false by default). + ([] + ;; Close/release resources, etc. )) ;; (Advanced) optional default handler dispatch opts, diff --git a/wiki/6-FAQ.md b/wiki/6-FAQ.md index 3f06b8c..cf2d7a2 100644 --- a/wiki/6-FAQ.md +++ b/wiki/6-FAQ.md @@ -97,6 +97,16 @@ Note that you can even use `format` or any other formatter/s of your choice. You See also [`msg-skip`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#msg-skip) and [`msg-splice`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#msg-splice) for some handy utils. +# How does Telemere compare to Mulog? + +> [Mulog](https://github.com/BrunoBonacci/mulog) is an excellent "micro-logging library" for Clojure that shares many of the same capabilities and objectives as Telemere! + +TODO - will add a comparison here before Telemere's final release. + +# How to use Telemere from a library? + +TODO - will add advice here before Telemere's final release. + # Other questions? -Please [open a Github issue](https://github.com/taoensso/telemere/issues). I'll regularly update the FAQ to add common questions. \ No newline at end of file +Please [open a Github issue](https://github.com/taoensso/telemere/issues) or ping on Telemere's [Slack channel](https://www.taoensso.com/telemere/slack). I'll regularly update the FAQ to add common questions. \ No newline at end of file diff --git a/wiki/7-Tips.md b/wiki/7-Tips.md index a856f15..11f3582 100644 --- a/wiki/7-Tips.md +++ b/wiki/7-Tips.md @@ -54,7 +54,7 @@ Consider the **quantities** of data that'd best suit your needs *for that data*: Telemere sampling -Telemere offers [extensive filtering](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) capabilities that help you easily express the **conditions** and **quantities** that make sense for your needs. *Use these* both for their effects and as a *form of documentation*. +Telemere offers [extensive filtering](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) capabilities that help you easily express the **conditions** and **quantities** that make sense for your needs. *Use these* both for their effects and as a *form of documentation*. ## Consider evolution