luciano/telemere
1
0
Fork 0
mirror of https://github.com/taoensso/telemere.git synced 2025-12-17 01:51:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

[new] Updates for latest Encore signal toolkit changes

Browse source
This commit is contained in:
Peter Taoussanis 2024-05-14 13:48:35 +02:00
parent 2810ed79a1
commit ca9b27f895
30 changed files with 135 additions and 207 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -30,6 +30,7 @@ Big thanks to those that have patiently helped report and debug issues, especial
* \[new] Ongoing [API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere) and [wiki](https://github.com/taoensso/telemere/wiki) doc improvements
* \[new] [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) can now specify per-handler `:max-shutdown-msecs` (beta12)
* \[new] [`with-handler`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-handler) and [`with-handler+`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-handler+) now auto shutdown handlers after use (beta12)
* \[new] (Advanced) Handler fns can now include `:dispatch-opts` metadata, useful for handler authors that want to set defaults for use by [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) (beta8)
* \[new] Added [Slack handler](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) (beta8)
* \[new] Added [TCP](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) and [UDP](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) socket handlers (beta7)

34
README.md
View file

@ -98,8 +98,8 @@ See for intro and usage:
;;; A quick taste of filtering...
(t/set-ns-filter! {:deny "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter
(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter
(t/set-min-level! :warn) ; Set minimum level
(t/set-min-level! :log :debug) ; Set minimul level for `:log` signals
@ -114,7 +114,7 @@ See [examples.cljc](https://github.com/taoensso/telemere/blob/master/examples.cl
See relevant docstrings (links below) for usage info-
### Signal creators
### Creating signals
| Name | Signal kind | Main arg | Optional arg | Returns |
| :---------------------------------------------------------------------------------------------------------- | :---------- | :------- | :------------- | :--------------------------- |
@ -126,25 +126,19 @@ See relevant docstrings (links below) for usage info-
| [`catch->error!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#catch-%3Eerror!) | `:error` | `form` | `opts`/`id` | Form value or given fallback |
| [`signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#signal!) | `<arb>` | `opts` | - | Depends on opts |
### Signal filters
| Global | Dynamic | Filters by |
| :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------- |
| [`set-kind-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-kind-filter!) | [`with-kind-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-kind-filter) | Signal kind (`:log`, `:event`, etc.) |
| [`set-ns-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-ns-filter!) | [`with-ns-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-ns-filter) | Signal namespace |
| [`set-id-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-id-filter!) | [`with-id-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-id-filter) | Signal id |
| [`set-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-min-level) | [`with-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-min-level) | Signal level (minimum can be specified by kind and/or ns) |
### Internal help
| Var | Help with |
| :---------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content |
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
Help is available without leaving your IDE:
| Var | Help with |
| :---------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Signal options |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) |
| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal and handler filters |
| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handlers |
| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options |
| [`help:environmental-config`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:environmental-config) | Config via JVM properties, environment variables, or classpath resources. |
### Included handlers

21
examples.cljc
View file

@ -39,8 +39,8 @@
;;; A quick taste of filtering...
(t/set-ns-filter! {:deny "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter
(t/set-ns-filter! {:disallow "taoensso.*" :allow "taoensso.sente.*"}) ; Set namespace filter
(t/set-id-filter! {:allow #{::my-particular-id "my-app/*"}}) ; Set id filter
(t/set-min-level! :warn) ; Set minimum level
(t/set-min-level! :log :debug) ; Set minimul level for `log!` signals
@ -185,17 +185,18 @@
(let [handler-fn
(fn a-handler:my-handler ; Note naming convention
;; Shutdown arity - called by Telemere exactly once when the handler
;; is to be shut down.
([]
;; Can no-op, or finalize/free resources as necessary.
)
;; Main arity - called by Telemere whenever the handler should handle
;; the given signal. Never called after shutdown.
;; Main arity, called by Telemere when handler should process given signal
([signal]
;; Do something with given signal (write to console/file/queue/db, etc.).
;; Return value is ignored.
)
;; Optional stop arity for handlers that need to close/release resources or
;; otherwise finalize themselves during system shutdown, etc. Called by
;; Telemere when appropriate, but ONLY IF the handler's dispatch options
;; include a truthy `:needs-stopping?` value (false by default).
([]
;; Close/release resources, etc.
))
;; (Advanced) optional default handler dispatch opts,

2
project.clj
View file

@ -8,7 +8,7 @@
:url "https://www.eclipse.org/legal/epl-v10.html"}
:dependencies
[[com.taoensso/encore "3.110.0"]]
[[com.taoensso/encore "3.112.0"]]
:test-paths ["test" #_"src"]

4
resources/signal-docstrings/catch-to-error!.txt
View file

@ -31,8 +31,8 @@ Tips:
See also `error!`.
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

7
resources/signal-docstrings/filter-env-config.txt → resources/signal-docstrings/environmental-config.txt
View file

@ -1,3 +1,6 @@
Telemere supports extensive environmental config via JVM properties,
environment variables, or classpath resources.
Environmental filter config includes:
Kind filter:
@ -23,10 +26,12 @@ Environmental filter config includes:
Examples:
`taoensso.telemere.rt-min-level.edn` -> ":info"
`TAOENSSO_TELEMERE_RT_NS_FILTER_EDN` -> "{:deny \"taoensso.*\"}"
`TAOENSSO_TELEMERE_RT_NS_FILTER_EDN` -> "{:disallow \"taoensso.*\"}"
`taoensso.telemere.rt-id-filter.cljs.edn` -> "#{:my-id1 :my-id2}"
`TAOENSSO_TELEMERE_RT_KIND_FILTER_CLJ_EDN` -> "nil"
For other (non-filter) environmental config, see the relevant docstrings.
Tips:
- The above ids are for runtime filters (the most common).

4
resources/signal-docstrings/error!.txt
View file

@ -24,8 +24,8 @@ Tips:
- `error` arg is a platform error (`java.lang.Throwable` or `js/Error`).
- Can conveniently be wrapped by `throw`: (throw (error! ...)).
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

4
resources/signal-docstrings/event!.txt
View file

@ -29,8 +29,8 @@ Tips:
Mnemonic: the arg that's typically larger is *always* in the rightmost
position, and for `event!` that's the `level-or-opts` arg.
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

4
resources/signal-docstrings/log!.txt
View file

@ -29,8 +29,8 @@ Tips:
- `msg` arg may be a string, or vector of strings to join with `\space`.
- See also `msg-splice`, `msg-skip` utils.
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

4
resources/signal-docstrings/signal!.txt
View file

@ -21,8 +21,8 @@ Tips:
- Test using `with-signal`: (with-signal (signal! ...)).
- Supports the same options [2] as other signals [1].
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

4
resources/signal-docstrings/signal-creators.txt
View file

@ -26,7 +26,7 @@ and return values), making them more/less convenient for certain use cases:
- `log!` emphasizes messages, while `event!` emphasizes ids.
- `signal!` is the generic creator, and is used by all the others.
-------------------------------------------------------------------
----------------------------------------------------------------------
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

49
resources/signal-docstrings/signal-flow.txt
View file

@ -1,49 +0,0 @@
A signal will be provided to a handler iff ALL of the following are true:
1. Signal (creation) is allowed by compile-time filters
2. Signal (creation) is allowed by runtime filters
3. Signal (handling) is allowed by handler filters
4. Signal middleware does not suppress the signal (return nil)
5. Handler middleware does not suppress the signal (return nil)
For 1-3, filtering may depend on (in order):
Sample rate → namespace → kind → id → level → when form/fn → rate limit
Compile-time vs runtime filtering:
Compile-time filtering is an advanced feature that can be tricky to set
and use correctly. Most folks will want ONLY runtime filtering.
Compile-time filtering works by eliding (completely removing the code for)
disallowed signals. This means zero performance cost for these signals,
but also means that compile-time filtering is PERMANENT once applied.
So if you set `:info` as the compile-time minimum level, that'll REMOVE
CODE for every signal call below `:info` level. To decrease that minimum
level, you'll need to rebuild.
Compile-time filtering can be set ONLY with environmental config
(JVM properties, environment variables, or classpath resources).
Signal and handler sampling is multiplicative:
Both signals and handlers can have independent sample rates, and these
MULTIPLY!
If a signal is created with 20% sampling and a handler handles 50%
of received signals, then 10% of possible signals will be handled
(50% of 20%).
The multiplicative rate is helpfully reflected in each signal's final
`:sample-rate` value.
For more info:
- Signal visual flowchart, Ref. <https://www.taoensso.com/telemere/flow>
- On signal filters, see: `help:signal-filters` docstring
- On handler filters, see: `help:signal-handlers` docstring
If anything is unclear, please ping me (@ptaoussanis) so that I can
improve these docs!

4
resources/signal-docstrings/spy!.txt
View file

@ -34,8 +34,8 @@ Tips:
- Can be useful to wrap with `catch->error!`:
(catch->error! ::error-id (spy! ...)).
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

4
resources/signal-docstrings/trace!.txt
View file

@ -38,8 +38,8 @@ Tips:
refers to the general action of tracing program flow rather than to the
common logging level of the same name.
-------------------------------------------------------------------
----------------------------------------------------------------------
[1] See `help:signal-creators` - (`signal!`, `log!`, `event!`, ...)
[2] See `help:signal-options` - {:keys [kind level id data ...]}
[3] See `help:signal-content` - {:keys [kind level id data ...]}
[4] See `help:signal-flow` - (filters, handling, etc.)
[4] See `help:signal-filters` - (by ns/kind/id/level, sampling, etc.)

2
shadow-cljs.edn
View file

@ -1,7 +1,7 @@
{;;:lein true
:source-paths ["src" "test"]
:dependencies
[[com.taoensso/encore "3.110.0"]
[[com.taoensso/encore "3.112.0"]
[cider/cider-nrepl "0.47.0"]
[binaryage/devtools "1.0.7"]]

32
src/taoensso/telemere.cljc
View file

@ -32,11 +32,12 @@
(remove-ns 'taoensso.telemere)
(:api (enc/interns-overview)))
(enc/assert-min-encore-version [3 110 0])
(enc/assert-min-encore-version [3 112 0])
;;;; TODO
;; - Add handlers: Logstash, Carmine, Datadog, Kafka
;; - Native OpenTelemetry traces and spans
;; - Solution and docs for lib authors
;; - Add handlers: Logstash, Carmine, Datadog, Kafka
;; - Update Tufte (signal API, config API, signal keys, etc.)
;; - Update Timbre (signal API, config API, signal keys, backport improvements)
@ -47,21 +48,7 @@
:sf-arity 4
:ct-sig-filter impl/ct-sig-filter
:*rt-sig-filter* impl/*rt-sig-filter*
:*sig-handlers* impl/*sig-handlers*
:sig-filter-env-config-help
(impl/signal-docstring :filter-env-config)})
(comment help:filters)
(comment
[level-aliases
help:handlers get-handlers add-handler! remove-handler!
with-handler with-handler+ shut-down-handlers!
help:filtering get-filters get-min-level
set-kind-filter! set-ns-filter! set-id-filter! set-min-level!
with-kind-filter with-ns-filter with-id-filter with-min-level])
:*sig-handlers* impl/*sig-handlers*})
;;;; Aliases
@ -90,12 +77,11 @@
;;;; Help
(impl/defhelp help:signal-creators :signal-creators)
(impl/defhelp help:signal-options :signal-options)
(impl/defhelp help:signal-flow :signal-flow)
(impl/defhelp help:signal-content :signal-content)
(enc/defalias help:signal-filters help:filters) ; Via Encore
(enc/defalias help:signal-handlers help:handlers) ; Via Encore
(do
(impl/defhelp help:signal-creators :signal-creators)
(impl/defhelp help:signal-options :signal-options)
(impl/defhelp help:signal-content :signal-content)
(impl/defhelp help:environmental-config :environmental-config))
;;;; Context

6
src/taoensso/telemere/consoles.cljc
View file

@ -35,7 +35,7 @@
(let [error-signal? utils/error-signal?]
(fn a-handler:console
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(let [^java.io.Writer stream
(case stream
@ -70,7 +70,7 @@
(let [js-console-logger utils/js-console-logger]
(fn a-handler:console
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(when-let [output (output-fn signal)]
(let [logger (js-console-logger (get signal :level))]
@ -115,7 +115,7 @@
:raw-error? true})]
(fn a-handler:console-raw
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(let [{:keys [level error]} signal
logger (js-console-logger level)]

9
src/taoensso/telemere/impl.cljc
View file

@ -106,7 +106,7 @@
(def ^:public msg-skip
"For use within signal message vectors.
Special value that will be ignored (no-op) when creating message.
Special value that will be ignored (noop) when creating message.
Useful for conditionally skipping parts of message content, etc.:
(signal! {:msg [\"Hello\" (if <cond> <then> msg-skip) \"world\"] <...>}) or
@ -242,13 +242,10 @@
[ns kind id level signal-value_]
sigs/IFilterableSignal
(allow-signal? [_ sig-filter] (sig-filter ns kind id level))
(signal-value [_ handler-context]
(signal-value [_ handler-sample-rate]
(let [sig-val @signal-value_]
(or
(when-let [handler-sample-rate
(when-let [^taoensso.encore.signals.HandlerContext hc handler-context]
(.-sample-rate hc))]
(when handler-sample-rate
(when (map? sig-val)
;; Replace call sample rate with combined (call * handler) sample rate
(assoc sig-val :sample-rate

2
src/taoensso/telemere/open_telemetry.clj
View file

@ -212,7 +212,7 @@
(let []
(fn a-handler:open-telemetry-logger
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(let [{:keys [ns inst level msg_]} signal
logger (.get logger-provider (or ns "default"))

2
src/taoensso/telemere/postal.clj
View file

@ -107,7 +107,7 @@
(let [handler-fn
(fn a-handler:postal
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(enc/when-let [subject (subject-fn signal)
body (body-fn signal)]

2
src/taoensso/telemere/slack.clj
View file

@ -74,7 +74,7 @@
handler-fn
(fn a-handler:slack
([]) ; Shut down (no-op)
([]) ; Shut down (noop)
([signal]
(when-let [output (output-fn signal)]
(slack.chat/post-message conn-opts channel-id

4
src/taoensso/telemere/timbre.cljc
View file

@ -159,5 +159,5 @@
#?(:clj (defmacro with-context+ "Prefer `telemere/with-ctx+`." [context & body] `(tel/with-ctx+ ~context (do ~@body))))
(defn shutdown-appenders!
"Prefer `telemere/shut-down-handlers!`."
[] (tel/shut-down-handlers!))
"Prefer `telemere/stop-handlers!`."
[] (tel/stop-handlers!))

4
src/taoensso/telemere/utils.cljc
View file

@ -176,7 +176,7 @@
(defn file-writer
"Experimental, subject to change.
Opens the specified file and returns a stateful fn of 2 arities:
[content] => Writes given content to file, or no-ops if closed.
[content] => Writes given content to file, or noops if closed.
[] => Closes the writer.
Useful for basic handlers that write to a file, etc.
@ -273,7 +273,7 @@
(defn tcp-socket-writer
"Experimental, subject to change.
Connects to specified TCP socket and returns a stateful fn of 2 arities:
[content] => Writes given content to socket, or no-ops if closed.
[content] => Writes given content to socket, or noops if closed.
[] => Closes the writer.
Useful for basic handlers that write to a TCP socket, etc.

4
test/taoensso/telemere_tests.cljc
View file

@ -325,8 +325,8 @@
c6 (enc/counter)]
(tel/with-handler :hid1
(fn ([_] (c5)) ([ ] (c6)))
dispatch-opts
(fn ([_] (c5)) ([] (c6)))
(assoc dispatch-opts :needs-stopping? true)
(do
(dotimes [_ n] (fp (fn [] (c1) (tel/event! ::ev-id1 {:run (c2), :do (c3)}) (c4))))
(fp)))

57
wiki/1-Getting-started.md
View file

@ -89,7 +89,7 @@ And setup your namespace imports:
Telemere is configured sensibly out-the-box.
See section [3-Config](./3-Config) for customization.
**Default minimum level**: `:info` (signals with lower levels will no-op).
**Default minimum level**: `:info` (signals with lower levels will noop).
**Default signal handlers**:
@ -122,7 +122,7 @@ Run [`check-intakes`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/tao
# Usage
## Create signals
## Creating signals
Use whichever signal creator is most convenient for your needs:
@ -136,12 +136,12 @@ Use whichever signal creator is most convenient for your needs:
| [`catch->error!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#catch-%3Eerror!) | `:error` | `form` | `opts`/`id` | Form value or given fallback
| [`signal!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#signal!) | `<arb>` | `opts` | - | Depends on opts
- See [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) for more info on signal creators.
- See [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) for signal options (shared by all creators).
- See relevant docstrings (links above) for usage info.
- See [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) for more about signal creators.
- See [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) for options shared by all signal creators.
- See [examples.cljc](https://github.com/taoensso/telemere/blob/master/examples.cljc) for REPL-ready examples.
## Check signals
## Checking signals
Use the [`with-signal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-signal) or (advanced) [`with-signals`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-signals) utils to help test/debug the signals that you're creating:
@ -160,17 +160,18 @@ Use the [`with-signal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/t
Both have several options, see their docstrings (links above) for details.
## Filter signals
## Filtering
A signal will be provided to a handler iff ALL of the following are true:
1. Signal **creation** is allowed by **compile-time** filter config
2. Signal **creation** is allowed by **runtime** filter config
3. Signal **handling** is allowed by **handler** filter config
1. Signal **creation** is allowed by **compile-time** "signal filters"
2. Signal **creation** is allowed by **runtime** "signal filters"
3. Signal **handling** is allowed by **runtime** "handler filters"
4. Signal **middleware** does not suppress the signal (return nil)
5. Handler **middleware** does not suppress the signal (return nil)
For 1-3, filtering may depend on (in order):
All filters (1-3) may depend on (in order):
Sample rate → namespace → kind → id → level → when form/fn → rate limit
Quick examples of some basic filtering:
@ -183,36 +184,24 @@ Quick examples of some basic filtering:
(t/with-min-level :trace ; Override global minimum level
(t/with-signal (t/event! ::my-id1 :debug))) ; => {:keys [inst id ...]}
;; Deny all signals in matching namespaces
(t/set-ns-filter! {:deny "some.nosy.namespace.*"})
;; Disallow all signals in matching namespaces
(t/set-ns-filter! {:disallow "some.nosy.namespace.*"})
```
- Filtering is always O(1), except for rate limits which are O(n_windows).
- Signal and handler sampling is *multiplicative*: if a signal is created with *20%* sampling and a handler handles *50%* of received signals, then *10%* of possible signals will be handled. This multiplicative rate is helpfully reflected in each signal's final `:sample-rate` value.
- See [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) for internal docs on signal flow.
- See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering.
- See section [2-Architecture](./2-Architecture) for a flowchart / visual aid.
Runtime signal filters can be set with:
| Global | Dynamic | Filters by |
| :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------- |
| [`set-kind-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-kind-filter!) | [`with-kind-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-kind-filter) | Signal kind (`:log`, `:event`, etc.) |
| [`set-ns-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-ns-filter!) | [`with-ns-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-ns-filter) | Signal namespace |
| [`set-id-filter!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-id-filter!) | [`with-id-filter`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-id-filter) | Signal id |
| [`set-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#set-min-level) | [`with-min-level`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#with-min-level) | Signal level (minimum can be specified by kind and/or ns) |
- See relevant docstrings (links above) for usage info.
- Compile-time filters are set with environmental config, see section [3-Config](./3-Config).
# Internal help
Telemere includes extensive internal help docstrings:
| Var | Help with |
| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content |
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| Var | Help with |
| :---------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------ |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | Creating signals |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Signal options |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal content (map given to middleware/handlers) |
| [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) | Signal and handler filters |
| [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handlers) | Signal handlers |
| [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) | Signal handler dispatch options |
| [`help:environmental-config`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:environmental-config) | Config via JVM properties, environment variables, or classpath resources.

17
wiki/2-Architecture.md
View file

@ -11,20 +11,9 @@ console/file/queue/db, etc.).
So you *call* a *signal creator* to (conditionally) create a *signal* (map) which is then dispatched to registered _signal handlers_ for (conditional) handling.
This flow is described by [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow), and is visualized below:
This flow is visualized below:
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/signal-flow.svg" alt="Telemere signal flowchart" width="640"/>
- `A/sync queue` semantics are configured per handler when calling [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!). Default semantics are: async with a dropping buffer, and 1 handler thread.
- The shared **signal middleware cache** is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.).
For more info see:
| Var | Help with |
| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content |
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
- `A/sync queue` semantics are specified via [handler dispatch options](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options).
- The shared **signal middleware cache** is super useful when doing signal transformations that are expensive and/or involve side effects (like syncing with another service/db to get a unique tx id, etc.).

14
wiki/3-Config.md
View file

@ -1,20 +1,20 @@
See below for config by topic-
# Signal filtering
# Filtering
A signal will be provided to a handler iff ALL of the following are true:
1. Signal **creation** is allowed by **compile-time** filter config
2. Signal **creation** is allowed by **runtime** filter config
3. Signal **handling** is allowed by **handler** filter config
1. Signal **creation** is allowed by **compile-time** "signal filters"
2. Signal **creation** is allowed by **runtime** "signal filters"
3. Signal **handling** is allowed by **runtime** "handler filters"
4. Signal **middleware** does not suppress the signal (return nil)
5. Handler **middleware** does not suppress the signal (return nil)
For 1-3, filtering may depend on (in order):
All filters (1-3) may depend on (in order):
Sample rate → namespace → kind → id → level → when form/fn → rate limit
- See [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) for info on signal creation filters, **environmental config**, etc.
- See [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) for info on signal handler filters.
See [`help:filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) for more about filtering.
# Signal handlers

27
wiki/4-Handlers.md
View file

@ -38,9 +38,9 @@ There's two kinds of config relevant to all signal handlers:
## Dispatch opts
Handler dispatch opts includes dispatch priority, handler filtering, handler middleware, queue semantics, back-pressure opts, etc.
Handler dispatch opts includes dispatch priority (determines order in which handlers are called), handler filtering, handler middleware, a/sync queue semantics, back-pressure opts, etc.
This is all specified when calling [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) - and documented there.
See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for full info, and [`default-handler-dispatch-opts`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#default-handler-dispatch-opts) for defaults.
Note that handler middleware in particular is an often overlooked but powerful feature, allowing you to arbitrarily transform and/or filter every [signal map](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) before it is given to the handler.
@ -127,7 +127,7 @@ These user-level data/opts are typically NOT included by default in handler outp
# Managing handlers
See [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) for info on handler management.
See [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) for info on handler management.
## Managing handlers on startup
@ -141,6 +141,10 @@ If you want to manage handlers **conditionally** based on **environmental config
Use this to easily define your own arbitrary cross-platform config, and make whatever conditional handler management decisions you'd like.
## Handler stats
By default, Telemere handlers maintain comprehensive internal info about their handling times and outcomes. See [`get-handlers-stats`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#get-handlers-stats) for more.
# Writing handlers
Writing your own signal handlers for Telemere is straightforward, and a reasonable choice if you prefer customizing behaviour that way, or want to write signals to a DB/format/service for which a ready-made handler isn't available.
@ -180,17 +184,18 @@ For more complex cases, or for handlers that you want to make available for use
(let [handler-fn
(fn a-handler:my-handler ; Note naming convention
;; Shutdown arity - called by Telemere exactly once when the handler
;; is to be shut down.
([]
;; Can no-op, or finalize/free resources as necessary.
)
;; Main arity - called by Telemere whenever the handler should handle
;; the given signal. Never called after shutdown.
;; Main arity, called by Telemere when handler should process given signal
([signal]
;; Do something with given signal (write to console/file/queue/db, etc.).
;; Return value is ignored.
)
;; Optional stop arity for handlers that need to close/release resources or
;; otherwise finalize themselves during system shutdown, etc. Called by
;; Telemere when appropriate, but ONLY IF the handler's dispatch options
;; include a truthy `:needs-stopping?` value (false by default).
([]
;; Close/release resources, etc.
))
;; (Advanced) optional default handler dispatch opts,

12
wiki/6-FAQ.md
View file

@ -97,6 +97,16 @@ Note that you can even use `format` or any other formatter/s of your choice. You
See also [`msg-skip`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#msg-skip) and [`msg-splice`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#msg-splice) for some handy utils.
# How does Telemere compare to Mulog?
> [Mulog](https://github.com/BrunoBonacci/mulog) is an excellent "micro-logging library" for Clojure that shares many of the same capabilities and objectives as Telemere!
TODO - will add a comparison here before Telemere's final release.
# How to use Telemere from a library?
TODO - will add advice here before Telemere's final release.
# Other questions?
Please [open a Github issue](https://github.com/taoensso/telemere/issues). I'll regularly update the FAQ to add common questions.
Please [open a Github issue](https://github.com/taoensso/telemere/issues) or ping on Telemere's [Slack channel](https://www.taoensso.com/telemere/slack). I'll regularly update the FAQ to add common questions.

2
wiki/7-Tips.md
View file

@ -54,7 +54,7 @@ Consider the **quantities** of data that'd best suit your needs *for that data*:
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/signal-sampling.svg" alt="Telemere sampling" width="640"/>
Telemere offers [extensive filtering](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) capabilities that help you easily express the **conditions** and **quantities** that make sense for your needs. *Use these* both for their effects and as a *form of documentation*.
Telemere offers [extensive filtering](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:filters) capabilities that help you easily express the **conditions** and **quantities** that make sense for your needs. *Use these* both for their effects and as a *form of documentation*.
## Consider evolution