telemere/wiki/4-Handlers.md

266 lines
16 KiB
Markdown
Raw Normal View History

2024-05-06 14:15:29 +00:00
Signal handlers process created signals to **do something with them** (analyse them, write them to console/file/queue/db, etc.).
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
Telemere includes a number of signal handlers out-the-box, and more may be available via the [community](./8-Community#handlers).
2024-05-08 07:54:08 +00:00
2024-08-05 10:59:18 +00:00
You can also easily [write your own handlers](#writing-handlers) for any output or integration you need.
2024-05-08 07:54:08 +00:00
2024-08-05 10:59:18 +00:00
# Included handlers
2024-05-08 07:54:08 +00:00
2024-08-20 12:00:45 +00:00
> See ✅ links for **features and usage**
> See 👍 links to **vote on handler** for future addition
| Target (↓) | Clj | Cljs |
| :--------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------: |
| [Apache Kafka](https://kafka.apache.org/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| [AWS Kinesis](https://aws.amazon.com/kinesis/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| Console | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) |
| Console (raw) | - | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) |
| [Datadog](https://www.datadoghq.com/) | [👍](https://github.com/taoensso/roadmap/issues/12) | [👍](https://github.com/taoensso/roadmap/issues/12) |
| Email | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.postal#handler:postal) | - |
| [Graylog](https://graylog.org/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| [Jaeger](https://www.jaegertracing.io/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| [Logstash](https://www.elastic.co/logstash) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| [OpenTelemetry](https://opentelemetry.io/) | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.open-telemetry#handler:open-telemetry) | [👍](https://github.com/taoensso/roadmap/issues/12) |
2024-08-20 12:00:45 +00:00
| [Redis](https://redis.io/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| SQL | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
| [Slack](https://slack.com/) | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) | - |
| TCP socket | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) | - |
| UDP socket | [](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) | - |
| [Zipkin](https://zipkin.io/) | [👍](https://github.com/taoensso/roadmap/issues/12) | - |
2024-08-05 10:59:18 +00:00
2024-04-15 07:57:40 +00:00
# Configuring handlers
There's two kinds of config relevant to all signal handlers:
1. **Dispatch** opts (common to all handlers), and
2. **Handler-specific** opts
## Dispatch opts
Handler dispatch opts includes dispatch priority (determines order in which handlers are called), handler filtering, handler middleware, a/sync queue semantics, back-pressure opts, etc.
2024-04-15 07:57:40 +00:00
See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for full info, and [`default-handler-dispatch-opts`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#default-handler-dispatch-opts) for defaults.
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
Note that handler middleware in particular is an often overlooked but powerful feature, allowing you to arbitrarily transform and/or filter every [signal map](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) before it is given to each handler.
2024-04-15 07:57:40 +00:00
## Handler-specific opts
Handler-specific opts are specified when calling a particular **handler constructor** (like [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CONSOLE/api/taoensso.telemere#handler:console)) - and documented by the constructor.
Note that it's common for Telemere handlers to be customized by providing *Clojure/Script functions* to the relevant handler constructor call.
See the [utils namespace](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.utils) for tools useful for customizing and writing signal handlers.
### Example
The standard Clj/s console handler ([`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console)) writes signals **as strings** to `*out*`/`*err` or browser console.
By default it writes formatted strings intended for human consumption:
```clojure
;; Create a test signal
(def my-signal
(t/with-signal
(t/log! {:id ::my-id, :data {:x1 :x2}} "My message")))
;; Create console handler with default opts (writes formatted string)
(def my-handler (t/handler:console))
;; Test handler, remember it's just a (fn [signal])
2024-08-05 10:59:18 +00:00
(my-handler my-signal) ; %>
2024-04-15 07:57:40 +00:00
;; 2024-04-11T10:54:57.202869Z INFO LOG Schrebermann.local examples(56,1) ::my-id - My message
;; data: {:x1 :x2}
```
To instead writes signals as edn:
```clojure
;; Create console which writes edn
(def my-handler
(t/handler:console
{:output-fn (t/pr-signal-fn {:pr-fn :edn})}))
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
(my-handler my-signal) ; %>
2024-04-15 07:57:40 +00:00
;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...}
```
To instead writes signals as JSON:
```clojure
;; Create console which writes signals as JSON
#?(:clj (require '[jsonista.core :as jsonista]))
2024-04-15 07:57:40 +00:00
(def my-handler
(t/handler:console
{:output-fn
(t/pr-signal-fn
{:pr-fn
#?(:cljs :json
:clj jsonista.core/write-value-as-string)})}))
2024-08-05 10:59:18 +00:00
(my-handler my-signal) ; %>
;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...}
2024-04-15 07:57:40 +00:00
```
Note that when writing JSON with Clojure, you *must* provide an appropriate `pr-fn`. This lets you plug in the JSON serializer of your choice ([jsonista](https://github.com/metosin/jsonista) is my default recommendation).
2024-04-15 07:57:40 +00:00
2024-05-03 06:53:16 +00:00
### Handler-specific per-signal kvs
2024-08-05 10:59:18 +00:00
Telemere includes a handy mechanism for including arbitrary app-level data/opts in individual signals for use by custom middleware and/or handlers.
2024-05-03 06:53:16 +00:00
2024-08-05 10:59:18 +00:00
Any *non-standard* (app-level) keys you include in your signal constructor opts will automatically be included in created signals, e.g.:
2024-05-03 06:53:16 +00:00
```clojure
(t/with-signal
(t/event! ::my-id
{:my-middleware-data "foo"
:my-handler-data "bar"}))
;; %>
2024-08-05 10:59:18 +00:00
;; {;; App-level kvs included inline (assoc'd to signal root)
2024-05-03 06:53:16 +00:00
;; :my-middleware-data "foo"
;; :my-handler-data "bar"
;; :kvs ; And also collected together under ":kvs" key
2024-08-05 10:59:18 +00:00
;; {:my-middleware-data "foo"
;; :my-handler-data "bar"}
2024-05-03 06:53:16 +00:00
;; ... }
```
2024-08-05 10:59:18 +00:00
These app-level data/opts are typically NOT included by default in handler output, making them a great way to convey data/opts to custom middleware/handlers.
2024-05-03 06:53:16 +00:00
2024-04-15 07:57:40 +00:00
# Managing handlers
2024-08-05 10:59:18 +00:00
See [`help:handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) for info on signal handler management.
2024-04-15 07:57:40 +00:00
## Managing handlers on startup
Want to add or remove a particular handler when your application starts?
Just make an appropriate call to [`add-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#add-handler!) or [`remove-handler!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#remove-handler!).
2024-08-05 10:59:18 +00:00
### Environmental config
2024-04-15 07:57:40 +00:00
2024-04-25 07:37:55 +00:00
If you want to manage handlers **conditionally** based on **environmental config** (JVM properties, environment variables, or classpath resources) - Telemere provides the highly flexible [`get-env`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#get-env) util.
2024-04-15 07:57:40 +00:00
2024-04-25 07:37:55 +00:00
Use this to easily define your own arbitrary cross-platform config, and make whatever conditional handler management decisions you'd like.
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
## Stopping handlers
Telemere supports complex handlers that may use internal state, buffers, etc.
For this reason, you should **always manually call** [`stop-handlers!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#stop-handlers!) somewhere appropriate to give registered handlers the opportunity to flush buffers, close files, etc.
The best place to do this is usually near the end of your application's `-main` or shutdown procedure, **AFTER** all other code has completed that could create signals.
You can use [`call-on-shutdown!`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#call-on-shutdown!) to create a JVM shutdown hook.
Note that `stop-handlers!` will conveniently **block** to finish async handling of any pending signals. The max blocking time can be configured *per-handler* via the `:drain-msecs` [handler dispatch option](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) and defaults to 6 seconds.
## Handler stats
2024-08-05 10:59:18 +00:00
By default, Telemere handlers maintain **comprehensive internal stats** including handling times and outcome counters.
This can be **really useful** for debugging handlers, and understanding handler performance and back-pressure behaviour in practice.
See [`get-handlers-stats`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#get-handlers-stats) for an output example, etc.
2024-04-15 07:57:40 +00:00
# Writing handlers
Writing your own signal handlers for Telemere is straightforward, and a reasonable choice if you prefer customizing behaviour that way, or want to write signals to a DB/format/service for which a ready-made handler isn't available.
2024-08-05 10:59:18 +00:00
- Signals are just plain Clojure/Script [maps](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content).
- Handlers just plain Clojure/Script fns of 2 arities:
2024-04-15 07:57:40 +00:00
```clojure
2024-08-05 10:59:18 +00:00
(defn my-basic-handler
([signal] (println signal)) ; Arity-1 called when handling a signal
([]) ; Arity-0 called when stopping the handler
)
2024-04-15 07:57:40 +00:00
```
2024-08-05 10:59:18 +00:00
If you're making a customizable handler for use by others, it's often handy to define a handler **constructor**:
2024-04-15 07:57:40 +00:00
```clojure
2024-08-05 10:59:18 +00:00
(defn handler:my-fancy-handler ; Note constructor naming convention
2024-05-05 12:15:09 +00:00
"Needs `some-lib`, Ref. <https://github.com/example/some-lib>.
2024-08-05 10:59:18 +00:00
Returns a signal handler that:
2024-05-05 12:15:09 +00:00
- Takes a Telemere signal (map).
2024-08-05 10:59:18 +00:00
- Does something useful with the signal!
2024-04-15 07:57:40 +00:00
Options:
2024-05-05 12:15:09 +00:00
`:option1` - Option description
`:option2` - Option description
Tips:
- Tip 1
- Tip 2"
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
([] (handler:my-fancy-handler nil)) ; Use default opts (iff defaults viable)
2024-04-15 07:57:40 +00:00
([{:as constructor-opts}]
2024-08-05 10:59:18 +00:00
;; Do option validation and other prep here, i.e. try to keep
;; expensive work outside handler function when possible!
2024-05-05 12:15:09 +00:00
2024-08-05 10:59:18 +00:00
(let [handler-fn ; Fn of exactly 2 arities
(fn a-handler:my-fancy-handler ; Note fn naming convention
2024-05-05 12:15:09 +00:00
2024-08-05 10:59:18 +00:00
([signal] ; Arity-1 called when handling a signal
;; Do something useful with the given signal (write to
;; console/file/queue/db, etc.). Return value is ignored.
)
2024-08-05 10:59:18 +00:00
([] ; Arity-0 called when stopping the handler
;; Flush buffers, close files, etc. May just noop.
;; Return value is ignored.
))]
;; (Advanced, optional) You can use metadata to provide default
;; handler dispatch options (see `help:handler-dispatch-options`)
(with-meta handler-fn
{:dispatch-opts
{:min-level :info
:rate-limit
[[1 1000] ; Max 1 signal per second
[10 60000] ; Max 10 signals per minute
]}}))))
2024-04-15 07:57:40 +00:00
```
- See [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) for signal map content.
2024-08-05 10:59:18 +00:00
- See [`help:handler-dispatch-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:handler-dispatch-options) for dispatch options.
2024-04-15 07:57:40 +00:00
- See the [utils namespace](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.utils) for tools useful for customizing and writing signal handlers.
2024-08-05 10:59:18 +00:00
- Feel free to [ping me](https://github.com/taoensso/telemere/issues) for assistance, or ask on the [`#telemere` Slack channel](https://www.taoensso.com/telemere/slack).
- [PRs](https://github.com/taoensso/telemere/pulls) are **very welcome** for additions to Telemere's included handlers, or to Telemere's [community resources](./8-Community)!
2024-04-15 07:57:40 +00:00
# Example output
```clojure
(t/log! {:id ::my-id, :data {:x1 :x2}} "My message") =>
```
## Clj console handler
2024-08-05 10:59:18 +00:00
[API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | string output:
2024-04-15 07:57:40 +00:00
```
2024-04-11T10:54:57.202869Z INFO LOG Schrebermann.local examples(56,1) ::my-id - My message
data: {:x1 :x2}
```
## Cljs console handler
2024-08-05 10:59:18 +00:00
[API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Chrome console:
2024-04-15 07:57:40 +00:00
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/handler-output-cljs-console.png" alt="Default ClojureScript console handler output" width="640"/>
## Cljs raw console handler
2024-08-05 10:59:18 +00:00
[API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) | Chrome console, with [cljs-devtools](https://github.com/binaryage/cljs-devtools):
2024-04-15 07:57:40 +00:00
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/handler-output-cljs-console-raw.png" alt="Raw ClojureScript console handler output" width="640"/>
## Clj file handler
2024-08-05 10:59:18 +00:00
[API](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:file) | MacOS terminal:
2024-04-15 07:57:40 +00:00
2024-08-05 10:59:18 +00:00
<img src="https://raw.githubusercontent.com/taoensso/telemere/master/imgs/handler-output-clj-file.png" alt="Default Clojure file handler output" width="640"/>