[mod] Rename, refactor signal formatting utils

- Simplified some util name (only relevant to folks customizing handler behaviour)
- Merged `format-signal->edn-fn`, `format-signal->json-fn` to single `pr-signal-fn`
This commit is contained in:
Peter Taoussanis 2024-05-03 13:43:10 +02:00
parent f3659146bf
commit 21cb44e709
14 changed files with 245 additions and 249 deletions

View file

@ -134,14 +134,13 @@ See relevant docstrings (links below) for usage info-
### Internal help ### Internal help
| Var | Help with | | Var | Help with |
| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- | | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators | | [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators | | [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content | | [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content |
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |
### Example handler output ### Example handler output

View file

@ -141,17 +141,19 @@
;; Create console which writes signals as edn ;; Create console which writes signals as edn
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn (taoensso.telemere.utils/format-signal->edn-fn)})) {:output-fn (t/pr-signal-fn :edn)}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...} ;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...}
;; Create console which writes signals as JSON ;; Create console which writes signals as JSON
#?(:clj (require '[jsonista.core :as jsonista]))
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn {:output-fn
(taoensso.telemere.utils/format-signal->json-fn (t/pr-signal-fn
{:pr-json-fn jsonista.core/write-value-as-string})})) #?(:cljs :json
:clj jsonista.core/write-value-as-string))}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...} ;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...}

View file

@ -53,6 +53,7 @@
[io.opentelemetry/opentelemetry-api "1.37.0"] [io.opentelemetry/opentelemetry-api "1.37.0"]
#_[io.opentelemetry/opentelemetry-sdk-extension-autoconfigure "1.37.0"] #_[io.opentelemetry/opentelemetry-sdk-extension-autoconfigure "1.37.0"]
#_[io.opentelemetry/opentelemetry-exporter-otlp "1.37.0"] #_[io.opentelemetry/opentelemetry-exporter-otlp "1.37.0"]
[metosin/jsonista "0.3.8"]
[com.draines/postal "2.0.5"]] [com.draines/postal "2.0.5"]]
:plugins :plugins

View file

@ -1,6 +0,0 @@
Common signal formatters include:
(utils/format-signal->str-fn) {<opts>}) ; For human-readable string output (default)
(utils/format-signal->edn-fn) {<opts>}) ; For edn output
(utils/format-signal->json-fn {<opts>}) ; For JSON output
See relevant docstrings for details.

View file

@ -79,7 +79,9 @@
#?(:clj impl/with-signal) #?(:clj impl/with-signal)
#?(:clj impl/with-signals) #?(:clj impl/with-signals)
#?(:clj impl/signal!) #?(:clj impl/signal!)
utils/error-signal?) utils/error-signal?
utils/pr-signal-fn
utils/format-signal-fn)
;;;; Help ;;;; Help
@ -89,7 +91,6 @@
(impl/defhelp help:signal-content :signal-content) (impl/defhelp help:signal-content :signal-content)
(enc/defalias help:signal-filters help:filters) ; Via Encore (enc/defalias help:signal-filters help:filters) ; Via Encore
(enc/defalias help:signal-handlers help:handlers) ; Via Encore (enc/defalias help:signal-handlers help:handlers) ; Via Encore
(impl/defhelp help:signal-formatters :signal-formatters)
;;;; Context ;;;; Context

View file

@ -24,14 +24,13 @@
signals formatted as edn, JSON, or human-readable strings. signals formatted as edn, JSON, or human-readable strings.
Options: Options:
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters` `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:stream` - `java.io.writer` `:stream` - `java.io.writer`
Defaults to `*err*` if `utils/error-signal?` is true, and `*out*` otherwise." Defaults to `*err*` if `utils/error-signal?` is true, and `*out*` otherwise."
([] (handler:console nil)) ([] (handler:console nil))
([{:keys [format-signal-fn stream] ([{:keys [output-fn stream]
:or {format-signal-fn (utils/format-signal->str-fn)}}] :or {output-fn (utils/format-signal-fn)}}]
(let [stream (case stream :*out* *out*, :*err* *err* stream) (let [stream (case stream :*out* *out*, :*err* *err* stream)
error-signal? utils/error-signal?] error-signal? utils/error-signal?]
@ -41,7 +40,7 @@
([signal] ([signal]
(let [^java.io.Writer stream (let [^java.io.Writer stream
(or stream (if (error-signal? signal) *err* *out*))] (or stream (if (error-signal? signal) *err* *out*))]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(.write stream (str output)) (.write stream (str output))
(.flush stream)))))))) (.flush stream))))))))
@ -57,11 +56,11 @@
signals formatted as edn, JSON, or human-readable strings. signals formatted as edn, JSON, or human-readable strings.
Options: Options:
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters`" `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`"
([] (handler:console nil)) ([] (handler:console nil))
([{:keys [format-signal-fn] ([{:keys [output-fn]
:or {format-signal-fn (utils/format-signal->str-fn)}}] :or {output-fn (utils/format-signal-fn)}}]
(when (exists? js/console) (when (exists? js/console)
(let [js-console-logger utils/js-console-logger] (let [js-console-logger utils/js-console-logger]
@ -69,7 +68,7 @@
(fn a-handler:console (fn a-handler:console
([]) ; Shut down (no-op) ([]) ; Shut down (no-op)
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(let [logger (js-console-logger (get signal :level))] (let [logger (js-console-logger (get signal :level))]
(.call logger logger (str output))))))))))) (.call logger logger (str output)))))))))))
@ -94,16 +93,16 @@
Ref. <https://github.com/binaryage/cljs-devtools>." Ref. <https://github.com/binaryage/cljs-devtools>."
([] (handler:console-raw nil)) ([] (handler:console-raw nil))
([{:keys [format-signal->prelude-fn format-nsecs-fn] :as opts ([{:keys [preamble-fn format-nsecs-fn] :as opts
:or :or
{format-signal->prelude-fn (utils/format-signal->prelude-fn) ; (fn [signal]) {preamble-fn (utils/signal-preamble-fn)
format-nsecs-fn (utils/format-nsecs-fn) ; (fn [nanosecs]) format-nsecs-fn (utils/format-nsecs-fn) ; (fn [nanosecs])
}}] }}]
(when (and (exists? js/console) (exists? js/console.group)) (when (and (exists? js/console) (exists? js/console.group))
(let [js-console-logger utils/js-console-logger (let [js-console-logger utils/js-console-logger
signal-content-handler ; (fn [signal hf vf] content-fn ; (fn [signal append-fn val-fn])
(utils/signal-content-handler (utils/signal-content-fn
{:format-nsecs-fn format-nsecs-fn {:format-nsecs-fn format-nsecs-fn
:format-error-fn nil :format-error-fn nil
:raw-error? true})] :raw-error? true})]
@ -115,8 +114,8 @@
logger (js-console-logger level)] logger (js-console-logger level)]
;; Unfortunately groups have no level ;; Unfortunately groups have no level
(.group js/console (format-signal->prelude-fn signal)) (.group js/console (preamble-fn signal))
(signal-content-handler signal (logger-fn logger) identity) (content-fn signal (logger-fn logger) identity)
(when-let [stack (and error (.-stack (enc/ex-root error)))] (when-let [stack (and error (.-stack (enc/ex-root error)))]
(.call logger logger stack)) (.call logger logger stack))

View file

@ -284,9 +284,10 @@
`/logs/telemere.log-2020-01-01m.8.gz` ; Archive for Jan 2020, part 8 (oldest entries) `/logs/telemere.log-2020-01-01m.8.gz` ; Archive for Jan 2020, part 8 (oldest entries)
Options: Options:
`:format-signal-fn`- (fn [signal]) => output, see `help:signal-formatters`. `:output-fn`- (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:path` - Path string of the target output file (default `logs/telemere.log`). `:path` - Path string of the target output file (default `logs/telemere.log`)
`:interval` - #{nil :daily :weekly :monthly} (default `:monthly`).
`:interval` - #{nil :daily :weekly :monthly} (default `:monthly`)
When non-nil, causes interval-based archives to be maintained. When non-nil, causes interval-based archives to be maintained.
`:max-file-size` #{nil <pos-int>} (default 4MB) `:max-file-size` #{nil <pos-int>} (default 4MB)
@ -300,7 +301,7 @@
([] (handler:file nil)) ([] (handler:file nil))
([{:keys ([{:keys
[format-signal-fn [output-fn
path interval path interval
max-file-size max-file-size
max-num-parts max-num-parts
@ -308,7 +309,7 @@
gzip-archives?] gzip-archives?]
:or :or
{format-signal-fn (utils/format-signal->str-fn) {output-fn (utils/format-signal-fn)
path "logs/telemere.log" ; Main path, we'll ALWAYS write to this exact file path "logs/telemere.log" ; Main path, we'll ALWAYS write to this exact file
interval :monthly interval :monthly
max-file-size (* 1024 1024 4) ; 4MB max-file-size (* 1024 1024 4) ; 4MB
@ -362,7 +363,7 @@
(fn a-handler:file (fn a-handler:file
([] (locking lock (fw))) ; Close writer ([] (locking lock (fw))) ; Close writer
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(let [new-interval? (when interval (new-interval!?)) (let [new-interval? (when interval (new-interval!?))
>max-file-size? (when max-file-size (>max-file-size?)) >max-file-size? (when max-file-size (>max-file-size?))
reset-stream? (or new-interval? >max-file-size?)] reset-stream? (or new-interval? >max-file-size?)]

View file

@ -13,23 +13,23 @@
;;;; Implementation ;;;; Implementation
(defn format-signal->subject-fn (defn signal-subject-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format [signal]) that: Returns a (fn format [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns a formatted email subject like: - Returns a formatted email subject like:
\"INFO EVENT :taoensso.telemere.postal/ev-id1 - msg\"" \"INFO EVENT :taoensso.telemere.postal/ev-id1 - msg\""
([] (format-signal->subject-fn nil)) ([] (signal-subject-fn nil))
([{:keys [max-len subject-signal-key] ([{:keys [max-len subject-signal-key]
:or :or
{max-len 128 {max-len 128
subject-signal-key :postal/subject}}] subject-signal-key :postal/subject}}]
(fn format-signal->subject [signal] (fn signal-subject [signal]
(or (or
(get signal subject-signal-key) ; Custom subject (get signal subject-signal-key) ; Custom subject
;; Simplified `format-signal->prelude-fn` ;; Simplified `utils/signal-preamble-fn`
(let [{:keys [level kind #_ns id msg_]} signal (let [{:keys [level kind #_ns id msg_]} signal
sb (enc/str-builder) sb (enc/str-builder)
s+spc (enc/sb-appender sb " ")] s+spc (enc/sb-appender sb " ")]
@ -41,9 +41,7 @@
(enc/get-substr-by-len (str sb) 0 max-len)))))) (enc/get-substr-by-len (str sb) 0 max-len))))))
(comment (comment ((signal-subject-fn) (tel/with-signal (tel/event! ::ev-id1 #_{:postal/subject "My subject"}))))
((format-signal->subject-fn)
(tel/with-signal (tel/event! ::ev-id1 #_{:postal/subject "My subject"}))))
;;;; Handler ;;;; Handler
@ -80,8 +78,8 @@
:cc \"engineering@example.com\" :cc \"engineering@example.com\"
:X-MyHeader \"A custom header\"} :X-MyHeader \"A custom header\"}
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters` `:subject-fn` - (fn [signal]) => email subject string
`:format-signal->subject-fn` - (fn [signal]) => email subject string `:body-fn` - (fn [signal]) => email body content string, see `format-signal-fn` or `pr-signal-fn`
Tips: Tips:
@ -107,12 +105,12 @@
([{:keys ([{:keys
[postal/conn-opts [postal/conn-opts
postal/msg-opts postal/msg-opts
format-signal-fn subject-fn
format-signal->subject-fn] body-fn]
:or :or
{format-signal-fn (utils/format-signal->str-fn) {subject-fn (signal-subject-fn)
format-signal->subject-fn (format-signal->subject-fn)}}] body-fn (utils/format-signal-fn)}}]
(when-not conn-opts (throw (ex-info "No `:postal/conn-opts` was provided" {}))) (when-not conn-opts (throw (ex-info "No `:postal/conn-opts` was provided" {})))
(when-not msg-opts (throw (ex-info "No `:postal/msg-opts` was provided" {}))) (when-not msg-opts (throw (ex-info "No `:postal/msg-opts` was provided" {})))
@ -121,14 +119,16 @@
(defn a-handler:postal (defn a-handler:postal
([]) ; Shut down (no-op) ([]) ; Shut down (no-op)
([signal] ([signal]
(enc/when-let [content (format-signal-fn signal) (enc/when-let [subject (subject-fn signal)
subject (format-signal->subject-fn signal)] body (body-fn signal)]
(let [msg (let [msg
(assoc msg-opts (assoc msg-opts
:subject (str subject) :subject (str subject)
:body :body
(if (string? body)
[{:type "text/plain; charset=utf-8" [{:type "text/plain; charset=utf-8"
:content (str content)}]) :content (str body)}]
body))
[result ex] [result ex]
(try (try

View file

@ -30,7 +30,7 @@
`port` - Destination TCP socket port int `port` - Destination TCP socket port int
`:socket-opts` - {:keys [ssl? connect-timeout-msecs]} `:socket-opts` - {:keys [ssl? connect-timeout-msecs]}
`:format-signal-fn`- (fn [signal]) => output, see `help:signal-formatters`. `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
Limitations: Limitations:
- Failed writes will be retried only once. - Failed writes will be retried only once.
@ -39,14 +39,14 @@
([host port] (handler:tcp-socket host port nil)) ([host port] (handler:tcp-socket host port nil))
([host port ([host port
{:keys [socket-opts format-signal-fn] {:keys [socket-opts output-fn]
:or {format-signal-fn (utils/format-signal->str-fn)}}] :or {output-fn (utils/format-signal-fn)}}]
(let [sw (utils/tcp-socket-writer host port socket-opts)] (let [sw (utils/tcp-socket-writer host port socket-opts)]
(defn a-handler:tcp-socket (defn a-handler:tcp-socket
([] (sw)) ; Shut down ([] (sw)) ; Shut down
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(sw output))))))) (sw output)))))))
(defn handler:udp-socket (defn handler:udp-socket
@ -60,7 +60,7 @@
`host` - Destination UDP socket hostname string `host` - Destination UDP socket hostname string
`port` - Destination UDP socket port int `port` - Destination UDP socket port int
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters`. `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:max-packet-bytes` - Max packet size (in bytes) before truncating output (default 512) `:max-packet-bytes` - Max packet size (in bytes) before truncating output (default 512)
`:truncation-warning-fn` - Optional (fn [{:keys [max actual signal]}]) to call whenever `:truncation-warning-fn` - Optional (fn [{:keys [max actual signal]}]) to call whenever
output is truncated. Should be appropriately rate-limited! output is truncated. Should be appropriately rate-limited!
@ -75,10 +75,10 @@
([host port] (handler:udp-socket host port nil)) ([host port] (handler:udp-socket host port nil))
([host port ([host port
{:keys [max-packet-bytes truncation-warning-fn format-signal-fn] {:keys [output-fn max-packet-bytes truncation-warning-fn]
:or :or
{max-packet-bytes 512 {output-fn (utils/format-signal-fn)
format-signal-fn (utils/format-signal->str-fn)}}] max-packet-bytes 512}}]
(let [max-packet-bytes (int max-packet-bytes) (let [max-packet-bytes (int max-packet-bytes)
socket (DatagramSocket.) ; No need to change socket once created socket (DatagramSocket.) ; No need to change socket once created
@ -89,7 +89,7 @@
(defn a-handler:udp-socket (defn a-handler:udp-socket
([] (.close socket)) ; Shut down ([] (.close socket)) ; Shut down
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(let [ba (enc/str->utf8-ba (str output)) (let [ba (enc/str->utf8-ba (str output))
ba-len (alength ba) ba-len (alength ba)
packet (DatagramPacket. ba (min ba-len max-packet-bytes))] packet (DatagramPacket. ba (min ba-len max-packet-bytes))]

View file

@ -452,17 +452,21 @@
(do (enc/ex-map (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"})))) (do (enc/ex-map (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"}))))
(println (str "--\n" ((format-error-fn) (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"})))))) (println (str "--\n" ((format-error-fn) (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"}))))))
(defn format-signal->prelude-fn ;;;;
(defn signal-preamble-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format [signal]) that: Returns a (fn preamble [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns a formatted prelude string like: - Returns a signal preamble ?string like:
\"2024-03-26T11:14:51.806Z INFO EVENT Hostname taoensso.telemere(2,21) ::ev-id - msg\"" \"2024-03-26T11:14:51.806Z INFO EVENT Hostname taoensso.telemere(2,21) ::ev-id - msg\"
([] (format-signal->prelude-fn nil))
See arglists for options."
([] (signal-preamble-fn nil))
([{:keys [format-inst-fn] ([{:keys [format-inst-fn]
:or {format-inst-fn (format-inst-fn)}}] :or {format-inst-fn (format-inst-fn)}}]
(fn format-signal->prelude [signal] (fn signal-preamble [signal]
(let [{:keys [inst level kind ns id msg_]} signal (let [{:keys [inst level kind ns id msg_]} signal
sb (enc/str-builder) sb (enc/str-builder)
s+spc (enc/sb-appender sb " ")] s+spc (enc/sb-appender sb " ")]
@ -484,39 +488,54 @@
(when id (s+spc (format-id ns id))) (when id (s+spc (format-id ns id)))
(when-let [msg (force msg_)] (s+spc "- " msg)) (when-let [msg (force msg_)] (s+spc "- " msg))
(str sb)))))
(comment ((format-signal->prelude-fn) (tel/with-signal (tel/event! ::ev-id)))) (when-not (zero? (enc/sb-length sb))
(str sb))))))
(defn ^:no-doc signal-content-handler (comment ((signal-preamble-fn) (tel/with-signal (tel/event! ::ev-id))))
"Private, don't use.
Returns a (fn handle [signal handle-fn value-fn]) for internal use. (defn signal-content-fn
Content equivalent to `format-signal->prelude-fn`." "Experimental, subject to change.
([] (signal-content-handler nil)) Returns a (fn content [signal]) that:
- Takes a Telemere signal.
- Returns a signal content ?string (incl. data, ctx, etc.)
See arglists for options."
([] (signal-content-fn nil))
([{:keys ([{:keys
[format-nsecs-fn [incl-thread? incl-kvs? raw-error?,
format-error-fn format-nsecs-fn format-error-fn]
raw-error?
incl-thread?
incl-kvs?]
:or :or
{format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs]) {format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs])
format-error-fn (format-error-fn) ; (fn [error]) format-error-fn (format-error-fn) ; (fn [error])
}}] }}]
(let [err-start (str newline "<<< error <<<" newline) (let [nl newline
err-stop (str newline ">>> error >>>")] err-start (str nl "<<< error <<<" nl)
err-stop (str nl ">>> error >>>")]
(fn signal-content
([signal]
(let [sb (enc/str-builder)
s++ (enc/sb-appender sb nl)]
(signal-content signal s++ enc/pr-edn*)
(when-not (zero? (enc/sb-length sb))
(str sb))))
;; Undocumented, advanced arity
([signal append-fn val-fn]
(let [af append-fn
vf val-fn]
(fn a-signal-content-handler [signal hf vf]
(let [{:keys [uid parent data kvs ctx #?(:clj thread) sample-rate]} signal] (let [{:keys [uid parent data kvs ctx #?(:clj thread) sample-rate]} signal]
(when sample-rate (hf "sample: " (vf sample-rate))) (when sample-rate (af " sample: " (vf sample-rate)))
(when uid (hf " uid: " (vf uid))) (when uid (af " uid: " (vf uid)))
(when parent (hf "parent: " (vf parent))) (when parent (af " parent: " (vf parent)))
#?(:clj (when (and thread incl-thread?) (hf "thread: " (vf thread)))) #?(:clj (when (and thread incl-thread?) (af " thread: " (vf thread))))
(when data (hf " data: " (vf data))) (when data (af " data: " (vf data)))
(when (and kvs incl-kvs?) (hf " kvs: " (vf kvs))) (when (and kvs incl-kvs?) (af " kvs: " (vf kvs)))
(when ctx (hf " ctx: " (vf ctx)))) (when ctx (af " ctx: " (vf ctx))))
(let [{:keys [run-form error]} signal] (let [{:keys [run-form error]} signal]
(when run-form (when run-form
@ -533,124 +552,105 @@
:nsecs run-nsecs :nsecs run-nsecs
:val run-val :val run-val
#?@(:clj [:val-type (enc/class-sym run-val)])})] #?@(:clj [:val-type (enc/class-sym run-val)])})]
(af " run: " (vf run-info))))
(hf " run: " (vf run-info))))
(when error (when error
(if raw-error? (if raw-error?
(hf " error: " error) (af " error: " error)
(when-let [ff format-error-fn] (when-let [ff format-error-fn]
(hf err-start (ff error) err-stop))))))))) (af err-start (ff error) err-stop)))))))))))
;;;; Signal formatters (comment
((signal-content-fn) (tel/with-signal (tel/event! ::ev-id)))
((signal-content-fn) (tel/with-signal (tel/event! ::ev-id {:data {:k1 "v1"}}))))
(defn format-signal->edn-fn (defn pr-signal-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format->edn [signal]) that: Returns a (fn pr-signal [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns edn string of the (minified) signal." - Returns machine-readable serialized string of the (minified) signal.
([] (format-signal->edn-fn nil))
([{:keys
[incl-kvs? end-with-newline?,
pr-edn-fn prep-fn]
:or Options include:
{end-with-newline? true, `pr-fn` ∈ #{<unary-fn> :edn :json (Cljs only)}
pr-edn-fn pr-edn See arglists for more.
prep-fn (comp error-in-signal->maps minify-signal)}}]
(let [nl newline] Examples:
(fn format-signal->edn [signal] (pr-signal-fn :edn {<opts>})
(let [signal (if (or incl-kvs? (not (map? signal))) signal (dissoc signal :kvs)) (pr-signal-fn :json {<opts>}) ; Cljs only
signal (if prep-fn (prep-fn signal) signal)
output (pr-edn-fn signal)]
(if end-with-newline?
(str output nl)
(do output)))))))
(comment ((format-signal->edn-fn) {:level :info, :msg "msg", :kvs {:k1 :v1}}))
(defn format-signal->json-fn
"Experimental, subject to change.
Returns a (fn format->json [signal]) that:
- Takes a Telemere signal.
- Returns JSON string of the (minified) signal.
(Clj only): An appropriate `:pr-json-fn` MUST be provided.
jsonista is one good option, Ref. <https://github.com/metosin/jsonista>:
;; To output JSON for Clj, you must provide an appropriate `pr-fn`.
;; `jsonista` is a good option, Ref. <https://github.com/metosin/jsonista>:
(require '[jsonista.core :as jsonista]) (require '[jsonista.core :as jsonista])
(format-signal->json-fn {:pr-json-fn jsonista/write-value-as-string ...})" (pr-signal-fn jsonista/write-value-as-string {<opts>})
([] (format-signal->json-fn nil))
([{:keys
[incl-kvs? end-with-newline?,
pr-json-fn prep-fn]
See also `format-signal-fn` for human-readable output."
([pr-fn] (pr-signal-fn pr-fn nil))
([pr-fn
{:keys [incl-thread? incl-kvs? incl-newline?, prep-fn]
:or :or
{end-with-newline? true, {incl-newline? true
#?@(:cljs [pr-json-fn pr-json]) prep-fn
prep-fn (comp error-in-signal->maps minify-signal)}}] (comp error-in-signal->maps
minify-signal)}}]
(when-not pr-json-fn
(throw
(ex-info (str "No `" `format-signal->json-fn "` `:pr-json-fn` was provided") {})))
(let [nl newline]
(fn format-signal->json [signal]
(let [signal (if (or incl-kvs? (not (map? signal))) signal (dissoc signal :kvs))
signal (if prep-fn (prep-fn signal) signal)
output (pr-json-fn signal)]
(if end-with-newline?
(str output nl)
(do output)))))))
(comment ((format-signal->json-fn) {:level :info, :msg "msg", :kvs {:k1 :v1}}))
(defn format-signal->str-fn
"Experimental, subject to change.
Returns a (fn format->str [signal]) that:
- Takes a Telemere signal.
- Returns a formatted string intended for text consoles, etc."
([] (format-signal->str-fn nil))
([{:keys
[format-signal->prelude-fn
format-nsecs-fn
format-error-fn
incl-thread?
incl-kvs?
end-with-newline?]
:or
{format-signal->prelude-fn (format-signal->prelude-fn) ; (fn [signal])
format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs])
format-error-fn (format-error-fn) ; (fn [error])
end-with-newline? true}}]
(let [nl newline (let [nl newline
signal-content-handler ; (fn [signal hf vf] pr-fn
(signal-content-handler (or
{:format-nsecs-fn format-nsecs-fn (case pr-fn
:format-error-fn format-error-fn :edn pr-edn
:incl-thread? incl-thread? #?@(:cljs [:json pr-json])
:incl-kvs? incl-kvs?})]
(fn format-signal->str [signal] (if (fn? pr-fn)
(let [sb (enc/str-builder) (do pr-fn)
s+ (partial enc/sb-append sb) (enc/unexpected-arg! pr-fn
s++ (partial enc/sb-append sb (str newline " "))] {:context `pr-signal-fn
:param 'pr-fn
:expected
#?(:clj '#{:edn unary-fn}
:cljs '#{:edn :json unary-fn})}))
(when-let [ff format-signal->prelude-fn] (s+ (ff signal))) ; Prelude (have fn? pr-fn)))]
(signal-content-handler signal s++ enc/pr-edn*) ; Content
(when end-with-newline? (enc/sb-append sb nl)) (fn pr-signal [signal]
(str sb)))))) (let [not-map? (not (map? signal))
signal (if (or incl-kvs? not-map?) signal (dissoc signal :kvs))
signal (if (or incl-thread? not-map?) signal (dissoc signal :thread))
signal (if prep-fn (prep-fn signal) signal)
output (pr-fn signal)]
(if incl-newline?
(str output nl)
(do output)))))))
(comment ((pr-signal-fn :edn) (tel/with-signal (tel/event! ::ev-id {:kvs {:k1 "v1"}}))))
(defn format-signal-fn
"Experimental, subject to change.
Returns a (fn format [signal]) that:
- Takes a Telemere signal.
- Returns human-readable formatted string.
See also `pr-signal-fn` for machine-readable output."
([] (format-signal-fn nil))
([{:keys [incl-newline? preamble-fn content-fn]
:or
{incl-newline? true
preamble-fn (signal-preamble-fn)
content-fn (signal-content-fn)}}]
(let [nl newline]
(fn format-signal [signal]
(let [preamble (when preamble-fn (preamble-fn signal))
content (when content-fn (content-fn signal))]
(if (and preamble content)
(str preamble nl content (when incl-newline? nl))
(str preamble content (when incl-newline? nl))))))))
(comment (comment
(tel/with-ctx {:c :C} (tel/with-ctx {:c :C}
(println (println
((format-signal->str-fn) ((format-signal-fn)
(tel/with-signal (tel/with-signal
(tel/event! ::ev-id (tel/event! ::ev-id
{:user-k1 #{:a :b :c} {:user-k1 #{:a :b :c}

View file

@ -674,15 +674,16 @@
#?(:clj " Root: clojure.lang.ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: clojure.lang.ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n" #?(:clj " Root: clojure.lang.ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: clojure.lang.ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n"
:cljs " Root: cljs.core/ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: cljs.core/ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n"))) :cljs " Root: cljs.core/ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: cljs.core/ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n")))
(testing "signal-preamble-fn"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0})) (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))
prelude ((utils/format-signal->prelude-fn) sig)] ; "2024-06-09T21:15:20.170Z INFO EVENT taoensso.telemere-tests(592,35) ::ev-id" preamble ((utils/signal-preamble-fn) sig)] ; "2024-06-09T21:15:20.170Z INFO EVENT taoensso.telemere-tests(592,35) ::ev-id"
[(is (enc/str-starts-with? prelude "2024-06-09T21:15:20.170Z INFO EVENT")) [(is (enc/str-starts-with? preamble "2024-06-09T21:15:20.170Z INFO EVENT"))
(is (enc/str-ends-with? prelude "::ev-id")) (is (enc/str-ends-with? preamble "::ev-id"))
(is (string? (re-find #"taoensso.telemere-tests\(\d+,\d+\)" prelude)))]) (is (string? (re-find #"taoensso.telemere-tests\(\d+,\d+\)" preamble)))]))
(testing "format-signal->edn-fn" (testing "pr-signal-fn/edn"
(let [sig (update (with-sig (tel/event! ::ev-id {:inst t0})) :inst enc/inst->udt) (let [sig (update (with-sig (tel/event! ::ev-id {:inst t0})) :inst enc/inst->udt)
sig* (enc/read-edn ((utils/format-signal->edn-fn) sig))] sig* (enc/read-edn ((tel/pr-signal-fn :edn) sig))]
(is (is
(enc/submap? sig* (enc/submap? sig*
{:schema 1, :kind :event, :id ::ev-id, :level :info, {:schema 1, :kind :event, :id ::ev-id, :level :info,
@ -692,9 +693,9 @@
:column pnat-int?})))) :column pnat-int?}))))
#?(:cljs #?(:cljs
(testing "format-signal->json-fn" (testing "pr-signal-fn/json"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0})) (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))
sig* (enc/read-json ((utils/format-signal->json-fn) sig))] sig* (enc/read-json ((tel/pr-signal-fn :json) sig))]
(is (is
(enc/submap? sig* (enc/submap? sig*
{"schema" 1, "kind" "event", "id" "taoensso.telemere-tests/ev-id", {"schema" 1, "kind" "event", "id" "taoensso.telemere-tests/ev-id",
@ -703,9 +704,9 @@
"line" pnat-int? "line" pnat-int?
"column" pnat-int?}))))) "column" pnat-int?})))))
(testing "format-signal->str-fn" (testing "format-signal-fn"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0}))] (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))]
(is (enc/str-starts-with? ((utils/format-signal->str-fn) sig) (is (enc/str-starts-with? ((tel/format-signal-fn) sig)
"2024-06-09T21:15:20.170Z INFO EVENT"))))])]) "2024-06-09T21:15:20.170Z INFO EVENT"))))])])
;;;; File handler ;;;; File handler

View file

@ -216,4 +216,3 @@ Telemere includes extensive internal help docstrings:
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |

View file

@ -28,4 +28,3 @@ For more info see:
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |

View file

@ -6,20 +6,21 @@ A number of signal handlers are included out-the box. Alphabetically:
| Name | Platform | Output target | Output format | | Name | Platform | Output target | Output format |
| :------------------------------------------------------------------------------------------------------------------------------------------------------- | :------- | :------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------- | | :------------------------------------------------------------------------------------------------------------------------------------------------------- | :------- | :------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------- |
| [`handler:carmine`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.carmine#handler:carmine) | Clj | [Redis](https://redis.io/) (via [Carmine](https://www.taoensso.com/carmine)) | Serialized signals [1] | | [`handler:carmine`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.carmine#handler:carmine) [0] | Clj | [Redis](https://redis.io/) (via [Carmine](https://www.taoensso.com/carmine)) | Serialized signals [1] |
| [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Clj | `*out*` or `*err*` | Formatted string [2] | | [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Clj | `*out*` or `*err*` | String [2] |
| [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Cljs | Browser console | Formatted string [2] | | [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Cljs | Browser console | String [2] |
| [`handler:console-raw`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) | Cljs | Browser console | Raw signals [3] | | [`handler:console-raw`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) | Cljs | Browser console | Raw signals [3] |
| [`handler:file`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:file) | Clj | File/s on disk | Formatted string [2] | | [`handler:file`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:file) | Clj | File/s on disk | String [2] |
| [`handler:logstash`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.logstash#handler:logstash) | Clj | [Logstash](https://www.elastic.co/logstash) | TODO | | [`handler:logstash`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.logstash#handler:logstash) [0] | Clj | [Logstash](https://www.elastic.co/logstash) | TODO |
| [`handler:open-telemetry-logger`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.open-telemetry#handler:open-telemetry-logger) | Clj | [OpenTelemetry](https://opentelemetry.io/) [Java client](https://github.com/open-telemetry/opentelemetry-java) | [LogRecord](https://opentelemetry.io/docs/specs/otel/logs/data-model/) | | [`handler:open-telemetry-logger`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.open-telemetry#handler:open-telemetry-logger) | Clj | [OpenTelemetry](https://opentelemetry.io/) [Java client](https://github.com/open-telemetry/opentelemetry-java) | [LogRecord](https://opentelemetry.io/docs/specs/otel/logs/data-model/) |
| [`handler:postal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.postal#handler:postal) | Clj | Email (via [postal](https://github.com/drewr/postal)) | Formatted string [2] | | [`handler:postal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.postal#handler:postal) | Clj | Email (via [postal](https://github.com/drewr/postal)) | String [2] |
| [`handler:slack`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) | Clj | [Slack](https://slack.com/) (via [clj-slack](https://github.com/julienXX/clj-slack)) | Formatted string [2] | | [`handler:slack`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) [0] | Clj | [Slack](https://slack.com/) (via [clj-slack](https://github.com/julienXX/clj-slack)) | String [2] |
| [`handler:tcp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) | Clj | TCP socket | Formatted string [2] | | [`handler:tcp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) | Clj | TCP socket | String [2] |
| [`handler:udp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) | Clj | UDP socket | Formatted string [2] | | [`handler:udp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) | Clj | UDP socket | String [2] |
- \[0] Coming soon
- \[1] Uses [Nippy](https://taoensso.com/nippy) to support all Clojure's rich data types - \[1] Uses [Nippy](https://taoensso.com/nippy) to support all Clojure's rich data types
- \[2] [Configurable](https://cljdoc.org/d/com.taoensso/telemere/1.0.0-beta3/api/taoensso.telemere#help:signal-formatters): human-readable (default), [edn](https://github.com/edn-format/edn), [JSON](https://www.json.org/), etc. - \[2] [Human-readable](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#format-signal-fn) (default), or [machine-readable](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#pr-signal-fn) ([edn](https://github.com/edn-format/edn), [JSON](https://www.json.org/), etc.).
- \[3] For use with browser formatting tools like [cljs-devtools](https://github.com/binaryage/cljs-devtools). - \[3] For use with browser formatting tools like [cljs-devtools](https://github.com/binaryage/cljs-devtools).
- See relevant docstrings (links above) for features, usage, etc. - See relevant docstrings (links above) for features, usage, etc.
- See section [8-Community](8-Community.md) for more (community-supported) handlers. - See section [8-Community](8-Community.md) for more (community-supported) handlers.
@ -75,7 +76,7 @@ To instead writes signals as edn:
;; Create console which writes edn ;; Create console which writes edn
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn (taoensso.telemere.utils/format-signal->edn-fn)})) {:output-fn (t/pr-signal-fn :edn)}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...} ;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...}
@ -84,18 +85,17 @@ To instead writes signals as edn:
To instead writes signals as JSON: To instead writes signals as JSON:
```clojure ```clojure
;; Create console which writes JSON ;; Create console which writes signals as JSON
#?(:clj (require '[jsonista.core :as jsonista]))
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn {:output-fn
(taoensso.telemere.utils/format-signal->json-fn (t/pr-signal-fn
{:pr-json-fn jsonista.core/write-value-as-string})})) #?(:cljs :json
:clj jsonista.core/write-value-as-string))}))
(my-handler my-signal) ; =>
;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...}
``` ```
Note that when writing JSON with Clojure, you *must* specify a `pr-json-fn`. This lets you plug in the JSON serializer of your choice ([jsonista](https://github.com/metosin/jsonista) is my default recommendation). Note that when writing JSON with Clojure, you *must* provide an appropriate `pr-fn`. This lets you plug in the JSON serializer of your choice ([jsonista](https://github.com/metosin/jsonista) is my default recommendation).
### Handler-specific per-signal kvs ### Handler-specific per-signal kvs