Compare commits

...

11 commits

Author SHA1 Message Date
Peter Taoussanis
4011560c60 v1.0.0-beta6 (2024-05-05) 2024-05-05 13:17:58 +02:00
Peter Taoussanis
d62bab2247 [mod] Rename, refactor signal formatting utils
- Simplified some util name (only relevant to folks customizing handler behaviour)
- Merged `format-signal->edn-fn`, `format-signal->json-fn` to single `pr-signal-fn`
2024-05-05 13:17:58 +02:00
Peter Taoussanis
7b7782e340 [doc] Doc improvements 2024-05-05 12:28:44 +02:00
Peter Taoussanis
ea6a039980 [new] Add :incl-kvs? opt to edn and JSON formatters 2024-05-05 12:28:44 +02:00
Peter Taoussanis
3e1f453d06 [new] Add :incl-thread?, :incl-kvs? opts to format-signal->str-fn 2024-05-05 12:28:44 +02:00
Peter Taoussanis
eed702a480 [new] Add :end-with-newline opt to signal formatters
Instead allow format-signal-fn to decide whether or not to end with a newline
2024-05-05 12:28:44 +02:00
Peter Taoussanis
cf22ddf861 [new] Add TCP, UDP socket handlers 2024-05-05 12:28:44 +02:00
Peter Taoussanis
cbd786be66 [fix] Broken postal handler subject 2024-05-03 14:09:53 +02:00
Peter Taoussanis
b6e8c5fd4a [new] Add experimental :thread key to Clj signals
Only downside/hesitation is that this info *must* be collected at the callsite,
which means that it affects the performance of *all* created signals.

Adds ~30-50 nsecs per signal.
2024-05-03 14:09:53 +02:00
Peter Taoussanis
b271c4f7b8 [mod] Simplify middleware - don't auto compose
Previously:

  It was possible to provide a vector of middleware fns when creating
  signals or adding handlers, e.g.:

    (event! ::ev-id1 {:middleware [fn1 fn2 ...]}),
    (add-handler! ::handler-id1 <handler-fn> {:middleware [fn1 fn2 ...]})

After this commit:

  Middleware is always expected to be a single fn, or nil.
  A `comp-middleware` util has been added to make it easy to compose multiple
  middleware fns into one.

Motivation:

  The previous (auto-composition) behaviour was nice when adding handlers,
  but incurred a (small but non-trivial) runtime cost when creating signals.

  The actual benefit (convenience) of auto-composition is very small, so
  I've decided to just remove this feature and add the `comp-middleware` util.

  Note that I ruled out the option of doing auto-comp only when adding handlers
  since that've meant an inconsistency and so complexity for little benefit.
2024-05-03 14:09:53 +02:00
Peter Taoussanis
195c16b476 [nop] Housekeeping 2024-05-03 14:09:53 +02:00
21 changed files with 680 additions and 301 deletions

View file

@ -2,6 +2,50 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
--- ---
# `v1.0.0-beta6` (2024-05-05)
> **Dep/s**: [Telemere](https://clojars.org/com.taoensso/telemere/versions/1.0.0-beta6) and [Telemere SLF4J provider](https://clojars.org/com.taoensso/slf4j-telemere/versions/1.0.0-beta6) are on Clojars.
> **Versioning**: Telemere uses [Break Versioning](https://www.taoensso.com/break-versioning).
This is a **maintenance pre-release** intended to fix issues that have come up during the beta. See below for details, and please **report any unexpected problems** on [GitHub](https://github.com/taoensso/telemere/issues) or the [Slack channel](https://www.taoensso.com/telemere/slack), thank you! 🙏
\- Peter Taoussanis
## Changes since `v1.0.0-beta1` (2024-04-19)
* d0a15bac [mod] Don't auto add OpenTelemetry handler
* 6d545dfc [mod] Move (simplify) OpenTelemetry ns
* c4d9dd09 [mod] Don't include user-level kvs in default signal content handler
* d3c63e17 [mod] Rename `clojure.tools.logging` sys val
* b271c4f7 [mod] Simplify middleware - don't auto compose
## Fixes since `v1.0.0-beta1` (2024-04-19)
* ffea1a30 [fix] Fix broken AOT support, add AOT tests
* e222297a [fix] SLF4J broken timestamps, add tests
## New since `v1.0.0-beta1` (2024-04-19)
* 2ba23ee7 [new] Add postal (email) handler
* cf22ddf8 [new] Add TCP, UDP socket handlers
* b6e8c5fd [new] Add experimental `:thread` key to Clj signals
* Handlers will now drain their signal queues on shutdown (configurable)
* Rate limiter performance improvements (via Encore)
* Doc improvements based on questions that've come up on Slack, etc.
## Everything since `v1.0.-beta5` (2024-04-29)
* ae823f0d [mod] Rename, refactor signal formatting utils
* b271c4f7 [mod] Simplify middleware - don't auto compose
* cbd786be [fix] Broken postal handler subject
* cf22ddf8 [new] Add TCP, UDP socket handlers
* b6e8c5fd [new] Add experimental `:thread` key to Clj signals
* ea6a0399 [new] Add `:incl-kvs?` opt to edn and JSON formatters
* 3e1f453d [new] Add `:incl-thread?`, `:incl-kvs?` opts to `format-signal->str-fn`
* eed702a4 [new] Add `:end-with-newline` opt to signal formatters
---
# `v1.0.0-beta5` (2024-04-29) # `v1.0.0-beta5` (2024-04-29)
> **Dep/s**: [Telemere](https://clojars.org/com.taoensso/telemere/versions/1.0.0-beta5) and [Telemere SLF4J provider](https://clojars.org/com.taoensso/slf4j-telemere/versions/1.0.0-beta5) are on Clojars. > **Dep/s**: [Telemere](https://clojars.org/com.taoensso/telemere/versions/1.0.0-beta5) and [Telemere SLF4J provider](https://clojars.org/com.taoensso/slf4j-telemere/versions/1.0.0-beta5) are on Clojars.

View file

@ -32,6 +32,7 @@ See [here][GitHub releases] for earlier releases.
- 1st-class **out-the-box interop** with [SLF4J v2](https://www.slf4j.org/), [clojure.tools.logging](https://github.com/clojure/tools.logging), [OpenTelemetry](https://opentelemetry.io/), and [Tufte](https://www.taoensso.com/tufte). - 1st-class **out-the-box interop** with [SLF4J v2](https://www.slf4j.org/), [clojure.tools.logging](https://github.com/clojure/tools.logging), [OpenTelemetry](https://opentelemetry.io/), and [Tufte](https://www.taoensso.com/tufte).
- Included [shim](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.timbre) for easy/gradual [migration from Timbre](../../wiki/5-Migrating). - Included [shim](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.timbre) for easy/gradual [migration from Timbre](../../wiki/5-Migrating).
- Included [handlers](../../wiki/4-Handlers#included-handlers) for consoles, files, email, Redis, Slack, TCP/UDP sockets, Logstash, etc.
#### Scaling #### Scaling
@ -133,14 +134,13 @@ See relevant docstrings (links below) for usage info-
### Internal help ### Internal help
| Var | Help with | | Var | Help with |
| :-------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- | | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------- |
| [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators | | [`help:signal-creators`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-creators) | List of signal creators |
| [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators | | [`help:signal-options`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-options) | Options for signal creators |
| [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content | | [`help:signal-content`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-content) | Signal map content |
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |
### Example handler output ### Example handler output

View file

@ -141,17 +141,19 @@
;; Create console which writes signals as edn ;; Create console which writes signals as edn
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn (taoensso.telemere.utils/format-signal->edn-fn)})) {:output-fn (t/pr-signal-fn :edn)}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...} ;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...}
;; Create console which writes signals as JSON ;; Create console which writes signals as JSON
#?(:clj (require '[jsonista.core :as jsonista]))
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn {:output-fn
(taoensso.telemere.utils/format-signal->json-fn (t/pr-signal-fn
{:pr-json-fn jsonista.core/write-value-as-string})})) #?(:cljs :json
:clj jsonista.core/write-value-as-string))}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...} ;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...}
@ -219,3 +221,19 @@
(t/log! (format "This message was built by `%s`" "format")) (t/log! (format "This message was built by `%s`" "format"))
;; %> {:msg "This message was built by `format`"} ;; %> {:msg "This message was built by `format`"}
;;; User kvs
(t/with-signal
(t/event! ::my-id
{:my-middleware-data "foo"
:my-handler-data "bar"}))
;; %>
;; {;; User kvs included inline (assoc'd to signal root)
;; :my-middleware-data "foo"
;; :my-handler-data "bar"
;; :kvs ; And also collected together under ":kvs" key
;; {:my-middleware-data "foo"
;; :my-handler-data "bar"}
;; ... }

View file

@ -1,4 +1,4 @@
(defproject com.taoensso/telemere "1.0.0-beta5" (defproject com.taoensso/telemere "1.0.0-beta6"
:author "Peter Taoussanis <https://www.taoensso.com>" :author "Peter Taoussanis <https://www.taoensso.com>"
:description "Structured telemetry library for Clojure/Script" :description "Structured telemetry library for Clojure/Script"
:url "https://www.taoensso.com/telemere" :url "https://www.taoensso.com/telemere"
@ -8,7 +8,7 @@
:url "https://www.eclipse.org/legal/epl-v10.html"} :url "https://www.eclipse.org/legal/epl-v10.html"}
:dependencies :dependencies
[[com.taoensso/encore "3.105.1"]] [[com.taoensso/encore "3.106.0"]]
:test-paths ["test" #_"src"] :test-paths ["test" #_"src"]
@ -16,7 +16,7 @@
{;; :default [:base :system :user :provided :dev] {;; :default [:base :system :user :provided :dev]
:provided {:dependencies [[org.clojure/clojurescript "1.11.132"] :provided {:dependencies [[org.clojure/clojurescript "1.11.132"]
[org.clojure/clojure "1.11.3"]]} [org.clojure/clojure "1.11.3"]]}
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0-alpha10"]]} :c1.12 {:dependencies [[org.clojure/clojure "1.12.0-alpha11"]]}
:c1.11 {:dependencies [[org.clojure/clojure "1.11.3"]]} :c1.11 {:dependencies [[org.clojure/clojure "1.11.3"]]}
:c1.10 {:dependencies [[org.clojure/clojure "1.10.1"]]} :c1.10 {:dependencies [[org.clojure/clojure "1.10.1"]]}
@ -45,13 +45,16 @@
[[org.clojure/test.check "1.1.1"] [[org.clojure/test.check "1.1.1"]
[org.clojure/tools.logging "1.3.0"] [org.clojure/tools.logging "1.3.0"]
[org.slf4j/slf4j-api "2.0.13"] [org.slf4j/slf4j-api "2.0.13"]
[com.taoensso/slf4j-telemere "1.0.0-beta5"] [com.taoensso/slf4j-telemere "1.0.0-beta6"]
#_[org.slf4j/slf4j-simple "2.0.13"] #_[org.slf4j/slf4j-simple "2.0.13"]
#_[org.slf4j/slf4j-nop "2.0.13"] #_[org.slf4j/slf4j-nop "2.0.13"]
[com.draines/postal "2.0.5"]
;;; For optional handlers
[io.opentelemetry/opentelemetry-api "1.37.0"] [io.opentelemetry/opentelemetry-api "1.37.0"]
#_[io.opentelemetry/opentelemetry-sdk-extension-autoconfigure "1.37.0"] #_[io.opentelemetry/opentelemetry-sdk-extension-autoconfigure "1.37.0"]
#_[io.opentelemetry/opentelemetry-exporter-otlp "1.37.0"]] #_[io.opentelemetry/opentelemetry-exporter-otlp "1.37.0"]
[metosin/jsonista "0.3.8"]
[com.draines/postal "2.0.5"]]
:plugins :plugins
[[lein-pprint "1.3.2"] [[lein-pprint "1.3.2"]

View file

@ -26,6 +26,7 @@ Default signal keys:
`:line` -------- ?int line of signal creator callsite, same as (:line location) `:line` -------- ?int line of signal creator callsite, same as (:line location)
`:column` ------ ?int column of signal creator callsite, same as (:column location) `:column` ------ ?int column of signal creator callsite, same as (:column location)
`:file` -------- ?str filename of signal creator callsite, same as (:file location) `:file` -------- ?str filename of signal creator callsite, same as (:file location)
`:thread` ------ (Clj only) {:keys [group name id]} thread info for thread that called signal creator
`:sample-rate` - ?rate ∈ℝ[0,1] for combined signal AND handler sampling (0.75 => allow 75% of signals, nil => allow all) `:sample-rate` - ?rate ∈ℝ[0,1] for combined signal AND handler sampling (0.75 => allow 75% of signals, nil => allow all)
<kvs> ---------- Other arb user-level ?kvs given to signal creator. Typically NOT included <kvs> ---------- Other arb user-level ?kvs given to signal creator. Typically NOT included

View file

@ -1,6 +0,0 @@
Common signal formatters include:
(utils/format-signal-str->fn) {<opts>}) ; For human-readable string output (default)
(utils/format-signal->edn-fn) {<opts>}) ; For edn output
(utils/format-signal->json-fn {<opts>}) ; For JSON output
See relevant docstrings for details.

View file

@ -22,7 +22,7 @@ Signal options (shared by all signal creators):
`:sample-rate` - ?rate ∈ℝ[0,1] for signal sampling (0.75 => allow 75% of signals, nil => allow all) `:sample-rate` - ?rate ∈ℝ[0,1] for signal sampling (0.75 => allow 75% of signals, nil => allow all)
`:when` -------- Arb ?form; when present, form must return truthy to allow signal `:when` -------- Arb ?form; when present, form must return truthy to allow signal
`:rate-limit` -- ?spec as given to `taoensso.telemere/rate-limiter`, see its docstring for details `:rate-limit` -- ?spec as given to `taoensso.telemere/rate-limiter`, see its docstring for details
`:middleware` -- ?[(fn [signal])=>modified-signal ...] signal middleware `:middleware` -- Optional (fn [signal]) => ?modified-signal to apply when signal is created
`:trace?` ------ Should tracing be enabled for `:run` form? `:trace?` ------ Should tracing be enabled for `:run` form?
<kvs> ---------- Other arb user-level ?kvs to incl. in signal. Typically NOT included in <kvs> ---------- Other arb user-level ?kvs to incl. in signal. Typically NOT included in

View file

@ -1,7 +1,7 @@
{;;:lein true {;;:lein true
:source-paths ["src" "test"] :source-paths ["src" "test"]
:dependencies :dependencies
[[com.taoensso/encore "3.105.1"] [[com.taoensso/encore "3.106.0"]
[cider/cider-nrepl "0.47.0"] [cider/cider-nrepl "0.47.0"]
[binaryage/devtools "1.0.7"]] [binaryage/devtools "1.0.7"]]

View file

@ -1,4 +1,4 @@
(defproject com.taoensso/slf4j-telemere "1.0.0-beta5" (defproject com.taoensso/slf4j-telemere "1.0.0-beta6"
:author "Peter Taoussanis <https://www.taoensso.com>" :author "Peter Taoussanis <https://www.taoensso.com>"
:description "Telemere backend/provider for SLF4J API v2" :description "Telemere backend/provider for SLF4J API v2"
:url "https://www.taoensso.com/telemere" :url "https://www.taoensso.com/telemere"
@ -16,7 +16,7 @@
{:dependencies {:dependencies
[[org.clojure/clojure "1.11.3"] [[org.clojure/clojure "1.11.3"]
[org.slf4j/slf4j-api "2.0.13"] [org.slf4j/slf4j-api "2.0.13"]
[com.taoensso/telemere "1.0.0-beta5"]]} [com.taoensso/telemere "1.0.0-beta6"]]}
:dev :dev
{:plugins {:plugins

View file

@ -32,9 +32,10 @@
(remove-ns 'taoensso.telemere) (remove-ns 'taoensso.telemere)
(:api (enc/interns-overview))) (:api (enc/interns-overview)))
(enc/assert-min-encore-version [3 105 1]) (enc/assert-min-encore-version [3 106 0])
;;;; TODO ;;;; TODO
;; - Add handlers: Logstash, Slack, Carmine, Datadog, Kafka
;; - Native OpenTelemetry traces and spans ;; - Native OpenTelemetry traces and spans
;; - Update Tufte (signal API, config API, signal keys, etc.) ;; - Update Tufte (signal API, config API, signal keys, etc.)
;; - Update Timbre (signal API, config API, signal keys, backport improvements) ;; - Update Timbre (signal API, config API, signal keys, backport improvements)
@ -71,13 +72,16 @@
enc/chance enc/chance
enc/rate-limiter enc/rate-limiter
enc/newline enc/newline
enc/comp-middleware
impl/msg-splice impl/msg-splice
impl/msg-skip impl/msg-skip
#?(:clj impl/with-signal) #?(:clj impl/with-signal)
#?(:clj impl/with-signals) #?(:clj impl/with-signals)
#?(:clj impl/signal!) #?(:clj impl/signal!)
utils/error-signal?) utils/error-signal?
utils/pr-signal-fn
utils/format-signal-fn)
;;;; Help ;;;; Help
@ -87,7 +91,6 @@
(impl/defhelp help:signal-content :signal-content) (impl/defhelp help:signal-content :signal-content)
(enc/defalias help:signal-filters help:filters) ; Via Encore (enc/defalias help:signal-filters help:filters) ; Via Encore
(enc/defalias help:signal-handlers help:handlers) ; Via Encore (enc/defalias help:signal-handlers help:handlers) ; Via Encore
(impl/defhelp help:signal-formatters :signal-formatters)
;;;; Context ;;;; Context
@ -118,12 +121,12 @@
#?(:clj #?(:clj
(defmacro set-ctx! (defmacro set-ctx!
"Set `*ctx*` var's root (base) value. See `*ctx*` for details." "Set `*ctx*` var's root (base) value. See `*ctx*` for details."
[root-val] `(enc/set-var-root! *ctx* ~root-val))) [root-ctx-val] `(enc/set-var-root! *ctx* ~root-ctx-val)))
#?(:clj #?(:clj
(defmacro with-ctx (defmacro with-ctx
"Evaluates given form with given `*ctx*` value. See `*ctx*` for details." "Evaluates given form with given `*ctx*` value. See `*ctx*` for details."
[init-val form] `(binding [*ctx* ~init-val] ~form))) [ctx-val form] `(binding [*ctx* ~ctx-val] ~form)))
(comment (with-ctx "my-ctx" *ctx*)) (comment (with-ctx "my-ctx" *ctx*))
@ -142,14 +145,13 @@
(comment (with-ctx {:a :A1 :b :B1} (with-ctx+ {:a :A2} *ctx*))) (comment (with-ctx {:a :A1 :b :B1} (with-ctx+ {:a :A2} *ctx*)))
;;;; Middleware ;;;; Signal middleware
(enc/defonce ^:dynamic *middleware* (enc/defonce ^:dynamic *middleware*
"Optional vector of unary middleware fns to apply (sequentially/left-to-right) "Optional (fn [signal]) => ?modified-signal to apply (once) when
to each signal before passing it to handlers. If any middleware fn returns nil, signal is created. When middleware returns nil, skips all handlers.
aborts immediately without calling handlers.
Useful for transforming each signal before handling. Compose multiple middleware fns together with `comp-middleware.
Re/bind dynamic value using `with-middleware`, `binding`. Re/bind dynamic value using `with-middleware`, `binding`.
Modify root (base) value using `set-middleware!`." Modify root (base) value using `set-middleware!`."
@ -158,13 +160,13 @@
#?(:clj #?(:clj
(defmacro set-middleware! (defmacro set-middleware!
"Set `*middleware*` var's root (base) value. See `*middleware*` for details." "Set `*middleware*` var's root (base) value. See `*middleware*` for details."
[root-val] `(enc/set-var-root! *middleware* ~root-val))) [?root-middleware-fn] `(enc/set-var-root! *middleware* ~?root-middleware-fn)))
#?(:clj #?(:clj
(defmacro with-middleware (defmacro with-middleware
"Evaluates given form with given `*middleware*` value. "Evaluates given form with given `*middleware*` value.
See `*middleware*` for details." See `*middleware*` for details."
[init-val form] `(binding [*middleware* ~init-val] ~form))) [?middleware-fn form] `(binding [*middleware* ~?middleware-fn] ~form)))
;;;; Signal creators ;;;; Signal creators
;; - signal! [ opts] ; => allowed? / run result (value or throw) ;; - signal! [ opts] ; => allowed? / run result (value or throw)

View file

@ -18,20 +18,19 @@
Returns a (fn handler [signal]) that: Returns a (fn handler [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Writes a formatted signal string to stream. - Writes formatted signal string to stream.
A general-purpose `println`-style handler that's well suited for outputting A general-purpose `println`-style handler that's well suited for outputting
signals formatted as edn, JSON, or human-readable strings. signals formatted as edn, JSON, or human-readable strings.
Options: Options:
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters` `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:stream` - `java.io.writer` `:stream` - `java.io.writer`
Defaults to `*err*` if `utils/error-signal?` is true, and `*out*` otherwise." Defaults to `*err*` if `utils/error-signal?` is true, and `*out*` otherwise."
([] (handler:console nil)) ([] (handler:console nil))
([{:keys [format-signal-fn stream] ([{:keys [output-fn stream]
:or {format-signal-fn (utils/format-signal->str-fn)}}] :or {output-fn (utils/format-signal-fn)}}]
(let [stream (case stream :*out* *out*, :*err* *err* stream) (let [stream (case stream :*out* *out*, :*err* *err* stream)
error-signal? utils/error-signal? error-signal? utils/error-signal?
@ -42,8 +41,8 @@
([signal] ([signal]
(let [^java.io.Writer stream (let [^java.io.Writer stream
(or stream (if (error-signal? signal) *err* *out*))] (or stream (if (error-signal? signal) *err* *out*))]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(.write stream (str output nl)) (.write stream (str output))
(.flush stream)))))))) (.flush stream))))))))
:cljs :cljs
@ -52,17 +51,17 @@
If `js/console` exists, returns a (fn handler [signal]) that: If `js/console` exists, returns a (fn handler [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Writes a formatted signal string to JavaScript console. - Writes formatted signal string to JavaScript console.
A general-purpose `println`-style handler that's well suited for outputting A general-purpose `println`-style handler that's well suited for outputting
signals formatted as edn, JSON, or human-readable strings. signals formatted as edn, JSON, or human-readable strings.
Options: Options:
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters`" `:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`"
([] (handler:console nil)) ([] (handler:console nil))
([{:keys [format-signal-fn] ([{:keys [output-fn]
:or {format-signal-fn (utils/format-signal->str-fn)}}] :or {output-fn (utils/format-signal-fn)}}]
(when (exists? js/console) (when (exists? js/console)
(let [js-console-logger utils/js-console-logger (let [js-console-logger utils/js-console-logger
@ -71,9 +70,9 @@
(fn a-handler:console (fn a-handler:console
([]) ; Shut down (no-op) ([]) ; Shut down (no-op)
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(let [logger (js-console-logger (get signal :level))] (let [logger (js-console-logger (get signal :level))]
(.call logger logger (str output nl))))))))))) (.call logger logger (str output)))))))))))
#?(:cljs #?(:cljs
(defn- logger-fn [logger] (defn- logger-fn [logger]
@ -96,16 +95,16 @@
Ref. <https://github.com/binaryage/cljs-devtools>." Ref. <https://github.com/binaryage/cljs-devtools>."
([] (handler:console-raw nil)) ([] (handler:console-raw nil))
([{:keys [format-signal->prelude-fn format-nsecs-fn] :as opts ([{:keys [preamble-fn format-nsecs-fn] :as opts
:or :or
{format-signal->prelude-fn (utils/format-signal->prelude-fn) ; (fn [signal]) {preamble-fn (utils/signal-preamble-fn)
format-nsecs-fn (utils/format-nsecs-fn) ; (fn [nanosecs]) format-nsecs-fn (utils/format-nsecs-fn) ; (fn [nanosecs])
}}] }}]
(when (and (exists? js/console) (exists? js/console.group)) (when (and (exists? js/console) (exists? js/console.group))
(let [js-console-logger utils/js-console-logger (let [js-console-logger utils/js-console-logger
signal-content-handler ; (fn [signal hf vf] content-fn ; (fn [signal append-fn val-fn])
(utils/signal-content-handler (utils/signal-content-fn
{:format-nsecs-fn format-nsecs-fn {:format-nsecs-fn format-nsecs-fn
:format-error-fn nil :format-error-fn nil
:raw-error? true})] :raw-error? true})]
@ -117,8 +116,8 @@
logger (js-console-logger level)] logger (js-console-logger level)]
;; Unfortunately groups have no level ;; Unfortunately groups have no level
(.group js/console (format-signal->prelude-fn signal)) (.group js/console (preamble-fn signal))
(signal-content-handler signal (logger-fn logger) identity) (content-fn signal (logger-fn logger) identity)
(when-let [stack (and error (.-stack (enc/ex-root error)))] (when-let [stack (and error (.-stack (enc/ex-root error)))]
(.call logger logger stack)) (.call logger logger stack))

View file

@ -270,7 +270,7 @@
Returns a (fn handler [signal]) that: Returns a (fn handler [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Writes a formatted signal string to file. - Writes formatted signal string to file.
Signals will be appended to file specified by `path`. Signals will be appended to file specified by `path`.
Depending on options, archives may be maintained: Depending on options, archives may be maintained:
@ -284,9 +284,10 @@
`/logs/telemere.log-2020-01-01m.8.gz` ; Archive for Jan 2020, part 8 (oldest entries) `/logs/telemere.log-2020-01-01m.8.gz` ; Archive for Jan 2020, part 8 (oldest entries)
Options: Options:
`:format-signal-fn`- (fn [signal]) => output, see `help:signal-formatters`. `:output-fn`- (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:path` - Path string of the target output file (default `logs/telemere.log`). `:path` - Path string of the target output file (default `logs/telemere.log`)
`:interval` - #{nil :daily :weekly :monthly} (default `:monthly`).
`:interval` - #{nil :daily :weekly :monthly} (default `:monthly`)
When non-nil, causes interval-based archives to be maintained. When non-nil, causes interval-based archives to be maintained.
`:max-file-size` #{nil <pos-int>} (default 4MB) `:max-file-size` #{nil <pos-int>} (default 4MB)
@ -300,7 +301,7 @@
([] (handler:file nil)) ([] (handler:file nil))
([{:keys ([{:keys
[format-signal-fn [output-fn
path interval path interval
max-file-size max-file-size
max-num-parts max-num-parts
@ -308,7 +309,7 @@
gzip-archives?] gzip-archives?]
:or :or
{format-signal-fn (utils/format-signal->str-fn) {output-fn (utils/format-signal-fn)
path "logs/telemere.log" ; Main path, we'll ALWAYS write to this exact file path "logs/telemere.log" ; Main path, we'll ALWAYS write to this exact file
interval :monthly interval :monthly
max-file-size (* 1024 1024 4) ; 4MB max-file-size (* 1024 1024 4) ; 4MB
@ -362,9 +363,8 @@
(fn a-handler:file (fn a-handler:file
([] (locking lock (fw))) ; Close writer ([] (locking lock (fw))) ; Close writer
([signal] ([signal]
(when-let [output (format-signal-fn signal)] (when-let [output (output-fn signal)]
(let [output-str (str output utils/newline) (let [new-interval? (when interval (new-interval!?))
new-interval? (when interval (new-interval!?))
>max-file-size? (when max-file-size (>max-file-size?)) >max-file-size? (when max-file-size (>max-file-size?))
reset-stream? (or new-interval? >max-file-size?)] reset-stream? (or new-interval? >max-file-size?)]
@ -387,7 +387,7 @@
max-num-parts gzip-archives? nil))) max-num-parts gzip-archives? nil)))
(when reset-stream? (fw :writer/reset!)) (when reset-stream? (fw :writer/reset!))
(do (fw output-str)))))))))) (do (fw output))))))))))
(comment (comment
(manage-test-files! :create) (manage-test-files! :create)

View file

@ -201,14 +201,6 @@
(defmacro with-tracing (defmacro with-tracing
"Wraps `form` with tracing iff const boolean `trace?` is true." "Wraps `form` with tracing iff const boolean `trace?` is true."
[trace? id uid form] [trace? id uid form]
;; Not much motivation to support runtime `trace?` form, but easy
;; to add support later if desired
(when-not (enc/const-form? trace?)
(enc/unexpected-arg! trace?
{:msg "Expected constant (compile-time) `:trace?` value"
:context `with-tracing}))
(if trace? (if trace?
`(binding [*trace-parent* (TraceParent. ~id ~uid)] ~form) `(binding [*trace-parent* (TraceParent. ~id ~uid)] ~form)
(do form)))) (do form))))
@ -223,7 +215,7 @@
(defrecord Signal (defrecord Signal
;; Telemere's main public data type, we avoid nesting and duplication ;; Telemere's main public data type, we avoid nesting and duplication
[^long schema inst uid, [^long schema inst uid,
location ns line column file, location ns line column file #?(:clj thread),
sample-rate, kind id level, ctx parent, sample-rate, kind id level, ctx parent,
data msg_ error run-form run-val, data msg_ error run-form run-val,
end-inst run-nsecs kvs] end-inst run-nsecs kvs]
@ -359,7 +351,7 @@
^Signal ^Signal
;; Note all dynamic vals passed as explicit args for better control ;; Note all dynamic vals passed as explicit args for better control
[inst uid, [inst uid,
location ns line column file, location ns line column file #?(:clj thread :cljs _thread),
sample-rate, kind id level, ctx parent, sample-rate, kind id level, ctx parent,
kvs data msg_, kvs data msg_,
run-form run-result error] run-form run-result error]
@ -379,14 +371,14 @@
msg_)] msg_)]
(Signal. 1 inst uid, (Signal. 1 inst uid,
location ns line column file, location ns line column file #?(:clj thread),
sample-rate, kind id level, ctx parent, sample-rate, kind id level, ctx parent,
data msg_, data msg_,
run-err run-form run-val, run-err run-form run-val,
end-inst run-nsecs kvs)) end-inst run-nsecs kvs))
(Signal. 1 inst uid, (Signal. 1 inst uid,
location ns line column file, location ns line column file #?(:clj thread),
sample-rate, kind id level, ctx parent, sample-rate, kind id level, ctx parent,
data msg_, error nil nil nil nil kvs))] data msg_, error nil nil nil nil kvs))]
@ -395,10 +387,10 @@
(do signal)))) (do signal))))
(comment (comment
(enc/qb 1e6 ; 55.67 (enc/qb 1e6 ; 66.8
(new-signal (new-signal
nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil
nil nil nil nil nil nil nil nil nil))) nil nil nil nil nil nil nil nil nil nil)))
;;;; Signal API helpers ;;;; Signal API helpers
@ -548,6 +540,17 @@
;;;; Signal macro ;;;; Signal macro
#?(:clj
(defn thread-info
"Returns {:keys [group name id]} for current thread."
[]
(when-let [t (Thread/currentThread)]
{:group (when-let [g (.getThreadGroup t)] (.getName g))
:name (.getName t)
:id (.getId t)})))
(comment (enc/qb 1e6 (thread-info))) ; 44.49
#?(:clj #?(:clj
(defmacro ^:public signal! (defmacro ^:public signal!
"Generic low-level signal call, also aliased in Encore." "Generic low-level signal call, also aliased in Encore."
@ -557,6 +560,7 @@
(have? map? opts) ; We require const map keys, but vals may require eval (have? map? opts) ; We require const map keys, but vals may require eval
(let [defaults (get opts :defaults) (let [defaults (get opts :defaults)
opts (merge defaults (dissoc opts :defaults)) opts (merge defaults (dissoc opts :defaults))
clj? (not (:ns &env))
{run-form :run} opts {run-form :run} opts
{:keys [#_expansion-id location elide? allow?]} {:keys [#_expansion-id location elide? allow?]}
@ -581,6 +585,13 @@
id-form :id} opts id-form :id} opts
trace? (get opts :trace? (boolean run-form)) trace? (get opts :trace? (boolean run-form))
_
(when-not (contains? #{true false nil} trace?)
;; Not much motivation to support runtime `trace?` form, but easy
;; to add support later if desired
(enc/unexpected-arg! trace?
{:msg "Expected constant (compile-time) `:trace?` boolean"
:context `with-tracing}))
inst-form (get opts :inst :auto) inst-form (get opts :inst :auto)
inst-form (if (= inst-form :auto) `(enc/now-inst*) inst-form) inst-form (if (= inst-form :auto) `(enc/now-inst*) inst-form)
@ -588,7 +599,9 @@
uid-form (get opts :uid (when trace? :auto/uuid)) uid-form (get opts :uid (when trace? :auto/uuid))
uid-form (parse-uid-form uid-form) uid-form (parse-uid-form uid-form)
signal-form thread-form (if clj? `(thread-info) nil)
signal-delay-form
(let [{do-form :do (let [{do-form :do
let-form :let let-form :let
msg-form :msg msg-form :msg
@ -601,6 +614,7 @@
ctx-form (get opts :ctx `taoensso.telemere/*ctx*) ctx-form (get opts :ctx `taoensso.telemere/*ctx*)
parent-form (get opts :parent (when trace? `taoensso.telemere.impl/*trace-parent*)) parent-form (get opts :parent (when trace? `taoensso.telemere.impl/*trace-parent*))
middleware-form (get opts :middleware `taoensso.telemere/*middleware*)
kvs-form kvs-form
(not-empty (not-empty
@ -610,6 +624,7 @@
:ctx :parent #_:trace?, :do :let :data :msg :error :run :ctx :parent #_:trace?, :do :let :data :msg :error :run
:elide? :allow? #_:expansion-id))] :elide? :allow? #_:expansion-id))]
;; Compile-time validation
(when (and run-form error-form) (when (and run-form error-form)
(throw ; Prevent ambiguity re: source of error (throw ; Prevent ambiguity re: source of error
(ex-info "Signals cannot have both `:run` and `:error` opts at the same time" (ex-info "Signals cannot have both `:run` and `:error` opts at the same time"
@ -618,23 +633,29 @@
:location location :location location
:other-opts (dissoc opts :run :error)}))) :other-opts (dissoc opts :run :error)})))
;; Eval let bindings AFTER call filtering but BEFORE data, msg `(delay
`(do ;; Delay (cache) shared by all handlers. Covers signal `:let` eval, signal construction,
;; middleware (possibly expensive), etc. Throws here will be caught by handler.
~do-form ~do-form
(let ~let-form ; Allow to throw during `signal-value_` deref (let [~@let-form ; Allow to throw, eval BEFORE data, msg, etc.
~'__signal
(new-signal ~'__inst ~'__uid (new-signal ~'__inst ~'__uid
~location ~'__ns ~line-form ~column-form ~file-form, ~location ~'__ns ~line-form ~column-form ~file-form ~'__thread,
~sample-rate-form, ~'__kind ~'__id ~'__level, ~ctx-form ~parent-form, ~sample-rate-form, ~'__kind ~'__id ~'__level, ~ctx-form ~parent-form,
~kvs-form ~data-form ~msg-form, ~kvs-form ~data-form ~msg-form,
'~run-form ~'__run-result ~error-form)))) '~run-form ~'__run-result ~error-form)]
run-fn-form (when run-form `(fn [] (~run-form)))] ;; Final unwrapped signal value visible to users/handler-fns, allow to throw
(if-let [sig-middleware# ~middleware-form]
(sig-middleware# ~'__signal) ; Apply signal middleware, can throw
(do ~'__signal)))))]
;; Could avoid double `run-form` expansion with a fn wrap (>0 cost) ;; Could avoid double `run-form` expansion with a fn wrap (>0 cost)
;; (let [run-fn-form (when run-form `(fn [] (~run-form)))]
;; `(let [~'run-fn-form ~run-fn-form] ;; `(let [~'run-fn-form ~run-fn-form]
;; (if-not ~allow? ;; (if-not ~allow?
;; (run-fn-form) ;; (run-fn-form)
;; (let [...]))) ;; (let [...]))))
`(enc/if-not ~allow? ; Allow to throw at call `(enc/if-not ~allow? ; Allow to throw at call
~run-form ~run-form
@ -644,34 +665,25 @@
~'__id ~id-form ; '' ~'__id ~id-form ; ''
~'__uid ~uid-form ; '' ~'__uid ~uid-form ; ''
~'__ns ~ns-form ; '' ~'__ns ~ns-form ; ''
~'__thread ~thread-form ; ''
~'__call-middleware ~(get opts :middleware `taoensso.telemere/*middleware*)
~'__run-result ; Non-throwing (traps) ~'__run-result ; Non-throwing (traps)
~(when run-form ~(when run-form
`(let [~'__t0 (enc/now-nano*)] `(let [t0# (enc/now-nano*)]
(with-tracing ~trace? ~'__id ~'__uid (with-tracing ~trace? ~'__id ~'__uid
(enc/try* (enc/try*
(do (RunResult. ~run-form nil (- (enc/now-nano*) ~'__t0))) (do (RunResult. ~run-form nil (- (enc/now-nano*) t0#)))
(catch :all ~'__t (RunResult. nil ~'__t (- (enc/now-nano*) ~'__t0))))))) (catch :all t# (RunResult. nil t# (- (enc/now-nano*) t0#)))))))
~'__signal_ signal_# ~signal-delay-form]
(delay
;; Cache shared by all handlers. Covers signal `:let` eval, signal construction,
;; middleware (possibly expensive), etc.
;; The unwrapped signal value actually visible to users/handler-fns, realized only (dispatch-signal! ; Runner preserves dynamic bindings when async.
;; AFTER handler filtering. Allowed to throw on deref (handler will catch). ;; Unconditionally send same wrapped signal to all handlers. Each handler will
(let [~'__signal ~signal-form] ; Can throw ;; use wrapper for handler filtering, unwrapping (realizing) only allowed signals.
(if ~'__call-middleware (WrappedSignal. ~'__ns ~'__kind ~'__id ~'__level signal_#))
((sigs/get-middleware-fn ~'__call-middleware) ~'__signal) ; Can throw
(do ~'__signal))))]
;; Unconditionally send same wrapped signal to all handlers.
;; Each handler will then use wrapper for filtering, unwrapping allowed signals.
(dispatch-signal! (WrappedSignal. ~'__ns ~'__kind ~'__id ~'__level ~'__signal_))
(if ~'__run-result (if ~'__run-result
(do (~'__run-result ~'__signal_)) (do (~'__run-result signal_#))
true)))))))) true))))))))
(comment (comment

View file

@ -13,23 +13,23 @@
;;;; Implementation ;;;; Implementation
(defn format-signal->subject-fn (defn signal-subject-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format [signal]) that: Returns a (fn format [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns a formatted email subject like: - Returns a formatted email subject like:
\"INFO EVENT :taoensso.telemere.postal/ev-id1 - msg\"" \"INFO EVENT :taoensso.telemere.postal/ev-id1 - msg\""
([] (format-signal->subject-fn nil)) ([] (signal-subject-fn nil))
([{:keys [max-len subject-signal-key] ([{:keys [max-len subject-signal-key]
:or :or
{max-len 128 {max-len 128
subject-signal-key :postal/subject}}] subject-signal-key :postal/subject}}]
(fn format-signal->subject [signal] (fn signal-subject [signal]
(or (or
(get signal subject-signal-key) ; Custom subject (get signal subject-signal-key) ; Custom subject
;; Simplified `format-signal->prelude-fn` ;; Simplified `utils/signal-preamble-fn`
(let [{:keys [level kind #_ns id msg_]} signal (let [{:keys [level kind #_ns id msg_]} signal
sb (enc/str-builder) sb (enc/str-builder)
s+spc (enc/sb-appender sb " ")] s+spc (enc/sb-appender sb " ")]
@ -41,9 +41,7 @@
(enc/get-substr-by-len (str sb) 0 max-len)))))) (enc/get-substr-by-len (str sb) 0 max-len))))))
(comment (comment ((signal-subject-fn) (tel/with-signal (tel/event! ::ev-id1 #_{:postal/subject "My subject"}))))
((format-signal->subject-fn)
(tel/with-signal (tel/event! ::ev-id1 #_{:postal/subject "My subject"}))))
;;;; Handler ;;;; Handler
@ -55,7 +53,7 @@
Returns a (fn handler [signal]) that: Returns a (fn handler [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Sends an email with formatted signal content to the configured recipient. - Sends formatted signal string as email to specified recipient.
Useful for emailing important alerts to admins, etc. Useful for emailing important alerts to admins, etc.
@ -80,8 +78,8 @@
:cc \"engineering@example.com\" :cc \"engineering@example.com\"
:X-MyHeader \"A custom header\"} :X-MyHeader \"A custom header\"}
`:format-signal-fn` - (fn [signal]) => output, see `help:signal-formatters` `:subject-fn` - (fn [signal]) => email subject string
`:format-signal->subject-fn` - (fn [signal]) => email subject string `:body-fn` - (fn [signal]) => email body content string, see `format-signal-fn` or `pr-signal-fn`
Tips: Tips:
@ -107,12 +105,12 @@
([{:keys ([{:keys
[postal/conn-opts [postal/conn-opts
postal/msg-opts postal/msg-opts
format-signal-fn subject-fn
format-signal->subject-fn] body-fn]
:or :or
{format-signal-fn (utils/format-signal->str-fn) {subject-fn (signal-subject-fn)
format-signal->subject-fn (format-signal->subject-fn)}}] body-fn (utils/format-signal-fn)}}]
(when-not conn-opts (throw (ex-info "No `:postal/conn-opts` was provided" {}))) (when-not conn-opts (throw (ex-info "No `:postal/conn-opts` was provided" {})))
(when-not msg-opts (throw (ex-info "No `:postal/msg-opts` was provided" {}))) (when-not msg-opts (throw (ex-info "No `:postal/msg-opts` was provided" {})))
@ -121,12 +119,16 @@
(defn a-handler:postal (defn a-handler:postal
([]) ; Shut down (no-op) ([]) ; Shut down (no-op)
([signal] ([signal]
(enc/when-let [subject (subject-fn signal)
body (body-fn signal)]
(let [msg (let [msg
(assoc msg-opts (assoc msg-opts
:subject (format-signal->subject-fn) :subject (str subject)
:body :body
(if (string? body)
[{:type "text/plain; charset=utf-8" [{:type "text/plain; charset=utf-8"
:content (format-signal-fn signal)}]) :content (str body)}]
body))
[result ex] [result ex]
(try (try
@ -136,4 +138,4 @@
success? (= (get result :code) 0)] success? (= (get result :code) 0)]
(when-not success? (when-not success?
(throw (ex-info "Failed to send email" result ex))))))))) (throw (ex-info "Failed to send email" result ex))))))))))

View file

@ -0,0 +1,106 @@
(ns taoensso.telemere.sockets
"Basic TCP/UDP socket handlers."
(:require
[taoensso.encore :as enc :refer [have have?]]
[taoensso.telemere.utils :as utils])
(:import
[java.net Socket InetAddress]
[java.net DatagramSocket DatagramPacket InetSocketAddress]
[java.io PrintWriter]))
(comment
(require '[taoensso.telemere :as tel])
(remove-ns 'taoensso.telemere.sockets)
(:api (enc/interns-overview)))
;;;; Implementation
;;;; Handlers
(defn handler:tcp-socket
"Experimental, subject to change. Feedback welcome!
Returns a (fn handler [signal]) that:
- Takes a Telemere signal.
- Sends formatted signal string to specified TCP socket.
Options:
`host` - Destination TCP socket hostname string
`port` - Destination TCP socket port int
`:socket-opts` - {:keys [ssl? connect-timeout-msecs]}
`:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
Limitations:
- Failed writes will be retried only once.
- Writes lock on a single underlying socket, so IO won't benefit from adding
extra handler threads. Let me know if there's demand for socket pooling."
([host port] (handler:tcp-socket host port nil))
([host port
{:keys [socket-opts output-fn]
:or {output-fn (utils/format-signal-fn)}}]
(let [sw (utils/tcp-socket-writer host port socket-opts)]
(defn a-handler:tcp-socket
([] (sw)) ; Shut down
([signal]
(when-let [output (output-fn signal)]
(sw output)))))))
(defn handler:udp-socket
"Experimental, subject to change. Feedback welcome!
Returns a (fn handler [signal]) that:
- Takes a Telemere signal.
- Sends formatted signal string to specified UDP socket.
Options:
`host` - Destination UDP socket hostname string
`port` - Destination UDP socket port int
`:output-fn` - (fn [signal]) => output string, see `format-signal-fn` or `pr-signal-fn`
`:max-packet-bytes` - Max packet size (in bytes) before truncating output (default 512)
`:truncation-warning-fn` - Optional (fn [{:keys [max actual signal]}]) to call whenever
output is truncated. Should be appropriately rate-limited!
Limitations:
- Due to UDP limitations, truncates output to `max-packet-bytes`!
- Failed writes will be retried only once.
- Writes lock on a single underlying socket, so IO won't benefit from adding
extra handler threads. Let me know if there's demand for socket pooling.
- No DTLS (Datagram Transport Layer Security) support,
please let me know if there's demand."
([host port] (handler:udp-socket host port nil))
([host port
{:keys [output-fn max-packet-bytes truncation-warning-fn]
:or
{output-fn (utils/format-signal-fn)
max-packet-bytes 512}}]
(let [max-packet-bytes (int max-packet-bytes)
socket (DatagramSocket.) ; No need to change socket once created
lock (Object.)]
(.connect socket (InetSocketAddress. (str host) (int port)))
(defn a-handler:udp-socket
([] (.close socket)) ; Shut down
([signal]
(when-let [output (output-fn signal)]
(let [ba (enc/str->utf8-ba (str output))
ba-len (alength ba)
packet (DatagramPacket. ba (min ba-len max-packet-bytes))]
(when (and truncation-warning-fn (> ba-len max-packet-bytes))
;; Fn should be appropriately rate-limited
(truncation-warning-fn {:max max-packet-bytes, :actual ba-len, :signal signal}))
(locking lock
(try
(.send (DatagramSocket.) packet)
(catch Exception _ ; Retry once
(Thread/sleep 250)
(.send (DatagramSocket.) packet)))))))))))

View file

@ -4,7 +4,8 @@
(:require (:require
[clojure.string :as str] [clojure.string :as str]
#?(:clj [clojure.java.io :as jio]) #?(:clj [clojure.java.io :as jio])
[taoensso.encore :as enc :refer [have have?]])) [taoensso.encore :as enc :refer [have have?]]
[taoensso.telemere.impl :as impl]))
(comment (comment
(require '[taoensso.telemere :as tel]) (require '[taoensso.telemere :as tel])
@ -52,7 +53,7 @@
;;;; Public misc ;;;; Public misc
(enc/defaliases enc/newline enc/pr-edn #?(:cljs enc/pr-json)) (enc/defaliases enc/newline enc/pr-edn #?(:cljs enc/pr-json) #?(:clj impl/thread-info))
#?(:clj (defn thread-name "Returns string name of current thread." ^String [] (.getName (Thread/currentThread)))) #?(:clj (defn thread-name "Returns string name of current thread." ^String [] (.getName (Thread/currentThread))))
#?(:clj (defn thread-id "Returns long id of current thread." ^long [] (.getId (Thread/currentThread)))) #?(:clj (defn thread-id "Returns long id of current thread." ^long [] (.getId (Thread/currentThread))))
@ -249,7 +250,7 @@
:writer/state {:file file, :stream (.deref stream_)} :writer/state {:file file, :stream (.deref stream_)}
(when (open?_) (when (open?_)
(let [content content-or-action (let [content content-or-action
ba (.getBytes (str content) java.nio.charset.StandardCharsets/UTF_8)] ba (enc/str->utf8-ba (str content))]
(locking lock (locking lock
(try (try
(file-exists!) (file-exists!)
@ -260,6 +261,133 @@
(comment (def fw1 (file-writer "test.txt" true)) (fw1 "x") (fw1)) (comment (def fw1 (file-writer "test.txt" true)) (fw1 "x") (fw1))
;;;; Sockets
#?(:clj
(defn- default-socket-fn
"Returns conected `java.net.Socket`, or throws."
^java.net.Socket [host port connect-timeout-msecs]
(let [addr (java.net.InetSocketAddress. ^String host (int port))
socket (java.net.Socket.)]
(if connect-timeout-msecs
(.connect socket addr (int connect-timeout-msecs))
(.connect socket addr))
socket)))
#?(:clj
(let [factory_ (delay (javax.net.ssl.SSLSocketFactory/getDefault))]
(defn- default-ssl-socket-fn
"Returns connected SSL `java.net.Socket`, or throws."
^java.net.Socket [^java.net.Socket socket ^String host port]
(.createSocket ^javax.net.ssl.SSLSocketFactory @factory_
socket host (int port) true))))
#?(:clj
(defn tcp-socket-writer
"Experimental, subject to change. Feedback welcome!
Connects to specified TCP socket and returns a stateful fn of 2 arities:
[content] => Writes given content to socket, or no-ops if closed.
[] => Closes the writer.
Useful for basic handlers that write to a TCP socket, etc.
Options:
`:ssl?` - Use SSL/TLS?
`:connect-timeout-msecs` - Connection timeout (default 3000 msecs)
`:socket-fn` - (fn [host port timeout]) => `java.net.Socket`
`:ssl-socket-fn` - (fn [socket host port]) => `java.net.Socket`
Notes:
- Writer should be manually closed after use (with zero-arity call).
- Flushes after every write.
- Will retry failed writes once, then drop.
- Thread safe, locks on single socket stream.
- Advanced users may want a custom implementation using a connection
pool and/or more sophisticated retry semantics, etc."
[host port
{:keys
[ssl? connect-timeout-msecs,
socket-fn ssl-socket-fn] :as opts
:or
{connect-timeout-msecs 3000
socket-fn default-socket-fn
ssl-socket-fn default-ssl-socket-fn}}]
(let [new-conn! ; => [<java.net.Socket> <java.io.OutputStream>], or throws
(fn []
(try
(let [^java.net.Socket socket
(let [socket (socket-fn host port connect-timeout-msecs)]
(if ssl?
(ssl-socket-fn socket host port)
(do socket)))]
[socket (.getOutputStream socket)])
(catch Exception ex
(throw (ex-info "Failed to create connection" opts ex)))))
conn_ (volatile! (new-conn!))
open?_ (enc/latom true)
close!
(fn []
(when (compare-and-set! open?_ true false)
(when-let [[^java.net.Socket socket] (.deref conn_)]
(.close socket)
(vreset! conn_ nil)
true)))
reset!
(fn []
(close!)
(vreset! conn_ (new-conn!))
(reset! open?_ true)
true)
write-ba!
(fn [^bytes ba-content]
(when-let [[_ ^java.io.OutputStream output] (.deref conn_)]
(.write output ba-content)
(.flush output)
true))
conn-okay!
(let [rl (enc/rate-limiter-once-per 250)]
(fn []
(or
(rl)
(when-let [[^java.net.Socket socket] (.deref conn_)]
(and
(not (.isClosed socket))
(do (.isConnected socket))))
(throw (java.io.IOException. "Bad connection")))))
lock (Object.)]
(fn a-tcp-socket-writer
([] (when (open?_) (locking lock (close!))))
([content-or-action]
(case content-or-action ; Undocumented, for dev/testing
:writer/open? (open?_)
:writer/reset! (locking lock (reset!))
:writer/state {:conn (.deref conn_)}
(when (open?_)
(let [content content-or-action
ba (enc/str->utf8-ba (str content))]
(locking lock
(try
(conn-okay!)
(write-ba! ba)
(catch Exception _ ; Retry once
(reset!)
(write-ba! ba))))))))))))
;;;; Formatters ;;;; Formatters
(defn format-nsecs-fn (defn format-nsecs-fn
@ -324,17 +452,21 @@
(do (enc/ex-map (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"})))) (do (enc/ex-map (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"}))))
(println (str "--\n" ((format-error-fn) (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"})))))) (println (str "--\n" ((format-error-fn) (ex-info "Ex2" {:k2 "v2"} (ex-info "Ex1" {:k1 "v1"}))))))
(defn format-signal->prelude-fn ;;;;
(defn signal-preamble-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format [signal]) that: Returns a (fn preamble [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns a formatted prelude string like: - Returns a signal preamble string like:
\"2024-03-26T11:14:51.806Z INFO EVENT Hostname taoensso.telemere(2,21) ::ev-id - msg\"" \"2024-03-26T11:14:51.806Z INFO EVENT Hostname taoensso.telemere(2,21) ::ev-id - msg\"
([] (format-signal->prelude-fn nil))
See arglists for options."
([] (signal-preamble-fn nil))
([{:keys [format-inst-fn] ([{:keys [format-inst-fn]
:or {format-inst-fn (format-inst-fn)}}] :or {format-inst-fn (format-inst-fn)}}]
(fn format-signal->prelude [signal] (fn signal-preamble [signal]
(let [{:keys [inst level kind ns id msg_]} signal (let [{:keys [inst level kind ns id msg_]} signal
sb (enc/str-builder) sb (enc/str-builder)
s+spc (enc/sb-appender sb " ")] s+spc (enc/sb-appender sb " ")]
@ -358,30 +490,49 @@
(when-let [msg (force msg_)] (s+spc "- " msg)) (when-let [msg (force msg_)] (s+spc "- " msg))
(str sb))))) (str sb)))))
(comment ((format-signal->prelude-fn) (tel/with-signal (tel/event! ::ev-id)))) (comment ((signal-preamble-fn) (tel/with-signal (tel/event! ::ev-id))))
(defn signal-content-fn
"Experimental, subject to change.
Returns a (fn content [signal]) that:
- Takes a Telemere signal.
- Returns a signal content string (incl. data, ctx, etc.)
See arglists for options."
([] (signal-content-fn nil))
([{:keys
[incl-thread? incl-kvs? raw-error?,
format-nsecs-fn format-error-fn]
(defn ^:no-doc signal-content-handler
"Private, don't use.
Returns a (fn handle [signal handle-fn value-fn]) for internal use.
Content equivalent to `format-signal->prelude-fn`."
([] (signal-content-handler nil))
([{:keys [format-nsecs-fn format-error-fn raw-error?]
:or :or
{format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs]) {format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs])
format-error-fn (format-error-fn) ; (fn [error]) format-error-fn (format-error-fn) ; (fn [error])
}}] }}]
(let [err-start (str newline "<<< error <<<" newline) (let [nl newline
err-stop (str newline ">>> error >>>")] err-start (str nl "<<< error <<<" nl)
err-stop (str nl ">>> error >>>")]
(fn a-signal-content-handler [signal hf vf] (fn signal-content
(let [{:keys [uid parent data #_kvs ctx sample-rate]} signal] ([signal]
(when sample-rate (hf "sample: " (vf sample-rate))) (let [sb (enc/str-builder)
(when uid (hf " uid: " (vf uid))) s++ (enc/sb-appender sb nl)]
(when parent (hf "parent: " (vf parent))) (signal-content signal s++ enc/pr-edn*)
(when data (hf " data: " (vf data))) (str sb)))
#_(when kvs (hf " kvs: " (vf kvs))) ; Don't auto include in output
(when ctx (hf " ctx: " (vf ctx)))) ;; Undocumented, advanced arity
([signal append-fn val-fn]
(let [af append-fn
vf val-fn]
(let [{:keys [uid parent data kvs ctx #?(:clj thread) sample-rate]} signal]
(when sample-rate (af " sample: " (vf sample-rate)))
(when uid (af " uid: " (vf uid)))
(when parent (af " parent: " (vf parent)))
#?(:clj (when (and thread incl-thread?) (af " thread: " (vf thread))))
(when data (af " data: " (vf data)))
(when (and kvs incl-kvs?) (af " kvs: " (vf kvs)))
(when ctx (af " ctx: " (vf ctx))))
(let [{:keys [run-form error]} signal] (let [{:keys [run-form error]} signal]
(when run-form (when run-form
@ -398,89 +549,103 @@
:nsecs run-nsecs :nsecs run-nsecs
:val run-val :val run-val
#?@(:clj [:val-type (enc/class-sym run-val)])})] #?@(:clj [:val-type (enc/class-sym run-val)])})]
(af " run: " (vf run-info))))
(hf " run: " (vf run-info))))
(when error (when error
(if raw-error? (if raw-error?
(hf " error: " error) (af " error: " error)
(when-let [ff format-error-fn] (when-let [ff format-error-fn]
(hf err-start (ff error) err-stop))))))))) (af err-start (ff error) err-stop)))))))))))
;;;; Signal formatters (comment ((signal-content-fn) (tel/with-signal (tel/event! ::ev-id {:data {:k1 "v1"}}))))
(defn format-signal->edn-fn (defn pr-signal-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format->edn [signal]) that: Returns a (fn pr-signal [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns edn string of the (minified) signal." - Returns machine-readable serialized string of the (minified) signal.
([] (format-signal->edn-fn nil))
([{:keys [pr-edn-fn prep-fn] Options include:
`pr-fn` ∈ #{<unary-fn> :edn :json (Cljs only)}
See arglists for more.
Examples:
(pr-signal-fn :edn {<opts>})
(pr-signal-fn :json {<opts>}) ; Cljs only
;; To output JSON for Clj, you must provide an appropriate `pr-fn`.
;; `jsonista` is a good option, Ref. <https://github.com/metosin/jsonista>:
(require '[jsonista.core :as jsonista])
(pr-signal-fn jsonista/write-value-as-string {<opts>})
See also `format-signal-fn` for human-readable output."
([pr-fn] (pr-signal-fn pr-fn nil))
([pr-fn
{:keys [incl-thread? incl-kvs? incl-newline?, prep-fn]
:or :or
{pr-edn-fn pr-edn {incl-newline? true
prep-fn (comp error-in-signal->maps minify-signal)}}] prep-fn
(comp error-in-signal->maps
minify-signal)}}]
(fn format-signal->edn [signal] (let [nl newline
(let [signal* (if prep-fn (prep-fn signal) signal)] pr-fn
(pr-edn-fn signal*))))) (or
(case pr-fn
:edn pr-edn
#?@(:cljs [:json pr-json])
(comment ((format-signal->edn-fn) {:level :info, :msg "msg"})) (if (fn? pr-fn)
(do pr-fn)
(enc/unexpected-arg! pr-fn
{:context `pr-signal-fn
:param 'pr-fn
:expected
#?(:clj '#{:edn unary-fn}
:cljs '#{:edn :json unary-fn})}))
(defn format-signal->json-fn (have fn? pr-fn)))]
(fn pr-signal [signal]
(let [not-map? (not (map? signal))
signal (if (or incl-kvs? not-map?) signal (dissoc signal :kvs))
signal (if (or incl-thread? not-map?) signal (dissoc signal :thread))
signal (if prep-fn (prep-fn signal) signal)
output (pr-fn signal)]
(if incl-newline?
(str output nl)
(do output)))))))
(comment ((pr-signal-fn :edn) (tel/with-signal (tel/event! ::ev-id {:kvs {:k1 "v1"}}))))
(defn format-signal-fn
"Experimental, subject to change. "Experimental, subject to change.
Returns a (fn format->json [signal]) that: Returns a (fn format [signal]) that:
- Takes a Telemere signal. - Takes a Telemere signal.
- Returns JSON string of the (minified) signal. - Returns human-readable formatted string.
(Clj only): An appropriate `:pr-json-fn` MUST be provided." See also `pr-signal-fn` for machine-readable output."
([] (format-signal->json-fn nil)) ([] (format-signal-fn nil))
([{:keys [pr-json-fn prep-fn] ([{:keys [incl-newline? preamble-fn content-fn]
:or :or
{#?@(:cljs [pr-json-fn pr-json]) {incl-newline? true
prep-fn (comp error-in-signal->maps minify-signal)}}] preamble-fn (signal-preamble-fn)
content-fn (signal-content-fn)}}]
(when-not pr-json-fn (let [nl newline]
(throw (fn format-signal [signal]
(ex-info (str "No `" `format-signal->json-fn "` `:pr-json-fn` was provided") {}))) (let [preamble (when preamble-fn (preamble-fn signal))
content (when content-fn (content-fn signal))]
(fn format-signal->json [signal] (if preamble
(let [signal* (if prep-fn (prep-fn signal) signal)] (if incl-newline? (str preamble nl content nl) (str preamble nl content))
(pr-json-fn signal*))))) (if incl-newline? (str content nl) (str content))))))))
(comment ((format-signal->json-fn) {:level :info, :msg "msg"}))
(defn format-signal->str-fn
"Experimental, subject to change.
Returns a (fn format->str [signal]) that:
- Takes a Telemere signal.
- Returns a formatted string intended for text consoles, etc."
([] (format-signal->str-fn nil))
([{:keys [format-signal->prelude-fn
format-nsecs-fn format-error-fn]
:or
{format-signal->prelude-fn (format-signal->prelude-fn) ; (fn [signal])
format-nsecs-fn (format-nsecs-fn) ; (fn [nanosecs])
format-error-fn (format-error-fn) ; (fn [error])
}}]
(let [signal-content-handler ; (fn [signal hf vf]
(signal-content-handler
{:format-nsecs-fn format-nsecs-fn
:format-error-fn format-error-fn})]
(fn format-signal->str [signal]
(let [sb (enc/str-builder)
s+ (partial enc/sb-append sb)
s++ (partial enc/sb-append sb (str newline " "))]
(when-let [ff format-signal->prelude-fn] (s+ (ff signal))) ; Prelude
(signal-content-handler signal s++ enc/pr-edn*) ; Content
(str sb))))))
(comment (comment
(tel/with-ctx {:c :C} (tel/with-ctx {:c :C}
(println (println
((format-signal->str-fn) ((format-signal-fn)
(tel/with-signal (tel/with-signal
(tel/event! ::ev-id (tel/event! ::ev-id
{:user-k1 #{:a :b :c} {:user-k1 #{:a :b :c}

View file

@ -229,10 +229,10 @@
(testing "Call middleware" (testing "Call middleware"
(let [c (enc/counter) (let [c (enc/counter)
[[rv1 _] [sv1]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))]})) [[rv1 _] [sv1]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))}))
[[rv2 _] [sv2]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))], :allow? false})) [[rv2 _] [sv2]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false}))
[[rv3 _] [sv3]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))]})) [[rv3 _] [sv3]] (with-sigs :raw nil (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))}))
[[rv4 _] [sv4]] (with-sigs :raw nil (sig! {:level :info, :middleware [(fn [_] "signal-value")]}))] [[rv4 _] [sv4]] (with-sigs :raw nil (sig! {:level :info, :middleware (fn [_] "signal-value")}))]
[(is (= rv1 0)) (is (sm? sv1 {:m1 1 :m2 2})) [(is (= rv1 0)) (is (sm? sv1 {:m1 1 :m2 2}))
(is (= rv2 3)) (is (nil? sv2)) (is (= rv2 3)) (is (nil? sv2))
@ -240,6 +240,10 @@
(is (= rv4 true)) (is (= sv4 "signal-value")) (is (= rv4 true)) (is (= sv4 "signal-value"))
(is (= @c 7) "3x run + 4x middleware")])) (is (= @c 7) "3x run + 4x middleware")]))
(testing "Binding conveyance"
(binding [*dynamic-var* :foo]
(is (sm? (with-sig (sig! {:level :info, :data {:dynamic-var *dynamic-var*}})) {:data {:dynamic-var :foo}}))))
#?(:clj #?(:clj
(testing "Printing" (testing "Printing"
(let [sv1 (with-sig (sig! {:level :info, :run (+ 1 2), :my-k1 :my-v1})) (let [sv1 (with-sig (sig! {:level :info, :run (+ 1 2), :my-k1 :my-v1}))
@ -256,31 +260,31 @@
(let [c (enc/counter) (let [c (enc/counter)
sv-h1_ (atom nil) sv-h1_ (atom nil)
sv-h2_ (atom nil) sv-h2_ (atom nil)
wh1 (sigs/wrap-handler :hid1 (fn [sv] (reset! sv-h1_ sv)) nil {:async nil, :middleware [#(assoc % :hm1 (c)) #(assoc % :hm2 (c))]}) wh1 (sigs/wrap-handler :hid1 (fn [sv] (reset! sv-h1_ sv)) nil {:async nil, :middleware (tel/comp-middleware #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))})
wh2 (sigs/wrap-handler :hid2 (fn [sv] (reset! sv-h2_ sv)) nil {:async nil, :middleware [#(assoc % :hm1 (c)) #(assoc % :hm2 (c))]})] wh2 (sigs/wrap-handler :hid2 (fn [sv] (reset! sv-h2_ sv)) nil {:async nil, :middleware (tel/comp-middleware #(assoc % :hm1 (c)) #(assoc % :hm2 (c)))})]
;; Note that call middleware output is cached and shared across all handlers ;; Note that call middleware output is cached and shared across all handlers
(binding [impl/*sig-handlers* [wh1 wh2]] (binding [impl/*sig-handlers* [wh1 wh2]]
(let [;; 1x run + 4x handler middleware + 2x call middleware = 7x (let [;; 1x run + 4x handler middleware + 2x call middleware = 7x
rv1 (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))]}) rv1 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))})
sv1-h1 @sv-h1_ sv1-h1 @sv-h1_
sv1-h2 @sv-h2_ sv1-h2 @sv-h2_
c1 @c c1 @c
;; 1x run ;; 1x run
rv2 (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))], :allow? false}) rv2 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c))), :allow? false})
sv2-h1 @sv-h1_ sv2-h1 @sv-h1_
sv2-h2 @sv-h2_ sv2-h2 @sv-h2_
c2 @c ; 8 c2 @c ; 8
;; 1x run + 4x handler middleware + 2x call middleware = 7x ;; 1x run + 4x handler middleware + 2x call middleware = 7x
rv3 (sig! {:level :info, :run (c), :middleware [#(assoc % :m1 (c)) #(assoc % :m2 (c))]}) rv3 (sig! {:level :info, :run (c), :middleware (tel/comp-middleware #(assoc % :m1 (c)) #(assoc % :m2 (c)))})
sv3-h1 @sv-h1_ sv3-h1 @sv-h1_
sv3-h2 @sv-h2_ sv3-h2 @sv-h2_
c3 @c ; 15 c3 @c ; 15
;; 4x handler middleware ;; 4x handler middleware
rv4 (sig! {:level :info, :middleware [(fn [_] {:my-sig-val? true})]}) rv4 (sig! {:level :info, :middleware (fn [_] {:my-sig-val? true})})
sv4-h1 @sv-h1_ sv4-h1 @sv-h1_
sv4-h2 @sv-h2_ sv4-h2 @sv-h2_
c4 @c] c4 @c]
@ -317,7 +321,7 @@
(tel/with-handler :hid1 (tel/with-handler :hid1
(fn [sv] (force (:data sv)) (reset! sv_ sv)) (fn [sv] (force (:data sv)) (reset! sv_ sv))
{:async nil, :error-fn (fn [x] (reset! error_ x)), :rl-error nil, {:async nil, :error-fn (fn [x] (reset! error_ x)), :rl-error nil,
:middleware [(fn [sv] (if *throwing-handler-middleware?* (ex1!) sv))]} :middleware (fn [sv] (if *throwing-handler-middleware?* (ex1!) sv))}
[(is (->> (sig! {:level :info, :when (ex1!)}) (throws? :ex-info "Ex1")) "`~filterable-expansion/allow` throws at call") [(is (->> (sig! {:level :info, :when (ex1!)}) (throws? :ex-info "Ex1")) "`~filterable-expansion/allow` throws at call")
(is (->> (sig! {:level :info, :inst (ex1!)}) (throws? :ex-info "Ex1")) "`~inst-form` throws at call") (is (->> (sig! {:level :info, :inst (ex1!)}) (throws? :ex-info "Ex1")) "`~inst-form` throws at call")
@ -335,7 +339,7 @@
(testing "Throwing call middleware" (testing "Throwing call middleware"
(reset-state!) (reset-state!)
[(is (true? (sig! {:level :info, :middleware [(fn [_] (ex1!))]}))) [(is (true? (sig! {:level :info, :middleware (fn [_] (ex1!))})))
(is (= @sv_ :nx)) (is (= @sv_ :nx))
(is (sm? @error_ {:handler-id :hid1, :error pex1?}))]) (is (sm? @error_ {:handler-id :hid1, :error pex1?}))])
@ -670,15 +674,16 @@
#?(:clj " Root: clojure.lang.ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: clojure.lang.ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n" #?(:clj " Root: clojure.lang.ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: clojure.lang.ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n"
:cljs " Root: cljs.core/ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: cljs.core/ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n"))) :cljs " Root: cljs.core/ExceptionInfo - Ex1\n data: {:k1 \"v1\"}\n\nCaused: cljs.core/ExceptionInfo - Ex2\n data: {:k2 \"v2\"}\n\nRoot stack trace:\n")))
(testing "signal-preamble-fn"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0})) (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))
prelude ((utils/format-signal->prelude-fn) sig)] ; "2024-06-09T21:15:20.170Z INFO EVENT taoensso.telemere-tests(592,35) ::ev-id" preamble ((utils/signal-preamble-fn) sig)] ; "2024-06-09T21:15:20.170Z INFO EVENT taoensso.telemere-tests(592,35) ::ev-id"
[(is (enc/str-starts-with? prelude "2024-06-09T21:15:20.170Z INFO EVENT")) [(is (enc/str-starts-with? preamble "2024-06-09T21:15:20.170Z INFO EVENT"))
(is (enc/str-ends-with? prelude "::ev-id")) (is (enc/str-ends-with? preamble "::ev-id"))
(is (string? (re-find #"taoensso.telemere-tests\(\d+,\d+\)" prelude)))]) (is (string? (re-find #"taoensso.telemere-tests\(\d+,\d+\)" preamble)))]))
(testing "format-signal->edn-fn" (testing "pr-signal-fn/edn"
(let [sig (update (with-sig (tel/event! ::ev-id {:inst t0})) :inst enc/inst->udt) (let [sig (update (with-sig (tel/event! ::ev-id {:inst t0})) :inst enc/inst->udt)
sig* (enc/read-edn ((utils/format-signal->edn-fn) sig))] sig* (enc/read-edn ((tel/pr-signal-fn :edn) sig))]
(is (is
(enc/submap? sig* (enc/submap? sig*
{:schema 1, :kind :event, :id ::ev-id, :level :info, {:schema 1, :kind :event, :id ::ev-id, :level :info,
@ -688,9 +693,9 @@
:column pnat-int?})))) :column pnat-int?}))))
#?(:cljs #?(:cljs
(testing "format-signal->json-fn" (testing "pr-signal-fn/json"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0})) (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))
sig* (enc/read-json ((utils/format-signal->json-fn) sig))] sig* (enc/read-json ((tel/pr-signal-fn :json) sig))]
(is (is
(enc/submap? sig* (enc/submap? sig*
{"schema" 1, "kind" "event", "id" "taoensso.telemere-tests/ev-id", {"schema" 1, "kind" "event", "id" "taoensso.telemere-tests/ev-id",
@ -699,9 +704,9 @@
"line" pnat-int? "line" pnat-int?
"column" pnat-int?}))))) "column" pnat-int?})))))
(testing "format-signal->str-fn" (testing "format-signal-fn"
(let [sig (with-sig (tel/event! ::ev-id {:inst t0}))] (let [sig (with-sig (tel/event! ::ev-id {:inst t0}))]
(is (enc/str-starts-with? ((utils/format-signal->str-fn) sig) (is (enc/str-starts-with? ((tel/format-signal-fn) sig)
"2024-06-09T21:15:20.170Z INFO EVENT"))))])]) "2024-06-09T21:15:20.170Z INFO EVENT"))))])])
;;;; File handler ;;;; File handler

View file

@ -216,4 +216,3 @@ Telemere includes extensive internal help docstrings:
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |

View file

@ -28,4 +28,3 @@ For more info see:
| [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling | | [`help:signal-flow`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-flow) | Ordered flow from signal creation to handling |
| [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters | | [`help:signal-filters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-filters) | API for configuring signal filters |
| [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers | | [`help:signal-handlers`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-handlers) | API for configuring signal handlers |
| [`help:signal-formatters`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#help:signal-formatters) | Signal formatters for use by handlers |

View file

@ -2,19 +2,26 @@ Signal handlers process created signals to *do something with them* (analyse the
# Included handlers # Included handlers
The following signal handlers are currently included out-the-box: A number of signal handlers are included out-the box. Alphabetically:
| Name | Platform | Output target | Output format | | Name | Platform | Output target | Output format |
| :------------------------------------------------------------------------------------------------------------------------------------------------------- | :------- | :------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------- | | :------------------------------------------------------------------------------------------------------------------------------------------------------- | :------- | :------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------- |
| [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Clj | `*out*` or `*err*` | Formatted string [1] | | [`handler:carmine`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.carmine#handler:carmine) [0] | Clj | [Redis](https://redis.io/) (via [Carmine](https://www.taoensso.com/carmine)) | Serialized signals [1] |
| [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Cljs | Browser console | Formatted string [1] | | [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Clj | `*out*` or `*err*` | String [2] |
| [`handler:console-raw`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) | Cljs | Browser console | Raw signal data [2] | | [`handler:console`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console) | Cljs | Browser console | String [2] |
| [`handler:file`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:file) | Clj | File/s on disk | Formatted string [1] | | [`handler:console-raw`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:console-raw) | Cljs | Browser console | Raw signals [3] |
| [`handler:postal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.postal#handler:postal) | Clj | Email (via [postal](https://github.com/drewr/postal)) | Formatted string [1] | | [`handler:file`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#handler:file) | Clj | File/s on disk | String [2] |
| [`handler:logstash`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.logstash#handler:logstash) [0] | Clj | [Logstash](https://www.elastic.co/logstash) | TODO |
| [`handler:open-telemetry-logger`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.open-telemetry#handler:open-telemetry-logger) | Clj | [OpenTelemetry](https://opentelemetry.io/) [Java client](https://github.com/open-telemetry/opentelemetry-java) | [LogRecord](https://opentelemetry.io/docs/specs/otel/logs/data-model/) | | [`handler:open-telemetry-logger`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.open-telemetry#handler:open-telemetry-logger) | Clj | [OpenTelemetry](https://opentelemetry.io/) [Java client](https://github.com/open-telemetry/opentelemetry-java) | [LogRecord](https://opentelemetry.io/docs/specs/otel/logs/data-model/) |
| [`handler:postal`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.postal#handler:postal) | Clj | Email (via [postal](https://github.com/drewr/postal)) | String [2] |
| [`handler:slack`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.slack#handler:slack) [0] | Clj | [Slack](https://slack.com/) (via [clj-slack](https://github.com/julienXX/clj-slack)) | String [2] |
| [`handler:tcp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:tcp-socket) | Clj | TCP socket | String [2] |
| [`handler:udp-socket`](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere.sockets#handler:udp-socket) | Clj | UDP socket | String [2] |
- \[1] [Configurable](https://cljdoc.org/d/com.taoensso/telemere/1.0.0-beta3/api/taoensso.telemere#help:signal-formatters): human-readable (default), [edn](https://github.com/edn-format/edn), [JSON](https://www.json.org/), etc. - \[0] Coming soon
- \[2] For use with browser formatting tools like [cljs-devtools](https://github.com/binaryage/cljs-devtools). - \[1] Uses [Nippy](https://taoensso.com/nippy) to support all Clojure's rich data types
- \[2] [Human-readable](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#format-signal-fn) (default), or [machine-readable](https://cljdoc.org/d/com.taoensso/telemere/CURRENT/api/taoensso.telemere#pr-signal-fn) ([edn](https://github.com/edn-format/edn), [JSON](https://www.json.org/), etc.).
- \[3] For use with browser formatting tools like [cljs-devtools](https://github.com/binaryage/cljs-devtools).
- See relevant docstrings (links above) for features, usage, etc. - See relevant docstrings (links above) for features, usage, etc.
- See section [8-Community](8-Community.md) for more (community-supported) handlers. - See section [8-Community](8-Community.md) for more (community-supported) handlers.
- If there's other handlers you'd like to see, feel free to [ping me](https://github.com/taoensso/telemere/issues), or ask on the [`#telemere` Slack channel](https://www.taoensso.com/telemere/slack). It helps to know what people most need! - If there's other handlers you'd like to see, feel free to [ping me](https://github.com/taoensso/telemere/issues), or ask on the [`#telemere` Slack channel](https://www.taoensso.com/telemere/slack). It helps to know what people most need!
@ -69,7 +76,7 @@ To instead writes signals as edn:
;; Create console which writes edn ;; Create console which writes edn
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn (taoensso.telemere.utils/format-signal->edn-fn)})) {:output-fn (t/pr-signal-fn :edn)}))
(my-handler my-signal) ; => (my-handler my-signal) ; =>
;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...} ;; {:inst #inst "2024-04-11T10:54:57.202869Z", :msg_ "My message", :ns "examples", ...}
@ -78,18 +85,41 @@ To instead writes signals as edn:
To instead writes signals as JSON: To instead writes signals as JSON:
```clojure ```clojure
;; Create console which writes JSON ;; Create console which writes signals as JSON
#?(:clj (require '[jsonista.core :as jsonista]))
(def my-handler (def my-handler
(t/handler:console (t/handler:console
{:format-signal-fn {:output-fn
(taoensso.telemere.utils/format-signal->json-fn (t/pr-signal-fn
{:pr-json-fn jsonista.core/write-value-as-string})})) #?(:cljs :json
:clj jsonista.core/write-value-as-string))}))
(my-handler my-signal) ; =>
;; {"inst":"2024-04-11T10:54:57.202869Z","msg_":"My message","ns":"examples", ...}
``` ```
Note that when writing JSON with Clojure, you *must* specify a `pr-json-fn`. This lets you plug in the JSON serializer of your choice ([jsonista](https://github.com/metosin/jsonista) is my default recommendation). Note that when writing JSON with Clojure, you *must* provide an appropriate `pr-fn`. This lets you plug in the JSON serializer of your choice ([jsonista](https://github.com/metosin/jsonista) is my default recommendation).
### Handler-specific per-signal kvs
Telemere includes a handy mechanism for including arbitrary user-level data/opts in individual signals for use by custom middleware and/or handlers.
Any *non-standard* (user) keys you include in your signal constructor opts will automatically be included in created signals, e.g.:
```clojure
(t/with-signal
(t/event! ::my-id
{:my-middleware-data "foo"
:my-handler-data "bar"}))
;; %>
;; {;; User kvs included inline (assoc'd to signal root)
;; :my-middleware-data "foo"
;; :my-handler-data "bar"
;; :kvs ; And also collected together under ":kvs" key
;; {:my-middleware-data "foo"
;; :my-handler-data "bar"}
;; ... }
```
These user-level data/opts are typically NOT included by default in handler output, making them a great way to convey data/opts to custom middleware/handlers.
# Managing handlers # Managing handlers

View file

@ -18,11 +18,11 @@ Why release Telemere as a *new library* instead of just updating Timbre?
Timbre was first released 12+ years ago, and has mostly attempted to keep breaks in that time minimal. Which means that its fundamental design is now 12+ years old. Timbre was first released 12+ years ago, and has mostly attempted to keep breaks in that time minimal. Which means that its fundamental design is now 12+ years old.
I've learnt a lot since then, and would write Timbre differently if I were doing it again today. There's many improvements I've wanted to make over the years, but held back both because of the effort involved and because of not wanting to break Timbre users that are happy with it the way it is. I've learnt a lot since then, and would write Timbre differently if I were doing it again today. There's many refinements I've wanted to make over the years, but held back both because of the effort involved and because of not wanting to break Timbre users that are happy with it the way it is.
Since receiving [open source funding](https://www.taoensso.com/my-work), undertaking larger projects became feasible - so I decided to experiment with a proof-of-concept rewrite free of all historical constraints. Since receiving [open source funding](https://www.taoensso.com/my-work), undertaking larger projects became feasible - so I decided to experiment with a proof-of-concept rewrite free of all historical constraints.
That eventually grew into Telemere. That eventually grew into Telemere. And I'm happy enough with the result that I feel confident in saying that there's nothing Timbre does better than Telemere, but plenty that Telemere does better than Timbre. Telemere is easier to use, faster, more robust, and significantly more flexible. It offers a better platform for what will be (I hope) the next many years of service.
I will **continue to maintain and support** Timbre for users that are happy with it, though I've also tried to make [migration](./5-Migrating#from-timbre) as easy as possible. I will **continue to maintain and support** Timbre for users that are happy with it, though I've also tried to make [migration](./5-Migrating#from-timbre) as easy as possible.