\ No newline at end of file
diff --git a/taoensso.nippy.benchmarks.html b/taoensso.nippy.benchmarks.html
index ca73328..e6bce04 100644
--- a/taoensso.nippy.benchmarks.html
+++ b/taoensso.nippy.benchmarks.html
@@ -1,3 +1,5 @@
-taoensso.nippy.benchmarks documentation
Map of data suitable for benching, a subset of
+`nippy/stress-data-comparable`.
\ No newline at end of file
diff --git a/taoensso.nippy.html b/taoensso.nippy.html
index be5be17..67fe128 100644
--- a/taoensso.nippy.html
+++ b/taoensso.nippy.html
@@ -1,9 +1,9 @@
-taoensso.nippy documentation
High-performance serialization library for Clojure
*auto-freeze-compressor*
dynamic
(fn [byte-array])->compressor used by `(freeze <x> {:compressor :auto}),
nil => default
*custom-readers*
dynamic
{<hash-or-byte-id> (fn [data-input])->read}
-
*final-freeze-fallback*
dynamic
DEPRECATED: prefer `*freeze-fallback`.
+
*final-freeze-fallback*
dynamic
deprecated
DEPRECATED: prefer `*freeze-fallback`.
*freeze-fallback*
dynamic
(fn [data-output x])->freeze, nil => default
*freeze-serializable-allowlist*
dynamic
Used when attempting to <freeze/thaw> an object that:
- Does NOT implement Nippy's Freezable protocol.
@@ -70,7 +70,7 @@ identifying this vulnerability.
[1] https://github.com/ptaoussanis/nippy/issues/130
[2] See `allow-and-record-any-serializable-class-unsafe`.
*incl-metadata?*
dynamic
Include metadata when freezing/thawing?
-
*serializable-whitelist*
dynamic
DEPRECATED, now called `*thaw-serializable-allowlist*`
+
*serializable-whitelist*
dynamic
deprecated
DEPRECATED, now called `*thaw-serializable-allowlist*`
*thaw-serializable-allowlist*
dynamic
Used when attempting to <freeze/thaw> an object that:
- Does NOT implement Nippy's Freezable protocol.
- DOES implement Java's Serializable interface.
@@ -135,7 +135,33 @@ Thanks to Timo Mihaljov (@solita-timo-mihaljov) for an excellent report
identifying this vulnerability.
[1] https://github.com/ptaoussanis/nippy/issues/130
-[2] See `allow-and-record-any-serializable-class-unsafe`.
-cache-proxy
{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing.
+[2] See `allow-and-record-any-serializable-class-unsafe`.
*thaw-xform*
dynamic
Experimental, subject to change. Feedback welcome.
+
+Transducer to use when thawing standard Clojure collection types
+(vectors, maps, sets, lists, etc.).
+
+Allows fast+flexible inspection and manipulation of data being thawed.
+
+Key-val style data structures like maps will provide `MapEntry` args
+to reducing function. Use `map-entry?`, `key`, `val` utils for these.
+
+Example transducers:
+
+ (map (fn [x] (println x) x)) ; Print each coll item thawed
+
+ (comp
+ (map (fn [x] (if (= x :secret) :redacted x))) ; Replace secrets
+ (remove (fn [x] ; Remove maps with a truthy :remove?
+ (or
+ (and (map? x) (:remove? x))
+ (and (map-entry? x) (= (key x) :remove?) (val y)))))))
+
+Note that while this is a very powerful feature, correctly writing
+and debugging transducers and reducing fns can be tricky.
+
+To help, if Nippy encounters an errors while applying your xform, it
+will throw a detailed `ExceptionInfo` with message
+"Error thrown via `*thaw-xform*`" to help you debug.
-cache-proxy
{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing.
aes128-cbc-encryptor
Default 128bit AES-CBC encryptor with many-round SHA-512 key-gen.
See also `aes-128-cbc-encryptor`.
aes128-encryptor
Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.
@@ -227,7 +253,7 @@ then merge the recorded classes into Nippy's default allowlist/s, e.g.:
(alter-var-root #'thaw-serializable-allowlist*
(fn [_] (into default-thaw-serializable-allowlist
- (keys (get-recorded-serializable-classes)))))
cache
(cache x)
Experimental, subject to change.
+ (keys (get-recorded-serializable-classes)))))
cache
(cache x)
Experimental, subject to change. Feedback welcome.
Wraps value so that future writes of the same wrapped value with same
metadata will be efficiently encoded as references to this one.
@@ -277,15 +303,15 @@ Returns truthy iff Nippy *appears* to support freezing the given argument.
used to enable the relevant roundtrip fallback test(s). These tests are
only **moderately reliable** since they're cached by arg type and don't
test for pre/post serialization value equality (there's no good general
-way of doing so).
Serializes arg (any Clojure data type) to a byte array.
+To freeze custom types, extend the Clojure reader or see `extend-freeze`.
freeze-to-file
(freeze-to-file file x)(freeze-to-file file x freeze-opts)
Convenience util: like `freeze`, but writes to `(clojure.java.io/file <file>)`.
freeze-to-out!
(freeze-to-out! data-output x)
Serializes arg (any Clojure data type) to a DataOutput.
This is a low-level util: in most cases you'll want `freeze` instead.
freeze-to-string
(freeze-to-string x)(freeze-to-string x freeze-opts)
Convenience util: like `freeze`, but returns a Base64-encoded string.
See also `thaw-from-string`.
get-recorded-serializable-classes
(get-recorded-serializable-classes)
Returns {<class-name> <frequency>} of the <=1000 classes that ~most
frequently made use of Nippy's Serializable support via
`allow-and-record-any-serializable-class-unsafe`.
-See that function's docstring for more info.
IFreezable1
protocol
members
-freeze-without-meta!
(-freeze-without-meta! x data-output)
IFreezable2
protocol
members
-freeze-with-meta!
(-freeze-with-meta! x data-output)
inspect-ba
(inspect-ba ba)(inspect-ba ba thaw-opts)
Alpha - subject to change
+See that function's docstring for more info.
IFreezable1
protocol
members
-freeze-without-meta!
(-freeze-without-meta! x data-output)
IFreezable2
protocol
members
-freeze-with-meta!
(-freeze-with-meta! x data-output)
inspect-ba
(inspect-ba ba)(inspect-ba ba thaw-opts)
Experimental, subject to change. Feedback welcome.
lz4-compressor
Default net.jpountz.lz4 compressor:
Ratio: low.
Write speed: very high.
@@ -301,7 +327,43 @@ Write speed: _very_ slow (also currently single-threaded).
Read speed: slow.
A specialized compressor for large, low-write data in space-sensitive
-environments.
read-quarantined-serializable-object-unsafe!
(read-quarantined-serializable-object-unsafe! m)
Given a quarantined Serializable object like
+environments.
public-types-spec
Public representation of Nippy's internal type schema.
+For use by tooling and advanced users.
+
+**HIGHLY EXPERIMENTAL!**
+Subject to breaking change without notice.
+Currently completely untested, may contain bugs.
+Intended for use only by early adopters to give design feedback.
+
+Format:
+ {<type-id> {:keys [type-id type-kw payload-spec deprecated?]}},
+
+ - `type-id`: A +ive single-byte identifier like `110`.
+ -ive type ids are reserved for custom user-defined types.
+
+ - `type-kw`: A keyword like `:kw-sm`,
+ suffixes used to differentiate subtypes of different sizes:
+ -0 ; Empty => 0 byte payload / element-count
+ -sm ; Small => 1 byte (byte) payload / element-count
+ -md ; Medium => 2 byte (short) payload / element-count
+ -lg ; Large => 4 byte (int) payload / element-count
+ -xl ; Extra large => 8 byte (long) payload / element-count
+
+ - `payload-spec` examples:
+ - nil ; No spec available (e.g. unpredictable payload)
+ - [] ; Type has no payload
+ - [[:bytes 4]] ; Type has a payload of exactly 4 bytes
+ - [[:bytes 2] [:elements 2]] ; Type has a payload of exactly 2 bytes, then
+ ; 2 elements
+
+ - [[:bytes {:read 2}]
+ [:elements {:read 4 :multiplier 2 :unsigned? true}]]
+
+ ; Type has payload of <short-count> bytes, then
+ ; <unsigned-int-count>*2 (multiplier) elements
+
+ Note that `payload-spec` can be handy for skipping over items in
+ data stream without fully reading every item.
read-quarantined-serializable-object-unsafe!
(read-quarantined-serializable-object-unsafe! m)
Given a quarantined Serializable object like
{:nippy/unthawable {:class-name <> :content <quarantined-ba>}}, reads and
returns the object WITHOUT regard for `*thaw-serializable-allowlist*`.
@@ -315,8 +377,8 @@ Write speed: very high.
A good general-purpose compressor.
stress-data
Reference data used for tests & benchmarks
stress-data-benchable
Reference data with stuff removed that breaks reader or other utils we'll
-be benching against
stress-data-comparable
Reference data with stuff removed that breaks roundtrip equality
-
Deserializes a frozen Nippy byte array to its original Clojure data type.
To thaw custom types, extend the Clojure reader or see `extend-thaw`.
** By default, supports data frozen with Nippy v2+ ONLY **
diff --git a/taoensso.nippy.tools.html b/taoensso.nippy.tools.html
index d452a4f..52788e7 100644
--- a/taoensso.nippy.tools.html
+++ b/taoensso.nippy.tools.html
@@ -1,17 +1,19 @@
-taoensso.nippy.tools documentation
Utils for community tools that want to add user-configurable Nippy support.
+Used by Carmine, Faraday, etc.
*freeze-opts*
dynamic
*thaw-opts*
dynamic
freeze
(freeze x)(freeze x default-opts)
Like `nippy/freeze` but uses as options the following, merged in
+order of ascending preference:
- - Optional `default-opts` arg given to this fn (default nil).
- - Optional `*freeze-opts*` dynamic value (default nil).
- - Optional opts provided to `wrap-for-freezing` (default nil).
thaw
(thaw ba)(thaw ba default-opts)
Like `nippy/thaw` but uses as opts the following merged in order of
-ascending preference:
+ 1. `default-opts` given to this fn (default nil).
+ 2. `tools/*freeze-opts*` dynamic value (default nil).
+ 3. Opts captured by `tools/wrap-for-freezing` (default nil).
- - Optional `default-opts` arg given to this fn (default nil).
- - Optional `*thaw-opts*` dynamic value (default nil).
with-freeze-opts
macro
(with-freeze-opts opts & body)
with-thaw-opts
macro
(with-thaw-opts opts & body)
wrap-for-freezing
(wrap-for-freezing x)(wrap-for-freezing x opts)
Ensures that given arg (any freezable data type) is wrapped so that
-(tools/freeze <wrapped-arg>) will serialize as
-(nippy/freeze <unwrapped-arg> <opts>).
+See also `tools/wrap-for-freezing`.
thaw
(thaw ba)(thaw ba default-opts)
Like `nippy/thaw` but uses as options the following, merged in
+order of ascending preference:
-See also `nippy.tools/freeze`, `nippy.tools/thaw`.
wrapped-for-freezing?
(wrapped-for-freezing? x)
\ No newline at end of file
+ 1. `default-opts` given to this fn (default nil).
+ 2. `tools/*thaw-opts*` dynamic value (default nil).
with-freeze-opts
macro
(with-freeze-opts opts & body)
with-freeze-opts+
macro
(with-freeze-opts+ opts & body)
with-thaw-opts
macro
(with-thaw-opts opts & body)
with-thaw-opts+
macro
(with-thaw-opts+ opts & body)
wrap-for-freezing
(wrap-for-freezing x)(wrap-for-freezing x wrap-opts)
Captures (merge `tools/*thaw-opts*` `wrap-opts`), and returns
+the given argument in a wrapped form so that `tools/freeze` will
+use the captured options when freezing the wrapper argument.
+
+See also `tools/freeze`.