luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
nippy/taoensso.nippy.html
Peter Taoussanis e012719c78 v3.4.0 (2024-04-30)
2024-04-30 11:20:23 +02:00

420 lines
No EOL
39 KiB
HTML
Raw Blame History

<!DOCTYPE html PUBLIC ""
"">
<html><head><meta charset="UTF-8" /><title>taoensso.nippy documentation</title><link rel="stylesheet" type="text/css" href="css/default.css" /><link rel="stylesheet" type="text/css" href="css/highlight.css" /><script type="text/javascript" src="js/highlight.min.js"></script><script type="text/javascript" src="js/jquery.min.js"></script><script type="text/javascript" src="js/page_effects.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body><div id="header"><h2>Generated by <a href="https://github.com/weavejester/codox">Codox</a></h2><h1><a href="index.html"><span class="project-title"><span class="project-name">Nippy</span> <span class="project-version">3.4.0</span></span></a></h1></div><div class="sidebar primary"><h3 class="no-link"><span class="inner">Project</span></h3><ul class="index-link"><li class="depth-1 "><a href="index.html"><div class="inner">Index</div></a></li></ul><h3 class="no-link"><span class="inner">Namespaces</span></h3><ul><li class="depth-1"><div class="no-link"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>taoensso</span></div></div></li><li class="depth-2 current"><a href="taoensso.nippy.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>nippy</span></div></a></li><li class="depth-3"><a href="taoensso.nippy.tools.html"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>tools</span></div></a></li></ul></div><div class="sidebar secondary"><h3><a href="#top"><span class="inner">Public Vars</span></a></h3><ul><li class="depth-1"><a href="taoensso.nippy.html#var-*auto-freeze-compressor*"><div class="inner"><span>*auto-freeze-compressor*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*custom-readers*"><div class="inner"><span>*custom-readers*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*freeze-fallback*"><div class="inner"><span>*freeze-fallback*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*freeze-serializable-allowlist*"><div class="inner"><span>*freeze-serializable-allowlist*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*incl-metadata.3F*"><div class="inner"><span>*incl-metadata?*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*thaw-serializable-allowlist*"><div class="inner"><span>*thaw-serializable-allowlist*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-*thaw-xform*"><div class="inner"><span>*thaw-xform*</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var--cache-proxy"><div class="inner"><span>-cache-proxy</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-aes128-cbc-encryptor"><div class="inner"><span>aes128-cbc-encryptor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-aes128-encryptor"><div class="inner"><span>aes128-encryptor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-aes128-gcm-encryptor"><div class="inner"><span>aes128-gcm-encryptor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-allow-and-record-any-serializable-class-unsafe"><div class="inner"><span>allow-and-record-any-serializable-class-unsafe</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-cache"><div class="inner"><span>cache</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-compress"><div class="inner"><span>compress</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-decompress"><div class="inner"><span>decompress</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-decrypt"><div class="inner"><span>decrypt</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-default-freeze-serializable-allowlist"><div class="inner"><span>default-freeze-serializable-allowlist</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-default-thaw-serializable-allowlist"><div class="inner"><span>default-thaw-serializable-allowlist</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-encrypt"><div class="inner"><span>encrypt</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-extend-freeze"><div class="inner"><span>extend-freeze</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-extend-thaw"><div class="inner"><span>extend-thaw</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-fast-freeze"><div class="inner"><span>fast-freeze</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-fast-thaw"><div class="inner"><span>fast-thaw</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-freezable.3F"><div class="inner"><span>freezable?</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-freeze"><div class="inner"><span>freeze</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-freeze-to-file"><div class="inner"><span>freeze-to-file</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-freeze-to-out.21"><div class="inner"><span>freeze-to-out!</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-freeze-to-string"><div class="inner"><span>freeze-to-string</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-get-recorded-serializable-classes"><div class="inner"><span>get-recorded-serializable-classes</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-IFreezable"><div class="inner"><span>IFreezable</span></div></a></li><li class="depth-2 branch"><a href="taoensso.nippy.html#var--freezable.3F"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>-freezable?</span></div></a></li><li class="depth-2"><a href="taoensso.nippy.html#var--freeze-without-meta.21"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>-freeze-without-meta!</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-IFreezableWithMeta"><div class="inner"><span>IFreezableWithMeta</span></div></a></li><li class="depth-2"><a href="taoensso.nippy.html#var--freeze-with-meta.21"><div class="inner"><span class="tree"><span class="top"></span><span class="bottom"></span></span><span>-freeze-with-meta!</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-inspect-ba"><div class="inner"><span>inspect-ba</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-lz4-compressor"><div class="inner"><span>lz4-compressor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-lzma2-compressor"><div class="inner"><span>lzma2-compressor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-public-types-spec"><div class="inner"><span>public-types-spec</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-read-quarantined-serializable-object-unsafe.21"><div class="inner"><span>read-quarantined-serializable-object-unsafe!</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-snappy-compressor"><div class="inner"><span>snappy-compressor</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-stress-data"><div class="inner"><span>stress-data</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-thaw"><div class="inner"><span>thaw</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-thaw-from-file"><div class="inner"><span>thaw-from-file</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-thaw-from-in.21"><div class="inner"><span>thaw-from-in!</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-thaw-from-resource"><div class="inner"><span>thaw-from-resource</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-thaw-from-string"><div class="inner"><span>thaw-from-string</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-throw-unfreezable"><div class="inner"><span>throw-unfreezable</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-try-write-readable"><div class="inner"><span>try-write-readable</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-try-write-serializable"><div class="inner"><span>try-write-serializable</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-write-id"><div class="inner"><span>write-id</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-write-unfreezable"><div class="inner"><span>write-unfreezable</span></div></a></li><li class="depth-1"><a href="taoensso.nippy.html#var-zstd-compressor"><div class="inner"><span>zstd-compressor</span></div></a></li></ul></div><div class="namespace-docs" id="content"><h1 class="anchor" id="top">taoensso.nippy</h1><div class="doc"><pre class="plaintext">High-performance serialization library for Clojure.
</pre></div><div class="public anchor" id="var-*auto-freeze-compressor*"><h3>*auto-freeze-compressor*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">(fn [byte-array])-&gt;compressor used by `(freeze &lt;x&gt; {:compressor :auto}),
nil =&gt; default</pre></div></div><div class="public anchor" id="var-*custom-readers*"><h3>*custom-readers*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">{&lt;hash-or-byte-id&gt; (fn [data-input])-&gt;read}
</pre></div></div><div class="public anchor" id="var-*freeze-fallback*"><h3>*freeze-fallback*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Controls Nippy's behaviour when trying to freeze an item for which Nippy
doesn't currently have a native freeze/thaw implementation.
Possible values:
1. `nil` (no freeze-fallback, default)
Tries the following in order:
- Freeze with Java's `Serializable` interface if possible
- Freeze with Clojure's reader if possible
- Throw
2. `:write-unfreezable` keyword
Tries the following in order:
- Freeze with Java's `Serializable` interface if possible
- Freeze with Clojure's reader if possible
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
3. [Advanced] Custom (fn [^java.io.DataOutput out item]) that must
write exactly one value to the given `DataOutput` stream</pre></div></div><div class="public anchor" id="var-*freeze-serializable-allowlist*"><h3>*freeze-serializable-allowlist*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Used when attempting to &lt;freeze/thaw&gt; an object that:
- Does NOT implement Nippy's `Freezable` protocol.
- DOES implement Java's `Serializable` interface.
In this case, an allowlist will be checked to see if Java's
`Serializable` interface may be used.
This is a security measure to prevent possible Remote Code Execution
(RCE) when thawing malicious payloads. See [1] for details.
If `freeze` encounters a disallowed `Serializable` class, it will throw.
If `thaw` encounters a disallowed `Serializable` class, it will:
- Throw if it's not possible to safely quarantine the object
(object was frozen with Nippy &lt; v2.15.0-final).
- Otherwise it will return a safely quarantined object of form
`{:nippy/unthawable {:class-name &lt;&gt; :content &lt;quarantined-ba&gt;}}`.
- Quarantined objects may be manually unquarantined with
`read-quarantined-serializable-object-unsafe!`.
There are 2x allowlists:
- `*freeze-serializable-allowlist*` ; Checked when freezing
- `*thaw-serializable-allowlist*` ; Checked when thawing
Example allowlist values:
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
- `#{"java.lang.Throwable", "clojure.lang.*"}` ; Set of class-names
- `"allow-and-record"` ; Special value, see [2]
Note that class-names in sets may contain "*" wildcards.
Default allowlist values are:
- default-freeze-serializable-allowlist ; `{"*"}` =&gt; allow any class
- default-thaw-serializable-allowlist ; A set of common safe classes
Allowlist values may be overridden with `binding`, `alter-var-root`, or:
- `taoensso.nippy.&lt;freeze/thaw&gt;-serializable-allowlist-base` JVM property value
- `taoensso.nippy.&lt;freeze/thaw&gt;-serializable-allowlist-add` JVM property value
- `TAOENSSO_NIPPY_&lt;FREEZE/THAW&gt;_SERIALIZABLE_ALLOWLIST_BASE` Environment variable value
- `TAOENSSO_NIPPY_&lt;FREEZE/THAW&gt;_SERIALIZABLE_ALLOWLIST_ADD` Environment variable value
If present, these will be read as comma-separated lists of class names
and formed into sets. Each initial allowlist value will then be:
(into (or &lt;?base&gt; &lt;default&gt;) &lt;?additions&gt;).
I.e. you can use:
- The "base" property/var to REPLACE Nippy's default allowlists.
- The "add" property/var to ADD TO Nippy's default allowlists.
The special `"allow-and-record"` value is also possible, see [2].
Upgrading from an older version of Nippy and unsure whether you've been
using Nippy's `Serializable` support, or which classes to allow? See [2].
See also `taoensso.encore/name-filter` for a util to help easily build
more advanced predicate functions.
Thanks to Timo Mihaljov (@solita-timo-mihaljov) for an excellent report
identifying this vulnerability.
[1] <a href="https://github.com/ptaoussanis/nippy/issues/130">https://github.com/ptaoussanis/nippy/issues/130</a>
[2] See `allow-and-record-any-serializable-class-unsafe`.</pre></div></div><div class="public anchor" id="var-*incl-metadata.3F*"><h3>*incl-metadata?*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Include metadata when freezing/thawing?
</pre></div></div><div class="public anchor" id="var-*thaw-serializable-allowlist*"><h3>*thaw-serializable-allowlist*</h3><h4 class="dynamic">dynamic</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Used when attempting to &lt;freeze/thaw&gt; an object that:
- Does NOT implement Nippy's `Freezable` protocol.
- DOES implement Java's `Serializable` interface.
In this case, an allowlist will be checked to see if Java's
`Serializable` interface may be used.
This is a security measure to prevent possible Remote Code Execution
(RCE) when thawing malicious payloads. See [1] for details.
If `freeze` encounters a disallowed `Serializable` class, it will throw.
If `thaw` encounters a disallowed `Serializable` class, it will:
- Throw if it's not possible to safely quarantine the object
(object was frozen with Nippy &lt; v2.15.0-final).
- Otherwise it will return a safely quarantined object of form
`{:nippy/unthawable {:class-name &lt;&gt; :content &lt;quarantined-ba&gt;}}`.
- Quarantined objects may be manually unquarantined with
`read-quarantined-serializable-object-unsafe!`.
There are 2x allowlists:
- `*freeze-serializable-allowlist*` ; Checked when freezing
- `*thaw-serializable-allowlist*` ; Checked when thawing
Example allowlist values:
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
- `#{"java.lang.Throwable", "clojure.lang.*"}` ; Set of class-names
- `"allow-and-record"` ; Special value, see [2]
Note that class-names in sets may contain "*" wildcards.
Default allowlist values are:
- default-freeze-serializable-allowlist ; `{"*"}` =&gt; allow any class
- default-thaw-serializable-allowlist ; A set of common safe classes
Allowlist values may be overridden with `binding`, `alter-var-root`, or:
- `taoensso.nippy.&lt;freeze/thaw&gt;-serializable-allowlist-base` JVM property value
- `taoensso.nippy.&lt;freeze/thaw&gt;-serializable-allowlist-add` JVM property value
- `TAOENSSO_NIPPY_&lt;FREEZE/THAW&gt;_SERIALIZABLE_ALLOWLIST_BASE` Environment variable value
- `TAOENSSO_NIPPY_&lt;FREEZE/THAW&gt;_SERIALIZABLE_ALLOWLIST_ADD` Environment variable value
If present, these will be read as comma-separated lists of class names
and formed into sets. Each initial allowlist value will then be:
(into (or &lt;?base&gt; &lt;default&gt;) &lt;?additions&gt;).
I.e. you can use:
- The "base" property/var to REPLACE Nippy's default allowlists.
- The "add" property/var to ADD TO Nippy's default allowlists.
The special `"allow-and-record"` value is also possible, see [2].
Upgrading from an older version of Nippy and unsure whether you've been
using Nippy's `Serializable` support, or which classes to allow? See [2].
See also `taoensso.encore/name-filter` for a util to help easily build
more advanced predicate functions.
Thanks to Timo Mihaljov (@solita-timo-mihaljov) for an excellent report
identifying this vulnerability.
[1] <a href="https://github.com/ptaoussanis/nippy/issues/130">https://github.com/ptaoussanis/nippy/issues/130</a>
[2] See `allow-and-record-any-serializable-class-unsafe`.</pre></div></div><div class="public anchor" id="var-*thaw-xform*"><h3>*thaw-xform*</h3><h4 class="dynamic">dynamic</h4><h4 class="added">added in v3.3.0-RC1 (2023-08-02)</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Experimental, subject to change. Feedback welcome!
Transducer to use when thawing standard Clojure collection types
(vectors, maps, sets, lists, etc.).
Allows fast+flexible inspection and manipulation of data being thawed.
Key-val style data structures like maps will provide `MapEntry` args
to reducing function. Use `map-entry?`, `key`, `val` utils for these.
Example transducers:
(map (fn [x] (println x) x)) ; Print each coll item thawed
(comp
(map (fn [x] (if (= x :secret) :redacted x))) ; Replace secrets
(remove (fn [x] ; Remove maps with a truthy :remove?
(or
(and (map? x) (:remove? x))
(and (map-entry? x) (= (key x) :remove?) (val y)))))))
Note that while this is a very powerful feature, correctly writing
and debugging transducers and reducing fns can be tricky.
To help, if Nippy encounters an error while applying your xform, it
will throw a detailed `ExceptionInfo` with message
"Error thrown via `*thaw-xform*`" to help you debug.</pre></div></div><div class="public anchor" id="var--cache-proxy"><h3>-cache-proxy</h3><div class="usage"></div><div class="doc"><pre class="plaintext">{[&lt;x&gt; &lt;meta&gt;] &lt;idx&gt;} for freezing, {&lt;idx&gt; &lt;x-with-meta&gt;} for thawing.
</pre></div></div><div class="public anchor" id="var-aes128-cbc-encryptor"><h3>aes128-cbc-encryptor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default 128bit AES-CBC encryptor with many-round SHA-512 key-gen.
See also `aes-128-cbc-encryptor`.</pre></div></div><div class="public anchor" id="var-aes128-encryptor"><h3>aes128-encryptor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.
Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
item encrypted will use a unique user-provided password).
IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
every key.
PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
encryption/decryption speed and the difficulty of attacking any
particular key.
Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.
Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
number of staff/admins, or you'll be using a single password to
encrypt many items).
IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.
PROS: Great amortized encryption/decryption speed. Expensive key hash makes
attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
key _is_ somehow compromised, _all_ items encrypted with that key are
compromised.
Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.</pre></div></div><div class="public anchor" id="var-aes128-gcm-encryptor"><h3>aes128-gcm-encryptor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default 128bit AES-GCM encryptor with many-round SHA-512 key-gen.
Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
item encrypted will use a unique user-provided password).
IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
every key.
PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
encryption/decryption speed and the difficulty of attacking any
particular key.
Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.
Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
number of staff/admins, or you'll be using a single password to
encrypt many items).
IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.
PROS: Great amortized encryption/decryption speed. Expensive key hash makes
attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
key _is_ somehow compromised, _all_ items encrypted with that key are
compromised.
Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.</pre></div></div><div class="public anchor" id="var-allow-and-record-any-serializable-class-unsafe"><h3>allow-and-record-any-serializable-class-unsafe</h3><div class="usage"><code>(allow-and-record-any-serializable-class-unsafe class-name)</code></div><div class="doc"><pre class="plaintext">A predicate (fn allow-class? [class-name]) fn that can be assigned
to `*freeze-serializable-allowlist*` and/or
`*thaw-serializable-allowlist*` that:
- Will allow ANY class to use Nippy's `Serializable` support (unsafe).
- And will record {&lt;class-name&gt; &lt;frequency-allowed&gt;} for the &lt;=1000
classes that ~most frequently made use of this support.
`get-recorded-serializable-classes` returns the recorded state.
This predicate is provided as a convenience for users upgrading from
previous versions of Nippy that allowed the use of `Serializable` for all
classes by default.
While transitioning from an unsafe-&gt;safe configuration, you can use
this predicate (unsafe) to record information about which classes have
been using Nippy's `Serializable` support in your environment.
Once some time has passed, you can check the recorded state. If you're
satisfied that all recorded classes are safely `Serializable`, you can
then merge the recorded classes into Nippy's default allowlist/s, e.g.:
(alter-var-root #'thaw-serializable-allowlist*
(fn [_] (into default-thaw-serializable-allowlist
(keys (get-recorded-serializable-classes)))))</pre></div></div><div class="public anchor" id="var-cache"><h3>cache</h3><div class="usage"><code>(cache x)</code></div><div class="doc"><pre class="plaintext">Experimental, subject to change. Feedback welcome!
Wraps value so that future writes of the same wrapped value with same
metadata will be efficiently encoded as references to this one.
(freeze [(cache "foo") (cache "foo") (cache "foo")])
will incl. a single "foo", plus 2x single-byte references to "foo".</pre></div></div><div class="public anchor" id="var-compress"><h3>compress</h3><div class="usage"><code>(compress compressor ba)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-decompress"><h3>decompress</h3><div class="usage"><code>(decompress compressor ba)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-decrypt"><h3>decrypt</h3><div class="usage"><code>(decrypt encryptor pwd ba)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-default-freeze-serializable-allowlist"><h3>default-freeze-serializable-allowlist</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Allows *any* class-name to be frozen using Java's `Serializable` interface.
This is generally safe since RCE risk is present only when thawing.
See also `*freeze-serializable-allowlist*`.</pre></div></div><div class="public anchor" id="var-default-thaw-serializable-allowlist"><h3>default-thaw-serializable-allowlist</h3><div class="usage"></div><div class="doc"><pre class="plaintext">A set of common safe class-names to allow to be frozen using Java's
`Serializable` interface. PRs welcome for additions.
See also `*thaw-serializable-allowlist*`.</pre></div></div><div class="public anchor" id="var-encrypt"><h3>encrypt</h3><div class="usage"><code>(encrypt encryptor pwd ba)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-extend-freeze"><h3>extend-freeze</h3><h4 class="type">macro</h4><div class="usage"><code>(extend-freeze type custom-type-id [x out] &amp; body)</code></div><div class="doc"><pre class="plaintext">Extends Nippy to support freezing of a custom type (ideally concrete) with
given id of form:
* Keyword - 2 byte overhead, keywords hashed to 16 bit id
* ℕ∈[1, 128] - 0 byte overhead
NB: be careful about extending to interfaces, Ref. &lt;<a href="http://goo.gl/6gGRlU">http://goo.gl/6gGRlU</a>&gt;.
(defrecord MyRec [data])
(extend-freeze MyRec :foo/my-type [x data-output] ; Keyword id
(.writeUTF [data-output] (:data x)))
;; or
(extend-freeze MyRec 1 [x data-output] ; Byte id
(.writeUTF [data-output] (:data x)))</pre></div></div><div class="public anchor" id="var-extend-thaw"><h3>extend-thaw</h3><h4 class="type">macro</h4><div class="usage"><code>(extend-thaw custom-type-id [in] &amp; body)</code></div><div class="doc"><pre class="plaintext">Extends Nippy to support thawing of a custom type with given id:
(extend-thaw :foo/my-type [data-input] ; Keyword id
(MyRec. (.readUTF data-input)))
;; or
(extend-thaw 1 [data-input] ; Byte id
(MyRec. (.readUTF data-input)))</pre></div></div><div class="public anchor" id="var-fast-freeze"><h3>fast-freeze</h3><div class="usage"><code>(fast-freeze x)</code></div><div class="doc"><pre class="plaintext">Like `freeze` but:
- Writes data without a Nippy header
- Drops all support for compression and encryption
- Must be thawed with `fast-thaw`
Equivalent to (but a little faster than) `freeze` with opts:
- :compressor nil
- :encryptor nil
- :no-header? true</pre></div></div><div class="public anchor" id="var-fast-thaw"><h3>fast-thaw</h3><div class="usage"><code>(fast-thaw ba)</code></div><div class="doc"><pre class="plaintext">Like `thaw` but:
- Drops all support for compression and encryption
- Supports only data frozen with `fast-freeze`
Equivalent to (but a little faster than) `thaw` with opts:
- :compressor nil
- :encryptor nil
- :no-header? true</pre></div></div><div class="public anchor" id="var-freezable.3F"><h3>freezable?</h3><div class="usage"><code>(freezable? x)</code><code>(freezable? x {:as opts, :keys [recursive? allow-clojure-reader? allow-java-serializable?], :or {recursive? true}})</code></div><div class="doc"><pre class="plaintext">Alpha, subject to change.
Returns ∈ #{:native :maybe-clojure-reader :maybe-java-serializable nil},
truthy iff Nippy seems to support freezing for the given argument.
Important: result can be inaccurate in some cases. To be completely sure you
unfortunately need to try freeze then thaw the argument, and check the thawed
value.
Options include:
`recursive?` - Check recursively into given arg?
`allow-clojure-reader?` - Allow freezing with Clojure's reader?
`allow-java-serializable?` - Allow freezing with Java's `Serializable`?</pre></div></div><div class="public anchor" id="var-freeze"><h3>freeze</h3><div class="usage"><code>(freeze x)</code><code>(freeze x {:as opts, :keys [compressor encryptor password serializable-allowlist incl-metadata?], :or {compressor :auto, encryptor aes128-gcm-encryptor}})</code></div><div class="doc"><pre class="plaintext">Serializes arg (any Clojure data type) to a byte array.
To freeze custom types, extend the Clojure reader or see `extend-freeze`.</pre></div></div><div class="public anchor" id="var-freeze-to-file"><h3>freeze-to-file</h3><div class="usage"><code>(freeze-to-file file x)</code><code>(freeze-to-file file x freeze-opts)</code></div><div class="doc"><pre class="plaintext">Convenience util: like `freeze`, but writes to `(clojure.java.io/file &lt;file&gt;)`.
</pre></div></div><div class="public anchor" id="var-freeze-to-out.21"><h3>freeze-to-out!</h3><div class="usage"><code>(freeze-to-out! data-output x)</code></div><div class="doc"><pre class="plaintext">Serializes arg (any Clojure data type) to a DataOutput.
This is a low-level util: in most cases you'll want `freeze` instead.</pre></div></div><div class="public anchor" id="var-freeze-to-string"><h3>freeze-to-string</h3><div class="usage"><code>(freeze-to-string x)</code><code>(freeze-to-string x freeze-opts)</code></div><div class="doc"><pre class="plaintext">Convenience util: like `freeze`, but returns a Base64-encoded string.
See also `thaw-from-string`.</pre></div></div><div class="public anchor" id="var-get-recorded-serializable-classes"><h3>get-recorded-serializable-classes</h3><div class="usage"><code>(get-recorded-serializable-classes)</code></div><div class="doc"><pre class="plaintext">Returns {&lt;class-name&gt; &lt;frequency&gt;} of the &lt;=1000 classes that ~most
frequently made use of Nippy's `Serializable` support via
`allow-and-record-any-serializable-class-unsafe`.
See that function's docstring for more info.</pre></div></div><div class="public anchor" id="var-IFreezable"><h3>IFreezable</h3><h4 class="type">protocol</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Private implementation detail.
Protocol that types must implement to support native freezing by Nippy.
Don't use this directly, instead see `extend-freeze`.</pre></div><div class="members"><h4>members</h4><div class="inner"><div class="public anchor" id="var--freezable.3F"><h3>-freezable?</h3><div class="usage"><code>(-freezable? _)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var--freeze-without-meta.21"><h3>-freeze-without-meta!</h3><div class="usage"><code>(-freeze-without-meta! _ data-output)</code></div><div class="doc"><pre class="plaintext"></pre></div></div></div></div></div><div class="public anchor" id="var-IFreezableWithMeta"><h3>IFreezableWithMeta</h3><h4 class="type">protocol</h4><div class="usage"></div><div class="doc"><pre class="plaintext">Private implementation detail.
Wrapper protocol around `IFreezable` used to handle possible metadata.</pre></div><div class="members"><h4>members</h4><div class="inner"><div class="public anchor" id="var--freeze-with-meta.21"><h3>-freeze-with-meta!</h3><div class="usage"><code>(-freeze-with-meta! _ data-output)</code></div><div class="doc"><pre class="plaintext"></pre></div></div></div></div></div><div class="public anchor" id="var-inspect-ba"><h3>inspect-ba</h3><div class="usage"><code>(inspect-ba ba)</code><code>(inspect-ba ba thaw-opts)</code></div><div class="doc"><pre class="plaintext">Experimental, subject to change. Feedback welcome!
</pre></div></div><div class="public anchor" id="var-lz4-compressor"><h3>lz4-compressor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default `LZ4` compressor:
- Compression ratio: `C` (0.58 on reference benchmark).
- Compression speed: `A` (238 msecs on reference benchmark).
- Decompression speed: `A+` (31 msecs on reference benchmark).
Good general-purpose compressor, favours speed.
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks.</pre></div></div><div class="public anchor" id="var-lzma2-compressor"><h3>lzma2-compressor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default `LZMA2` compressor:
- Compression ratio: `A+` (0.4 on reference benchmark).
- Compression speed: `E` (18.5 secs on reference benchmark).
- Decompression speed: `D` (11.8 secs on reference benchmark).
Specialized compressor, strongly favours ratio.
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks.</pre></div></div><div class="public anchor" id="var-public-types-spec"><h3>public-types-spec</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Public representation of Nippy's internal type schema.
For use by tooling and advanced users.
**HIGHLY EXPERIMENTAL!**
Subject to breaking change without notice.
Currently completely untested, may contain bugs.
Intended for use only by early adopters to give design feedback.
Format:
{&lt;type-id&gt; {:keys [type-id type-kw payload-spec deprecated?]}},
- `type-id`: A +ive single-byte identifier like `110`.
-ive type ids are reserved for custom user-defined types.
- `type-kw`: A keyword like `:kw-sm`,
suffixes used to differentiate subtypes of different sizes:
-0 ; Empty =&gt; 0 byte payload / element-count
-sm ; Small =&gt; 1 byte (byte) payload / element-count
-md ; Medium =&gt; 2 byte (short) payload / element-count
-lg ; Large =&gt; 4 byte (int) payload / element-count
-xl ; Extra large =&gt; 8 byte (long) payload / element-count
- `payload-spec` examples:
- nil ; No spec available (e.g. unpredictable payload)
- [] ; Type has no payload
- [[:bytes 4]] ; Type has a payload of exactly 4 bytes
- [[:bytes 2] [:elements 2]] ; Type has a payload of exactly 2 bytes, then
; 2 elements
- [[:bytes {:read 2}]
[:elements {:read 4 :multiplier 2 :unsigned? true}]]
; Type has payload of &lt;short-count&gt; bytes, then
; &lt;unsigned-int-count&gt;*2 (multiplier) elements
Note that `payload-spec` can be handy for skipping over items in
data stream without fully reading every item.</pre></div></div><div class="public anchor" id="var-read-quarantined-serializable-object-unsafe.21"><h3>read-quarantined-serializable-object-unsafe!</h3><div class="usage"><code>(read-quarantined-serializable-object-unsafe! m)</code></div><div class="doc"><pre class="plaintext">Given a quarantined Serializable object like
{:nippy/unthawable {:class-name &lt;&gt; :content &lt;quarantined-ba&gt;}}, reads and
returns the object WITHOUT regard for `*thaw-serializable-allowlist*`.
**MAY BE UNSAFE!** Don't call this unless you absolutely trust the payload
to not contain any malicious code.
See `*thaw-serializable-allowlist*` for more info.</pre></div></div><div class="public anchor" id="var-snappy-compressor"><h3>snappy-compressor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default `Snappy` compressor:
- Compression ratio: `C` (0.58 on reference benchmark).
- Compression speed: `A+` (206 msecs on reference benchmark).
- Decompression speed: `B` (134 msecs on reference benchmark).
Good general-purpose compressor, favours speed.
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks.</pre></div></div><div class="public anchor" id="var-stress-data"><h3>stress-data</h3><div class="usage"><code>(stress-data {:keys [comparable?], :as opts})</code></div><div class="doc"><pre class="plaintext">Returns map of reference stress data for use by tests, benchmarks, etc.
</pre></div></div><div class="public anchor" id="var-thaw"><h3>thaw</h3><div class="usage"><code>(thaw ba)</code><code>(thaw ba {:as opts, :keys [v1-compatibility? compressor encryptor password serializable-allowlist incl-metadata? thaw-xform], :or {compressor :auto, encryptor :auto}})</code></div><div class="doc"><pre class="plaintext">Deserializes a frozen Nippy byte array to its original Clojure data type.
To thaw custom types, extend the Clojure reader or see `extend-thaw`.
** By default, supports data frozen with Nippy v2+ ONLY **
Add `{:v1-compatibility? true}` option to support thawing of data frozen with
legacy versions of Nippy.
Options include:
:v1-compatibility? - support data frozen by legacy versions of Nippy?
:compressor - :auto (checks header, default) an ICompressor, or nil
:encryptor - :auto (checks header, default), an IEncryptor, or nil</pre></div></div><div class="public anchor" id="var-thaw-from-file"><h3>thaw-from-file</h3><div class="usage"><code>(thaw-from-file file)</code><code>(thaw-from-file file thaw-opts)</code></div><div class="doc"><pre class="plaintext">Convenience util: like `thaw`, but reads from `(clojure.java.io/file &lt;file&gt;)`.
</pre></div></div><div class="public anchor" id="var-thaw-from-in.21"><h3>thaw-from-in!</h3><div class="usage"><code>(thaw-from-in! data-input)</code></div><div class="doc"><pre class="plaintext">Deserializes a frozen object from given DataInput to its original Clojure
data type.
This is a low-level util: in most cases you'll want `thaw` instead.</pre></div></div><div class="public anchor" id="var-thaw-from-resource"><h3>thaw-from-resource</h3><div class="usage"><code>(thaw-from-resource res)</code><code>(thaw-from-resource res thaw-opts)</code></div><div class="doc"><pre class="plaintext">Convenience util: like `thaw`, but reads from `(clojure.java.io/resource &lt;res&gt;)`.
</pre></div></div><div class="public anchor" id="var-thaw-from-string"><h3>thaw-from-string</h3><div class="usage"><code>(thaw-from-string s)</code><code>(thaw-from-string s thaw-opts)</code></div><div class="doc"><pre class="plaintext">Convenience util: like `thaw`, but takes a Base64-encoded string.
See also `freeze-to-string`.</pre></div></div><div class="public anchor" id="var-throw-unfreezable"><h3>throw-unfreezable</h3><div class="usage"><code>(throw-unfreezable x)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-try-write-readable"><h3>try-write-readable</h3><div class="usage"><code>(try-write-readable out x)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-try-write-serializable"><h3>try-write-serializable</h3><div class="usage"><code>(try-write-serializable out x)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-write-id"><h3>write-id</h3><h4 class="type">macro</h4><div class="usage"><code>(write-id out id)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-write-unfreezable"><h3>write-unfreezable</h3><div class="usage"><code>(write-unfreezable out x)</code></div><div class="doc"><pre class="plaintext"></pre></div></div><div class="public anchor" id="var-zstd-compressor"><h3>zstd-compressor</h3><div class="usage"></div><div class="doc"><pre class="plaintext">Default `Zstd` (`Zstandard`) compressor:
- Compression ratio: `B` (0.53 on reference benchmark).
- Compression speed: `C` (1300 msecs on reference benchmark).
- Decompression speed: `B` (400 msecs on reference benchmark).
Good general-purpose compressor, balances ratio &amp; speed.
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks.</pre></div></div></div></body></html>
Reference in a new issue View git blame Copy permalink