taoensso.nippy
High-performance JVM Clojure serialization library. Originally adapted from
Deep-Freeze.
*final-freeze-fallback*
dynamic
Alpha - subject to change.
->Compressable-LZMA2
(->Compressable-LZMA2 value)
Positional factory function for class taoensso.nippy.Compressable-LZMA2.
->StressRecord
(->StressRecord data)
Positional factory function for class taoensso.nippy.StressRecord.
aes128-encryptor
Default 128bit AES encryptor with multi-round SHA-512 key-gen.
Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
item encrypted will use a unique user-provided password).
IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
every key.
PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
encryption/decryption speed and the difficulty of attacking any
particular key.
Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.
Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
number of staff/admins, or you'll be using a single password to
encrypt many items).
IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.
PROS: Great amortized encryption/decryption speed. Expensive key hash makes
attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
key _is_ somehow compromised, _all_ items encrypted with that key are
compromised.
Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.default-freeze-compressor-selector
(default-freeze-compressor-selector ba)
Strategy:
* Prioritize speed, but allow lz4.
* Skip lz4 unless it's likely that lz4's space benefit will outweigh its
space overhead.
default-freeze-compressor-selector_
EXPERIMENTAL.
Determines the global default default compressor selector
(fn [^bytes ba])->compressor used by `(freeze <x> {:compressor :auto <...>}).extend-freeze
macro
(extend-freeze type custom-type-id [x out] & body)
Extends Nippy to support freezing of a custom type (ideally concrete) with
given id of form:
* Keyword - 2 byte overhead, resistent to id collisions.
* Byte ∈[1, 128] - no overhead, subject to id collisions.
(defrecord MyType [data])
(extend-freeze MyType :foo/my-type [x data-output] ; Keyword id
(.writeUTF [data-output] (:data x)))
;; or
(extend-freeze MyType 1 [x data-output] ; Byte id
(.writeUTF [data-output] (:data x)))
extend-thaw
macro
(extend-thaw custom-type-id [in] & body)
Extends Nippy to support thawing of a custom type with given id:
(extend-thaw :foo/my-type [data-input] ; Keyword id
(->MyType (.readUTF data-input)))
;; or
(extend-thaw 1 [data-input] ; Byte id
(->MyType (.readUTF data-input)))
Freezable
protocol
members
freeze-to-out*
(freeze-to-out* this out)
freezable?
(freezable? x & [{:keys [allow-clojure-reader? allow-java-serializable?]}])
Alpha - subject to change, MAY BE BUGGY!
Returns truthy value iff Nippy supports de/serialization of given argument.
Conservative with default options.
`:allow-clojure-reader?` and `:allow-java-serializable?` options may be used
to also enable the relevant roundtrip fallback test(s). These tests are only
**moderately reliable** since they're cached by arg type and don't test for
pre/post serialization equality (there's no good general way of doing so).
freeze
(freeze x & [{:keys [compressor encryptor password skip-header?], :or {compressor :auto, encryptor aes128-encryptor}, :as opts}])
Serializes arg (any Clojure data type) to a byte array. To freeze custom
types, extend the Clojure reader or see `extend-freeze`.
freeze-fallback-as-str
(freeze-fallback-as-str x out)
freeze-to-out!
(freeze-to-out! data-output x & _)
Low-level API. Serializes arg (any Clojure data type) to a DataOutput.
freeze-to-stream!
DEPRECATED: Use `freeze-to-out!` instead.
inspect-ba
(inspect-ba ba & [thaw-opts])
Alpha - subject to change.
lz4-compressor
Default net.jpountz.lz4 compressor:
Ratio: low.
Write speed: very high.
Read speed: very high.
A good general-purpose compressor, competitive with Snappy.
Thanks to Max Penet (@mpenet) for our first implementation,
Ref. https://github.com/mpenet/nippy-lz4lz4hc-compressor
Like `lz4-compressor` but trades some write speed for ratio.
lzma2-compressor
Default org.tukaani.xz.LZMA2 compressor:
Ratio: high.
Write speed: _very_ slow (also currently single-threaded).
Read speed: slow.
A specialized compressor for large, low-write data in space-sensitive
environments.map->Compressable-LZMA2
(map->Compressable-LZMA2 m__6249__auto__)
Factory function for class taoensso.nippy.Compressable-LZMA2, taking a map of keywords to field values.
map->StressRecord
(map->StressRecord m__6249__auto__)
Factory function for class taoensso.nippy.StressRecord, taking a map of keywords to field values.
read-biginteger
macro
(read-biginteger in)
read-bytes
macro
(read-bytes in & [small?])
read-compact-long
macro
(read-compact-long in)
read-utf8
macro
(read-utf8 in & [small?])
snappy-compressor
Default org.iq80.snappy.Snappy compressor:
Ratio: low.
Write speed: very high.
Read speed: very high.
A good general-purpose compressor.stress-data
Reference data used for tests & benchmarks.
stress-data-benchable
Reference data with stuff removed that breaks reader or other utils we'll
be benching against.
stress-data-comparable
Reference data with stuff removed that breaks roundtrip equality.
thaw
(thaw ba & [{:keys [compressor encryptor password v1-compatibility?], :or {compressor :auto, encryptor :auto, v1-compatibility? true}, :as opts}])
Deserializes a frozen object from given byte array to its original Clojure
data type. Supports data frozen with current and all previous versions of
Nippy. To thaw custom types, extend the Clojure reader or see `extend-thaw`.
Options include:
:compressor - An ICompressor, :auto (requires Nippy header), or nil.
:encryptor - An IEncryptor, :auto (requires Nippy header), or nil.
thaw-from-in!
(thaw-from-in! data-input & _)
Low-level API. Deserializes a frozen object from given DataInput to its
original Clojure data type.
thaw-from-stream!
DEPRECATED: Use `thaw-from-in!` instead.
when-debug-mode
macro
(when-debug-mode & body)
write-biginteger
macro
(write-biginteger out x)
write-bytes
macro
(write-bytes out ba & [small?])
write-compact-long
macro
(write-compact-long out x)
write-id
macro
(write-id out id)
write-utf8
macro
(write-utf8 out x & [small?])