taoensso.nippy

High-performance serialization library for Clojure.
Originally adapted from Deep-Freeze (Ref. https://goo.gl/OePPGr).

*custom-readers*

dynamic

{<hash-or-byte-id> (fn [data-input])}

*default-freeze-compressor-selector*

dynamic

(fn selector [^bytes ba])->compressor used by `(freeze <x> {:compressor :auto}).

*final-freeze-fallback*

dynamic

->Compressable-LZMA2

(->Compressable-LZMA2 value)
Positional factory function for class taoensso.nippy.Compressable-LZMA2.

->StressRecord

(->StressRecord data)
Positional factory function for class taoensso.nippy.StressRecord.

aes128-encryptor

Default 128bit AES encryptor with many-round SHA-512 key-gen.

Password form [:salted "my-password"]
---------------------------------------
USE CASE: You want more than a small, finite number of passwords (e.g. each
          item encrypted will use a unique user-provided password).

IMPLEMENTATION: Uses a relatively cheap key hash, but automatically salts
                every key.

PROS: Each key is independent so would need to be attacked independently.
CONS: Key caching impossible, so there's an inherent trade-off between
      encryption/decryption speed and the difficulty of attacking any
      particular key.

Slower than `aes128-cached`, and easier to attack any particular key - but
keys are independent.

Password form [:cached "my-password"]
---------------------------------------
USE CASE: You want only a small, finite number of passwords (e.g. a limited
          number of staff/admins, or you'll be using a single password to
          encrypt many items).

IMPLEMENTATION: Uses a _very_ expensive (but cached) key hash, and no salt.

PROS: Great amortized encryption/decryption speed. Expensive key hash makes
      attacking any particular key very difficult.
CONS: Using a small number of keys for many encrypted items means that if any
      key _is_ somehow compromised, _all_ items encrypted with that key are
      compromised.

Faster than `aes128-salted`, and harder to attack any particular key - but
increased danger if a key is somehow compromised.

byte-sized?

(byte-sized? n)

compress

(compress compressor ba)

decompress

(decompress compressor ba)

decrypt

(decrypt encryptor pwd ba)

default-freeze-compressor-selector

(default-freeze-compressor-selector ba)
Strategy:
* Prioritize speed, but allow lz4.
* Skip lz4 unless it's likely that lz4's space benefit will outweigh its
  space overhead.

encrypt

(encrypt encryptor pwd ba)

extend-freeze

macro

(extend-freeze type custom-type-id [x out] & body)
Extends Nippy to support freezing of a custom type (ideally concrete) with
given id of form:
  * Keyword           - 2 byte overhead, resistent to id collisions
  * Integer ∈[1, 128] - no overhead, subject to id collisions

(defrecord MyType [data])
(extend-freeze MyType :foo/my-type [x data-output] ; Keyword id
  (.writeUTF [data-output] (:data x)))
;; or
(extend-freeze MyType 1 [x data-output] ; Byte id
  (.writeUTF [data-output] (:data x)))

extend-thaw

macro

(extend-thaw custom-type-id [in] & body)
Extends Nippy to support thawing of a custom type with given id:
(extend-thaw :foo/my-type [data-input] ; Keyword id
  (->MyType (.readUTF data-input)))
;; or
(extend-thaw 1 [data-input] ; Byte id
  (->MyType (.readUTF data-input)))

Freezable

protocol

Implementation detail. Be careful about extending to interfaces,
Ref. http://goo.gl/6gGRlU.

members

-freeze-to-out

(-freeze-to-out this out)

freezable?

(freezable? x & [{:keys [allow-clojure-reader? allow-java-serializable?]}])
Alpha - subject to change, MAY BE BUGGY!
Returns truthy value iff Nippy supports de/serialization of given argument.
Conservative with default options.

`:allow-clojure-reader?` and `:allow-java-serializable?` options may be used
to also enable the relevant roundtrip fallback test(s). These tests are only
**moderately reliable** since they're cached by arg type and don't test for
pre/post serialization equality (there's no good general way of doing so).

freeze

(freeze x)(freeze x {:keys [compressor encryptor password], :or {compressor :auto, encryptor aes128-encryptor}, :as opts})
Serializes arg (any Clojure data type) to a byte array. To freeze custom
types, extend the Clojure reader or see `extend-freeze`.

freeze-fallback-as-str

(freeze-fallback-as-str out x)

freeze-to-out!

(freeze-to-out! data-output x)
Serializes arg (any Clojure data type) to a DataOutput

id-bigdec

id-bigint

id-biginteger

id-boolean

id-byte

id-byte-as-long

id-bytes

id-char

id-date

id-double

id-float

id-int-as-long

id-integer

id-keyword

id-keyword-depr1

id-list

id-long

id-map

id-map-depr1

id-map-depr2

id-meta

id-nil

id-prefixed-custom

id-queue

id-ratio

id-reader

id-reader-depr1

id-record

id-reserved

id-seq

id-serializable

id-set

id-short

id-short-as-long

id-sm-bytes

id-sm-keyword

id-sm-map

id-sm-set

id-sm-string

id-sm-vec

id-sorted-map

id-sorted-map-depr1

id-sorted-set

id-string

id-string-depr1

id-uuid

id-vec

inspect-ba

(inspect-ba ba)(inspect-ba ba thaw-opts)
Alpha - subject to change

lz4-compressor

Default net.jpountz.lz4 compressor:
      Ratio: low.
Write speed: very high.
 Read speed: very high.

A good general-purpose compressor, competitive with Snappy.

Thanks to Max Penet (@mpenet) for our first implementation,
Ref. https://github.com/mpenet/nippy-lz4

lz4hc-compressor

Like `lz4-compressor` but trades some write speed for ratio.

lzma2-compressor

Default org.tukaani.xz.LZMA2 compressor:
      Ratio: high.
Write speed: _very_ slow (also currently single-threaded).
 Read speed: slow.

A specialized compressor for large, low-write data in space-sensitive
environments.

map->Compressable-LZMA2

(map->Compressable-LZMA2 m__6518__auto__)
Factory function for class taoensso.nippy.Compressable-LZMA2, taking a map of keywords to field values.

map->StressRecord

(map->StressRecord m__6518__auto__)
Factory function for class taoensso.nippy.StressRecord, taking a map of keywords to field values.

read-biginteger

(read-biginteger in)

read-bytes

(read-bytes in)

read-coll

(read-coll in to-coll)

read-kvs

(read-kvs in to-coll)

read-kvs-depr1

(read-kvs-depr1 in to-coll)

read-sm-bytes

(read-sm-bytes in)

read-sm-coll

(read-sm-coll in to-coll)

read-sm-kvs

(read-sm-kvs in to-coll)

read-sm-utf8

(read-sm-utf8 in)

read-utf8

(read-utf8 in)

set-default-freeze-compressor-selector!

(set-default-freeze-compressor-selector! selector)
Sets root binding of `*default-freeze-compressor-selector*`

short-sized?

(short-sized? n)

snappy-compressor

Default org.iq80.snappy.Snappy compressor:
      Ratio: low.
Write speed: very high.
 Read speed: very high.

A good general-purpose compressor.

stress-data

Reference data used for tests & benchmarks

stress-data-benchable

Reference data with stuff removed that breaks reader or other utils we'll
be benching against

stress-data-comparable

Reference data with stuff removed that breaks roundtrip equality

swap-custom-readers!

(swap-custom-readers! f)

thaw

(thaw ba)(thaw ba {:keys [v1-compatibility? compressor encryptor password], :or {compressor :auto, encryptor :auto}, :as opts})
Deserializes a frozen object from given byte array to its original Clojure
data type. To thaw custom types, extend the Clojure reader or see `extend-thaw`.

** By default, supports data frozen with Nippy v2+ ONLY **
Add `{:v1-compatibility? true}` option to support thawing of data frozen with
legacy versions of Nippy.

Options include:
  :v1-compatibility? - support data frozen by legacy versions of Nippy?
  :compressor - :auto (checks header, default)  an ICompressor, or nil
  :encryptor  - :auto (checks header, default), an IEncryptor,  or nil

thaw-from-in!

(thaw-from-in! data-input)
Deserializes a frozen object from given DataInput to its original Clojure
data type

when-debug

macro

(when-debug & body)

write-biginteger

(write-biginteger out n)

write-bytes

(write-bytes out ba)

write-id

macro

(write-id out id)

write-ided-bytes

(write-ided-bytes out ba)(write-ided-bytes out id-sm id ba)

write-ided-coll

(write-ided-coll out ?id-sm id coll)

write-ided-keyword

(write-ided-keyword out kw)

write-ided-kvs

(write-ided-kvs out ?id-sm id coll)

write-ided-long

(write-ided-long out n)

write-ided-map

(write-ided-map out m)

write-ided-set

(write-ided-set out s)

write-ided-string

(write-ided-string out s)

write-ided-vec

(write-ided-vec out v)

write-sm-bytes

(write-sm-bytes out ba)

write-sm-utf8

(write-sm-utf8 out s)

write-utf8

(write-utf8 out s)