luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Fast serialization library for Clojure
673 commits 2 branches 109 tags 2 MiB
Clojure 100%
Find a file
Peter Taoussanis 578c585bbf [mod] Remove nippy/snappy-compressor 
Details:

  - Nippy will continue to support thawing OLD data that was originally compressed with Snappy.
  - But Nippy will no longer support freezing NEW data with Snappy.

Motivation:

  - The current Snappy implementation can cause JVM crashes in some cases [1].

  - The only alternative JVM implementation that seems to be safe [2] uses JNI and
    so would introduce possible incompatibility issues even for folks not using Snappy.

  - Nippy already moved to the superior LZ4 as its default compression scheme in v2.7.0,
    more than 9 years ago.

[1] Ref. <https://github.com/airlift/aircompressor/issues/183>
[2] Ref. <https://github.com/xerial/snappy-java>
2024-02-06 16:01:13 +01:00
.github/workflows [nop] Update project template 2023-08-02 13:50:40 +02:00
bb [nop] Changes for dependency bumps 2023-09-25 11:50:06 +02:00
doc [new] Add cljdoc.edn config 2023-07-31 16:18:29 +02:00
resources/META-INF/native-image/com.taoensso/nippy [#146] [New] Add Graal native configurations (@FieryCod) 2022-06-23 12:35:47 +02:00
src/taoensso [mod] Remove nippy/snappy-compressor 2024-02-06 16:01:13 +01:00
test/taoensso [mod] Remove nippy/snappy-compressor 2024-02-06 16:01:13 +01:00
wiki [mod] Refactor stress data 2024-02-06 14:30:59 +01:00
.gitignore [nop] Update project template 2023-09-25 11:50:06 +02:00
bb.edn [nop] Update project template 2023-08-02 13:50:40 +02:00
benchmarks.png [new] [#164] Update benchmarks 2024-02-06 14:30:59 +01:00
CHANGELOG.md [nop] [#163] Update CHANGELOG 2024-02-06 09:28:48 +01:00
FUNDING.yml Add FUNDING.yml 2020-09-12 11:55:29 +02:00
LICENSE.txt [nop] Update README template 2023-08-02 13:50:40 +02:00
project.clj [mod] Remove nippy/snappy-compressor 2024-02-06 16:01:13 +01:00
README.md [new] [#164] Update benchmarks 2024-02-06 14:30:59 +01:00

README.md

Taoensso open source
Documentation | Latest releases | Get support

Nippy

The fastest serialization library for Clojure

Clojure's rich data types are awesome. And its reader allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).

Nippy is an attempt to provide a reliable, high-performance drop-in alternative to the reader.

Used by Carmine, Faraday, PigPen, Onyx, XTDB, Datalevin, and others.

Latest release/s

  • 2023-10-11 3.3.0 (stable): changes
  • 2023-09-26 3.4.0-beta1 (dev): changes

Main tests Graal tests

See here for earlier releases.

Why Nippy?

  • Small, simple all-Clojure library
  • Terrific performance: the best for Clojure that I'm aware of
  • Comprehensive support for all standard data types
  • Easily extendable to custom data types
  • Robust test suite, incl. full coverage for every supported type
  • Auto fallback to Java Serializable when available
  • Auto fallback to Clojure Reader for all other types (including tagged literals)
  • Pluggable compression with built-in LZ4, Zstandard, etc.
  • Pluggable encryption with built-in AES128
  • Tools for easy + robust integration into 3rd-party libraries, etc.
  • Powerful thaw transducer for flexible data inspection and transformation

Performance

Since its earliest versions, Nippy has consistently been the fastest serialization library for Clojure that I'm aware of. Latest benchmark results:

benchmarks-png

Documentation

Funding

You can help support continued work on this project, thank you!! 🙏

License

Copyright © 2012-2024 Peter Taoussanis.
Licensed under EPL 1.0 (same as Clojure).