nippy/README.md

129 lines
5.7 KiB
Markdown
Raw Normal View History

2022-10-28 09:53:23 +00:00
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
2025-04-14 16:33:54 +00:00
[**API**][cljdoc] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
2016-01-23 04:45:24 +00:00
2022-10-28 09:53:23 +00:00
# Nippy
2012-07-01 17:13:32 +00:00
2023-08-02 12:47:28 +00:00
### The fastest serialization library for Clojure
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
2020-09-20 10:05:37 +00:00
2024-05-30 07:11:28 +00:00
Nippy is a mature, high-performance **drop-in alternative to the reader**.
2020-07-24 16:27:34 +00:00
2024-05-30 07:11:28 +00:00
It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx), [XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
2020-07-24 16:27:34 +00:00
2022-10-28 09:53:23 +00:00
## Latest release/s
2020-07-24 16:27:34 +00:00
2025-04-14 16:33:54 +00:00
- `2025-04-15` `v3.5.0`: [release info](../../releases/tag/v3.5.0)
2016-03-10 07:11:34 +00:00
2022-10-28 09:53:23 +00:00
[![Main tests][Main tests SVG]][Main tests URL]
[![Graal tests][Graal tests SVG]][Graal tests URL]
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
See [here][GitHub releases] for earlier releases.
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
## Why Nippy?
2012-07-06 12:53:02 +00:00
2024-05-30 07:11:28 +00:00
- Small, simple **pure-Clojure** library
2022-10-28 09:53:23 +00:00
- **Terrific performance**: the [best](#performance) for Clojure that I'm aware of
2023-08-02 12:47:28 +00:00
- Comprehensive support for [all standard data types](../../wiki/1-Getting-started#deserializing)
- Easily extendable to [custom data types](../../wiki/1-Getting-started#custom-types)
2024-05-30 07:11:28 +00:00
- **Robust test suite** incl. coverage of every supported type
- **Mature** and widely used in production for 12+ years
- Optional auto fallback to [Java Serializable](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*freeze-serializable-allowlist*) for [safe](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*freeze-serializable-allowlist*) types
- Optional auto fallback to Clojure Reader (including tagged literals)
- Optional smart **compression** with [LZ4](https://code.google.com/p/lz4/) or [Zstandard](https://facebook.github.io/zstd/)
- Optional [encryption](../../wiki/1-Getting-started#encryption) with AES128
2023-08-02 12:47:28 +00:00
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation
2024-05-30 07:11:28 +00:00
## Quick example
Nippy's super easy to use:
```clojure
(require '[taoensso.nippy :as nippy])
;; Freeze any Clojure value
(nippy/freeze <my-value>) ; => Serialized byte[]
;; Thaw the byte[] to get back the original value:
(nippy/thaw (nippy/freeze <my-value>)) ; => <my-value>
```
See the [wiki](https://github.com/taoensso/nippy/wiki/1-Getting-started#deserializing) for more.
## Operational considerations
### Data longevity
Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.
But please note that the **converse is not generally true**:
- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)
### Rolling updates and rollback
From time to time, Nippy may introduce:
- Support for serializing **new types**
- Optimizations to the serialization of **pre-existing types**
To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4) will always introduce such changes over **two version releases**:
- Release 1: to add **read support** for the new types
- Release 2: to add **write support** for the new types
Starting from v3.4, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.
### Stability of byte output
It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.
However, I know that a small minority of users *do* have specialized needs in this area.
So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.
2022-10-28 09:53:23 +00:00
## Performance
2013-06-12 08:19:09 +00:00
2024-05-30 07:11:28 +00:00
Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest results:
2013-07-29 08:59:24 +00:00
2023-08-02 12:47:28 +00:00
![benchmarks-png](../../raw/master/benchmarks.png)
2013-07-29 08:59:24 +00:00
2024-05-30 07:11:28 +00:00
PRs welcome to include other alternatives in the [benchmark suite](../../blob/master/test/taoensso/nippy_benchmarks.clj)!
2022-10-28 09:53:23 +00:00
## Documentation
2013-07-29 08:59:24 +00:00
2023-10-26 15:02:25 +00:00
- [Wiki][GitHub wiki] (getting started, usage, etc.)
2025-04-14 16:33:54 +00:00
- API reference via [cljdoc][cljdoc]
2013-07-29 08:59:24 +00:00
2022-10-28 09:53:23 +00:00
## Funding
2012-07-06 12:53:02 +00:00
2023-08-02 12:47:28 +00:00
You can [help support][sponsor] continued work on this project, thank you!! 🙏
2016-06-10 03:37:29 +00:00
2022-10-28 09:53:23 +00:00
## License
2012-07-06 12:53:02 +00:00
2025-04-14 16:33:54 +00:00
Copyright &copy; 2012-2025 [Peter Taoussanis][].
2022-10-28 09:53:23 +00:00
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
<!-- Common -->
2012-11-05 17:49:37 +00:00
2022-10-28 09:53:23 +00:00
[GitHub releases]: ../../releases
[GitHub issues]: ../../issues
[GitHub wiki]: ../../wiki
2014-02-25 07:37:27 +00:00
2022-10-28 09:53:23 +00:00
[Peter Taoussanis]: https://www.taoensso.com
[sponsor]: https://www.taoensso.com/sponsor
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
<!-- Project -->
2012-07-06 12:53:02 +00:00
2025-04-14 16:33:54 +00:00
[cljdoc]: https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy
2012-07-06 12:53:02 +00:00
2022-10-28 09:53:23 +00:00
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
[Clojars URL]: https://clojars.org/com.taoensso/nippy
2014-02-22 18:30:11 +00:00
2022-10-28 09:53:23 +00:00
[Main tests SVG]: https://github.com/taoensso/nippy/actions/workflows/main-tests.yml/badge.svg
[Main tests URL]: https://github.com/taoensso/nippy/actions/workflows/main-tests.yml
[Graal tests SVG]: https://github.com/taoensso/nippy/actions/workflows/graal-tests.yml/badge.svg
[Graal tests URL]: https://github.com/taoensso/nippy/actions/workflows/graal-tests.yml