luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[nop] Clarify CHANGELOG, data longevity

Browse source
This commit is contained in:
Peter Taoussanis 2024-05-30 09:11:28 +02:00
parent 9ce3f985b7
commit 3df9dd6378
4 changed files with 82 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
CHANGELOG.md
View file

@ -7,15 +7,25 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2).
> **Versioning**: Nippy uses [Break Versioning](https://www.taoensso.com/break-versioning).
This release updates some internal dependencies and is **recommended for all existing users**.
This release includes **important updates to internal dependencies** and is **recommended for all existing users**.
It should be a **non-breaking update** for almost all users of Nippy `v3.4.x`, `v3.3.x`, and `v3.2.x`.
It should be a **straight-forward and non-breaking update** for almost everyone:
Notes:
| Updating from Nippy version | API changes? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | New types |
| :-------------------------- | :----------- | :---------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------- |
| `v3.4.1` (2024-05-02) | - | - | - |
| `v3.4.0` (2024-04-30) | - | Yes | `clojure.lang.MapEntry` |
| `v3.3.0` (2023-10-11) | - | - | `java.lang.ClassCastException`, `java.sql.Date` |
| `v3.2.0` (2022-07-18) | - | - | `org.joda.time.DateTime` |
| `v3.1.3` (2022-06-23) | - | - | Several `java.time.X` types |
- May produce **different serialized output** to `v3.4.0` and `v3.3.0`. Most users won't care about this, but you could be affected if you depend on specific serialized byte values (for example by comparing serialized output between different versions of Nippy).
- When using Nippy version **X** to thaw data frozen by Nippy version **Y>X**, there is necessarily a risk of the thaw throwing when encountering unfamiliar types. This **can affect rolling updates** and/or **limit your ability to revert** a Nippy update - **so please ensure adequate testing** in your environment before updating against production data!
- As always, **please report any unexpected problems** 🙏
If updating from older versions of Nippy, please see the relevant release notes.
As always:
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
- **Please report any unexpected problems** 🙏
\- [Peter Taoussanis](https://www.taoensso.com)

38
README.md
View file

@ -7,10 +7,9 @@
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
Nippy is an attempt to provide a reliable, high-performance **drop-in alternative to the reader**.
Nippy is a mature, high-performance **drop-in alternative to the reader**.
Used by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx),
[XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx), [XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
## Latest release/s
@ -35,6 +34,39 @@ See [here][GitHub releases] for earlier releases.
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation
## Operational considerations
### Data longevity
Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.
But please note that the **converse is not generally true**:
- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)
### Rolling updates and rollback
From time to time, Nippy may introduce:
- Support for serializing **new types**
- Optimizations to the serialization of **pre-existing types**
To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4) will always introduce such changes over **two version releases**:
- Release 1: to add **read support** for the new types
- Release 2: to add **write support** for the new types
Starting from v3.4, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.
### Stability of byte output
It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.
However, I know that a small minority of users *do* have specialized needs in this area.
So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.
## Performance
Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:

30
wiki/2 Operational-considerations.md Normal file
View file

@ -0,0 +1,30 @@
# Data longevity
Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.
But please note that the **converse is not generally true**:
- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)
# Rolling updates and rollback
From time to time, Nippy may introduce:
- Support for serializing **new types**
- Optimizations to the serialization of **pre-existing types**
To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4.1) will always introduce such changes over **two version releases**:
- Release 1: to add **read support** for the new types
- Release 2: to add **write support** for the new types
Starting from v3.4.1, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.
# Stability of byte output
It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.
However, I know that a small minority of users *do* have specialized needs in this area.
So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.

2
wiki/2 Evolving-your-Nippy-data.md → wiki/3 Evolving-your-Nippy-data.md
View file

@ -1,4 +1,4 @@
> This article was kindly contributed by a Nippy user (@Outrovurt)
> This article is **community content** kindly contributed by a Nippy user (@Outrovurt)
This article describes a number of use cases where you need to make changes to your code which will have some impact on data you have already frozen using Nippy, and how best to manage each specific case. We will also discuss custom freezing and thawing.