When support is added for a new type in Nippy version X, it necessarily means
that data containing that new type and frozen with Nippy version X is unthawable
with Nippy versions < X.
Earlier versions of Nippy will throw an exception on thawing affected data:
\"Unrecognized type id (<n>). Data frozen with newer Nippy version?\"
This can present a challenge when updating to new versions of Nippy, e.g.:
- Rolling updates could lead to old and new versions of Nippy temporarily co-existing.
- Data written with new types could limit your ability to revert a Nippy update.
There's no easy solution to this in GENERAL, but we CAN at least help reduce the
burden related to CHANGES in core data types by introducing changes over 2 phases:
1. Nippy vX reads new (changed) type, writes old type
2. Nippy vX+1 writes new (changed) type
When relevant, we can then warn users in the CHANGELOG to not leapfrog
(e.g. Nippy vX -> Nippy vX+2) when doing rolling updates.
This commit bootstraps the new compatibility feature by initially targeting core type
compatibility with Nippy v3.2.0 (2022-07-18).
A future Nippy version (e.g. v3.5.0) will then target v3.4.0, with an appropriate
CHANGELOG instruction to update in phases for environments that involve rolling
updates.
Details:
- Nippy will continue to support thawing OLD data that was originally compressed with Snappy.
- But Nippy will no longer support freezing NEW data with Snappy.
Motivation:
- The current Snappy implementation can cause JVM crashes in some cases [1].
- The only alternative JVM implementation that seems to be safe [2] uses JNI and
so would introduce possible incompatibility issues even for folks not using Snappy.
- Nippy already moved to the superior LZ4 as its default compression scheme in v2.7.0,
more than 9 years ago.
[1] Ref. <https://github.com/airlift/aircompressor/issues/183>
[2] Ref. <https://github.com/xerial/snappy-java>
BREAKING for the very small minority of folks that use `nippy/stress-data`.
Changes:
1. Make `nippy/stress-data` a function
It's unnecessarily wasteful to generate and store all this data when it's not
being used in the common case.
2. Make data deterministic
The stress data will now generally be stable by default between different versions
of Nippy, etc. This will help support an upcoming test for stable serialized output.
