Compare commits
21 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ae4463a85c | ||
|
|
1b99516177 | ||
|
|
e68c2d56fe | ||
|
|
bfba59483a | ||
|
|
f7bb2824ac | ||
|
|
8d62dc2826 | ||
|
|
da57206b0d | ||
|
|
e0f49ced5a | ||
|
|
8d107650cd | ||
|
|
1026ea0ae7 | ||
|
|
c92457025f | ||
|
|
3cb29f3c2e | ||
|
|
a9ea13618c | ||
|
|
d415a2bf72 | ||
|
|
b217db5579 | ||
|
|
9022aad018 | ||
|
|
c0d1da1bb4 | ||
|
|
bb178f66fc | ||
|
|
9b380821cd | ||
|
|
c5209e32ce | ||
|
|
f6240582e1 |
13 changed files with 565 additions and 322 deletions
115
CHANGELOG.md
115
CHANGELOG.md
|
|
@ -2,6 +2,121 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
# `v3.5.0` (2025-04-15)
|
||||||
|
|
||||||
|
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0)
|
||||||
|
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
||||||
|
|
||||||
|
This is a **general maintenance release** focused on updating dependencies and laying groundwork (read support) for new array types coming in Nippy v3.6.
|
||||||
|
|
||||||
|
It **drops support for Clojure v1.9** but should otherwise be a safe update from (at least) all recent versions of Nippy.
|
||||||
|
|
||||||
|
## Since `v3.5.0-RC1` (2024-10-28)
|
||||||
|
|
||||||
|
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
|
||||||
|
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
|
||||||
|
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
|
||||||
|
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
|
||||||
|
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
|
||||||
|
|
||||||
|
## Since `v3.4.2` (2024-05-26)
|
||||||
|
|
||||||
|
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
|
||||||
|
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
|
||||||
|
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
|
||||||
|
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
|
||||||
|
- \[new] [#175] Mark [cache](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#cache) feature as stable \[b217db5]
|
||||||
|
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
|
||||||
|
|
||||||
|
## Migration info
|
||||||
|
|
||||||
|
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
||||||
|
| :------------------------ | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
||||||
|
| `v3.5.0-RC1` (2024-10-28) | - | - | - |
|
||||||
|
| `v3.4.2` (2024-05-26) | - | - | - |
|
||||||
|
| `v3.4.1` (2024-05-02) | - | - | - |
|
||||||
|
| `v3.4.0` (2024-04-30) | - | **Yes** | - |
|
||||||
|
| `v3.3.0` (2023-10-11) | - | - | - |
|
||||||
|
| `v3.2.0` (2022-07-18) | - | - | - |
|
||||||
|
| `v3.1.3` (2022-06-23) | - | - | - |
|
||||||
|
|
||||||
|
> [1] Relevant only when Nippy introduces support for new types **AND** you plan to update Nippy with a **rolling update** (coexisting new and old versions).
|
||||||
|
|
||||||
|
If updating from older versions of Nippy, please see the relevant release notes.
|
||||||
|
|
||||||
|
As always:
|
||||||
|
|
||||||
|
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
|
||||||
|
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
|
||||||
|
- **Please report any unexpected problems** 🙏
|
||||||
|
|
||||||
|
\- [Peter Taoussanis](https://www.taoensso.com)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# `v3.5.0-RC1` (2024-10-28)
|
||||||
|
|
||||||
|
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0-RC1)
|
||||||
|
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
||||||
|
|
||||||
|
This is a **non-breaking maintenance release** that updates dependencies and includes read support for more native array types to be introduced in a future v3.6 release.
|
||||||
|
|
||||||
|
It should be safe to update from (at least) all recent versions of Nippy.
|
||||||
|
|
||||||
|
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
||||||
|
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
||||||
|
| `v3.4.2` (2024-05-26) | - | - | - |
|
||||||
|
| `v3.4.1` (2024-05-02) | - | - | - |
|
||||||
|
| `v3.4.0` (2024-04-30) | - | Yes | - |
|
||||||
|
| `v3.3.0` (2023-10-11) | - | - | - |
|
||||||
|
| `v3.2.0` (2022-07-18) | - | - | - |
|
||||||
|
| `v3.1.3` (2022-06-23) | - | - | - |
|
||||||
|
|
||||||
|
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
|
||||||
|
|
||||||
|
If updating from older versions of Nippy, please see the relevant release notes.
|
||||||
|
|
||||||
|
As always:
|
||||||
|
|
||||||
|
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
|
||||||
|
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
|
||||||
|
- **Please report any unexpected problems** 🙏
|
||||||
|
|
||||||
|
\- [Peter Taoussanis](https://www.taoensso.com)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# `v3.4.2` (2024-05-26)
|
||||||
|
|
||||||
|
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2)
|
||||||
|
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
||||||
|
|
||||||
|
⚠️ This release addresses a [**security vulnerability**](https://github.com/taoensso/nippy/security/advisories/GHSA-vw78-267v-588h) in Nippy's upstream compression library and is **recommended for all existing users**.
|
||||||
|
|
||||||
|
It should be a **straight-forward and non-breaking update** for almost everyone:
|
||||||
|
|
||||||
|
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
||||||
|
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
||||||
|
| `v3.4.1` (2024-05-02) | - | - | - |
|
||||||
|
| `v3.4.0` (2024-04-30) | - | Yes | - |
|
||||||
|
| `v3.3.0` (2023-10-11) | - | - | - |
|
||||||
|
| `v3.2.0` (2022-07-18) | - | - | - |
|
||||||
|
| `v3.1.3` (2022-06-23) | - | - | - |
|
||||||
|
|
||||||
|
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
|
||||||
|
|
||||||
|
If updating from older versions of Nippy, please see the relevant release notes.
|
||||||
|
|
||||||
|
As always:
|
||||||
|
|
||||||
|
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
|
||||||
|
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
|
||||||
|
- **Please report any unexpected problems** 🙏
|
||||||
|
|
||||||
|
\- [Peter Taoussanis](https://www.taoensso.com)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
# `v3.4.1` (2024-05-02)
|
# `v3.4.1` (2024-05-02)
|
||||||
|
|
||||||
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.1).
|
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.1).
|
||||||
|
|
|
||||||
84
README.md
84
README.md
|
|
@ -1,20 +1,19 @@
|
||||||
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
|
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
|
||||||
[**Documentation**](#documentation) | [Latest releases](#latest-releases) | [Get support][GitHub issues]
|
[**API**][cljdoc] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
|
||||||
|
|
||||||
# Nippy
|
# Nippy
|
||||||
|
|
||||||
### The fastest serialization library for Clojure
|
### Fast serialization library for Clojure
|
||||||
|
|
||||||
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
|
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
|
||||||
|
|
||||||
Nippy is an attempt to provide a reliable, high-performance **drop-in alternative to the reader**.
|
Nippy is a mature, high-performance **drop-in alternative to the reader**.
|
||||||
|
|
||||||
Used by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx),
|
It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx), [XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
|
||||||
[XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
|
|
||||||
|
|
||||||
## Latest release/s
|
## Latest release/s
|
||||||
|
|
||||||
- `2024-05-02` `v3.4.1`: [release info](../../releases/tag/v3.4.1)
|
- `2025-04-15` `v3.5.0`: [release info](../../releases/tag/v3.5.0)
|
||||||
|
|
||||||
[![Main tests][Main tests SVG]][Main tests URL]
|
[![Main tests][Main tests SVG]][Main tests URL]
|
||||||
[![Graal tests][Graal tests SVG]][Graal tests URL]
|
[![Graal tests][Graal tests SVG]][Graal tests URL]
|
||||||
|
|
@ -23,28 +22,80 @@ See [here][GitHub releases] for earlier releases.
|
||||||
|
|
||||||
## Why Nippy?
|
## Why Nippy?
|
||||||
|
|
||||||
- Small, simple **all-Clojure** library
|
- Small, simple **pure-Clojure** library
|
||||||
- **Terrific performance**: the [best](#performance) for Clojure that I'm aware of
|
- **Terrific performance**: the [best](#performance) for Clojure that I'm aware of
|
||||||
- Comprehensive support for [all standard data types](../../wiki/1-Getting-started#deserializing)
|
- Comprehensive support for [all standard data types](../../wiki/1-Getting-started#deserializing)
|
||||||
- Easily extendable to [custom data types](../../wiki/1-Getting-started#custom-types)
|
- Easily extendable to [custom data types](../../wiki/1-Getting-started#custom-types)
|
||||||
- **Robust test suite**, incl. full coverage for every supported type
|
- **Robust test suite** incl. coverage of every supported type
|
||||||
- Auto fallback to [Java Serializable](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*freeze-serializable-allowlist*) when available
|
- **Mature** and widely used in production for 12+ years
|
||||||
- Auto fallback to Clojure Reader for all other types (including tagged literals)
|
- Optional auto fallback to [Java Serializable](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*freeze-serializable-allowlist*) for [safe](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*freeze-serializable-allowlist*) types
|
||||||
- Pluggable **compression** with built-in [LZ4](https://code.google.com/p/lz4/), [Zstandard](https://facebook.github.io/zstd/), etc.
|
- Optional auto fallback to Clojure Reader (including tagged literals)
|
||||||
- Pluggable [encryption](../../wiki/1-Getting-started#encryption) with built-in AES128
|
- Optional smart **compression** with [LZ4](https://code.google.com/p/lz4/) or [Zstandard](https://facebook.github.io/zstd/)
|
||||||
|
- Optional [encryption](../../wiki/1-Getting-started#encryption) with AES128
|
||||||
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
|
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
|
||||||
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation
|
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation
|
||||||
|
|
||||||
|
## Quick example
|
||||||
|
|
||||||
|
Nippy's super easy to use:
|
||||||
|
|
||||||
|
```clojure
|
||||||
|
(require '[taoensso.nippy :as nippy])
|
||||||
|
|
||||||
|
;; Freeze any Clojure value
|
||||||
|
(nippy/freeze <my-value>) ; => Serialized byte[]
|
||||||
|
|
||||||
|
;; Thaw the byte[] to get back the original value:
|
||||||
|
(nippy/thaw (nippy/freeze <my-value>)) ; => <my-value>
|
||||||
|
```
|
||||||
|
|
||||||
|
See the [wiki](https://github.com/taoensso/nippy/wiki/1-Getting-started#deserializing) for more.
|
||||||
|
|
||||||
|
## Operational considerations
|
||||||
|
|
||||||
|
### Data longevity
|
||||||
|
|
||||||
|
Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.
|
||||||
|
|
||||||
|
But please note that the **converse is not generally true**:
|
||||||
|
|
||||||
|
- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
|
||||||
|
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)
|
||||||
|
|
||||||
|
### Rolling updates and rollback
|
||||||
|
|
||||||
|
From time to time, Nippy may introduce:
|
||||||
|
|
||||||
|
- Support for serializing **new types**
|
||||||
|
- Optimizations to the serialization of **pre-existing types**
|
||||||
|
|
||||||
|
To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4) will always introduce such changes over **two version releases**:
|
||||||
|
|
||||||
|
- Release 1: to add **read support** for the new types
|
||||||
|
- Release 2: to add **write support** for the new types
|
||||||
|
|
||||||
|
Starting from v3.4, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.
|
||||||
|
|
||||||
|
### Stability of byte output
|
||||||
|
|
||||||
|
It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.
|
||||||
|
|
||||||
|
However, I know that a small minority of users *do* have specialized needs in this area.
|
||||||
|
|
||||||
|
So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.
|
||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:
|
Nippy is fast! Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
|
PRs welcome to include other alternatives in the bench suite!
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [Wiki][GitHub wiki] (getting started, usage, etc.)
|
- [Wiki][GitHub wiki] (getting started, usage, etc.)
|
||||||
- API reference: [cljdoc][cljdoc docs], [Codox][Codox docs]
|
- API reference via [cljdoc][cljdoc]
|
||||||
|
|
||||||
## Funding
|
## Funding
|
||||||
|
|
||||||
|
|
@ -52,7 +103,7 @@ You can [help support][sponsor] continued work on this project, thank you!! 🙏
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Copyright © 2012-2024 [Peter Taoussanis][].
|
Copyright © 2012-2025 [Peter Taoussanis][].
|
||||||
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
||||||
|
|
||||||
<!-- Common -->
|
<!-- Common -->
|
||||||
|
|
@ -66,8 +117,7 @@ Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
||||||
|
|
||||||
<!-- Project -->
|
<!-- Project -->
|
||||||
|
|
||||||
[Codox docs]: https://taoensso.github.io/nippy/
|
[cljdoc]: https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy
|
||||||
[cljdoc docs]: https://cljdoc.org/d/com.taoensso/nippy/
|
|
||||||
|
|
||||||
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
|
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
|
||||||
[Clojars URL]: https://clojars.org/com.taoensso/nippy
|
[Clojars URL]: https://clojars.org/com.taoensso/nippy
|
||||||
|
|
|
||||||
36
project.clj
36
project.clj
|
|
@ -1,6 +1,6 @@
|
||||||
(defproject com.taoensso/nippy "3.4.1"
|
(defproject com.taoensso/nippy "3.5.0"
|
||||||
:author "Peter Taoussanis <https://www.taoensso.com>"
|
:author "Peter Taoussanis <https://www.taoensso.com>"
|
||||||
:description "The fastest serialization library for Clojure"
|
:description "Fast serialization library for Clojure"
|
||||||
:url "https://www.taoensso.com/nippy"
|
:url "https://www.taoensso.com/nippy"
|
||||||
|
|
||||||
:license
|
:license
|
||||||
|
|
@ -8,20 +8,19 @@
|
||||||
:url "https://www.eclipse.org/legal/epl-v10.html"}
|
:url "https://www.eclipse.org/legal/epl-v10.html"}
|
||||||
|
|
||||||
:dependencies
|
:dependencies
|
||||||
[[org.clojure/tools.reader "1.4.2"]
|
[[org.clojure/tools.reader "1.5.2"]
|
||||||
[com.taoensso/encore "3.106.0"]
|
[com.taoensso/encore "3.142.0"]
|
||||||
[org.tukaani/xz "1.9"]
|
[org.tukaani/xz "1.10"]
|
||||||
[io.airlift/aircompressor "0.26"]]
|
[io.airlift/aircompressor "2.0.2"]]
|
||||||
|
|
||||||
:test-paths ["test" #_"src"]
|
:test-paths ["test" #_"src"]
|
||||||
|
|
||||||
:profiles
|
:profiles
|
||||||
{;; :default [:base :system :user :provided :dev]
|
{;; :default [:base :system :user :provided :dev]
|
||||||
:provided {:dependencies [[org.clojure/clojure "1.11.1"]]}
|
:provided {:dependencies [[org.clojure/clojure "1.11.4"]]}
|
||||||
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0-alpha11"]]}
|
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0"]]}
|
||||||
:c1.11 {:dependencies [[org.clojure/clojure "1.11.3"]]}
|
:c1.11 {:dependencies [[org.clojure/clojure "1.11.4"]]}
|
||||||
:c1.10 {:dependencies [[org.clojure/clojure "1.10.1"]]}
|
:c1.10 {:dependencies [[org.clojure/clojure "1.10.3"]]}
|
||||||
:c1.9 {:dependencies [[org.clojure/clojure "1.9.0"]]}
|
|
||||||
|
|
||||||
:graal-tests
|
:graal-tests
|
||||||
{:source-paths ["test"]
|
{:source-paths ["test"]
|
||||||
|
|
@ -38,7 +37,9 @@
|
||||||
"-Xms1024m" "-Xmx2048m"
|
"-Xms1024m" "-Xmx2048m"
|
||||||
"-Dtaoensso.elide-deprecated=true"
|
"-Dtaoensso.elide-deprecated=true"
|
||||||
"-Dtaoensso.nippy.thaw-serializable-allowlist-base=base.1, base.2"
|
"-Dtaoensso.nippy.thaw-serializable-allowlist-base=base.1, base.2"
|
||||||
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"]
|
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"
|
||||||
|
#_"-Dtaoensso.nippy.target-release=320"
|
||||||
|
#_"-Dtaoensso.nippy.target-release=350"]
|
||||||
|
|
||||||
:global-vars
|
:global-vars
|
||||||
{*warn-on-reflection* true
|
{*warn-on-reflection* true
|
||||||
|
|
@ -51,18 +52,13 @@
|
||||||
|
|
||||||
:plugins
|
:plugins
|
||||||
[[lein-pprint "1.3.2"]
|
[[lein-pprint "1.3.2"]
|
||||||
[lein-ancient "0.7.0"]
|
[lein-ancient "0.7.0"]]}}
|
||||||
[com.taoensso.forks/lein-codox "0.10.11"]]
|
|
||||||
|
|
||||||
:codox
|
|
||||||
{:language #{:clojure #_:clojurescript}
|
|
||||||
:base-language :clojure}}}
|
|
||||||
|
|
||||||
:aliases
|
:aliases
|
||||||
{"start-dev" ["with-profile" "+dev" "repl" ":headless"]
|
{"start-dev" ["with-profile" "+dev" "repl" ":headless"]
|
||||||
;; "build-once" ["do" ["clean"] ["cljsbuild" "once"]]
|
;; "build-once" ["do" ["clean"] ["cljsbuild" "once"]]
|
||||||
"deploy-lib" ["do" #_["build-once"] ["deploy" "clojars"] ["install"]]
|
"deploy-lib" ["do" #_["build-once"] ["deploy" "clojars"] ["install"]]
|
||||||
|
|
||||||
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10:+c1.9" "test"]
|
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10" "test"]
|
||||||
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
|
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
|
||||||
"test-all" ["do" ["clean"] ["test-clj"] #_["test-cljs"]]})
|
"test-all" ["do" ["clean"] ["test-clj"] #_["test-cljs"]]})
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,7 @@
|
||||||
(:require
|
(:require
|
||||||
[clojure.string :as str]
|
[clojure.string :as str]
|
||||||
[clojure.java.io :as jio]
|
[clojure.java.io :as jio]
|
||||||
|
[taoensso.truss :as truss]
|
||||||
[taoensso.encore :as enc]
|
[taoensso.encore :as enc]
|
||||||
[taoensso.nippy
|
[taoensso.nippy
|
||||||
[impl :as impl]
|
[impl :as impl]
|
||||||
|
|
@ -25,7 +26,7 @@
|
||||||
PersistentQueue PersistentTreeMap PersistentTreeSet PersistentList
|
PersistentQueue PersistentTreeMap PersistentTreeSet PersistentList
|
||||||
MapEntry LazySeq IRecord ISeq IType]))
|
MapEntry LazySeq IRecord ISeq IType]))
|
||||||
|
|
||||||
(enc/assert-min-encore-version [3 106 0])
|
(enc/assert-min-encore-version [3 142 0])
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(set! *unchecked-math* :warn-on-boxed)
|
(set! *unchecked-math* :warn-on-boxed)
|
||||||
|
|
@ -145,11 +146,6 @@
|
||||||
51 [:reader-md [[:bytes {:read 2}]]]
|
51 [:reader-md [[:bytes {:read 2}]]]
|
||||||
52 [:reader-lg [[:bytes {:read 4}]]]
|
52 [:reader-lg [[:bytes {:read 4}]]]
|
||||||
|
|
||||||
53 [:bytes-0 []]
|
|
||||||
7 [:bytes-sm [[:bytes {:read 1}]]]
|
|
||||||
15 [:bytes-md [[:bytes {:read 2}]]]
|
|
||||||
2 [:bytes-lg [[:bytes {:read 4}]]]
|
|
||||||
|
|
||||||
17 [:vec-0 []]
|
17 [:vec-0 []]
|
||||||
113 [:vec-2 [[:elements 2]]]
|
113 [:vec-2 [[:elements 2]]]
|
||||||
114 [:vec-3 [[:elements 3]]]
|
114 [:vec-3 [[:elements 3]]]
|
||||||
|
|
@ -182,7 +178,6 @@
|
||||||
28 [:sorted-set-lg [[:elements {:read 4}]]]
|
28 [:sorted-set-lg [[:elements {:read 4}]]]
|
||||||
31 [:sorted-map-lg [[:elements {:read 4 :multiplier 2}]]]
|
31 [:sorted-map-lg [[:elements {:read 4 :multiplier 2}]]]
|
||||||
26 [:queue-lg [[:elements {:read 4}]]]
|
26 [:queue-lg [[:elements {:read 4}]]]
|
||||||
115 [:objects-lg [[:elements {:read 4}]]]
|
|
||||||
|
|
||||||
25 [:meta [[:elements 1]]]
|
25 [:meta [[:elements 1]]]
|
||||||
58 [:regex [[:elements 1]]]
|
58 [:regex [[:elements 1]]]
|
||||||
|
|
@ -196,6 +191,21 @@
|
||||||
70 [:ratio [[:bytes {:read 4}]
|
70 [:ratio [[:bytes {:read 4}]
|
||||||
[:bytes {:read 4}]]]
|
[:bytes {:read 4}]]]
|
||||||
|
|
||||||
|
;; Arrays
|
||||||
|
53 [:byte-array-0 []]
|
||||||
|
7 [:byte-array-sm [[:elements {:read 1}]]]
|
||||||
|
15 [:byte-array-md [[:elements {:read 2}]]]
|
||||||
|
2 [:byte-array-lg [[:elements {:read 4}]]]
|
||||||
|
|
||||||
|
109 [:int-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
||||||
|
108 [:long-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
||||||
|
|
||||||
|
117 [:float-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
||||||
|
116 [:double-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
||||||
|
|
||||||
|
107 [:string-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
||||||
|
115 [:object-array-lg [[:elements {:read 4}]]]
|
||||||
|
|
||||||
;; Serializable
|
;; Serializable
|
||||||
75 [:sz-quarantined-sm [[:bytes {:read 1}] [:elements 1]]]
|
75 [:sz-quarantined-sm [[:bytes {:read 1}] [:elements 1]]]
|
||||||
76 [:sz-quarantined-md [[:bytes {:read 2}] [:elements 1]]]
|
76 [:sz-quarantined-md [[:bytes {:read 2}] [:elements 1]]]
|
||||||
|
|
@ -300,14 +310,14 @@
|
||||||
- `payload-spec` examples:
|
- `payload-spec` examples:
|
||||||
- nil ; No spec available (e.g. unpredictable payload)
|
- nil ; No spec available (e.g. unpredictable payload)
|
||||||
- [] ; Type has no payload
|
- [] ; Type has no payload
|
||||||
- [[:bytes 4]] ; Type has a payload of exactly 4 bytes
|
- [[:bytes 4]] ; Type has payload of exactly 4 bytes
|
||||||
- [[:bytes 2] [:elements 2]] ; Type has a payload of exactly 2 bytes, then
|
- [[:bytes 2] [:elements 2]] ; Type has payload of exactly 2 bytes,
|
||||||
; 2 elements
|
; followed by 2 elements
|
||||||
|
|
||||||
- [[:bytes {:read 2}]
|
- [[:bytes {:read 2}]
|
||||||
[:elements {:read 4 :multiplier 2 :unsigned? true}]]
|
[:elements {:read 4 :multiplier 2 :unsigned? true}]]
|
||||||
|
|
||||||
; Type has payload of <short-count> bytes, then
|
; Type has payload of <short-count> bytes, followed by
|
||||||
; <unsigned-int-count>*2 (multiplier) elements
|
; <unsigned-int-count>*2 (multiplier) elements
|
||||||
|
|
||||||
Note that `payload-spec` can be handy for skipping over items in
|
Note that `payload-spec` can be handy for skipping over items in
|
||||||
|
|
@ -356,25 +366,26 @@
|
||||||
;; For back compatibility (incl. Timbre's Carmine appender)
|
;; For back compatibility (incl. Timbre's Carmine appender)
|
||||||
(enc/defonce ^:dynamic ^:no-doc ^:deprecated *final-freeze-fallback* "Prefer `*freeze-fallback`." nil)
|
(enc/defonce ^:dynamic ^:no-doc ^:deprecated *final-freeze-fallback* "Prefer `*freeze-fallback`." nil)
|
||||||
(enc/defonce ^:dynamic *freeze-fallback*
|
(enc/defonce ^:dynamic *freeze-fallback*
|
||||||
"Controls Nippy's behaviour when trying to freeze an item for which Nippy
|
"Controls Nippy's behaviour when trying to freeze an object with a type for
|
||||||
doesn't currently have a native freeze/thaw implementation.
|
which Nippy doesn't currently have a native (protocol) implementation.
|
||||||
|
|
||||||
Possible values:
|
Possible values:
|
||||||
|
|
||||||
1. `nil` (no freeze-fallback, default)
|
1. `nil` (no fallback, default)
|
||||||
Tries the following in order:
|
Tries the following in order:
|
||||||
- Freeze with Java's `Serializable` interface if possible
|
- Freeze with Java's `Serializable` interface if this seems possible
|
||||||
- Freeze with Clojure's reader if possible
|
- Freeze with Clojure's reader if this seems possible
|
||||||
- Throw
|
- Throw
|
||||||
|
|
||||||
2. `:write-unfreezable` keyword
|
2. `:write-unfreezable` keyword
|
||||||
Tries the following in order:
|
Tries the following in order:
|
||||||
- Freeze with Java's `Serializable` interface if possible
|
- Freeze with Java's `Serializable` interface if this seems possible
|
||||||
- Freeze with Clojure's reader if possible
|
- Freeze with Clojure's reader if this seems possible
|
||||||
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
|
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
|
||||||
|
|
||||||
3. [Advanced] Custom (fn [^java.io.DataOutput out item]) that must
|
3. [Advanced] Custom (fn [^java.io.DataOutput out obj]) that must
|
||||||
write exactly one value to the given `DataOutput` stream"
|
write an appropriate object type id and payload to the given
|
||||||
|
`DataOutput` stream."
|
||||||
|
|
||||||
nil)
|
nil)
|
||||||
|
|
||||||
|
|
@ -431,16 +442,16 @@
|
||||||
;; Unfortunately quite a bit of complexity to do this safely
|
;; Unfortunately quite a bit of complexity to do this safely
|
||||||
|
|
||||||
(def default-freeze-serializable-allowlist
|
(def default-freeze-serializable-allowlist
|
||||||
"Allows *any* class-name to be frozen using Java's `Serializable` interface.
|
"Allows *any* class name to be frozen using Java's `Serializable` interface.
|
||||||
This is generally safe since RCE risk is present only when thawing.
|
This is generally safe since RCE risk is present only when thawing.
|
||||||
See also `*freeze-serializable-allowlist*`."
|
See also `*freeze-serializable-allowlist*`."
|
||||||
#{"*"})
|
#{"*"})
|
||||||
|
|
||||||
(def default-thaw-serializable-allowlist
|
(def default-thaw-serializable-allowlist
|
||||||
"A set of common safe class-names to allow to be frozen using Java's
|
"A set of common safe class names to allow to be frozen using Java's
|
||||||
`Serializable` interface. PRs welcome for additions.
|
`Serializable` interface. PRs welcome for additions.
|
||||||
See also `*thaw-serializable-allowlist*`."
|
See also `*thaw-serializable-allowlist*`."
|
||||||
#{"[I" "[F" "[Z" "[B" "[C" "[D" "[S" "[J"
|
#{"[Z" "[B" "[S" "[I" "[J" "[F" "[D" "[C" "[Ljava.lang.String;"
|
||||||
|
|
||||||
"java.lang.Throwable"
|
"java.lang.Throwable"
|
||||||
"java.lang.Exception"
|
"java.lang.Exception"
|
||||||
|
|
@ -504,10 +515,10 @@
|
||||||
|
|
||||||
Example allowlist values:
|
Example allowlist values:
|
||||||
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
|
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
|
||||||
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class-names
|
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class names
|
||||||
- `\"allow-and-record\"` ; Special value, see [2]
|
- `\"allow-and-record\"` ; Special value, see [2]
|
||||||
|
|
||||||
Note that class-names in sets may contain \"*\" wildcards.
|
Note that class names in sets may contain \"*\" wildcards.
|
||||||
|
|
||||||
Default allowlist values are:
|
Default allowlist values are:
|
||||||
- default-freeze-serializable-allowlist ; `{\"*\"}` => allow any class
|
- default-freeze-serializable-allowlist ; `{\"*\"}` => allow any class
|
||||||
|
|
@ -641,15 +652,23 @@
|
||||||
(defn- write-bytes [^DataOutput out ^bytes ba]
|
(defn- write-bytes [^DataOutput out ^bytes ba]
|
||||||
(let [len (alength ba)]
|
(let [len (alength ba)]
|
||||||
(if (zero? len)
|
(if (zero? len)
|
||||||
(write-id out id-bytes-0)
|
(write-id out id-byte-array-0)
|
||||||
(do
|
(do
|
||||||
(enc/cond
|
(enc/cond
|
||||||
(sm-count? len) (do (write-id out id-bytes-sm) (write-sm-count out len))
|
(sm-count? len) (do (write-id out id-byte-array-sm) (write-sm-count out len))
|
||||||
(md-count? len) (do (write-id out id-bytes-md) (write-md-count out len))
|
(md-count? len) (do (write-id out id-byte-array-md) (write-md-count out len))
|
||||||
:else (do (write-id out id-bytes-lg) (write-lg-count out len)))
|
:else (do (write-id out id-byte-array-lg) (write-lg-count out len)))
|
||||||
|
|
||||||
(.write out ba 0 len)))))
|
(.write out ba 0 len)))))
|
||||||
|
|
||||||
|
(defmacro ^:private -run! [proc coll] `(do (reduce #(~proc %2) nil ~coll) nil))
|
||||||
|
(defmacro ^:private -run-kv! [proc m] `(do (reduce-kv #(~proc %2 %3) nil ~m) nil))
|
||||||
|
|
||||||
|
(defn- write-array-lg [^DataOutput out array array-len id]
|
||||||
|
(write-id out id)
|
||||||
|
(write-lg-count out array-len)
|
||||||
|
(-run! (fn [in] (-freeze-with-meta! in out)) array))
|
||||||
|
|
||||||
(defn- write-biginteger [out ^BigInteger n] (write-bytes-lg out (.toByteArray n)))
|
(defn- write-biginteger [out ^BigInteger n] (write-bytes-lg out (.toByteArray n)))
|
||||||
|
|
||||||
(defn- write-str-sm* [^DataOutput out ^String s] (write-bytes-sm* out (.getBytes s StandardCharsets/UTF_8)))
|
(defn- write-str-sm* [^DataOutput out ^String s] (write-bytes-sm* out (.getBytes s StandardCharsets/UTF_8)))
|
||||||
|
|
@ -677,7 +696,7 @@
|
||||||
(sm-count? len) (do (write-id out id-kw-sm) (write-sm-count out len))
|
(sm-count? len) (do (write-id out id-kw-sm) (write-sm-count out len))
|
||||||
(md-count? len) (do (write-id out id-kw-md) (write-md-count out len))
|
(md-count? len) (do (write-id out id-kw-md) (write-md-count out len))
|
||||||
;; :else (do (write-id out id-kw-lg) (write-lg-count out len)) ; Unrealistic
|
;; :else (do (write-id out id-kw-lg) (write-lg-count out len)) ; Unrealistic
|
||||||
:else (throw (ex-info "Keyword too long" {:name s})))
|
:else (truss/ex-info! "Keyword too long" {:name s}))
|
||||||
|
|
||||||
(.write out ba 0 len)))
|
(.write out ba 0 len)))
|
||||||
|
|
||||||
|
|
@ -689,7 +708,7 @@
|
||||||
(sm-count? len) (do (write-id out id-sym-sm) (write-sm-count out len))
|
(sm-count? len) (do (write-id out id-sym-sm) (write-sm-count out len))
|
||||||
(md-count? len) (do (write-id out id-sym-md) (write-md-count out len))
|
(md-count? len) (do (write-id out id-sym-md) (write-md-count out len))
|
||||||
;; :else (do (write-id out id-sym-lg) (write-lg-count out len)) ; Unrealistic
|
;; :else (do (write-id out id-sym-lg) (write-lg-count out len)) ; Unrealistic
|
||||||
:else (throw (ex-info "Symbol too long" {:name s})))
|
:else (truss/ex-info! "Symbol too long" {:name s}))
|
||||||
|
|
||||||
(.write out ba 0 len)))
|
(.write out ba 0 len)))
|
||||||
|
|
||||||
|
|
@ -729,9 +748,6 @@
|
||||||
(<= y range-uint) (do (write-id out id-long-neg-lg) (.writeInt out (+ y Integer/MIN_VALUE)))
|
(<= y range-uint) (do (write-id out id-long-neg-lg) (.writeInt out (+ y Integer/MIN_VALUE)))
|
||||||
:else (do (write-id out id-long-xl) (.writeLong out n))))))
|
:else (do (write-id out id-long-xl) (.writeLong out n))))))
|
||||||
|
|
||||||
(defmacro ^:private -run! [proc coll] `(do (reduce #(~proc %2) nil ~coll) nil))
|
|
||||||
(defmacro ^:private -run-kv! [proc m] `(do (reduce-kv #(~proc %2 %3) nil ~m) nil))
|
|
||||||
|
|
||||||
(defn- write-vec [^DataOutput out v]
|
(defn- write-vec [^DataOutput out v]
|
||||||
(let [cnt (count v)]
|
(let [cnt (count v)]
|
||||||
(if (zero? cnt)
|
(if (zero? cnt)
|
||||||
|
|
@ -879,57 +895,45 @@
|
||||||
|
|
||||||
(-run! (fn [in] (-freeze-with-meta! in out)) s)))))
|
(-run! (fn [in] (-freeze-with-meta! in out)) s)))))
|
||||||
|
|
||||||
(defn- write-objects [^DataOutput out ^objects ary]
|
(defn- write-serializable [^DataOutput out x]
|
||||||
(let [len (alength ary)]
|
|
||||||
(write-id out id-objects-lg)
|
|
||||||
(write-lg-count out len)
|
|
||||||
(-run! (fn [in] (-freeze-with-meta! in out)) ary)))
|
|
||||||
|
|
||||||
(defn- write-serializable [^DataOutput out x ^String class-name]
|
|
||||||
(when-debug (println (str "write-serializable: " (type x))))
|
(when-debug (println (str "write-serializable: " (type x))))
|
||||||
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
|
(when (and (instance? Serializable x) (not (fn? x)))
|
||||||
len (alength class-name-ba)]
|
(let [class-name (.getName (class x))] ; Reflect
|
||||||
|
(when (freeze-serializable-allowed? class-name)
|
||||||
|
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
|
||||||
|
len (alength class-name-ba)]
|
||||||
|
|
||||||
(enc/cond
|
(enc/cond
|
||||||
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
|
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
|
||||||
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
|
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
|
||||||
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
||||||
:else (throw (ex-info "Serializable class name too long" {:name class-name})))
|
:else (truss/ex-info! "Serializable class name too long" {:name class-name}))
|
||||||
|
|
||||||
;; Legacy: write object directly to out.
|
;; Legacy: write object directly to out.
|
||||||
;; (.writeObject (ObjectOutputStream. out) x)
|
;; (.writeObject (ObjectOutputStream. out) x)
|
||||||
|
|
||||||
;; Quarantined: write object to ba, then ba to out.
|
;; Quarantined: write object to ba, then ba to out.
|
||||||
;; We'll have object length during thaw, allowing us to skip readObject.
|
;; We'll have object length during thaw, allowing us to skip readObject.
|
||||||
(let [quarantined-ba (ByteArrayOutputStream.)]
|
(let [quarantined-ba (ByteArrayOutputStream.)]
|
||||||
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
|
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
|
||||||
(write-bytes out (.toByteArray quarantined-ba)))))
|
(write-bytes out (.toByteArray quarantined-ba)))
|
||||||
|
|
||||||
|
true)))))
|
||||||
|
|
||||||
(defn- write-readable [^DataOutput out x]
|
(defn- write-readable [^DataOutput out x]
|
||||||
(when-debug (println (str "write-readable: " (type x))))
|
(when-debug (println (str "write-readable: " (type x))))
|
||||||
(let [edn (enc/pr-edn x)
|
|
||||||
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
|
|
||||||
len (alength edn-ba)]
|
|
||||||
(enc/cond
|
|
||||||
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
|
|
||||||
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
|
|
||||||
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))))
|
|
||||||
|
|
||||||
(defn try-write-serializable [out x]
|
|
||||||
(when (and (instance? Serializable x) (not (fn? x)))
|
|
||||||
(try
|
|
||||||
(let [class-name (.getName (class x))] ; Reflect
|
|
||||||
(when (freeze-serializable-allowed? class-name)
|
|
||||||
(write-serializable out x class-name)
|
|
||||||
true))
|
|
||||||
(catch Throwable _ nil))))
|
|
||||||
|
|
||||||
(defn try-write-readable [out x]
|
|
||||||
(when (impl/seems-readable? x)
|
(when (impl/seems-readable? x)
|
||||||
(try
|
(let [edn (enc/pr-edn x)
|
||||||
(write-readable out x)
|
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
|
||||||
true
|
len (alength edn-ba)]
|
||||||
(catch Throwable _ nil))))
|
(enc/cond
|
||||||
|
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
|
||||||
|
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
|
||||||
|
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))
|
||||||
|
true)))
|
||||||
|
|
||||||
|
(defn ^:deprecated try-write-serializable [out x] (truss/catching :all (write-serializable out x)))
|
||||||
|
(defn ^:deprecated try-write-readable [out x] (truss/catching :all (write-readable out x)))
|
||||||
|
|
||||||
(defn- try-pr-edn [x]
|
(defn- try-pr-edn [x]
|
||||||
(try
|
(try
|
||||||
|
|
@ -947,29 +951,22 @@
|
||||||
:content (try-pr-edn x)}}
|
:content (try-pr-edn x)}}
|
||||||
out))
|
out))
|
||||||
|
|
||||||
(defn throw-unfreezable [x]
|
|
||||||
(let [t (type x)]
|
|
||||||
(throw
|
|
||||||
(ex-info (str "Unfreezable type: " t)
|
|
||||||
{:type t
|
|
||||||
:as-str (try-pr-edn x)}))))
|
|
||||||
|
|
||||||
;; Public `-freeze-with-meta!` with different arg order
|
;; Public `-freeze-with-meta!` with different arg order
|
||||||
(defn freeze-to-out!
|
(defn freeze-to-out!
|
||||||
"Serializes arg (any Clojure data type) to a DataOutput.
|
"Serializes arg (any Clojure data type) to a DataOutput.
|
||||||
This is a low-level util: in most cases you'll want `freeze` instead."
|
This is a low-level util: in most cases you'll want `freeze` instead."
|
||||||
[^DataOutput data-output x] (-freeze-with-meta! x data-output))
|
[^DataOutput data-output x] (-freeze-with-meta! x data-output))
|
||||||
|
|
||||||
;;;; Caching ; Experimental
|
;;;; Caching
|
||||||
|
|
||||||
;; Nb: don't use an auto initialValue; can cause thread-local state to
|
|
||||||
;; accidentally hang around with the use of `freeze-to-out!`, etc.
|
|
||||||
;; Safer to require explicit activation through `with-cache`.
|
|
||||||
(def ^ThreadLocal -cache-proxy
|
(def ^ThreadLocal -cache-proxy
|
||||||
"{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing."
|
"{[<x> <meta>] <idx>} for freezing, {<idx> <x-with-meta>} for thawing."
|
||||||
|
;; Nb: don't use an auto initialValue; can cause thread-local state to
|
||||||
|
;; accidentally hang around with the use of `freeze-to-out!`, etc.
|
||||||
|
;; Safer to require explicit activation through `with-cache`.
|
||||||
(proxy [ThreadLocal] []))
|
(proxy [ThreadLocal] []))
|
||||||
|
|
||||||
(defmacro ^:private with-cache
|
(defmacro with-cache
|
||||||
"Executes body with support for freezing/thawing cached values.
|
"Executes body with support for freezing/thawing cached values.
|
||||||
|
|
||||||
This is a low-level util: you won't need to use this yourself unless
|
This is a low-level util: you won't need to use this yourself unless
|
||||||
|
|
@ -984,9 +981,7 @@
|
||||||
|
|
||||||
(deftype Cached [val])
|
(deftype Cached [val])
|
||||||
(defn cache
|
(defn cache
|
||||||
"Experimental, subject to change. Feedback welcome!
|
"Wraps value so that future writes of the same wrapped value with same
|
||||||
|
|
||||||
Wraps value so that future writes of the same wrapped value with same
|
|
||||||
metadata will be efficiently encoded as references to this one.
|
metadata will be efficiently encoded as references to this one.
|
||||||
|
|
||||||
(freeze [(cache \"foo\") (cache \"foo\") (cache \"foo\")])
|
(freeze [(cache \"foo\") (cache \"foo\") (cache \"foo\")])
|
||||||
|
|
@ -1034,7 +1029,7 @@
|
||||||
(when first-occurance? (-freeze-with-meta! x-val out)))
|
(when first-occurance? (-freeze-with-meta! x-val out)))
|
||||||
|
|
||||||
:else
|
:else
|
||||||
;; (throw (ex-info "Max cache size exceeded" {:idx idx}))
|
;; (truss/ex-info! "Max cache size exceeded" {:idx idx})
|
||||||
(-freeze-with-meta! x-val out) ; Just freeze uncached
|
(-freeze-with-meta! x-val out) ; Just freeze uncached
|
||||||
))
|
))
|
||||||
|
|
||||||
|
|
@ -1051,7 +1046,7 @@
|
||||||
(vswap! cache_ assoc idx x)
|
(vswap! cache_ assoc idx x)
|
||||||
x)
|
x)
|
||||||
v))
|
v))
|
||||||
(throw (ex-info "No cache_ established, can't thaw. See `with-cache`." {}))))))
|
(truss/ex-info! "Can't thaw without cache available. See `with-cache`." {})))))
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(thaw (freeze [(cache "foo") (cache "foo") (cache "foo")]))
|
(thaw (freeze [(cache "foo") (cache "foo") (cache "foo")]))
|
||||||
|
|
@ -1096,8 +1091,6 @@
|
||||||
(.writeLong out (.getLeastSignificantBits x))))
|
(.writeLong out (.getLeastSignificantBits x))))
|
||||||
|
|
||||||
(freezer Boolean nil true (if (boolean x) (write-id out id-true) (write-id out id-false)))
|
(freezer Boolean nil true (if (boolean x) (write-id out id-true) (write-id out id-false)))
|
||||||
(freezer (Class/forName "[B") nil true (write-bytes out x))
|
|
||||||
(freezer (Class/forName "[Ljava.lang.Object;") nil true (write-objects out x))
|
|
||||||
(freezer String nil true (write-str out x))
|
(freezer String nil true (write-str out x))
|
||||||
(freezer Keyword nil true (write-kw out x))
|
(freezer Keyword nil true (write-kw out x))
|
||||||
(freezer Symbol nil true (write-sym out x))
|
(freezer Symbol nil true (write-sym out x))
|
||||||
|
|
@ -1107,6 +1100,17 @@
|
||||||
(do (write-id out id-double-0))
|
(do (write-id out id-double-0))
|
||||||
(do (write-id out id-double) (.writeDouble out x))))
|
(do (write-id out id-double) (.writeDouble out x))))
|
||||||
|
|
||||||
|
;; Arrays
|
||||||
|
(freezer (Class/forName "[B") nil true (write-bytes out x))
|
||||||
|
(freezer (Class/forName "[Ljava.lang.Object;") nil true (write-array-lg out x (alength ^"[Ljava.lang.Object;" x) id-object-array-lg))
|
||||||
|
|
||||||
|
(when (impl/target-release>= 350)
|
||||||
|
(freezer (Class/forName "[I") nil true (write-array-lg out x (alength ^"[I" x) id-int-array-lg))
|
||||||
|
(freezer (Class/forName "[J") nil true (write-array-lg out x (alength ^"[J" x) id-long-array-lg))
|
||||||
|
(freezer (Class/forName "[F") nil true (write-array-lg out x (alength ^"[F" x) id-float-array-lg))
|
||||||
|
(freezer (Class/forName "[D") nil true (write-array-lg out x (alength ^"[D" x) id-double-array-lg))
|
||||||
|
(freezer (Class/forName "[Ljava.lang.String;") nil true (write-array-lg out x (alength ^"[Ljava.lang.String;" x) id-string-array-lg)))
|
||||||
|
|
||||||
(freezer PersistentQueue nil true (write-counted-coll out id-queue-lg x))
|
(freezer PersistentQueue nil true (write-counted-coll out id-queue-lg x))
|
||||||
(freezer PersistentTreeSet nil true (write-counted-coll out id-sorted-set-lg x))
|
(freezer PersistentTreeSet nil true (write-counted-coll out id-sorted-set-lg x))
|
||||||
(freezer PersistentTreeMap nil true (write-kvs out id-sorted-map-lg x))
|
(freezer PersistentTreeMap nil true (write-kvs out id-sorted-map-lg x))
|
||||||
|
|
@ -1124,7 +1128,7 @@
|
||||||
(sm-count? len) (do (write-id out id-record-sm) (write-bytes-sm out class-name-ba))
|
(sm-count? len) (do (write-id out id-record-sm) (write-bytes-sm out class-name-ba))
|
||||||
(md-count? len) (do (write-id out id-record-md) (write-bytes-md out class-name-ba))
|
(md-count? len) (do (write-id out id-record-md) (write-bytes-md out class-name-ba))
|
||||||
;; :else (do (write-id out id-record-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
;; :else (do (write-id out id-record-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
||||||
:else (throw (ex-info "Record class name too long" {:name class-name})))
|
:else (truss/ex-info! "Record class name too long" {:name class-name}))
|
||||||
|
|
||||||
(-freeze-without-meta! (into {} x) out)))
|
(-freeze-without-meta! (into {} x) out)))
|
||||||
|
|
||||||
|
|
@ -1179,13 +1183,20 @@
|
||||||
(write-unfreezable out x)))
|
(write-unfreezable out x)))
|
||||||
|
|
||||||
;; Without ff
|
;; Without ff
|
||||||
(or
|
(enc/cond
|
||||||
(try-write-serializable out x)
|
:let [[r1 e1] (try [(write-serializable out x)] (catch Throwable t [nil t]))], r1 r1
|
||||||
(try-write-readable out x)
|
:let [[r2 e2] (try [(write-readable out x)] (catch Throwable t [nil t]))], r2 r2
|
||||||
|
|
||||||
(when-let [fff *final-freeze-fallback*] (fff out x) true) ; Deprecated
|
:if-let [fff *final-freeze-fallback*] (fff out x) ; Deprecated
|
||||||
|
:else
|
||||||
(throw-unfreezable x)))))
|
(let [t (type x)]
|
||||||
|
(truss/ex-info! (str "Failed to freeze type: " t)
|
||||||
|
(enc/assoc-some
|
||||||
|
{:type t
|
||||||
|
:as-str (try-pr-edn x)}
|
||||||
|
{:serializable-error e1
|
||||||
|
:readable-error e2})
|
||||||
|
(or e1 e2)))))))
|
||||||
|
|
||||||
;;;;
|
;;;;
|
||||||
|
|
||||||
|
|
@ -1199,9 +1210,8 @@
|
||||||
(defn- wrap-header [data-ba head-meta]
|
(defn- wrap-header [data-ba head-meta]
|
||||||
(if-let [head-ba (get-head-ba head-meta)]
|
(if-let [head-ba (get-head-ba head-meta)]
|
||||||
(enc/ba-concat head-ba data-ba)
|
(enc/ba-concat head-ba data-ba)
|
||||||
(throw
|
(truss/ex-info! (str "Unrecognized header meta: " head-meta)
|
||||||
(ex-info (str "Unrecognized header meta: " head-meta)
|
{:head-meta head-meta})))
|
||||||
{:head-meta head-meta}))))
|
|
||||||
|
|
||||||
(comment (wrap-header (.getBytes "foo") {:compressor-id :lz4
|
(comment (wrap-header (.getBytes "foo") {:compressor-id :lz4
|
||||||
:encryptor-id nil}))
|
:encryptor-id nil}))
|
||||||
|
|
@ -1250,11 +1260,16 @@
|
||||||
- Drops all support for compression and encryption
|
- Drops all support for compression and encryption
|
||||||
- Must be thawed with `fast-thaw`
|
- Must be thawed with `fast-thaw`
|
||||||
|
|
||||||
Equivalent to (but a little faster than) `freeze` with opts:
|
Equivalent to (but a little faster than) `freeze` with opts
|
||||||
- :compressor nil
|
{:no-header? true, :compressor nil, :encryptor nil}.
|
||||||
- :encryptor nil
|
|
||||||
- :no-header? true"
|
Intended for use only by advanced users that clearly understand the tradeoffs.
|
||||||
[x]
|
I STRONGLY recommend that most users prefer the standard `freeze` since:
|
||||||
|
- The Nippy header is useful for data portability and preservation
|
||||||
|
- Compression is often benefitial at little/no cost
|
||||||
|
- The performance difference between `freeze` and `fast-freeze` is
|
||||||
|
often negligible in practice."
|
||||||
|
^bytes [x]
|
||||||
(let [baos (ByteArrayOutputStream. 64)
|
(let [baos (ByteArrayOutputStream. 64)
|
||||||
dos (DataOutputStream. baos)]
|
dos (DataOutputStream. baos)]
|
||||||
(with-cache (-freeze-with-meta! x dos))
|
(with-cache (-freeze-with-meta! x dos))
|
||||||
|
|
@ -1263,11 +1278,13 @@
|
||||||
(defn freeze
|
(defn freeze
|
||||||
"Serializes arg (any Clojure data type) to a byte array.
|
"Serializes arg (any Clojure data type) to a byte array.
|
||||||
To freeze custom types, extend the Clojure reader or see `extend-freeze`."
|
To freeze custom types, extend the Clojure reader or see `extend-freeze`."
|
||||||
([x] (freeze x nil))
|
(^bytes [x] (freeze x nil))
|
||||||
([x {:as opts
|
(^bytes
|
||||||
:keys [compressor encryptor password serializable-allowlist incl-metadata?]
|
[x
|
||||||
:or {compressor :auto
|
{:as opts
|
||||||
encryptor aes128-gcm-encryptor}}]
|
:keys [compressor encryptor password serializable-allowlist incl-metadata?]
|
||||||
|
:or {compressor :auto
|
||||||
|
encryptor aes128-gcm-encryptor}}]
|
||||||
|
|
||||||
(call-with-bindings :freeze opts
|
(call-with-bindings :freeze opts
|
||||||
(fn []
|
(fn []
|
||||||
|
|
@ -1323,10 +1340,21 @@
|
||||||
([^DataInput in len] (let [ba (byte-array len)] (.readFully in ba 0 len) ba))
|
([^DataInput in len] (let [ba (byte-array len)] (.readFully in ba 0 len) ba))
|
||||||
([^DataInput in ]
|
([^DataInput in ]
|
||||||
(enc/case-eval (.readByte in)
|
(enc/case-eval (.readByte in)
|
||||||
id-bytes-0 (byte-array 0)
|
id-byte-array-0 (byte-array 0)
|
||||||
id-bytes-sm (read-bytes in (read-sm-count in))
|
id-byte-array-sm (read-bytes in (read-sm-count in))
|
||||||
id-bytes-md (read-bytes in (read-md-count in))
|
id-byte-array-md (read-bytes in (read-md-count in))
|
||||||
id-bytes-lg (read-bytes in (read-lg-count in)))))
|
id-byte-array-lg (read-bytes in (read-lg-count in)))))
|
||||||
|
|
||||||
|
(defmacro ^:private read-array [in thaw-type array-type array]
|
||||||
|
(let [thawed-sym (with-meta 'thawed-sym {:tag thaw-type})
|
||||||
|
array-sym (with-meta 'array-sym {:tag array-type})]
|
||||||
|
`(let [~array-sym ~array]
|
||||||
|
(enc/reduce-n
|
||||||
|
(fn [_# idx#]
|
||||||
|
(let [~thawed-sym (thaw-from-in! ~in)]
|
||||||
|
(aset ~'array-sym idx# ~'thawed-sym)))
|
||||||
|
nil (alength ~'array-sym))
|
||||||
|
~'array-sym)))
|
||||||
|
|
||||||
(defn- read-str-sm* [^DataInput in] (String. ^bytes (read-bytes in (read-sm-count* in)) StandardCharsets/UTF_8))
|
(defn- read-str-sm* [^DataInput in] (String. ^bytes (read-bytes in (read-sm-count* in)) StandardCharsets/UTF_8))
|
||||||
(defn- read-str-sm [^DataInput in] (String. ^bytes (read-bytes in (read-sm-count in)) StandardCharsets/UTF_8))
|
(defn- read-str-sm [^DataInput in] (String. ^bytes (read-bytes in (read-sm-count in)) StandardCharsets/UTF_8))
|
||||||
|
|
@ -1346,7 +1374,7 @@
|
||||||
|
|
||||||
(defmacro ^:private editable? [coll] `(instance? clojure.lang.IEditableCollection ~coll))
|
(defmacro ^:private editable? [coll] `(instance? clojure.lang.IEditableCollection ~coll))
|
||||||
|
|
||||||
(defn- xform* [xform] (enc/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
|
(defn- xform* [xform] (truss/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
|
||||||
|
|
||||||
(let [rf! (fn rf! ([x] (persistent! x)) ([acc x] (conj! acc x)))
|
(let [rf! (fn rf! ([x] (persistent! x)) ([acc x] (conj! acc x)))
|
||||||
rf* (fn rf* ([x] x) ([acc x] (conj acc x)))]
|
rf* (fn rf* ([x] x) ([acc x] (conj acc x)))]
|
||||||
|
|
@ -1375,12 +1403,6 @@
|
||||||
(let [rf rf2 ] (rf (enc/reduce-n (fn [acc _] (rf acc (thaw-from-in! in) (thaw-from-in! in))) init n)))))))
|
(let [rf rf2 ] (rf (enc/reduce-n (fn [acc _] (rf acc (thaw-from-in! in) (thaw-from-in! in))) init n)))))))
|
||||||
|
|
||||||
(defn- read-kvs-depr [to ^DataInput in] (read-kvs-into to in (quot (.readInt in) 2)))
|
(defn- read-kvs-depr [to ^DataInput in] (read-kvs-into to in (quot (.readInt in) 2)))
|
||||||
(defn- read-objects [^objects ary ^DataInput in]
|
|
||||||
(enc/reduce-n
|
|
||||||
(fn [^objects ary i]
|
|
||||||
(aset ary i (thaw-from-in! in))
|
|
||||||
ary)
|
|
||||||
ary (alength ary)))
|
|
||||||
|
|
||||||
(def ^:private class-method-sig (into-array Class [IPersistentMap]))
|
(def ^:private class-method-sig (into-array Class [IPersistentMap]))
|
||||||
|
|
||||||
|
|
@ -1389,14 +1411,12 @@
|
||||||
(try
|
(try
|
||||||
(custom-reader in)
|
(custom-reader in)
|
||||||
(catch Exception e
|
(catch Exception e
|
||||||
(throw
|
(truss/ex-info!
|
||||||
(ex-info
|
(str "Reader exception for custom type id: " type-id)
|
||||||
(str "Reader exception for custom type id: " type-id)
|
{:type-id type-id, :prefixed? prefixed?} e)))
|
||||||
{:type-id type-id, :prefixed? prefixed?} e))))
|
(truss/ex-info!
|
||||||
(throw
|
(str "No reader provided for custom type id: " type-id)
|
||||||
(ex-info
|
{:type-id type-id, :prefixed? prefixed?})))
|
||||||
(str "No reader provided for custom type id: " type-id)
|
|
||||||
{:type-id type-id, :prefixed? prefixed?}))))
|
|
||||||
|
|
||||||
(defn- read-edn [edn]
|
(defn- read-edn [edn]
|
||||||
(try
|
(try
|
||||||
|
|
@ -1473,9 +1493,9 @@
|
||||||
[^DataInput in class-name]
|
[^DataInput in class-name]
|
||||||
(if (thaw-serializable-allowed? class-name)
|
(if (thaw-serializable-allowed? class-name)
|
||||||
(read-object in class-name)
|
(read-object in class-name)
|
||||||
(throw ; No way to skip bytes, so best we can do is throw
|
(truss/ex-info! ; No way to skip bytes, so best we can do is throw
|
||||||
(ex-info "Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
|
"Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
|
||||||
{:class-name class-name}))))
|
{:class-name class-name})))
|
||||||
|
|
||||||
(defn- read-record [in class-name]
|
(defn- read-record [in class-name]
|
||||||
(let [content (thaw-from-in! in)]
|
(let [content (thaw-from-in! in)]
|
||||||
|
|
@ -1573,12 +1593,19 @@
|
||||||
id-cached-sm (thaw-cached (read-sm-count in) in)
|
id-cached-sm (thaw-cached (read-sm-count in) in)
|
||||||
id-cached-md (thaw-cached (read-md-count in) in)
|
id-cached-md (thaw-cached (read-md-count in) in)
|
||||||
|
|
||||||
id-bytes-0 (byte-array 0)
|
id-byte-array-0 (byte-array 0)
|
||||||
id-bytes-sm (read-bytes in (read-sm-count in))
|
id-byte-array-sm (read-bytes in (read-sm-count in))
|
||||||
id-bytes-md (read-bytes in (read-md-count in))
|
id-byte-array-md (read-bytes in (read-md-count in))
|
||||||
id-bytes-lg (read-bytes in (read-lg-count in))
|
id-byte-array-lg (read-bytes in (read-lg-count in))
|
||||||
|
|
||||||
id-objects-lg (read-objects (object-array (read-lg-count in)) in)
|
id-long-array-lg (read-array in long "[J" (long-array (read-lg-count in)))
|
||||||
|
id-int-array-lg (read-array in int "[I" (int-array (read-lg-count in)))
|
||||||
|
|
||||||
|
id-double-array-lg (read-array in double "[D" (double-array (read-lg-count in)))
|
||||||
|
id-float-array-lg (read-array in float "[F" (float-array (read-lg-count in)))
|
||||||
|
|
||||||
|
id-string-array-lg (read-array in String "[Ljava.lang.String;" (make-array String (read-lg-count in)))
|
||||||
|
id-object-array-lg (read-array in Object "[Ljava.lang.Object;" (object-array (read-lg-count in)))
|
||||||
|
|
||||||
id-str-0 ""
|
id-str-0 ""
|
||||||
id-str-sm* (read-str in (read-sm-count* in))
|
id-str-sm* (read-str in (read-sm-count* in))
|
||||||
|
|
@ -1721,15 +1748,13 @@
|
||||||
|
|
||||||
(if (neg? type-id)
|
(if (neg? type-id)
|
||||||
(read-custom! in nil type-id) ; Unprefixed custom type
|
(read-custom! in nil type-id) ; Unprefixed custom type
|
||||||
(throw
|
(truss/ex-info!
|
||||||
(ex-info
|
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
|
||||||
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
|
{:type-id type-id})))
|
||||||
{:type-id type-id}))))
|
|
||||||
|
|
||||||
(catch Throwable t
|
(catch Throwable t
|
||||||
(throw
|
(truss/ex-info! (str "Thaw failed against type-id: " type-id)
|
||||||
(ex-info (str "Thaw failed against type-id: " type-id)
|
{:type-id type-id} t)))))
|
||||||
{:type-id type-id} t))))))
|
|
||||||
|
|
||||||
(let [head-sig head-sig] ; Not ^:const
|
(let [head-sig head-sig] ; Not ^:const
|
||||||
(defn- try-parse-header [^bytes ba]
|
(defn- try-parse-header [^bytes ba]
|
||||||
|
|
@ -1749,33 +1774,31 @@
|
||||||
:lzma2 lzma2-compressor
|
:lzma2 lzma2-compressor
|
||||||
:lz4 lz4-compressor
|
:lz4 lz4-compressor
|
||||||
:zstd zstd-compressor
|
:zstd zstd-compressor
|
||||||
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
|
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
|
||||||
:else (throw (ex-info ":auto not supported for non-standard compressors." {}))
|
:else (truss/ex-info! ":auto not supported for non-standard compressors." {})
|
||||||
(do (throw (ex-info (str "Unrecognized :auto compressor id: " compressor-id)
|
(do (truss/ex-info! (str "Unrecognized :auto compressor id: " compressor-id)
|
||||||
{:compressor-id compressor-id})))))
|
{:compressor-id compressor-id}))))
|
||||||
|
|
||||||
(defn- get-auto-encryptor [encryptor-id]
|
(defn- get-auto-encryptor [encryptor-id]
|
||||||
(case encryptor-id
|
(case encryptor-id
|
||||||
nil nil
|
nil nil
|
||||||
:aes128-gcm-sha512 aes128-gcm-encryptor
|
:aes128-gcm-sha512 aes128-gcm-encryptor
|
||||||
:aes128-cbc-sha512 aes128-cbc-encryptor
|
:aes128-cbc-sha512 aes128-cbc-encryptor
|
||||||
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
|
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
|
||||||
:else (throw (ex-info ":auto not supported for non-standard encryptors." {}))
|
:else (truss/ex-info! ":auto not supported for non-standard encryptors." {})
|
||||||
(do (throw (ex-info (str "Unrecognized :auto encryptor id: " encryptor-id)
|
(do (truss/ex-info! (str "Unrecognized :auto encryptor id: " encryptor-id)
|
||||||
{:encryptor-id encryptor-id})))))
|
{:encryptor-id encryptor-id}))))
|
||||||
|
|
||||||
(def ^:private err-msg-unknown-thaw-failure "Possible decryption/decompression error, unfrozen/damaged data, etc.")
|
(def ^:private err-msg-unknown-thaw-failure "Possible decryption/decompression error, unfrozen/damaged data, etc.")
|
||||||
(def ^:private err-msg-unrecognized-header "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?")
|
(def ^:private err-msg-unrecognized-header "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?")
|
||||||
|
|
||||||
(defn fast-thaw
|
(defn fast-thaw
|
||||||
"Like `thaw` but:
|
"Like `thaw` but:
|
||||||
- Drops all support for compression and encryption
|
|
||||||
- Supports only data frozen with `fast-freeze`
|
- Supports only data frozen with `fast-freeze`
|
||||||
|
- Drops all support for compression and encryption
|
||||||
|
|
||||||
Equivalent to (but a little faster than) `thaw` with opts:
|
Equivalent to (but a little faster than) `thaw` with opts:
|
||||||
- :compressor nil
|
{:no-header? true, :compressor nil, :encryptor nil}."
|
||||||
- :encryptor nil
|
|
||||||
- :no-header? true"
|
|
||||||
[^bytes ba]
|
[^bytes ba]
|
||||||
(let [dis (DataInputStream. (ByteArrayInputStream. ba))]
|
(let [dis (DataInputStream. (ByteArrayInputStream. ba))]
|
||||||
(with-cache (thaw-from-in! dis))))
|
(with-cache (thaw-from-in! dis))))
|
||||||
|
|
@ -1811,24 +1834,23 @@
|
||||||
(fn ex
|
(fn ex
|
||||||
([ msg] (ex nil msg))
|
([ msg] (ex nil msg))
|
||||||
([e msg]
|
([e msg]
|
||||||
(throw
|
(truss/ex-info! (str "Thaw failed. " msg)
|
||||||
(ex-info (str "Thaw failed. " msg)
|
{:opts
|
||||||
{:opts
|
(assoc opts
|
||||||
(assoc opts
|
:compressor compressor
|
||||||
:compressor compressor
|
:encryptor encryptor)
|
||||||
:encryptor encryptor)
|
|
||||||
|
|
||||||
:bindings
|
:bindings
|
||||||
(enc/assoc-some {}
|
(enc/assoc-some {}
|
||||||
'*freeze-fallback* *freeze-fallback*
|
'*freeze-fallback* *freeze-fallback*
|
||||||
'*final-freeze-fallback* *final-freeze-fallback*
|
'*final-freeze-fallback* *final-freeze-fallback*
|
||||||
'*auto-freeze-compressor* *auto-freeze-compressor*
|
'*auto-freeze-compressor* *auto-freeze-compressor*
|
||||||
'*custom-readers* *custom-readers*
|
'*custom-readers* *custom-readers*
|
||||||
'*incl-metadata?* *incl-metadata?*
|
'*incl-metadata?* *incl-metadata?*
|
||||||
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
|
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
|
||||||
'*thaw-xform* *thaw-xform*)}
|
'*thaw-xform* *thaw-xform*)}
|
||||||
|
|
||||||
e))))
|
e)))
|
||||||
|
|
||||||
thaw-data
|
thaw-data
|
||||||
(fn [data-ba compressor-id encryptor-id ex-fn]
|
(fn [data-ba compressor-id encryptor-id ex-fn]
|
||||||
|
|
@ -1918,8 +1940,9 @@
|
||||||
"Extends Nippy to support freezing of a custom type (ideally concrete) with
|
"Extends Nippy to support freezing of a custom type (ideally concrete) with
|
||||||
given id of form:
|
given id of form:
|
||||||
|
|
||||||
* Keyword - 2 byte overhead, keywords hashed to 16 bit id
|
* ℕ∈[1, 128] - 0 byte overhead. You are responsible for managing ids.
|
||||||
* ℕ∈[1, 128] - 0 byte overhead
|
* (Namespaced) keyword - 2 byte overhead. Keyword will be hashed to 16 bit int,
|
||||||
|
collisions will throw at compile-time.
|
||||||
|
|
||||||
NB: be careful about extending to interfaces, Ref. <http://goo.gl/6gGRlU>.
|
NB: be careful about extending to interfaces, Ref. <http://goo.gl/6gGRlU>.
|
||||||
|
|
||||||
|
|
@ -1984,7 +2007,7 @@
|
||||||
[{:keys [comparable?] :as opts}]
|
[{:keys [comparable?] :as opts}]
|
||||||
(let [rng (java.util.Random. 123456) ; Seeded for determinism
|
(let [rng (java.util.Random. 123456) ; Seeded for determinism
|
||||||
rand-nth (fn [coll] (nth coll (.nextInt rng (count coll))))
|
rand-nth (fn [coll] (nth coll (.nextInt rng (count coll))))
|
||||||
all
|
base
|
||||||
{:nil nil
|
{:nil nil
|
||||||
:true true
|
:true true
|
||||||
:false false
|
:false false
|
||||||
|
|
@ -2021,7 +2044,6 @@
|
||||||
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{[] ()}}] #{:a :b}}
|
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{[] ()}}] #{:a :b}}
|
||||||
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5 "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"] {} #{} [] ()]]]]]
|
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5 "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"] {} #{} [] ()]]]]]
|
||||||
|
|
||||||
:regex #"^(https?:)?//(www\?|\?)?"
|
|
||||||
:sorted-set (sorted-set 1 2 3 4 5)
|
:sorted-set (sorted-set 1 2 3 4 5)
|
||||||
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)
|
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)
|
||||||
:lazy-seq-empty (map identity ())
|
:lazy-seq-empty (map identity ())
|
||||||
|
|
@ -2029,12 +2051,10 @@
|
||||||
:queue (into clojure.lang.PersistentQueue/EMPTY [:a :b :c :d :e :f :g])
|
:queue (into clojure.lang.PersistentQueue/EMPTY [:a :b :c :d :e :f :g])
|
||||||
:queue-empty clojure.lang.PersistentQueue/EMPTY
|
:queue-empty clojure.lang.PersistentQueue/EMPTY
|
||||||
|
|
||||||
:uuid (java.util.UUID. 7232453380187312026 -7067939076204274491)
|
:uuid (java.util.UUID. 7232453380187312026 -7067939076204274491)
|
||||||
:uri (java.net.URI. "https://clojure.org")
|
:uri (java.net.URI. "https://clojure.org")
|
||||||
:defrecord (StressRecord. "data")
|
:defrecord (StressRecord. "data")
|
||||||
:deftype (StressType. "data")
|
:deftype (StressType. "data")
|
||||||
:bytes (byte-array [(byte 1) (byte 2) (byte 3)])
|
|
||||||
:objects (object-array [1 "two" {:data "data"}])
|
|
||||||
|
|
||||||
:util-date (java.util.Date. 1577884455500)
|
:util-date (java.util.Date. 1577884455500)
|
||||||
:sql-date (java.sql.Date. 1577884455500)
|
:sql-date (java.sql.Date. 1577884455500)
|
||||||
|
|
@ -2042,10 +2062,6 @@
|
||||||
:duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) ::skip)
|
:duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) ::skip)
|
||||||
:period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) ::skip)
|
:period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) ::skip)
|
||||||
|
|
||||||
:throwable (Throwable. "Msg")
|
|
||||||
:exception (Exception. "Msg")
|
|
||||||
:ex-info (ex-info "Msg" {:data "data"})
|
|
||||||
|
|
||||||
:many-longs (vec (repeatedly 512 #(rand-nth (range 10))))
|
:many-longs (vec (repeatedly 512 #(rand-nth (range 10))))
|
||||||
:many-doubles (vec (repeatedly 512 #(double (rand-nth (range 10)))))
|
:many-doubles (vec (repeatedly 512 #(double (rand-nth (range 10)))))
|
||||||
:many-strings (vec (repeatedly 512 #(rand-nth ["foo" "bar" "baz" "qux"])))
|
:many-strings (vec (repeatedly 512 #(rand-nth ["foo" "bar" "baz" "qux"])))
|
||||||
|
|
@ -2055,8 +2071,24 @@
|
||||||
(rand-nth ["foo" "bar" "baz" "qux" ]))))}]
|
(rand-nth ["foo" "bar" "baz" "qux" ]))))}]
|
||||||
|
|
||||||
(if comparable?
|
(if comparable?
|
||||||
(dissoc all :bytes :objects :throwable :exception :ex-info :regex)
|
base
|
||||||
(do all))))
|
(assoc base
|
||||||
|
:non-comparable
|
||||||
|
{:regex #"^(https?:)?//(www\?|\?)?"
|
||||||
|
:throwable (Throwable. "Msg")
|
||||||
|
:exception (Exception. "Msg")
|
||||||
|
:ex-info (ex-info "Msg" {:data "data"})
|
||||||
|
:arrays
|
||||||
|
{:boolean (boolean-array (mapv even? (range 32)))
|
||||||
|
:byte (byte-array (mapv byte (range 32)))
|
||||||
|
:short (short-array (mapv short (range 32)))
|
||||||
|
:int (int-array (mapv int (range 32)))
|
||||||
|
:long (long-array (mapv long (range 32)))
|
||||||
|
:float (float-array (mapv float (range 32)))
|
||||||
|
:double (double-array (mapv double (range 32)))
|
||||||
|
:char (char-array (mapv char (range 32)))
|
||||||
|
:str (into-array String (mapv str (range 32)))
|
||||||
|
:object (object-array (mapv vector (range 32)))}}))))
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
[(= (stress-data {:comparable? true}) (stress-data {:comparable? true}))
|
[(= (stress-data {:comparable? true}) (stress-data {:comparable? true}))
|
||||||
|
|
@ -2199,5 +2231,5 @@
|
||||||
(alter-var-root *thaw-serializable-allowlist* f) and/or
|
(alter-var-root *thaw-serializable-allowlist* f) and/or
|
||||||
(alter-var-root *freeze-serializable-allow-list* f) instead."
|
(alter-var-root *freeze-serializable-allow-list* f) instead."
|
||||||
[f]
|
[f]
|
||||||
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (enc/have set? old))))
|
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (truss/have set? old))))
|
||||||
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (enc/have set? old))))))
|
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (truss/have set? old))))))
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,8 @@
|
||||||
(ns ^:no-doc taoensso.nippy.compression
|
(ns ^:no-doc taoensso.nippy.compression
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require [taoensso.encore :as enc])
|
(:require
|
||||||
|
[taoensso.truss :as truss]
|
||||||
|
[taoensso.encore :as enc])
|
||||||
(:import
|
(:import
|
||||||
[java.nio ByteBuffer]
|
[java.nio ByteBuffer]
|
||||||
[java.io
|
[java.io
|
||||||
|
|
@ -156,7 +158,7 @@
|
||||||
(.read xzs ba 0 len-decomp)
|
(.read xzs ba 0 len-decomp)
|
||||||
(if (== -1 (.read xzs)) ; Good practice as extra safety measure
|
(if (== -1 (.read xzs)) ; Good practice as extra safety measure
|
||||||
nil
|
nil
|
||||||
(throw (ex-info "LZMA2 Decompress failed: corrupt data?" {:ba ba})))
|
(truss/ex-info! "LZMA2 Decompress failed: corrupt data?" {:ba ba}))
|
||||||
ba)))
|
ba)))
|
||||||
|
|
||||||
;;;; Public API
|
;;;; Public API
|
||||||
|
|
@ -174,8 +176,8 @@
|
||||||
(def lz4-compressor
|
(def lz4-compressor
|
||||||
"Default `LZ4` compressor:
|
"Default `LZ4` compressor:
|
||||||
- Compression ratio: `C` (0.58 on reference benchmark).
|
- Compression ratio: `C` (0.58 on reference benchmark).
|
||||||
- Compression speed: `A` (238 msecs on reference benchmark).
|
- Compression speed: `A` (240 msecs on reference benchmark).
|
||||||
- Decompression speed: `A+` (31 msecs on reference benchmark).
|
- Decompression speed: `A+` (30 msecs on reference benchmark).
|
||||||
|
|
||||||
Good general-purpose compressor, favours speed.
|
Good general-purpose compressor, favours speed.
|
||||||
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
||||||
|
|
@ -184,8 +186,8 @@
|
||||||
(def lzo-compressor
|
(def lzo-compressor
|
||||||
"Default `LZO` compressor:
|
"Default `LZO` compressor:
|
||||||
- Compression ratio: `C` (0.58 on reference benchmark).
|
- Compression ratio: `C` (0.58 on reference benchmark).
|
||||||
- Compression speed: `A` (216 msecs on reference benchmark).
|
- Compression speed: `A` (220 msecs on reference benchmark).
|
||||||
- Decompression speed: `A` (43 msecs on reference benchmark).
|
- Decompression speed: `A` (40 msecs on reference benchmark).
|
||||||
|
|
||||||
Good general-purpose compressor, favours speed.
|
Good general-purpose compressor, favours speed.
|
||||||
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
||||||
|
|
@ -195,7 +197,7 @@
|
||||||
"Default `LZMA2` compressor:
|
"Default `LZMA2` compressor:
|
||||||
- Compression ratio: `A+` (0.4 on reference benchmark).
|
- Compression ratio: `A+` (0.4 on reference benchmark).
|
||||||
- Compression speed: `E` (18.5 secs on reference benchmark).
|
- Compression speed: `E` (18.5 secs on reference benchmark).
|
||||||
- Decompression speed: `D` (11.8 secs on reference benchmark).
|
- Decompression speed: `D` (12 secs on reference benchmark).
|
||||||
|
|
||||||
Specialized compressor, strongly favours ratio.
|
Specialized compressor, strongly favours ratio.
|
||||||
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
||||||
|
|
@ -204,8 +206,8 @@
|
||||||
(enc/def* snappy-compressor
|
(enc/def* snappy-compressor
|
||||||
"Default `Snappy` compressor:
|
"Default `Snappy` compressor:
|
||||||
- Compression ratio: `C` (0.58 on reference benchmark).
|
- Compression ratio: `C` (0.58 on reference benchmark).
|
||||||
- Compression speed: `A+` (206 msecs on reference benchmark).
|
- Compression speed: `A+` (210 msecs on reference benchmark).
|
||||||
- Decompression speed: `B` (134 msecs on reference benchmark).
|
- Decompression speed: `B` (130 msecs on reference benchmark).
|
||||||
Good general-purpose compressor, favours speed.
|
Good general-purpose compressor, favours speed.
|
||||||
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
See `taoensso.nippy-benchmarks` for detailed comparative benchmarks."
|
||||||
(SnappyCompressor. false))
|
(SnappyCompressor. false))
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,9 @@
|
||||||
"Low-level crypto utils.
|
"Low-level crypto utils.
|
||||||
Private & alpha, very likely to change!"
|
Private & alpha, very likely to change!"
|
||||||
(:refer-clojure :exclude [rand-nth])
|
(:refer-clojure :exclude [rand-nth])
|
||||||
(:require [taoensso.encore :as enc]))
|
(:require
|
||||||
|
[taoensso.truss :as truss]
|
||||||
|
[taoensso.encore :as enc]))
|
||||||
|
|
||||||
;; Note that AES128 may be preferable to AES256 due to known attack
|
;; Note that AES128 may be preferable to AES256 due to known attack
|
||||||
;; vectors specific to AES256, Ref. <https://goo.gl/qU4CCV>
|
;; vectors specific to AES256, Ref. <https://goo.gl/qU4CCV>
|
||||||
|
|
@ -45,7 +47,7 @@
|
||||||
(defn take-ba ^bytes [n ^bytes ba] (java.util.Arrays/copyOf ba ^int n)) ; Pads if ba too small
|
(defn take-ba ^bytes [n ^bytes ba] (java.util.Arrays/copyOf ba ^int n)) ; Pads if ba too small
|
||||||
(defn utf8->ba ^bytes [^String s] (.getBytes s "UTF-8"))
|
(defn utf8->ba ^bytes [^String s] (.getBytes s "UTF-8"))
|
||||||
(defn- add-salt ^bytes [?salt-ba ba] (if ?salt-ba (enc/ba-concat ?salt-ba ba) ba))
|
(defn- add-salt ^bytes [?salt-ba ba] (if ?salt-ba (enc/ba-concat ?salt-ba ba) ba))
|
||||||
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (enc/have enc/bytes? utf8-or-ba)))
|
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (truss/have enc/bytes? utf8-or-ba)))
|
||||||
|
|
||||||
(comment (seq (pwd-as-ba "foo")))
|
(comment (seq (pwd-as-ba "foo")))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
(ns ^:no-doc taoensso.nippy.encryption
|
(ns ^:no-doc taoensso.nippy.encryption
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require
|
(:require
|
||||||
|
[taoensso.truss :as truss]
|
||||||
[taoensso.encore :as enc]
|
[taoensso.encore :as enc]
|
||||||
[taoensso.nippy.crypto :as crypto]))
|
[taoensso.nippy.crypto :as crypto]))
|
||||||
|
|
||||||
|
|
@ -14,11 +15,11 @@
|
||||||
(decrypt ^bytes [encryptor pwd ba]))
|
(decrypt ^bytes [encryptor pwd ba]))
|
||||||
|
|
||||||
(defn- throw-destructure-ex [typed-password]
|
(defn- throw-destructure-ex [typed-password]
|
||||||
(throw (ex-info
|
(truss/ex-info!
|
||||||
(str "Expected password form: "
|
(str "Expected password form: "
|
||||||
"[<#{:salted :cached}> <password-string>].\n "
|
"[<#{:salted :cached}> <password-string>].\n "
|
||||||
"See `aes128-encryptor` docstring for details!")
|
"See `aes128-encryptor` docstring for details!")
|
||||||
{:typed-password typed-password})))
|
{:typed-password typed-password}))
|
||||||
|
|
||||||
(defn- destructure-typed-pwd [typed-password]
|
(defn- destructure-typed-pwd [typed-password]
|
||||||
(if (vector? typed-password)
|
(if (vector? typed-password)
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,7 @@
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require
|
(:require
|
||||||
[clojure.string :as str]
|
[clojure.string :as str]
|
||||||
|
[taoensso.truss :as truss]
|
||||||
[taoensso.encore :as enc]))
|
[taoensso.encore :as enc]))
|
||||||
|
|
||||||
;;;; Fallback type tests
|
;;;; Fallback type tests
|
||||||
|
|
@ -63,7 +64,7 @@
|
||||||
(when x
|
(when x
|
||||||
(if (string? x)
|
(if (string? x)
|
||||||
(if (= x "") #{} (set (mapv str/trim (str/split x #"[,:]"))))
|
(if (= x "") #{} (set (mapv str/trim (str/split x #"[,:]"))))
|
||||||
(enc/have set? x))))
|
(truss/have set? x))))
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(mapv classname-set [nil #{"foo"} "" "foo, bar:baz"])
|
(mapv classname-set [nil #{"foo"} "" "foo, bar:baz"])
|
||||||
|
|
@ -151,25 +152,56 @@
|
||||||
See that function's docstring for more info."
|
See that function's docstring for more info."
|
||||||
[] (trim nmax (state_))))
|
[] (trim nmax (state_))))
|
||||||
|
|
||||||
|
(comment
|
||||||
|
(count (get-recorded-serializable-classes))
|
||||||
|
(enc/reduce-n
|
||||||
|
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
|
||||||
|
nil 0 1e5))
|
||||||
|
|
||||||
|
(let [compile
|
||||||
|
(enc/fmemoize
|
||||||
|
(fn [x]
|
||||||
|
(if (allow-and-record? x)
|
||||||
|
allow-and-record-any-serializable-class-unsafe
|
||||||
|
(enc/name-filter x))))
|
||||||
|
|
||||||
|
fn? fn?
|
||||||
|
conform?
|
||||||
|
(fn [x cn]
|
||||||
|
(if (fn? x)
|
||||||
|
(x cn) ; Intentionally uncached, can be handy
|
||||||
|
((compile x) cn)))]
|
||||||
|
|
||||||
|
(defn serializable-allowed? [allow-list class-name]
|
||||||
|
(conform? allow-list class-name)))
|
||||||
|
|
||||||
;;;; Release targeting
|
;;;; Release targeting
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(set! *print-length* nil)
|
(set! *print-length* nil)
|
||||||
(vec (sort (keys taoensso.nippy/public-types-spec)))
|
(vec (sort (keys taoensso.nippy/public-types-spec)))
|
||||||
|
|
||||||
;; To help support release targeting, we keep track of when new type ids are added
|
;; To help support release targeting, we track new type ids added over time
|
||||||
(let [id-history ; {<release> #{type-ids}}
|
(let [id-history ; {<release> #{type-ids}}
|
||||||
{340 ; v3.4.0 (2024-04-30), added 2
|
{350 ; v3.5.0 (2025-04-15), added 5x
|
||||||
;; New: map-entry meta-protocol-key
|
;; #{int-array-lg long-array-lg float-array-lg double-array-lg string-array-lg}
|
||||||
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
|
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104
|
||||||
|
105 106 107 108 109 110 111 112 113 114 115 116 117}
|
||||||
|
|
||||||
|
340 ; v3.4.0 (2024-04-30), added 2x
|
||||||
|
;; #{map-entry meta-protocol-key}
|
||||||
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104
|
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104
|
||||||
105 106 110 111 112 113 114 115}
|
105 106 110 111 112 113 114 115}
|
||||||
|
|
||||||
330 ; v3.3.0 (2023-10-11), added 11
|
330 ; v3.3.0 (2023-10-11), added 11x
|
||||||
;; New: long-pos-sm long-pos-md long-pos-lg long-neg-sm long-neg-md long-neg-lg
|
;; #{long-pos-sm long-pos-md long-pos-lg long-neg-sm long-neg-md long-neg-lg
|
||||||
;; str-sm* vec-sm* set-sm* map-sm* sql-date
|
;; str-sm* vec-sm* set-sm* map-sm* sql-date}
|
||||||
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
|
|
@ -182,8 +214,8 @@
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
81 82 83 84 85 86 90 91 100 101 102 105 106 110 111 112 113 114 115}
|
81 82 83 84 85 86 90 91 100 101 102 105 106 110 111 112 113 114 115}
|
||||||
|
|
||||||
313 ; v3.1.3 (2022-06-23), added 5
|
313 ; v3.1.3 (2022-06-23), added 5x
|
||||||
;; New: time-instant time-duration time-period kw-md sym-md
|
;; #{time-instant time-duration time-period kw-md sym-md}
|
||||||
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
|
|
@ -193,16 +225,15 @@
|
||||||
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 80
|
||||||
81 82 90 91 100 101 102 105 106 110 111 112 113 114 115}}]
|
81 82 90 91 100 101 102 105 106 110 111 112 113 114 115}}
|
||||||
|
|
||||||
(defn diff [new-release old-release]
|
diff
|
||||||
(vec (sort (clojure.set/difference (id-history new-release) (id-history old-release))))))
|
(fn [new-release old-release]
|
||||||
|
(vec (sort (clojure.set/difference (id-history new-release) (id-history old-release)))))]
|
||||||
|
|
||||||
(diff 340 330))
|
(diff 350 340)))
|
||||||
|
|
||||||
(let [;; Initially target compatibility with v3.2.0 (2020-07-18)
|
(let [target-release
|
||||||
;; Next release will target v3.4.0 (2024-04-30), etc.
|
|
||||||
target-release
|
|
||||||
(enc/get-env {:as :edn, :default 320}
|
(enc/get-env {:as :edn, :default 320}
|
||||||
:taoensso.nippy.target-release)
|
:taoensso.nippy.target-release)
|
||||||
|
|
||||||
|
|
@ -240,28 +271,3 @@
|
||||||
[min-release] (target>= min-release)))
|
[min-release] (target>= min-release)))
|
||||||
|
|
||||||
(comment (macroexpand '(target-release>= 340)))
|
(comment (macroexpand '(target-release>= 340)))
|
||||||
|
|
||||||
;;;
|
|
||||||
|
|
||||||
(comment
|
|
||||||
(count (get-recorded-serializable-classes))
|
|
||||||
(enc/reduce-n
|
|
||||||
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
|
|
||||||
nil 0 1e5))
|
|
||||||
|
|
||||||
(let [compile
|
|
||||||
(enc/fmemoize
|
|
||||||
(fn [x]
|
|
||||||
(if (allow-and-record? x)
|
|
||||||
allow-and-record-any-serializable-class-unsafe
|
|
||||||
(enc/name-filter x))))
|
|
||||||
|
|
||||||
fn? fn?
|
|
||||||
conform?
|
|
||||||
(fn [x cn]
|
|
||||||
(if (fn? x)
|
|
||||||
(x cn) ; Intentionally uncached, can be handy
|
|
||||||
((compile x) cn)))]
|
|
||||||
|
|
||||||
(defn serializable-allowed? [allow-list class-name]
|
|
||||||
(conform? allow-list class-name)))
|
|
||||||
|
|
|
||||||
|
|
@ -1,20 +1,18 @@
|
||||||
(ns taoensso.nippy.tools
|
(ns taoensso.nippy.tools
|
||||||
"Utils for community tools that want to add user-configurable Nippy support.
|
"Utils for community tools that want to add user-configurable Nippy support.
|
||||||
Used by Carmine, Faraday, etc."
|
Used by Carmine, Faraday, etc."
|
||||||
|
|
||||||
(:refer-clojure :exclude [binding])
|
|
||||||
(:require
|
(:require
|
||||||
[taoensso.encore :as enc :refer [binding]]
|
[taoensso.encore :as enc]
|
||||||
[taoensso.nippy :as nippy]))
|
[taoensso.nippy :as nippy]))
|
||||||
|
|
||||||
(def ^:dynamic *freeze-opts* nil)
|
(def ^:dynamic *freeze-opts* nil)
|
||||||
(def ^:dynamic *thaw-opts* nil)
|
(def ^:dynamic *thaw-opts* nil)
|
||||||
|
|
||||||
(do
|
(do
|
||||||
(defmacro with-freeze-opts [opts & body] `(binding [*freeze-opts* ~opts ] ~@body))
|
(defmacro with-freeze-opts [opts & body] `(binding [*freeze-opts* ~opts ] ~@body))
|
||||||
(defmacro with-freeze-opts+ [opts & body] `(binding [*freeze-opts* (enc/fast-merge *freeze-opts* ~opts)] ~@body))
|
(defmacro with-freeze-opts+ [opts & body] `(binding [*freeze-opts* (enc/merge *freeze-opts* ~opts)] ~@body))
|
||||||
(defmacro with-thaw-opts [opts & body] `(binding [*thaw-opts* ~opts ] ~@body))
|
(defmacro with-thaw-opts [opts & body] `(binding [*thaw-opts* ~opts ] ~@body))
|
||||||
(defmacro with-thaw-opts+ [opts & body] `(binding [*thaw-opts* (enc/fast-merge *thaw-opts* ~opts)] ~@body)))
|
(defmacro with-thaw-opts+ [opts & body] `(binding [*thaw-opts* (enc/merge *thaw-opts* ~opts)] ~@body)))
|
||||||
|
|
||||||
(deftype WrappedForFreezing [val opts])
|
(deftype WrappedForFreezing [val opts])
|
||||||
(defn wrapped-for-freezing? [x] (instance? WrappedForFreezing x))
|
(defn wrapped-for-freezing? [x] (instance? WrappedForFreezing x))
|
||||||
|
|
@ -27,7 +25,7 @@
|
||||||
See also `tools/freeze`."
|
See also `tools/freeze`."
|
||||||
([x ] (wrap-for-freezing x nil))
|
([x ] (wrap-for-freezing x nil))
|
||||||
([x wrap-opts]
|
([x wrap-opts]
|
||||||
(let [captured-opts (enc/fast-merge *freeze-opts* wrap-opts)] ; wrap > dynamic
|
(let [captured-opts (enc/merge *freeze-opts* wrap-opts)] ; wrap > dynamic
|
||||||
(if (instance? WrappedForFreezing x)
|
(if (instance? WrappedForFreezing x)
|
||||||
(let [^WrappedForFreezing x x]
|
(let [^WrappedForFreezing x x]
|
||||||
(if (= (.-opts x) captured-opts)
|
(if (= (.-opts x) captured-opts)
|
||||||
|
|
@ -46,13 +44,13 @@
|
||||||
See also `tools/wrap-for-freezing`."
|
See also `tools/wrap-for-freezing`."
|
||||||
([x ] (freeze x nil))
|
([x ] (freeze x nil))
|
||||||
([x default-opts]
|
([x default-opts]
|
||||||
(let [default-opts (get default-opts :default-opts default-opts) ; Back compatibility
|
(let [default-opts (get default-opts :default-opts default-opts) ; Back compatibility
|
||||||
active-opts (enc/fast-merge default-opts *freeze-opts*)] ; dynamic > default
|
active-opts (enc/merge default-opts *freeze-opts*)] ; dynamic > default
|
||||||
|
|
||||||
(if (instance? WrappedForFreezing x)
|
(if (instance? WrappedForFreezing x)
|
||||||
(let [^WrappedForFreezing x x]
|
(let [^WrappedForFreezing x x]
|
||||||
(nippy/freeze (.-val x) (enc/fast-merge active-opts (.-opts x)))) ; captured > active!
|
(nippy/freeze (.-val x) (enc/merge active-opts (.-opts x)))) ; captured > active!
|
||||||
(nippy/freeze x active-opts)))))
|
(nippy/freeze x active-opts)))))
|
||||||
|
|
||||||
(defn thaw
|
(defn thaw
|
||||||
"Like `nippy/thaw` but uses as options the following, merged in
|
"Like `nippy/thaw` but uses as options the following, merged in
|
||||||
|
|
@ -62,8 +60,8 @@
|
||||||
2. `tools/*thaw-opts*` dynamic value (default nil)."
|
2. `tools/*thaw-opts*` dynamic value (default nil)."
|
||||||
([ba ] (thaw ba nil))
|
([ba ] (thaw ba nil))
|
||||||
([ba default-opts]
|
([ba default-opts]
|
||||||
(let [default-opts (get default-opts :default-opts default-opts) ; Back compatibility
|
(let [default-opts (get default-opts :default-opts default-opts) ; Back compatibility
|
||||||
active-opts (enc/fast-merge default-opts *thaw-opts*)] ; dynamic > default
|
active-opts (enc/merge default-opts *thaw-opts*)] ; dynamic > default
|
||||||
|
|
||||||
(nippy/thaw ba active-opts))))
|
(nippy/thaw ba active-opts))))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,11 @@
|
||||||
[clojure.test.check :as tc]
|
[clojure.test.check :as tc]
|
||||||
[clojure.test.check.generators :as tc-gens]
|
[clojure.test.check.generators :as tc-gens]
|
||||||
[clojure.test.check.properties :as tc-props]
|
[clojure.test.check.properties :as tc-props]
|
||||||
|
[taoensso.truss :as truss :refer [throws?]]
|
||||||
[taoensso.encore :as enc :refer [ba=]]
|
[taoensso.encore :as enc :refer [ba=]]
|
||||||
[taoensso.nippy :as nippy :refer [freeze thaw]]
|
[taoensso.nippy :as nippy :refer [freeze thaw]]
|
||||||
|
[taoensso.nippy.impl :as impl]
|
||||||
|
[taoensso.nippy.tools :as tools]
|
||||||
[taoensso.nippy.compression :as compr]
|
[taoensso.nippy.compression :as compr]
|
||||||
[taoensso.nippy.crypto :as crypto]
|
[taoensso.nippy.crypto :as crypto]
|
||||||
[taoensso.nippy-benchmarks :as benchmarks]))
|
[taoensso.nippy-benchmarks :as benchmarks]))
|
||||||
|
|
@ -53,10 +56,6 @@
|
||||||
#(freeze % {:password [:salted "p"]}))
|
#(freeze % {:password [:salted "p"]}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
||||||
(let [d (nippy/stress-data {})]
|
|
||||||
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
|
|
||||||
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
|
|
||||||
|
|
||||||
(is (= test-data ((comp #(thaw % {:compressor nippy/lzma2-compressor})
|
(is (= test-data ((comp #(thaw % {:compressor nippy/lzma2-compressor})
|
||||||
#(freeze % {:compressor nippy/lzma2-compressor}))
|
#(freeze % {:compressor nippy/lzma2-compressor}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
@ -75,9 +74,9 @@
|
||||||
#(freeze % {:compressor nippy/zstd-compressor}))
|
#(freeze % {:compressor nippy/zstd-compressor}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
||||||
(is (enc/throws? Exception (thaw (freeze test-data {:password "malformed"}))))
|
(is (throws? Exception (thaw (freeze test-data {:password "malformed"}))))
|
||||||
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
||||||
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
||||||
|
|
||||||
(is
|
(is
|
||||||
(= "payload"
|
(= "payload"
|
||||||
|
|
@ -94,14 +93,23 @@
|
||||||
(let [n range-uint+] (= (thaw (freeze n)) n))
|
(let [n range-uint+] (= (thaw (freeze n)) n))
|
||||||
(let [n (- range-uint+)] (= (thaw (freeze n)) n))]))
|
(let [n (- range-uint+)] (= (thaw (freeze n)) n))]))
|
||||||
|
|
||||||
(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (fn []))))
|
(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (fn []))))
|
||||||
|
|
||||||
(testing "Clojure v1.10+ metadata protocol extensions"
|
(testing "Clojure v1.10+ metadata protocol extensions"
|
||||||
[(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
|
[(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
|
||||||
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
|
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
|
||||||
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
|
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
|
||||||
"Don't attach empty metadata")])
|
"Don't attach empty metadata")])
|
||||||
|
|
||||||
|
(let [d (nippy/stress-data {})]
|
||||||
|
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
|
||||||
|
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
|
||||||
|
|
||||||
|
(testing "Arrays"
|
||||||
|
(binding [nippy/*thaw-serializable-allowlist* nippy/default-freeze-serializable-allowlist]
|
||||||
|
(mapv (fn [[k aval]] (is (= (vec aval) (-> aval nippy/freeze nippy/thaw vec)) (name k)))
|
||||||
|
(get-in (nippy/stress-data {}) [:non-comparable :arrays]))))
|
||||||
|
|
||||||
(is (gen-test 1600 [gen-data] (= gen-data (thaw (freeze gen-data)))) "Generative")])
|
(is (gen-test 1600 [gen-data] (= gen-data (thaw (freeze gen-data)))) "Generative")])
|
||||||
|
|
||||||
;;;; Custom types & records
|
;;;; Custom types & records
|
||||||
|
|
@ -112,7 +120,7 @@
|
||||||
(deftest _types
|
(deftest _types
|
||||||
[(testing "Extend to custom type"
|
[(testing "Extend to custom type"
|
||||||
[(is
|
[(is
|
||||||
(enc/throws? Exception ; No thaw extension yet
|
(throws? Exception ; No thaw extension yet
|
||||||
(do
|
(do
|
||||||
(alter-var-root #'nippy/*custom-readers* (constantly {}))
|
(alter-var-root #'nippy/*custom-readers* (constantly {}))
|
||||||
(nippy/extend-freeze MyType 1 [x s]
|
(nippy/extend-freeze MyType 1 [x s]
|
||||||
|
|
@ -323,7 +331,7 @@
|
||||||
[(is (= nippy/*thaw-serializable-allowlist* #{"base.1" "base.2" "add.1" "add.2"})
|
[(is (= nippy/*thaw-serializable-allowlist* #{"base.1" "base.2" "add.1" "add.2"})
|
||||||
"JVM properties override initial allowlist values")
|
"JVM properties override initial allowlist values")
|
||||||
|
|
||||||
(is (enc/throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
|
(is (throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
|
||||||
"Can't freeze Serializable objects unless approved by allowlist")
|
"Can't freeze Serializable objects unless approved by allowlist")
|
||||||
|
|
||||||
(is (sem?
|
(is (sem?
|
||||||
|
|
@ -435,8 +443,8 @@
|
||||||
|
|
||||||
(is (= (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze []))) []) "rf not run on empty colls")
|
(is (= (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze []))) []) "rf not run on empty colls")
|
||||||
|
|
||||||
(let [ex (enc/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
|
(let [ex (truss/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
|
||||||
(is (= (-> ex enc/ex-cause enc/ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
|
(is (= (-> ex ex-cause ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
|
||||||
|
|
||||||
;;;; Compressors
|
;;;; Compressors
|
||||||
|
|
||||||
|
|
@ -453,7 +461,7 @@
|
||||||
(print ".") (flush)
|
(print ".") (flush)
|
||||||
(dotimes [_ 1000]
|
(dotimes [_ 1000]
|
||||||
(is
|
(is
|
||||||
(nil? (enc/catching (compr/decompress c (crypto/rand-bytes 1024))))
|
(nil? (truss/catching :all (compr/decompress c (crypto/rand-bytes 1024))))
|
||||||
"Decompression never crashes JVM, even against invalid data")))
|
"Decompression never crashes JVM, even against invalid data")))
|
||||||
(println)))
|
(println)))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -107,7 +107,10 @@ Deserialize it:
|
||||||
|
|
||||||
Couldn't be simpler!
|
Couldn't be simpler!
|
||||||
|
|
||||||
See also the lower-level [`freeze-to-out!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-freeze-to-out.21) and [`thaw-from-in!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-thaw-from-in.21) fns for operating on `DataOutput` and `DataInput` types directly.
|
# Streaming
|
||||||
|
|
||||||
|
- To serialize directly to a `java.io.DataInput`, see [`freeze-to-out!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-freeze-to-out.21).
|
||||||
|
- To deserialize directly from a `java.io.DataOutput`, see [`thaw-from-in!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-thaw-from-in.21).
|
||||||
|
|
||||||
# Encryption
|
# Encryption
|
||||||
|
|
||||||
|
|
|
||||||
30
wiki/2 Operational-considerations.md
Normal file
30
wiki/2 Operational-considerations.md
Normal file
|
|
@ -0,0 +1,30 @@
|
||||||
|
# Data longevity
|
||||||
|
|
||||||
|
Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.
|
||||||
|
|
||||||
|
But please note that the **converse is not generally true**:
|
||||||
|
|
||||||
|
- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
|
||||||
|
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)
|
||||||
|
|
||||||
|
# Rolling updates and rollback
|
||||||
|
|
||||||
|
From time to time, Nippy may introduce:
|
||||||
|
|
||||||
|
- Support for serializing **new types**
|
||||||
|
- Optimizations to the serialization of **pre-existing types**
|
||||||
|
|
||||||
|
To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4.1) will always introduce such changes over **two version releases**:
|
||||||
|
|
||||||
|
- Release 1: to add **read support** for the new types
|
||||||
|
- Release 2: to add **write support** for the new types
|
||||||
|
|
||||||
|
Starting from v3.4.1, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.
|
||||||
|
|
||||||
|
# Stability of byte output
|
||||||
|
|
||||||
|
It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.
|
||||||
|
|
||||||
|
However, I know that a small minority of users *do* have specialized needs in this area.
|
||||||
|
|
||||||
|
So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.
|
||||||
|
|
@ -1,4 +1,4 @@
|
||||||
> This article was kindly contributed by a Nippy user (@Outrovurt)
|
> This article is **community content** kindly contributed by a Nippy user (@Outrovurt)
|
||||||
|
|
||||||
This article describes a number of use cases where you need to make changes to your code which will have some impact on data you have already frozen using Nippy, and how best to manage each specific case. We will also discuss custom freezing and thawing.
|
This article describes a number of use cases where you need to make changes to your code which will have some impact on data you have already frozen using Nippy, and how best to manage each specific case. We will also discuss custom freezing and thawing.
|
||||||
|
|
||||||
Loading…
Reference in a new issue