Compare commits
No commits in common. "master" and "v3.5.0-RC1" have entirely different histories.
master
...
v3.5.0-RC1
10 changed files with 265 additions and 305 deletions
92
CHANGELOG.md
92
CHANGELOG.md
|
|
@ -2,77 +2,25 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# `v3.5.0` (2025-04-15)
|
|
||||||
|
|
||||||
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0)
|
|
||||||
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
|
||||||
|
|
||||||
This is a **general maintenance release** focused on updating dependencies and laying groundwork (read support) for new array types coming in Nippy v3.6.
|
|
||||||
|
|
||||||
It **drops support for Clojure v1.9** but should otherwise be a safe update from (at least) all recent versions of Nippy.
|
|
||||||
|
|
||||||
## Since `v3.5.0-RC1` (2024-10-28)
|
|
||||||
|
|
||||||
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
|
|
||||||
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
|
|
||||||
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
|
|
||||||
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
|
|
||||||
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
|
|
||||||
|
|
||||||
## Since `v3.4.2` (2024-05-26)
|
|
||||||
|
|
||||||
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
|
|
||||||
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
|
|
||||||
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
|
|
||||||
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
|
|
||||||
- \[new] [#175] Mark [cache](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#cache) feature as stable \[b217db5]
|
|
||||||
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
|
|
||||||
|
|
||||||
## Migration info
|
|
||||||
|
|
||||||
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
|
||||||
| :------------------------ | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
|
||||||
| `v3.5.0-RC1` (2024-10-28) | - | - | - |
|
|
||||||
| `v3.4.2` (2024-05-26) | - | - | - |
|
|
||||||
| `v3.4.1` (2024-05-02) | - | - | - |
|
|
||||||
| `v3.4.0` (2024-04-30) | - | **Yes** | - |
|
|
||||||
| `v3.3.0` (2023-10-11) | - | - | - |
|
|
||||||
| `v3.2.0` (2022-07-18) | - | - | - |
|
|
||||||
| `v3.1.3` (2022-06-23) | - | - | - |
|
|
||||||
|
|
||||||
> [1] Relevant only when Nippy introduces support for new types **AND** you plan to update Nippy with a **rolling update** (coexisting new and old versions).
|
|
||||||
|
|
||||||
If updating from older versions of Nippy, please see the relevant release notes.
|
|
||||||
|
|
||||||
As always:
|
|
||||||
|
|
||||||
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
|
|
||||||
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
|
|
||||||
- **Please report any unexpected problems** 🙏
|
|
||||||
|
|
||||||
\- [Peter Taoussanis](https://www.taoensso.com)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# `v3.5.0-RC1` (2024-10-28)
|
# `v3.5.0-RC1` (2024-10-28)
|
||||||
|
|
||||||
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0-RC1)
|
- 📦 **Dependency**: available [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0-RC1)
|
||||||
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
||||||
|
|
||||||
This is a **non-breaking maintenance release** that updates dependencies and includes read support for more native array types to be introduced in a future v3.6 release.
|
This is a **non-breaking maintenance release** that updates dependencies and includes read support for more native array types to be introduced in a future v3.6 release.
|
||||||
|
|
||||||
It should be safe to update from (at least) all recent versions of Nippy.
|
It should be safe to update from (at least) all recent versions of Nippy.
|
||||||
|
|
||||||
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Recommended update sequence [1]
|
||||||
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
| :-- | :-- | :-- | :--
|
||||||
| `v3.4.2` (2024-05-26) | - | - | - |
|
| `v3.4.2` (2024-05-26) | - | - | -
|
||||||
| `v3.4.1` (2024-05-02) | - | - | - |
|
| `v3.4.1` (2024-05-02) | - | - | -
|
||||||
| `v3.4.0` (2024-04-30) | - | Yes | - |
|
| `v3.4.0` (2024-04-30) | - | Yes | -
|
||||||
| `v3.3.0` (2023-10-11) | - | - | - |
|
| `v3.3.0` (2023-10-11) | - | - | -
|
||||||
| `v3.2.0` (2022-07-18) | - | - | - |
|
| `v3.2.0` (2022-07-18) | - | - | -
|
||||||
| `v3.1.3` (2022-06-23) | - | - | - |
|
| `v3.1.3` (2022-06-23) | - | - | -
|
||||||
|
|
||||||
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
|
> [1] Relevant only when introducing support for new types, to help with rolling updates
|
||||||
|
|
||||||
If updating from older versions of Nippy, please see the relevant release notes.
|
If updating from older versions of Nippy, please see the relevant release notes.
|
||||||
|
|
||||||
|
|
@ -88,22 +36,22 @@ As always:
|
||||||
|
|
||||||
# `v3.4.2` (2024-05-26)
|
# `v3.4.2` (2024-05-26)
|
||||||
|
|
||||||
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2)
|
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2).
|
||||||
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
|
> **Versioning**: Nippy uses [Break Versioning](https://www.taoensso.com/break-versioning).
|
||||||
|
|
||||||
⚠️ This release addresses a [**security vulnerability**](https://github.com/taoensso/nippy/security/advisories/GHSA-vw78-267v-588h) in Nippy's upstream compression library and is **recommended for all existing users**.
|
⚠️ This release addresses a [**security vulnerability**](https://github.com/taoensso/nippy/security/advisories/GHSA-vw78-267v-588h) in Nippy's upstream compression library and is **recommended for all existing users**.
|
||||||
|
|
||||||
It should be a **straight-forward and non-breaking update** for almost everyone:
|
It should be a **straight-forward and non-breaking update** for almost everyone:
|
||||||
|
|
||||||
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
|
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Recommended update sequence [1]
|
||||||
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
|
| :-- | :-- | :-- | :--
|
||||||
| `v3.4.1` (2024-05-02) | - | - | - |
|
| `v3.4.1` (2024-05-02) | - | - | -
|
||||||
| `v3.4.0` (2024-04-30) | - | Yes | - |
|
| `v3.4.0` (2024-04-30) | - | Yes | -
|
||||||
| `v3.3.0` (2023-10-11) | - | - | - |
|
| `v3.3.0` (2023-10-11) | - | - | -
|
||||||
| `v3.2.0` (2022-07-18) | - | - | - |
|
| `v3.2.0` (2022-07-18) | - | - | -
|
||||||
| `v3.1.3` (2022-06-23) | - | - | - |
|
| `v3.1.3` (2022-06-23) | - | - | -
|
||||||
|
|
||||||
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
|
> [1] Relevant only when introducing support for new types, to help with rolling updates
|
||||||
|
|
||||||
If updating from older versions of Nippy, please see the relevant release notes.
|
If updating from older versions of Nippy, please see the relevant release notes.
|
||||||
|
|
||||||
|
|
|
||||||
18
README.md
18
README.md
|
|
@ -1,9 +1,9 @@
|
||||||
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
|
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
|
||||||
[**API**][cljdoc] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
|
[**API**][cljdoc docs] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
|
||||||
|
|
||||||
# Nippy
|
# Nippy
|
||||||
|
|
||||||
### Fast serialization library for Clojure
|
### The fastest serialization library for Clojure
|
||||||
|
|
||||||
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
|
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
|
||||||
|
|
||||||
|
|
@ -13,7 +13,8 @@ It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](ht
|
||||||
|
|
||||||
## Latest release/s
|
## Latest release/s
|
||||||
|
|
||||||
- `2025-04-15` `v3.5.0`: [release info](../../releases/tag/v3.5.0)
|
- `2024-05-26` `v3.4.2` (stable): [release info](../../releases/tag/v3.4.2)
|
||||||
|
- `2024-10-28` `v3.5.0-RC1` (dev): [release info](../../releases/tag/v3.5.0-RC1)
|
||||||
|
|
||||||
[![Main tests][Main tests SVG]][Main tests URL]
|
[![Main tests][Main tests SVG]][Main tests URL]
|
||||||
[![Graal tests][Graal tests SVG]][Graal tests URL]
|
[![Graal tests][Graal tests SVG]][Graal tests URL]
|
||||||
|
|
@ -86,16 +87,16 @@ So starting with Nippy v3.4, Nippy's release notes will **always clearly indicat
|
||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
Nippy is fast! Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:
|
Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest results:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
PRs welcome to include other alternatives in the bench suite!
|
PRs welcome to include other alternatives in the [benchmark suite](../../blob/master/test/taoensso/nippy_benchmarks.clj)!
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [Wiki][GitHub wiki] (getting started, usage, etc.)
|
- [Wiki][GitHub wiki] (getting started, usage, etc.)
|
||||||
- API reference via [cljdoc][cljdoc]
|
- API reference: [cljdoc][cljdoc docs], [Codox][Codox docs]
|
||||||
|
|
||||||
## Funding
|
## Funding
|
||||||
|
|
||||||
|
|
@ -103,7 +104,7 @@ You can [help support][sponsor] continued work on this project, thank you!! 🙏
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Copyright © 2012-2025 [Peter Taoussanis][].
|
Copyright © 2012-2024 [Peter Taoussanis][].
|
||||||
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
||||||
|
|
||||||
<!-- Common -->
|
<!-- Common -->
|
||||||
|
|
@ -117,7 +118,8 @@ Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
|
||||||
|
|
||||||
<!-- Project -->
|
<!-- Project -->
|
||||||
|
|
||||||
[cljdoc]: https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy
|
[Codox docs]: https://taoensso.github.io/nippy/
|
||||||
|
[cljdoc docs]: https://cljdoc.org/d/com.taoensso/nippy/
|
||||||
|
|
||||||
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
|
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
|
||||||
[Clojars URL]: https://clojars.org/com.taoensso/nippy
|
[Clojars URL]: https://clojars.org/com.taoensso/nippy
|
||||||
|
|
|
||||||
30
project.clj
30
project.clj
|
|
@ -1,6 +1,6 @@
|
||||||
(defproject com.taoensso/nippy "3.5.0"
|
(defproject com.taoensso/nippy "3.5.0-RC1"
|
||||||
:author "Peter Taoussanis <https://www.taoensso.com>"
|
:author "Peter Taoussanis <https://www.taoensso.com>"
|
||||||
:description "Fast serialization library for Clojure"
|
:description "The fastest serialization library for Clojure"
|
||||||
:url "https://www.taoensso.com/nippy"
|
:url "https://www.taoensso.com/nippy"
|
||||||
|
|
||||||
:license
|
:license
|
||||||
|
|
@ -8,8 +8,8 @@
|
||||||
:url "https://www.eclipse.org/legal/epl-v10.html"}
|
:url "https://www.eclipse.org/legal/epl-v10.html"}
|
||||||
|
|
||||||
:dependencies
|
:dependencies
|
||||||
[[org.clojure/tools.reader "1.5.2"]
|
[[org.clojure/tools.reader "1.5.0"]
|
||||||
[com.taoensso/encore "3.142.0"]
|
[com.taoensso/encore "3.127.0"]
|
||||||
[org.tukaani/xz "1.10"]
|
[org.tukaani/xz "1.10"]
|
||||||
[io.airlift/aircompressor "2.0.2"]]
|
[io.airlift/aircompressor "2.0.2"]]
|
||||||
|
|
||||||
|
|
@ -17,10 +17,11 @@
|
||||||
|
|
||||||
:profiles
|
:profiles
|
||||||
{;; :default [:base :system :user :provided :dev]
|
{;; :default [:base :system :user :provided :dev]
|
||||||
:provided {:dependencies [[org.clojure/clojure "1.11.4"]]}
|
:provided {:dependencies [[org.clojure/clojure "1.11.3"]]}
|
||||||
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0"]]}
|
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0-alpha12"]]}
|
||||||
:c1.11 {:dependencies [[org.clojure/clojure "1.11.4"]]}
|
:c1.11 {:dependencies [[org.clojure/clojure "1.11.3"]]}
|
||||||
:c1.10 {:dependencies [[org.clojure/clojure "1.10.3"]]}
|
:c1.10 {:dependencies [[org.clojure/clojure "1.10.1"]]}
|
||||||
|
:c1.9 {:dependencies [[org.clojure/clojure "1.9.0"]]}
|
||||||
|
|
||||||
:graal-tests
|
:graal-tests
|
||||||
{:source-paths ["test"]
|
{:source-paths ["test"]
|
||||||
|
|
@ -37,9 +38,7 @@
|
||||||
"-Xms1024m" "-Xmx2048m"
|
"-Xms1024m" "-Xmx2048m"
|
||||||
"-Dtaoensso.elide-deprecated=true"
|
"-Dtaoensso.elide-deprecated=true"
|
||||||
"-Dtaoensso.nippy.thaw-serializable-allowlist-base=base.1, base.2"
|
"-Dtaoensso.nippy.thaw-serializable-allowlist-base=base.1, base.2"
|
||||||
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"
|
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"]
|
||||||
#_"-Dtaoensso.nippy.target-release=320"
|
|
||||||
#_"-Dtaoensso.nippy.target-release=350"]
|
|
||||||
|
|
||||||
:global-vars
|
:global-vars
|
||||||
{*warn-on-reflection* true
|
{*warn-on-reflection* true
|
||||||
|
|
@ -52,13 +51,18 @@
|
||||||
|
|
||||||
:plugins
|
:plugins
|
||||||
[[lein-pprint "1.3.2"]
|
[[lein-pprint "1.3.2"]
|
||||||
[lein-ancient "0.7.0"]]}}
|
[lein-ancient "0.7.0"]
|
||||||
|
[com.taoensso.forks/lein-codox "0.10.11"]]
|
||||||
|
|
||||||
|
:codox
|
||||||
|
{:language #{:clojure #_:clojurescript}
|
||||||
|
:base-language :clojure}}}
|
||||||
|
|
||||||
:aliases
|
:aliases
|
||||||
{"start-dev" ["with-profile" "+dev" "repl" ":headless"]
|
{"start-dev" ["with-profile" "+dev" "repl" ":headless"]
|
||||||
;; "build-once" ["do" ["clean"] ["cljsbuild" "once"]]
|
;; "build-once" ["do" ["clean"] ["cljsbuild" "once"]]
|
||||||
"deploy-lib" ["do" #_["build-once"] ["deploy" "clojars"] ["install"]]
|
"deploy-lib" ["do" #_["build-once"] ["deploy" "clojars"] ["install"]]
|
||||||
|
|
||||||
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10" "test"]
|
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10:+c1.9" "test"]
|
||||||
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
|
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
|
||||||
"test-all" ["do" ["clean"] ["test-clj"] #_["test-cljs"]]})
|
"test-all" ["do" ["clean"] ["test-clj"] #_["test-cljs"]]})
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,6 @@
|
||||||
(:require
|
(:require
|
||||||
[clojure.string :as str]
|
[clojure.string :as str]
|
||||||
[clojure.java.io :as jio]
|
[clojure.java.io :as jio]
|
||||||
[taoensso.truss :as truss]
|
|
||||||
[taoensso.encore :as enc]
|
[taoensso.encore :as enc]
|
||||||
[taoensso.nippy
|
[taoensso.nippy
|
||||||
[impl :as impl]
|
[impl :as impl]
|
||||||
|
|
@ -26,7 +25,7 @@
|
||||||
PersistentQueue PersistentTreeMap PersistentTreeSet PersistentList
|
PersistentQueue PersistentTreeMap PersistentTreeSet PersistentList
|
||||||
MapEntry LazySeq IRecord ISeq IType]))
|
MapEntry LazySeq IRecord ISeq IType]))
|
||||||
|
|
||||||
(enc/assert-min-encore-version [3 142 0])
|
(enc/assert-min-encore-version [3 127 0])
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(set! *unchecked-math* :warn-on-boxed)
|
(set! *unchecked-math* :warn-on-boxed)
|
||||||
|
|
@ -197,13 +196,13 @@
|
||||||
15 [:byte-array-md [[:elements {:read 2}]]]
|
15 [:byte-array-md [[:elements {:read 2}]]]
|
||||||
2 [:byte-array-lg [[:elements {:read 4}]]]
|
2 [:byte-array-lg [[:elements {:read 4}]]]
|
||||||
|
|
||||||
109 [:int-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
108 [:long-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
|
||||||
108 [:long-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
109 [:int-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
|
||||||
|
|
||||||
117 [:float-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
116 [:double-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
|
||||||
116 [:double-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
117 [:float-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
|
||||||
|
|
||||||
107 [:string-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
|
107 [:string-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
|
||||||
115 [:object-array-lg [[:elements {:read 4}]]]
|
115 [:object-array-lg [[:elements {:read 4}]]]
|
||||||
|
|
||||||
;; Serializable
|
;; Serializable
|
||||||
|
|
@ -366,26 +365,25 @@
|
||||||
;; For back compatibility (incl. Timbre's Carmine appender)
|
;; For back compatibility (incl. Timbre's Carmine appender)
|
||||||
(enc/defonce ^:dynamic ^:no-doc ^:deprecated *final-freeze-fallback* "Prefer `*freeze-fallback`." nil)
|
(enc/defonce ^:dynamic ^:no-doc ^:deprecated *final-freeze-fallback* "Prefer `*freeze-fallback`." nil)
|
||||||
(enc/defonce ^:dynamic *freeze-fallback*
|
(enc/defonce ^:dynamic *freeze-fallback*
|
||||||
"Controls Nippy's behaviour when trying to freeze an object with a type for
|
"Controls Nippy's behaviour when trying to freeze an item for which Nippy
|
||||||
which Nippy doesn't currently have a native (protocol) implementation.
|
doesn't currently have a native freeze/thaw implementation.
|
||||||
|
|
||||||
Possible values:
|
Possible values:
|
||||||
|
|
||||||
1. `nil` (no fallback, default)
|
1. `nil` (no freeze-fallback, default)
|
||||||
Tries the following in order:
|
Tries the following in order:
|
||||||
- Freeze with Java's `Serializable` interface if this seems possible
|
- Freeze with Java's `Serializable` interface if possible
|
||||||
- Freeze with Clojure's reader if this seems possible
|
- Freeze with Clojure's reader if possible
|
||||||
- Throw
|
- Throw
|
||||||
|
|
||||||
2. `:write-unfreezable` keyword
|
2. `:write-unfreezable` keyword
|
||||||
Tries the following in order:
|
Tries the following in order:
|
||||||
- Freeze with Java's `Serializable` interface if this seems possible
|
- Freeze with Java's `Serializable` interface if possible
|
||||||
- Freeze with Clojure's reader if this seems possible
|
- Freeze with Clojure's reader if possible
|
||||||
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
|
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
|
||||||
|
|
||||||
3. [Advanced] Custom (fn [^java.io.DataOutput out obj]) that must
|
3. [Advanced] Custom (fn [^java.io.DataOutput out item]) that must
|
||||||
write an appropriate object type id and payload to the given
|
write exactly one value to the given `DataOutput` stream"
|
||||||
`DataOutput` stream."
|
|
||||||
|
|
||||||
nil)
|
nil)
|
||||||
|
|
||||||
|
|
@ -442,16 +440,16 @@
|
||||||
;; Unfortunately quite a bit of complexity to do this safely
|
;; Unfortunately quite a bit of complexity to do this safely
|
||||||
|
|
||||||
(def default-freeze-serializable-allowlist
|
(def default-freeze-serializable-allowlist
|
||||||
"Allows *any* class name to be frozen using Java's `Serializable` interface.
|
"Allows *any* class-name to be frozen using Java's `Serializable` interface.
|
||||||
This is generally safe since RCE risk is present only when thawing.
|
This is generally safe since RCE risk is present only when thawing.
|
||||||
See also `*freeze-serializable-allowlist*`."
|
See also `*freeze-serializable-allowlist*`."
|
||||||
#{"*"})
|
#{"*"})
|
||||||
|
|
||||||
(def default-thaw-serializable-allowlist
|
(def default-thaw-serializable-allowlist
|
||||||
"A set of common safe class names to allow to be frozen using Java's
|
"A set of common safe class-names to allow to be frozen using Java's
|
||||||
`Serializable` interface. PRs welcome for additions.
|
`Serializable` interface. PRs welcome for additions.
|
||||||
See also `*thaw-serializable-allowlist*`."
|
See also `*thaw-serializable-allowlist*`."
|
||||||
#{"[Z" "[B" "[S" "[I" "[J" "[F" "[D" "[C" "[Ljava.lang.String;"
|
#{"[I" "[F" "[Z" "[B" "[C" "[D" "[S" "[J"
|
||||||
|
|
||||||
"java.lang.Throwable"
|
"java.lang.Throwable"
|
||||||
"java.lang.Exception"
|
"java.lang.Exception"
|
||||||
|
|
@ -515,10 +513,10 @@
|
||||||
|
|
||||||
Example allowlist values:
|
Example allowlist values:
|
||||||
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
|
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
|
||||||
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class names
|
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class-names
|
||||||
- `\"allow-and-record\"` ; Special value, see [2]
|
- `\"allow-and-record\"` ; Special value, see [2]
|
||||||
|
|
||||||
Note that class names in sets may contain \"*\" wildcards.
|
Note that class-names in sets may contain \"*\" wildcards.
|
||||||
|
|
||||||
Default allowlist values are:
|
Default allowlist values are:
|
||||||
- default-freeze-serializable-allowlist ; `{\"*\"}` => allow any class
|
- default-freeze-serializable-allowlist ; `{\"*\"}` => allow any class
|
||||||
|
|
@ -696,7 +694,7 @@
|
||||||
(sm-count? len) (do (write-id out id-kw-sm) (write-sm-count out len))
|
(sm-count? len) (do (write-id out id-kw-sm) (write-sm-count out len))
|
||||||
(md-count? len) (do (write-id out id-kw-md) (write-md-count out len))
|
(md-count? len) (do (write-id out id-kw-md) (write-md-count out len))
|
||||||
;; :else (do (write-id out id-kw-lg) (write-lg-count out len)) ; Unrealistic
|
;; :else (do (write-id out id-kw-lg) (write-lg-count out len)) ; Unrealistic
|
||||||
:else (truss/ex-info! "Keyword too long" {:name s}))
|
:else (throw (ex-info "Keyword too long" {:name s})))
|
||||||
|
|
||||||
(.write out ba 0 len)))
|
(.write out ba 0 len)))
|
||||||
|
|
||||||
|
|
@ -708,7 +706,7 @@
|
||||||
(sm-count? len) (do (write-id out id-sym-sm) (write-sm-count out len))
|
(sm-count? len) (do (write-id out id-sym-sm) (write-sm-count out len))
|
||||||
(md-count? len) (do (write-id out id-sym-md) (write-md-count out len))
|
(md-count? len) (do (write-id out id-sym-md) (write-md-count out len))
|
||||||
;; :else (do (write-id out id-sym-lg) (write-lg-count out len)) ; Unrealistic
|
;; :else (do (write-id out id-sym-lg) (write-lg-count out len)) ; Unrealistic
|
||||||
:else (truss/ex-info! "Symbol too long" {:name s}))
|
:else (throw (ex-info "Symbol too long" {:name s})))
|
||||||
|
|
||||||
(.write out ba 0 len)))
|
(.write out ba 0 len)))
|
||||||
|
|
||||||
|
|
@ -895,11 +893,8 @@
|
||||||
|
|
||||||
(-run! (fn [in] (-freeze-with-meta! in out)) s)))))
|
(-run! (fn [in] (-freeze-with-meta! in out)) s)))))
|
||||||
|
|
||||||
(defn- write-serializable [^DataOutput out x]
|
(defn- write-serializable [^DataOutput out x ^String class-name]
|
||||||
(when-debug (println (str "write-serializable: " (type x))))
|
(when-debug (println (str "write-serializable: " (type x))))
|
||||||
(when (and (instance? Serializable x) (not (fn? x)))
|
|
||||||
(let [class-name (.getName (class x))] ; Reflect
|
|
||||||
(when (freeze-serializable-allowed? class-name)
|
|
||||||
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
|
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
|
||||||
len (alength class-name-ba)]
|
len (alength class-name-ba)]
|
||||||
|
|
||||||
|
|
@ -907,7 +902,7 @@
|
||||||
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
|
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
|
||||||
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
|
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
|
||||||
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
||||||
:else (truss/ex-info! "Serializable class name too long" {:name class-name}))
|
:else (throw (ex-info "Serializable class name too long" {:name class-name})))
|
||||||
|
|
||||||
;; Legacy: write object directly to out.
|
;; Legacy: write object directly to out.
|
||||||
;; (.writeObject (ObjectOutputStream. out) x)
|
;; (.writeObject (ObjectOutputStream. out) x)
|
||||||
|
|
@ -916,24 +911,33 @@
|
||||||
;; We'll have object length during thaw, allowing us to skip readObject.
|
;; We'll have object length during thaw, allowing us to skip readObject.
|
||||||
(let [quarantined-ba (ByteArrayOutputStream.)]
|
(let [quarantined-ba (ByteArrayOutputStream.)]
|
||||||
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
|
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
|
||||||
(write-bytes out (.toByteArray quarantined-ba)))
|
(write-bytes out (.toByteArray quarantined-ba)))))
|
||||||
|
|
||||||
true)))))
|
|
||||||
|
|
||||||
(defn- write-readable [^DataOutput out x]
|
(defn- write-readable [^DataOutput out x]
|
||||||
(when-debug (println (str "write-readable: " (type x))))
|
(when-debug (println (str "write-readable: " (type x))))
|
||||||
(when (impl/seems-readable? x)
|
|
||||||
(let [edn (enc/pr-edn x)
|
(let [edn (enc/pr-edn x)
|
||||||
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
|
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
|
||||||
len (alength edn-ba)]
|
len (alength edn-ba)]
|
||||||
(enc/cond
|
(enc/cond
|
||||||
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
|
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
|
||||||
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
|
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
|
||||||
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))
|
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))))
|
||||||
true)))
|
|
||||||
|
|
||||||
(defn ^:deprecated try-write-serializable [out x] (truss/catching :all (write-serializable out x)))
|
(defn try-write-serializable [out x]
|
||||||
(defn ^:deprecated try-write-readable [out x] (truss/catching :all (write-readable out x)))
|
(when (and (instance? Serializable x) (not (fn? x)))
|
||||||
|
(try
|
||||||
|
(let [class-name (.getName (class x))] ; Reflect
|
||||||
|
(when (freeze-serializable-allowed? class-name)
|
||||||
|
(write-serializable out x class-name)
|
||||||
|
true))
|
||||||
|
(catch Throwable _ nil))))
|
||||||
|
|
||||||
|
(defn try-write-readable [out x]
|
||||||
|
(when (impl/seems-readable? x)
|
||||||
|
(try
|
||||||
|
(write-readable out x)
|
||||||
|
true
|
||||||
|
(catch Throwable _ nil))))
|
||||||
|
|
||||||
(defn- try-pr-edn [x]
|
(defn- try-pr-edn [x]
|
||||||
(try
|
(try
|
||||||
|
|
@ -951,6 +955,13 @@
|
||||||
:content (try-pr-edn x)}}
|
:content (try-pr-edn x)}}
|
||||||
out))
|
out))
|
||||||
|
|
||||||
|
(defn throw-unfreezable [x]
|
||||||
|
(let [t (type x)]
|
||||||
|
(throw
|
||||||
|
(ex-info (str "Unfreezable type: " t)
|
||||||
|
{:type t
|
||||||
|
:as-str (try-pr-edn x)}))))
|
||||||
|
|
||||||
;; Public `-freeze-with-meta!` with different arg order
|
;; Public `-freeze-with-meta!` with different arg order
|
||||||
(defn freeze-to-out!
|
(defn freeze-to-out!
|
||||||
"Serializes arg (any Clojure data type) to a DataOutput.
|
"Serializes arg (any Clojure data type) to a DataOutput.
|
||||||
|
|
@ -1029,7 +1040,7 @@
|
||||||
(when first-occurance? (-freeze-with-meta! x-val out)))
|
(when first-occurance? (-freeze-with-meta! x-val out)))
|
||||||
|
|
||||||
:else
|
:else
|
||||||
;; (truss/ex-info! "Max cache size exceeded" {:idx idx})
|
;; (throw (ex-info "Max cache size exceeded" {:idx idx}))
|
||||||
(-freeze-with-meta! x-val out) ; Just freeze uncached
|
(-freeze-with-meta! x-val out) ; Just freeze uncached
|
||||||
))
|
))
|
||||||
|
|
||||||
|
|
@ -1046,7 +1057,7 @@
|
||||||
(vswap! cache_ assoc idx x)
|
(vswap! cache_ assoc idx x)
|
||||||
x)
|
x)
|
||||||
v))
|
v))
|
||||||
(truss/ex-info! "Can't thaw without cache available. See `with-cache`." {})))))
|
(throw (ex-info "Can't thaw without cache available. See `with-cache`." {}))))))
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(thaw (freeze [(cache "foo") (cache "foo") (cache "foo")]))
|
(thaw (freeze [(cache "foo") (cache "foo") (cache "foo")]))
|
||||||
|
|
@ -1105,10 +1116,12 @@
|
||||||
(freezer (Class/forName "[Ljava.lang.Object;") nil true (write-array-lg out x (alength ^"[Ljava.lang.Object;" x) id-object-array-lg))
|
(freezer (Class/forName "[Ljava.lang.Object;") nil true (write-array-lg out x (alength ^"[Ljava.lang.Object;" x) id-object-array-lg))
|
||||||
|
|
||||||
(when (impl/target-release>= 350)
|
(when (impl/target-release>= 350)
|
||||||
(freezer (Class/forName "[I") nil true (write-array-lg out x (alength ^"[I" x) id-int-array-lg))
|
|
||||||
(freezer (Class/forName "[J") nil true (write-array-lg out x (alength ^"[J" x) id-long-array-lg))
|
(freezer (Class/forName "[J") nil true (write-array-lg out x (alength ^"[J" x) id-long-array-lg))
|
||||||
(freezer (Class/forName "[F") nil true (write-array-lg out x (alength ^"[F" x) id-float-array-lg))
|
(freezer (Class/forName "[I") nil true (write-array-lg out x (alength ^"[I" x) id-int-array-lg))
|
||||||
|
|
||||||
(freezer (Class/forName "[D") nil true (write-array-lg out x (alength ^"[D" x) id-double-array-lg))
|
(freezer (Class/forName "[D") nil true (write-array-lg out x (alength ^"[D" x) id-double-array-lg))
|
||||||
|
(freezer (Class/forName "[F") nil true (write-array-lg out x (alength ^"[F" x) id-float-array-lg))
|
||||||
|
|
||||||
(freezer (Class/forName "[Ljava.lang.String;") nil true (write-array-lg out x (alength ^"[Ljava.lang.String;" x) id-string-array-lg)))
|
(freezer (Class/forName "[Ljava.lang.String;") nil true (write-array-lg out x (alength ^"[Ljava.lang.String;" x) id-string-array-lg)))
|
||||||
|
|
||||||
(freezer PersistentQueue nil true (write-counted-coll out id-queue-lg x))
|
(freezer PersistentQueue nil true (write-counted-coll out id-queue-lg x))
|
||||||
|
|
@ -1128,7 +1141,7 @@
|
||||||
(sm-count? len) (do (write-id out id-record-sm) (write-bytes-sm out class-name-ba))
|
(sm-count? len) (do (write-id out id-record-sm) (write-bytes-sm out class-name-ba))
|
||||||
(md-count? len) (do (write-id out id-record-md) (write-bytes-md out class-name-ba))
|
(md-count? len) (do (write-id out id-record-md) (write-bytes-md out class-name-ba))
|
||||||
;; :else (do (write-id out id-record-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
;; :else (do (write-id out id-record-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
|
||||||
:else (truss/ex-info! "Record class name too long" {:name class-name}))
|
:else (throw (ex-info "Record class name too long" {:name class-name})))
|
||||||
|
|
||||||
(-freeze-without-meta! (into {} x) out)))
|
(-freeze-without-meta! (into {} x) out)))
|
||||||
|
|
||||||
|
|
@ -1183,20 +1196,13 @@
|
||||||
(write-unfreezable out x)))
|
(write-unfreezable out x)))
|
||||||
|
|
||||||
;; Without ff
|
;; Without ff
|
||||||
(enc/cond
|
(or
|
||||||
:let [[r1 e1] (try [(write-serializable out x)] (catch Throwable t [nil t]))], r1 r1
|
(try-write-serializable out x)
|
||||||
:let [[r2 e2] (try [(write-readable out x)] (catch Throwable t [nil t]))], r2 r2
|
(try-write-readable out x)
|
||||||
|
|
||||||
:if-let [fff *final-freeze-fallback*] (fff out x) ; Deprecated
|
(when-let [fff *final-freeze-fallback*] (fff out x) true) ; Deprecated
|
||||||
:else
|
|
||||||
(let [t (type x)]
|
(throw-unfreezable x)))))
|
||||||
(truss/ex-info! (str "Failed to freeze type: " t)
|
|
||||||
(enc/assoc-some
|
|
||||||
{:type t
|
|
||||||
:as-str (try-pr-edn x)}
|
|
||||||
{:serializable-error e1
|
|
||||||
:readable-error e2})
|
|
||||||
(or e1 e2)))))))
|
|
||||||
|
|
||||||
;;;;
|
;;;;
|
||||||
|
|
||||||
|
|
@ -1210,8 +1216,9 @@
|
||||||
(defn- wrap-header [data-ba head-meta]
|
(defn- wrap-header [data-ba head-meta]
|
||||||
(if-let [head-ba (get-head-ba head-meta)]
|
(if-let [head-ba (get-head-ba head-meta)]
|
||||||
(enc/ba-concat head-ba data-ba)
|
(enc/ba-concat head-ba data-ba)
|
||||||
(truss/ex-info! (str "Unrecognized header meta: " head-meta)
|
(throw
|
||||||
{:head-meta head-meta})))
|
(ex-info (str "Unrecognized header meta: " head-meta)
|
||||||
|
{:head-meta head-meta}))))
|
||||||
|
|
||||||
(comment (wrap-header (.getBytes "foo") {:compressor-id :lz4
|
(comment (wrap-header (.getBytes "foo") {:compressor-id :lz4
|
||||||
:encryptor-id nil}))
|
:encryptor-id nil}))
|
||||||
|
|
@ -1345,7 +1352,7 @@
|
||||||
id-byte-array-md (read-bytes in (read-md-count in))
|
id-byte-array-md (read-bytes in (read-md-count in))
|
||||||
id-byte-array-lg (read-bytes in (read-lg-count in)))))
|
id-byte-array-lg (read-bytes in (read-lg-count in)))))
|
||||||
|
|
||||||
(defmacro ^:private read-array [in thaw-type array-type array]
|
(defmacro ^:private read-array [in thaw-type array array-type]
|
||||||
(let [thawed-sym (with-meta 'thawed-sym {:tag thaw-type})
|
(let [thawed-sym (with-meta 'thawed-sym {:tag thaw-type})
|
||||||
array-sym (with-meta 'array-sym {:tag array-type})]
|
array-sym (with-meta 'array-sym {:tag array-type})]
|
||||||
`(let [~array-sym ~array]
|
`(let [~array-sym ~array]
|
||||||
|
|
@ -1374,7 +1381,7 @@
|
||||||
|
|
||||||
(defmacro ^:private editable? [coll] `(instance? clojure.lang.IEditableCollection ~coll))
|
(defmacro ^:private editable? [coll] `(instance? clojure.lang.IEditableCollection ~coll))
|
||||||
|
|
||||||
(defn- xform* [xform] (truss/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
|
(defn- xform* [xform] (enc/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
|
||||||
|
|
||||||
(let [rf! (fn rf! ([x] (persistent! x)) ([acc x] (conj! acc x)))
|
(let [rf! (fn rf! ([x] (persistent! x)) ([acc x] (conj! acc x)))
|
||||||
rf* (fn rf* ([x] x) ([acc x] (conj acc x)))]
|
rf* (fn rf* ([x] x) ([acc x] (conj acc x)))]
|
||||||
|
|
@ -1411,12 +1418,14 @@
|
||||||
(try
|
(try
|
||||||
(custom-reader in)
|
(custom-reader in)
|
||||||
(catch Exception e
|
(catch Exception e
|
||||||
(truss/ex-info!
|
(throw
|
||||||
|
(ex-info
|
||||||
(str "Reader exception for custom type id: " type-id)
|
(str "Reader exception for custom type id: " type-id)
|
||||||
{:type-id type-id, :prefixed? prefixed?} e)))
|
{:type-id type-id, :prefixed? prefixed?} e))))
|
||||||
(truss/ex-info!
|
(throw
|
||||||
|
(ex-info
|
||||||
(str "No reader provided for custom type id: " type-id)
|
(str "No reader provided for custom type id: " type-id)
|
||||||
{:type-id type-id, :prefixed? prefixed?})))
|
{:type-id type-id, :prefixed? prefixed?}))))
|
||||||
|
|
||||||
(defn- read-edn [edn]
|
(defn- read-edn [edn]
|
||||||
(try
|
(try
|
||||||
|
|
@ -1493,9 +1502,9 @@
|
||||||
[^DataInput in class-name]
|
[^DataInput in class-name]
|
||||||
(if (thaw-serializable-allowed? class-name)
|
(if (thaw-serializable-allowed? class-name)
|
||||||
(read-object in class-name)
|
(read-object in class-name)
|
||||||
(truss/ex-info! ; No way to skip bytes, so best we can do is throw
|
(throw ; No way to skip bytes, so best we can do is throw
|
||||||
"Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
|
(ex-info "Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
|
||||||
{:class-name class-name})))
|
{:class-name class-name}))))
|
||||||
|
|
||||||
(defn- read-record [in class-name]
|
(defn- read-record [in class-name]
|
||||||
(let [content (thaw-from-in! in)]
|
(let [content (thaw-from-in! in)]
|
||||||
|
|
@ -1598,14 +1607,14 @@
|
||||||
id-byte-array-md (read-bytes in (read-md-count in))
|
id-byte-array-md (read-bytes in (read-md-count in))
|
||||||
id-byte-array-lg (read-bytes in (read-lg-count in))
|
id-byte-array-lg (read-bytes in (read-lg-count in))
|
||||||
|
|
||||||
id-long-array-lg (read-array in long "[J" (long-array (read-lg-count in)))
|
id-long-array-lg (read-array in long (long-array (read-lg-count in)) "[J")
|
||||||
id-int-array-lg (read-array in int "[I" (int-array (read-lg-count in)))
|
id-int-array-lg (read-array in int (int-array (read-lg-count in)) "[I")
|
||||||
|
|
||||||
id-double-array-lg (read-array in double "[D" (double-array (read-lg-count in)))
|
id-double-array-lg (read-array in double (double-array (read-lg-count in)) "[D")
|
||||||
id-float-array-lg (read-array in float "[F" (float-array (read-lg-count in)))
|
id-float-array-lg (read-array in float (float-array (read-lg-count in)) "[F")
|
||||||
|
|
||||||
id-string-array-lg (read-array in String "[Ljava.lang.String;" (make-array String (read-lg-count in)))
|
id-string-array-lg (read-array in String (make-array String (read-lg-count in)) "[Ljava.lang.String;")
|
||||||
id-object-array-lg (read-array in Object "[Ljava.lang.Object;" (object-array (read-lg-count in)))
|
id-object-array-lg (read-array in Object (object-array (read-lg-count in)) "[Ljava.lang.Object;")
|
||||||
|
|
||||||
id-str-0 ""
|
id-str-0 ""
|
||||||
id-str-sm* (read-str in (read-sm-count* in))
|
id-str-sm* (read-str in (read-sm-count* in))
|
||||||
|
|
@ -1748,13 +1757,15 @@
|
||||||
|
|
||||||
(if (neg? type-id)
|
(if (neg? type-id)
|
||||||
(read-custom! in nil type-id) ; Unprefixed custom type
|
(read-custom! in nil type-id) ; Unprefixed custom type
|
||||||
(truss/ex-info!
|
(throw
|
||||||
|
(ex-info
|
||||||
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
|
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
|
||||||
{:type-id type-id})))
|
{:type-id type-id}))))
|
||||||
|
|
||||||
(catch Throwable t
|
(catch Throwable t
|
||||||
(truss/ex-info! (str "Thaw failed against type-id: " type-id)
|
(throw
|
||||||
{:type-id type-id} t)))))
|
(ex-info (str "Thaw failed against type-id: " type-id)
|
||||||
|
{:type-id type-id} t))))))
|
||||||
|
|
||||||
(let [head-sig head-sig] ; Not ^:const
|
(let [head-sig head-sig] ; Not ^:const
|
||||||
(defn- try-parse-header [^bytes ba]
|
(defn- try-parse-header [^bytes ba]
|
||||||
|
|
@ -1774,20 +1785,20 @@
|
||||||
:lzma2 lzma2-compressor
|
:lzma2 lzma2-compressor
|
||||||
:lz4 lz4-compressor
|
:lz4 lz4-compressor
|
||||||
:zstd zstd-compressor
|
:zstd zstd-compressor
|
||||||
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
|
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
|
||||||
:else (truss/ex-info! ":auto not supported for non-standard compressors." {})
|
:else (throw (ex-info ":auto not supported for non-standard compressors." {}))
|
||||||
(do (truss/ex-info! (str "Unrecognized :auto compressor id: " compressor-id)
|
(do (throw (ex-info (str "Unrecognized :auto compressor id: " compressor-id)
|
||||||
{:compressor-id compressor-id}))))
|
{:compressor-id compressor-id})))))
|
||||||
|
|
||||||
(defn- get-auto-encryptor [encryptor-id]
|
(defn- get-auto-encryptor [encryptor-id]
|
||||||
(case encryptor-id
|
(case encryptor-id
|
||||||
nil nil
|
nil nil
|
||||||
:aes128-gcm-sha512 aes128-gcm-encryptor
|
:aes128-gcm-sha512 aes128-gcm-encryptor
|
||||||
:aes128-cbc-sha512 aes128-cbc-encryptor
|
:aes128-cbc-sha512 aes128-cbc-encryptor
|
||||||
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
|
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
|
||||||
:else (truss/ex-info! ":auto not supported for non-standard encryptors." {})
|
:else (throw (ex-info ":auto not supported for non-standard encryptors." {}))
|
||||||
(do (truss/ex-info! (str "Unrecognized :auto encryptor id: " encryptor-id)
|
(do (throw (ex-info (str "Unrecognized :auto encryptor id: " encryptor-id)
|
||||||
{:encryptor-id encryptor-id}))))
|
{:encryptor-id encryptor-id})))))
|
||||||
|
|
||||||
(def ^:private err-msg-unknown-thaw-failure "Possible decryption/decompression error, unfrozen/damaged data, etc.")
|
(def ^:private err-msg-unknown-thaw-failure "Possible decryption/decompression error, unfrozen/damaged data, etc.")
|
||||||
(def ^:private err-msg-unrecognized-header "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?")
|
(def ^:private err-msg-unrecognized-header "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?")
|
||||||
|
|
@ -1834,7 +1845,8 @@
|
||||||
(fn ex
|
(fn ex
|
||||||
([ msg] (ex nil msg))
|
([ msg] (ex nil msg))
|
||||||
([e msg]
|
([e msg]
|
||||||
(truss/ex-info! (str "Thaw failed. " msg)
|
(throw
|
||||||
|
(ex-info (str "Thaw failed. " msg)
|
||||||
{:opts
|
{:opts
|
||||||
(assoc opts
|
(assoc opts
|
||||||
:compressor compressor
|
:compressor compressor
|
||||||
|
|
@ -1850,7 +1862,7 @@
|
||||||
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
|
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
|
||||||
'*thaw-xform* *thaw-xform*)}
|
'*thaw-xform* *thaw-xform*)}
|
||||||
|
|
||||||
e)))
|
e))))
|
||||||
|
|
||||||
thaw-data
|
thaw-data
|
||||||
(fn [data-ba compressor-id encryptor-id ex-fn]
|
(fn [data-ba compressor-id encryptor-id ex-fn]
|
||||||
|
|
@ -2007,7 +2019,7 @@
|
||||||
[{:keys [comparable?] :as opts}]
|
[{:keys [comparable?] :as opts}]
|
||||||
(let [rng (java.util.Random. 123456) ; Seeded for determinism
|
(let [rng (java.util.Random. 123456) ; Seeded for determinism
|
||||||
rand-nth (fn [coll] (nth coll (.nextInt rng (count coll))))
|
rand-nth (fn [coll] (nth coll (.nextInt rng (count coll))))
|
||||||
base
|
all
|
||||||
{:nil nil
|
{:nil nil
|
||||||
:true true
|
:true true
|
||||||
:false false
|
:false false
|
||||||
|
|
@ -2044,6 +2056,7 @@
|
||||||
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{[] ()}}] #{:a :b}}
|
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{[] ()}}] #{:a :b}}
|
||||||
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5 "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"] {} #{} [] ()]]]]]
|
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5 "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"] {} #{} [] ()]]]]]
|
||||||
|
|
||||||
|
:regex #"^(https?:)?//(www\?|\?)?"
|
||||||
:sorted-set (sorted-set 1 2 3 4 5)
|
:sorted-set (sorted-set 1 2 3 4 5)
|
||||||
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)
|
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)
|
||||||
:lazy-seq-empty (map identity ())
|
:lazy-seq-empty (map identity ())
|
||||||
|
|
@ -2056,12 +2069,31 @@
|
||||||
:defrecord (StressRecord. "data")
|
:defrecord (StressRecord. "data")
|
||||||
:deftype (StressType. "data")
|
:deftype (StressType. "data")
|
||||||
|
|
||||||
|
:bytes (byte-array [(byte 1) (byte 2) (byte 3)])
|
||||||
|
:objects (object-array [1 "two" {:data "data"}])
|
||||||
|
|
||||||
|
;; TODO (target-release>= 350)
|
||||||
|
;; :byte-array (byte-array [(byte 1) (byte 2) (byte 3) (byte 4)])
|
||||||
|
|
||||||
|
;; :long-array (long-array [1 2 3 4])
|
||||||
|
;; :int-array (int-array [1 2 3 4])
|
||||||
|
|
||||||
|
;; :double-array (double-array [1.5 2.5 3.5 4.5])
|
||||||
|
;; :float-array (float-array [1.5 2.5 3.5 4.5])
|
||||||
|
|
||||||
|
;; :object-array (object-array [1 "two" {:data "data"}])
|
||||||
|
;; :string-array (into-array String ["a" "b" "c"])
|
||||||
|
|
||||||
:util-date (java.util.Date. 1577884455500)
|
:util-date (java.util.Date. 1577884455500)
|
||||||
:sql-date (java.sql.Date. 1577884455500)
|
:sql-date (java.sql.Date. 1577884455500)
|
||||||
:instant (enc/compile-if java.time.Instant (java.time.Instant/parse "2020-01-01T13:14:15.50Z") ::skip)
|
:instant (enc/compile-if java.time.Instant (java.time.Instant/parse "2020-01-01T13:14:15.50Z") ::skip)
|
||||||
:duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) ::skip)
|
:duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) ::skip)
|
||||||
:period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) ::skip)
|
:period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) ::skip)
|
||||||
|
|
||||||
|
:throwable (Throwable. "Msg")
|
||||||
|
:exception (Exception. "Msg")
|
||||||
|
:ex-info (ex-info "Msg" {:data "data"})
|
||||||
|
|
||||||
:many-longs (vec (repeatedly 512 #(rand-nth (range 10))))
|
:many-longs (vec (repeatedly 512 #(rand-nth (range 10))))
|
||||||
:many-doubles (vec (repeatedly 512 #(double (rand-nth (range 10)))))
|
:many-doubles (vec (repeatedly 512 #(double (rand-nth (range 10)))))
|
||||||
:many-strings (vec (repeatedly 512 #(rand-nth ["foo" "bar" "baz" "qux"])))
|
:many-strings (vec (repeatedly 512 #(rand-nth ["foo" "bar" "baz" "qux"])))
|
||||||
|
|
@ -2071,24 +2103,8 @@
|
||||||
(rand-nth ["foo" "bar" "baz" "qux" ]))))}]
|
(rand-nth ["foo" "bar" "baz" "qux" ]))))}]
|
||||||
|
|
||||||
(if comparable?
|
(if comparable?
|
||||||
base
|
(dissoc all :bytes :objects :throwable :exception :ex-info :regex)
|
||||||
(assoc base
|
(do all))))
|
||||||
:non-comparable
|
|
||||||
{:regex #"^(https?:)?//(www\?|\?)?"
|
|
||||||
:throwable (Throwable. "Msg")
|
|
||||||
:exception (Exception. "Msg")
|
|
||||||
:ex-info (ex-info "Msg" {:data "data"})
|
|
||||||
:arrays
|
|
||||||
{:boolean (boolean-array (mapv even? (range 32)))
|
|
||||||
:byte (byte-array (mapv byte (range 32)))
|
|
||||||
:short (short-array (mapv short (range 32)))
|
|
||||||
:int (int-array (mapv int (range 32)))
|
|
||||||
:long (long-array (mapv long (range 32)))
|
|
||||||
:float (float-array (mapv float (range 32)))
|
|
||||||
:double (double-array (mapv double (range 32)))
|
|
||||||
:char (char-array (mapv char (range 32)))
|
|
||||||
:str (into-array String (mapv str (range 32)))
|
|
||||||
:object (object-array (mapv vector (range 32)))}}))))
|
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
[(= (stress-data {:comparable? true}) (stress-data {:comparable? true}))
|
[(= (stress-data {:comparable? true}) (stress-data {:comparable? true}))
|
||||||
|
|
@ -2231,5 +2247,5 @@
|
||||||
(alter-var-root *thaw-serializable-allowlist* f) and/or
|
(alter-var-root *thaw-serializable-allowlist* f) and/or
|
||||||
(alter-var-root *freeze-serializable-allow-list* f) instead."
|
(alter-var-root *freeze-serializable-allow-list* f) instead."
|
||||||
[f]
|
[f]
|
||||||
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (truss/have set? old))))
|
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (enc/have set? old))))
|
||||||
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (truss/have set? old))))))
|
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (enc/have set? old))))))
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,6 @@
|
||||||
(ns ^:no-doc taoensso.nippy.compression
|
(ns ^:no-doc taoensso.nippy.compression
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require
|
(:require [taoensso.encore :as enc])
|
||||||
[taoensso.truss :as truss]
|
|
||||||
[taoensso.encore :as enc])
|
|
||||||
(:import
|
(:import
|
||||||
[java.nio ByteBuffer]
|
[java.nio ByteBuffer]
|
||||||
[java.io
|
[java.io
|
||||||
|
|
@ -158,7 +156,7 @@
|
||||||
(.read xzs ba 0 len-decomp)
|
(.read xzs ba 0 len-decomp)
|
||||||
(if (== -1 (.read xzs)) ; Good practice as extra safety measure
|
(if (== -1 (.read xzs)) ; Good practice as extra safety measure
|
||||||
nil
|
nil
|
||||||
(truss/ex-info! "LZMA2 Decompress failed: corrupt data?" {:ba ba}))
|
(throw (ex-info "LZMA2 Decompress failed: corrupt data?" {:ba ba})))
|
||||||
ba)))
|
ba)))
|
||||||
|
|
||||||
;;;; Public API
|
;;;; Public API
|
||||||
|
|
|
||||||
|
|
@ -2,9 +2,7 @@
|
||||||
"Low-level crypto utils.
|
"Low-level crypto utils.
|
||||||
Private & alpha, very likely to change!"
|
Private & alpha, very likely to change!"
|
||||||
(:refer-clojure :exclude [rand-nth])
|
(:refer-clojure :exclude [rand-nth])
|
||||||
(:require
|
(:require [taoensso.encore :as enc]))
|
||||||
[taoensso.truss :as truss]
|
|
||||||
[taoensso.encore :as enc]))
|
|
||||||
|
|
||||||
;; Note that AES128 may be preferable to AES256 due to known attack
|
;; Note that AES128 may be preferable to AES256 due to known attack
|
||||||
;; vectors specific to AES256, Ref. <https://goo.gl/qU4CCV>
|
;; vectors specific to AES256, Ref. <https://goo.gl/qU4CCV>
|
||||||
|
|
@ -47,7 +45,7 @@
|
||||||
(defn take-ba ^bytes [n ^bytes ba] (java.util.Arrays/copyOf ba ^int n)) ; Pads if ba too small
|
(defn take-ba ^bytes [n ^bytes ba] (java.util.Arrays/copyOf ba ^int n)) ; Pads if ba too small
|
||||||
(defn utf8->ba ^bytes [^String s] (.getBytes s "UTF-8"))
|
(defn utf8->ba ^bytes [^String s] (.getBytes s "UTF-8"))
|
||||||
(defn- add-salt ^bytes [?salt-ba ba] (if ?salt-ba (enc/ba-concat ?salt-ba ba) ba))
|
(defn- add-salt ^bytes [?salt-ba ba] (if ?salt-ba (enc/ba-concat ?salt-ba ba) ba))
|
||||||
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (truss/have enc/bytes? utf8-or-ba)))
|
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (enc/have enc/bytes? utf8-or-ba)))
|
||||||
|
|
||||||
(comment (seq (pwd-as-ba "foo")))
|
(comment (seq (pwd-as-ba "foo")))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,6 @@
|
||||||
(ns ^:no-doc taoensso.nippy.encryption
|
(ns ^:no-doc taoensso.nippy.encryption
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require
|
(:require
|
||||||
[taoensso.truss :as truss]
|
|
||||||
[taoensso.encore :as enc]
|
[taoensso.encore :as enc]
|
||||||
[taoensso.nippy.crypto :as crypto]))
|
[taoensso.nippy.crypto :as crypto]))
|
||||||
|
|
||||||
|
|
@ -15,11 +14,11 @@
|
||||||
(decrypt ^bytes [encryptor pwd ba]))
|
(decrypt ^bytes [encryptor pwd ba]))
|
||||||
|
|
||||||
(defn- throw-destructure-ex [typed-password]
|
(defn- throw-destructure-ex [typed-password]
|
||||||
(truss/ex-info!
|
(throw (ex-info
|
||||||
(str "Expected password form: "
|
(str "Expected password form: "
|
||||||
"[<#{:salted :cached}> <password-string>].\n "
|
"[<#{:salted :cached}> <password-string>].\n "
|
||||||
"See `aes128-encryptor` docstring for details!")
|
"See `aes128-encryptor` docstring for details!")
|
||||||
{:typed-password typed-password}))
|
{:typed-password typed-password})))
|
||||||
|
|
||||||
(defn- destructure-typed-pwd [typed-password]
|
(defn- destructure-typed-pwd [typed-password]
|
||||||
(if (vector? typed-password)
|
(if (vector? typed-password)
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,6 @@
|
||||||
"Private, implementation detail."
|
"Private, implementation detail."
|
||||||
(:require
|
(:require
|
||||||
[clojure.string :as str]
|
[clojure.string :as str]
|
||||||
[taoensso.truss :as truss]
|
|
||||||
[taoensso.encore :as enc]))
|
[taoensso.encore :as enc]))
|
||||||
|
|
||||||
;;;; Fallback type tests
|
;;;; Fallback type tests
|
||||||
|
|
@ -64,7 +63,7 @@
|
||||||
(when x
|
(when x
|
||||||
(if (string? x)
|
(if (string? x)
|
||||||
(if (= x "") #{} (set (mapv str/trim (str/split x #"[,:]"))))
|
(if (= x "") #{} (set (mapv str/trim (str/split x #"[,:]"))))
|
||||||
(truss/have set? x))))
|
(enc/have set? x))))
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
(mapv classname-set [nil #{"foo"} "" "foo, bar:baz"])
|
(mapv classname-set [nil #{"foo"} "" "foo, bar:baz"])
|
||||||
|
|
@ -152,29 +151,6 @@
|
||||||
See that function's docstring for more info."
|
See that function's docstring for more info."
|
||||||
[] (trim nmax (state_))))
|
[] (trim nmax (state_))))
|
||||||
|
|
||||||
(comment
|
|
||||||
(count (get-recorded-serializable-classes))
|
|
||||||
(enc/reduce-n
|
|
||||||
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
|
|
||||||
nil 0 1e5))
|
|
||||||
|
|
||||||
(let [compile
|
|
||||||
(enc/fmemoize
|
|
||||||
(fn [x]
|
|
||||||
(if (allow-and-record? x)
|
|
||||||
allow-and-record-any-serializable-class-unsafe
|
|
||||||
(enc/name-filter x))))
|
|
||||||
|
|
||||||
fn? fn?
|
|
||||||
conform?
|
|
||||||
(fn [x cn]
|
|
||||||
(if (fn? x)
|
|
||||||
(x cn) ; Intentionally uncached, can be handy
|
|
||||||
((compile x) cn)))]
|
|
||||||
|
|
||||||
(defn serializable-allowed? [allow-list class-name]
|
|
||||||
(conform? allow-list class-name)))
|
|
||||||
|
|
||||||
;;;; Release targeting
|
;;;; Release targeting
|
||||||
|
|
||||||
(comment
|
(comment
|
||||||
|
|
@ -183,8 +159,8 @@
|
||||||
|
|
||||||
;; To help support release targeting, we track new type ids added over time
|
;; To help support release targeting, we track new type ids added over time
|
||||||
(let [id-history ; {<release> #{type-ids}}
|
(let [id-history ; {<release> #{type-ids}}
|
||||||
{350 ; v3.5.0 (2025-04-15), added 5x
|
{350 ; v3.5.0 (YYYY-MM-DD), added 5x
|
||||||
;; #{int-array-lg long-array-lg float-array-lg double-array-lg string-array-lg}
|
;; #{string-array-lg long-array-lg int-array-lg double-array-lg float-array-lg}
|
||||||
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
|
||||||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
|
||||||
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
|
||||||
|
|
@ -271,3 +247,28 @@
|
||||||
[min-release] (target>= min-release)))
|
[min-release] (target>= min-release)))
|
||||||
|
|
||||||
(comment (macroexpand '(target-release>= 340)))
|
(comment (macroexpand '(target-release>= 340)))
|
||||||
|
|
||||||
|
;;;
|
||||||
|
|
||||||
|
(comment
|
||||||
|
(count (get-recorded-serializable-classes))
|
||||||
|
(enc/reduce-n
|
||||||
|
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
|
||||||
|
nil 0 1e5))
|
||||||
|
|
||||||
|
(let [compile
|
||||||
|
(enc/fmemoize
|
||||||
|
(fn [x]
|
||||||
|
(if (allow-and-record? x)
|
||||||
|
allow-and-record-any-serializable-class-unsafe
|
||||||
|
(enc/name-filter x))))
|
||||||
|
|
||||||
|
fn? fn?
|
||||||
|
conform?
|
||||||
|
(fn [x cn]
|
||||||
|
(if (fn? x)
|
||||||
|
(x cn) ; Intentionally uncached, can be handy
|
||||||
|
((compile x) cn)))]
|
||||||
|
|
||||||
|
(defn serializable-allowed? [allow-list class-name]
|
||||||
|
(conform? allow-list class-name)))
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,10 @@
|
||||||
(ns taoensso.nippy.tools
|
(ns taoensso.nippy.tools
|
||||||
"Utils for community tools that want to add user-configurable Nippy support.
|
"Utils for community tools that want to add user-configurable Nippy support.
|
||||||
Used by Carmine, Faraday, etc."
|
Used by Carmine, Faraday, etc."
|
||||||
|
|
||||||
|
(:refer-clojure :exclude [binding])
|
||||||
(:require
|
(:require
|
||||||
[taoensso.encore :as enc]
|
[taoensso.encore :as enc :refer [binding]]
|
||||||
[taoensso.nippy :as nippy]))
|
[taoensso.nippy :as nippy]))
|
||||||
|
|
||||||
(def ^:dynamic *freeze-opts* nil)
|
(def ^:dynamic *freeze-opts* nil)
|
||||||
|
|
|
||||||
|
|
@ -4,11 +4,8 @@
|
||||||
[clojure.test.check :as tc]
|
[clojure.test.check :as tc]
|
||||||
[clojure.test.check.generators :as tc-gens]
|
[clojure.test.check.generators :as tc-gens]
|
||||||
[clojure.test.check.properties :as tc-props]
|
[clojure.test.check.properties :as tc-props]
|
||||||
[taoensso.truss :as truss :refer [throws?]]
|
|
||||||
[taoensso.encore :as enc :refer [ba=]]
|
[taoensso.encore :as enc :refer [ba=]]
|
||||||
[taoensso.nippy :as nippy :refer [freeze thaw]]
|
[taoensso.nippy :as nippy :refer [freeze thaw]]
|
||||||
[taoensso.nippy.impl :as impl]
|
|
||||||
[taoensso.nippy.tools :as tools]
|
|
||||||
[taoensso.nippy.compression :as compr]
|
[taoensso.nippy.compression :as compr]
|
||||||
[taoensso.nippy.crypto :as crypto]
|
[taoensso.nippy.crypto :as crypto]
|
||||||
[taoensso.nippy-benchmarks :as benchmarks]))
|
[taoensso.nippy-benchmarks :as benchmarks]))
|
||||||
|
|
@ -56,6 +53,10 @@
|
||||||
#(freeze % {:password [:salted "p"]}))
|
#(freeze % {:password [:salted "p"]}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
||||||
|
(let [d (nippy/stress-data {})]
|
||||||
|
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
|
||||||
|
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
|
||||||
|
|
||||||
(is (= test-data ((comp #(thaw % {:compressor nippy/lzma2-compressor})
|
(is (= test-data ((comp #(thaw % {:compressor nippy/lzma2-compressor})
|
||||||
#(freeze % {:compressor nippy/lzma2-compressor}))
|
#(freeze % {:compressor nippy/lzma2-compressor}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
@ -74,9 +75,9 @@
|
||||||
#(freeze % {:compressor nippy/zstd-compressor}))
|
#(freeze % {:compressor nippy/zstd-compressor}))
|
||||||
test-data)))
|
test-data)))
|
||||||
|
|
||||||
(is (throws? Exception (thaw (freeze test-data {:password "malformed"}))))
|
(is (enc/throws? Exception (thaw (freeze test-data {:password "malformed"}))))
|
||||||
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
||||||
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
|
||||||
|
|
||||||
(is
|
(is
|
||||||
(= "payload"
|
(= "payload"
|
||||||
|
|
@ -93,23 +94,14 @@
|
||||||
(let [n range-uint+] (= (thaw (freeze n)) n))
|
(let [n range-uint+] (= (thaw (freeze n)) n))
|
||||||
(let [n (- range-uint+)] (= (thaw (freeze n)) n))]))
|
(let [n (- range-uint+)] (= (thaw (freeze n)) n))]))
|
||||||
|
|
||||||
(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (fn []))))
|
(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (fn []))))
|
||||||
|
|
||||||
(testing "Clojure v1.10+ metadata protocol extensions"
|
(testing "Clojure v1.10+ metadata protocol extensions"
|
||||||
[(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
|
[(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
|
||||||
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
|
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
|
||||||
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
|
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
|
||||||
"Don't attach empty metadata")])
|
"Don't attach empty metadata")])
|
||||||
|
|
||||||
(let [d (nippy/stress-data {})]
|
|
||||||
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
|
|
||||||
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
|
|
||||||
|
|
||||||
(testing "Arrays"
|
|
||||||
(binding [nippy/*thaw-serializable-allowlist* nippy/default-freeze-serializable-allowlist]
|
|
||||||
(mapv (fn [[k aval]] (is (= (vec aval) (-> aval nippy/freeze nippy/thaw vec)) (name k)))
|
|
||||||
(get-in (nippy/stress-data {}) [:non-comparable :arrays]))))
|
|
||||||
|
|
||||||
(is (gen-test 1600 [gen-data] (= gen-data (thaw (freeze gen-data)))) "Generative")])
|
(is (gen-test 1600 [gen-data] (= gen-data (thaw (freeze gen-data)))) "Generative")])
|
||||||
|
|
||||||
;;;; Custom types & records
|
;;;; Custom types & records
|
||||||
|
|
@ -120,7 +112,7 @@
|
||||||
(deftest _types
|
(deftest _types
|
||||||
[(testing "Extend to custom type"
|
[(testing "Extend to custom type"
|
||||||
[(is
|
[(is
|
||||||
(throws? Exception ; No thaw extension yet
|
(enc/throws? Exception ; No thaw extension yet
|
||||||
(do
|
(do
|
||||||
(alter-var-root #'nippy/*custom-readers* (constantly {}))
|
(alter-var-root #'nippy/*custom-readers* (constantly {}))
|
||||||
(nippy/extend-freeze MyType 1 [x s]
|
(nippy/extend-freeze MyType 1 [x s]
|
||||||
|
|
@ -331,7 +323,7 @@
|
||||||
[(is (= nippy/*thaw-serializable-allowlist* #{"base.1" "base.2" "add.1" "add.2"})
|
[(is (= nippy/*thaw-serializable-allowlist* #{"base.1" "base.2" "add.1" "add.2"})
|
||||||
"JVM properties override initial allowlist values")
|
"JVM properties override initial allowlist values")
|
||||||
|
|
||||||
(is (throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
|
(is (enc/throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
|
||||||
"Can't freeze Serializable objects unless approved by allowlist")
|
"Can't freeze Serializable objects unless approved by allowlist")
|
||||||
|
|
||||||
(is (sem?
|
(is (sem?
|
||||||
|
|
@ -443,8 +435,8 @@
|
||||||
|
|
||||||
(is (= (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze []))) []) "rf not run on empty colls")
|
(is (= (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze []))) []) "rf not run on empty colls")
|
||||||
|
|
||||||
(let [ex (truss/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
|
(let [ex (enc/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
|
||||||
(is (= (-> ex ex-cause ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
|
(is (= (-> ex enc/ex-cause enc/ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
|
||||||
|
|
||||||
;;;; Compressors
|
;;;; Compressors
|
||||||
|
|
||||||
|
|
@ -461,7 +453,7 @@
|
||||||
(print ".") (flush)
|
(print ".") (flush)
|
||||||
(dotimes [_ 1000]
|
(dotimes [_ 1000]
|
||||||
(is
|
(is
|
||||||
(nil? (truss/catching :all (compr/decompress c (crypto/rand-bytes 1024))))
|
(nil? (enc/catching (compr/decompress c (crypto/rand-bytes 1024))))
|
||||||
"Decompression never crashes JVM, even against invalid data")))
|
"Decompression never crashes JVM, even against invalid data")))
|
||||||
(println)))
|
(println)))
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue