Compare commits

...

11 commits

Author SHA1 Message Date
Peter Taoussanis
ae4463a85c [nop] Drop use of deprecated enc/binding 2025-05-04 15:17:13 +02:00
Peter Taoussanis
1b99516177 [doc] Misc improvements 2025-04-15 10:42:51 +02:00
Peter Taoussanis
e68c2d56fe v3.5.0 (2025-04-15) 2025-04-15 08:58:10 +02:00
Peter Taoussanis
bfba59483a [new] Add string array type to default thaw-serializable-allowlist 2025-04-14 23:03:17 +02:00
Peter Taoussanis
f7bb2824ac [new] Add value tests for non-comparable types 2025-04-14 23:03:17 +02:00
Peter Taoussanis
8d62dc2826 [new] Use Truss exceptions on errors
Ref. https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#*ctx*
2025-04-14 23:03:17 +02:00
Peter Taoussanis
da57206b0d [mod] Drop official Clojure v1.9 support
Clojure v1.9  was released December 8,  2017 (~7 years ago)
Clojure v1.10 was released December 17, 2018 (~6 years ago)

Official support continues for v1.10, v1.11, v1.12.
2025-04-14 23:03:17 +02:00
Peter Taoussanis
e0f49ced5a [nop] Bump deps 2025-04-14 23:03:17 +02:00
Peter Taoussanis
8d107650cd [new] [#184] Incl. cause on non-native freeze failures
Before this commit:

  - When freezing an item WITHOUT a native Nippy implementation,
    Nippy may try to use (1) Java Serializable or (2) Clojure's reader.
    If these also fail, an ex-info will be thrown.
    The ex-info does NOT include any info about possible exceptions
    from (1) or (2).

After this commit:

  - The thrown ex-info now includes info about possible exceptions
    from (1) and (2). These can be useful, e.g. when indicating
    an OOM error, etc.
2025-04-14 23:03:17 +02:00
Peter Taoussanis
1026ea0ae7 [doc] Clarify *freeze-fallback* docstring 2025-04-14 23:03:17 +02:00
Peter Taoussanis
c92457025f [nop] Housekeeping 2025-04-14 23:03:17 +02:00
10 changed files with 305 additions and 265 deletions

View file

@ -2,25 +2,77 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
---
# `v3.5.0` (2025-04-15)
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0)
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
This is a **general maintenance release** focused on updating dependencies and laying groundwork (read support) for new array types coming in Nippy v3.6.
It **drops support for Clojure v1.9** but should otherwise be a safe update from (at least) all recent versions of Nippy.
## Since `v3.5.0-RC1` (2024-10-28)
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
## Since `v3.4.2` (2024-05-26)
- **\[mod]** Drop official Clojure v1.9 support \[da57206]
- \[new] Add string array type to default [thaw-serializable-allowlist](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#*thaw-serializable-allowlist*) \[bfba594]
- \[new] Use [Truss exceptions](https://cljdoc.org/d/com.taoensso/truss/CURRENT/api/taoensso.truss#ex-info) on errors \[8d62dc2]
- \[new] [#184] Incl. cause on non-native freeze failures \[8d10765]
- \[new] [#175] Mark [cache](https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy#cache) feature as stable \[b217db5]
- \[doc] Clarify `*freeze-fallback*` docstring \[1026ea0]
## Migration info
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
| :------------------------ | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
| `v3.5.0-RC1` (2024-10-28) | - | - | - |
| `v3.4.2` (2024-05-26) | - | - | - |
| `v3.4.1` (2024-05-02) | - | - | - |
| `v3.4.0` (2024-04-30) | - | **Yes** | - |
| `v3.3.0` (2023-10-11) | - | - | - |
| `v3.2.0` (2022-07-18) | - | - | - |
| `v3.1.3` (2022-06-23) | - | - | - |
> [1] Relevant only when Nippy introduces support for new types **AND** you plan to update Nippy with a **rolling update** (coexisting new and old versions).
If updating from older versions of Nippy, please see the relevant release notes.
As always:
- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
- **Please report any unexpected problems** 🙏
\- [Peter Taoussanis](https://www.taoensso.com)
---
# `v3.5.0-RC1` (2024-10-28)
- 📦 **Dependency**: available [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0-RC1)
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.5.0-RC1)
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
This is a **non-breaking maintenance release** that updates dependencies and includes read support for more native array types to be introduced in a future v3.6 release.
It should be safe to update from (at least) all recent versions of Nippy.
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Recommended update sequence [1]
| :-- | :-- | :-- | :--
| `v3.4.2` (2024-05-26) | - | - | -
| `v3.4.1` (2024-05-02) | - | - | -
| `v3.4.0` (2024-04-30) | - | Yes | -
| `v3.3.0` (2023-10-11) | - | - | -
| `v3.2.0` (2022-07-18) | - | - | -
| `v3.1.3` (2022-06-23) | - | - | -
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
| `v3.4.2` (2024-05-26) | - | - | - |
| `v3.4.1` (2024-05-02) | - | - | - |
| `v3.4.0` (2024-04-30) | - | Yes | - |
| `v3.3.0` (2023-10-11) | - | - | - |
| `v3.2.0` (2022-07-18) | - | - | - |
| `v3.1.3` (2022-06-23) | - | - | - |
> [1] Relevant only when introducing support for new types, to help with rolling updates
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
If updating from older versions of Nippy, please see the relevant release notes.
@ -36,22 +88,22 @@ As always:
# `v3.4.2` (2024-05-26)
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2).
> **Versioning**: Nippy uses [Break Versioning](https://www.taoensso.com/break-versioning).
- **Dependency**: [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2)
- **Versioning**: [Break Versioning](https://www.taoensso.com/break-versioning)
⚠️ This release addresses a [**security vulnerability**](https://github.com/taoensso/nippy/security/advisories/GHSA-vw78-267v-588h) in Nippy's upstream compression library and is **recommended for all existing users**.
It should be a **straight-forward and non-breaking update** for almost everyone:
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Recommended update sequence [1]
| :-- | :-- | :-- | :--
| `v3.4.1` (2024-05-02) | - | - | -
| `v3.4.0` (2024-04-30) | - | Yes | -
| `v3.3.0` (2023-10-11) | - | - | -
| `v3.2.0` (2022-07-18) | - | - | -
| `v3.1.3` (2022-06-23) | - | - | -
| Updating from Nippy | Changes to API? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | Rolling update sequence [1] |
| :-------------------- | :-------------- | :---------------------------------------------------------------------------------------------------------------------- | :-------------------------- |
| `v3.4.1` (2024-05-02) | - | - | - |
| `v3.4.0` (2024-04-30) | - | Yes | - |
| `v3.3.0` (2023-10-11) | - | - | - |
| `v3.2.0` (2022-07-18) | - | - | - |
| `v3.1.3` (2022-06-23) | - | - | - |
> [1] Relevant only when introducing support for new types, to help with rolling updates
> [1] Relevant only when introducing support for new types, and for users that do rolling updates
If updating from older versions of Nippy, please see the relevant release notes.

View file

@ -1,9 +1,9 @@
<a href="https://www.taoensso.com/clojure" title="More stuff by @ptaoussanis at www.taoensso.com"><img src="https://www.taoensso.com/open-source.png" alt="Taoensso open source" width="340"/></a>
[**API**][cljdoc docs] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
[**API**][cljdoc] | [**Wiki**][GitHub wiki] | [Latest releases](#latest-releases) | [Get support][GitHub issues]
# Nippy
### The fastest serialization library for Clojure
### Fast serialization library for Clojure
Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).
@ -13,8 +13,7 @@ It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](ht
## Latest release/s
- `2024-05-26` `v3.4.2` (stable): [release info](../../releases/tag/v3.4.2)
- `2024-10-28` `v3.5.0-RC1` (dev): [release info](../../releases/tag/v3.5.0-RC1)
- `2025-04-15` `v3.5.0`: [release info](../../releases/tag/v3.5.0)
[![Main tests][Main tests SVG]][Main tests URL]
[![Graal tests][Graal tests SVG]][Graal tests URL]
@ -87,16 +86,16 @@ So starting with Nippy v3.4, Nippy's release notes will **always clearly indicat
## Performance
Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest results:
Nippy is fast! Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:
![benchmarks-png](../../raw/master/benchmarks.png)
PRs welcome to include other alternatives in the [benchmark suite](../../blob/master/test/taoensso/nippy_benchmarks.clj)!
PRs welcome to include other alternatives in the bench suite!
## Documentation
- [Wiki][GitHub wiki] (getting started, usage, etc.)
- API reference: [cljdoc][cljdoc docs], [Codox][Codox docs]
- API reference via [cljdoc][cljdoc]
## Funding
@ -104,7 +103,7 @@ You can [help support][sponsor] continued work on this project, thank you!! 🙏
## License
Copyright &copy; 2012-2024 [Peter Taoussanis][].
Copyright &copy; 2012-2025 [Peter Taoussanis][].
Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
<!-- Common -->
@ -118,8 +117,7 @@ Licensed under [EPL 1.0](LICENSE.txt) (same as Clojure).
<!-- Project -->
[Codox docs]: https://taoensso.github.io/nippy/
[cljdoc docs]: https://cljdoc.org/d/com.taoensso/nippy/
[cljdoc]: https://cljdoc.org/d/com.taoensso/nippy/CURRENT/api/taoensso.nippy
[Clojars SVG]: https://img.shields.io/clojars/v/com.taoensso/nippy.svg
[Clojars URL]: https://clojars.org/com.taoensso/nippy

View file

@ -1,6 +1,6 @@
(defproject com.taoensso/nippy "3.5.0-RC1"
(defproject com.taoensso/nippy "3.5.0"
:author "Peter Taoussanis <https://www.taoensso.com>"
:description "The fastest serialization library for Clojure"
:description "Fast serialization library for Clojure"
:url "https://www.taoensso.com/nippy"
:license
@ -8,8 +8,8 @@
:url "https://www.eclipse.org/legal/epl-v10.html"}
:dependencies
[[org.clojure/tools.reader "1.5.0"]
[com.taoensso/encore "3.127.0"]
[[org.clojure/tools.reader "1.5.2"]
[com.taoensso/encore "3.142.0"]
[org.tukaani/xz "1.10"]
[io.airlift/aircompressor "2.0.2"]]
@ -17,11 +17,10 @@
:profiles
{;; :default [:base :system :user :provided :dev]
:provided {:dependencies [[org.clojure/clojure "1.11.3"]]}
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0-alpha12"]]}
:c1.11 {:dependencies [[org.clojure/clojure "1.11.3"]]}
:c1.10 {:dependencies [[org.clojure/clojure "1.10.1"]]}
:c1.9 {:dependencies [[org.clojure/clojure "1.9.0"]]}
:provided {:dependencies [[org.clojure/clojure "1.11.4"]]}
:c1.12 {:dependencies [[org.clojure/clojure "1.12.0"]]}
:c1.11 {:dependencies [[org.clojure/clojure "1.11.4"]]}
:c1.10 {:dependencies [[org.clojure/clojure "1.10.3"]]}
:graal-tests
{:source-paths ["test"]
@ -38,7 +37,9 @@
"-Xms1024m" "-Xmx2048m"
"-Dtaoensso.elide-deprecated=true"
"-Dtaoensso.nippy.thaw-serializable-allowlist-base=base.1, base.2"
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"]
"-Dtaoensso.nippy.thaw-serializable-allowlist-add=add.1 , add.2"
#_"-Dtaoensso.nippy.target-release=320"
#_"-Dtaoensso.nippy.target-release=350"]
:global-vars
{*warn-on-reflection* true
@ -51,18 +52,13 @@
:plugins
[[lein-pprint "1.3.2"]
[lein-ancient "0.7.0"]
[com.taoensso.forks/lein-codox "0.10.11"]]
:codox
{:language #{:clojure #_:clojurescript}
:base-language :clojure}}}
[lein-ancient "0.7.0"]]}}
:aliases
{"start-dev" ["with-profile" "+dev" "repl" ":headless"]
;; "build-once" ["do" ["clean"] ["cljsbuild" "once"]]
"deploy-lib" ["do" #_["build-once"] ["deploy" "clojars"] ["install"]]
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10:+c1.9" "test"]
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
"test-clj" ["with-profile" "+c1.12:+c1.11:+c1.10" "test"]
;; "test-cljs" ["with-profile" "+c1.12" "cljsbuild" "test"]
"test-all" ["do" ["clean"] ["test-clj"] #_["test-cljs"]]})

View file

@ -4,6 +4,7 @@
(:require
[clojure.string :as str]
[clojure.java.io :as jio]
[taoensso.truss :as truss]
[taoensso.encore :as enc]
[taoensso.nippy
[impl :as impl]
@ -25,7 +26,7 @@
PersistentQueue PersistentTreeMap PersistentTreeSet PersistentList
MapEntry LazySeq IRecord ISeq IType]))
(enc/assert-min-encore-version [3 127 0])
(enc/assert-min-encore-version [3 142 0])
(comment
(set! *unchecked-math* :warn-on-boxed)
@ -196,13 +197,13 @@
15 [:byte-array-md [[:elements {:read 2}]]]
2 [:byte-array-lg [[:elements {:read 4}]]]
108 [:long-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
109 [:int-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
109 [:int-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
108 [:long-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
116 [:double-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
117 [:float-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
117 [:float-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
116 [:double-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
107 [:string-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (YYYY-MM-DD)
107 [:string-array-lg [[:elements {:read 4}]]] ; Added v3.5.0 (2025-04-15)
115 [:object-array-lg [[:elements {:read 4}]]]
;; Serializable
@ -365,25 +366,26 @@
;; For back compatibility (incl. Timbre's Carmine appender)
(enc/defonce ^:dynamic ^:no-doc ^:deprecated *final-freeze-fallback* "Prefer `*freeze-fallback`." nil)
(enc/defonce ^:dynamic *freeze-fallback*
"Controls Nippy's behaviour when trying to freeze an item for which Nippy
doesn't currently have a native freeze/thaw implementation.
"Controls Nippy's behaviour when trying to freeze an object with a type for
which Nippy doesn't currently have a native (protocol) implementation.
Possible values:
1. `nil` (no freeze-fallback, default)
1. `nil` (no fallback, default)
Tries the following in order:
- Freeze with Java's `Serializable` interface if possible
- Freeze with Clojure's reader if possible
- Freeze with Java's `Serializable` interface if this seems possible
- Freeze with Clojure's reader if this seems possible
- Throw
2. `:write-unfreezable` keyword
Tries the following in order:
- Freeze with Java's `Serializable` interface if possible
- Freeze with Clojure's reader if possible
- Freeze with Java's `Serializable` interface if this seems possible
- Freeze with Clojure's reader if this seems possible
- Freeze a {:nippy/unfreezable {:type _}} placeholder value
3. [Advanced] Custom (fn [^java.io.DataOutput out item]) that must
write exactly one value to the given `DataOutput` stream"
3. [Advanced] Custom (fn [^java.io.DataOutput out obj]) that must
write an appropriate object type id and payload to the given
`DataOutput` stream."
nil)
@ -440,16 +442,16 @@
;; Unfortunately quite a bit of complexity to do this safely
(def default-freeze-serializable-allowlist
"Allows *any* class-name to be frozen using Java's `Serializable` interface.
"Allows *any* class name to be frozen using Java's `Serializable` interface.
This is generally safe since RCE risk is present only when thawing.
See also `*freeze-serializable-allowlist*`."
#{"*"})
(def default-thaw-serializable-allowlist
"A set of common safe class-names to allow to be frozen using Java's
"A set of common safe class names to allow to be frozen using Java's
`Serializable` interface. PRs welcome for additions.
See also `*thaw-serializable-allowlist*`."
#{"[I" "[F" "[Z" "[B" "[C" "[D" "[S" "[J"
#{"[Z" "[B" "[S" "[I" "[J" "[F" "[D" "[C" "[Ljava.lang.String;"
"java.lang.Throwable"
"java.lang.Exception"
@ -513,10 +515,10 @@
Example allowlist values:
- `(fn allow-class? [class-name] true)` ; Arbitrary predicate fn
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class-names
- `#{\"java.lang.Throwable\", \"clojure.lang.*\"}` ; Set of class names
- `\"allow-and-record\"` ; Special value, see [2]
Note that class-names in sets may contain \"*\" wildcards.
Note that class names in sets may contain \"*\" wildcards.
Default allowlist values are:
- default-freeze-serializable-allowlist ; `{\"*\"}` => allow any class
@ -694,7 +696,7 @@
(sm-count? len) (do (write-id out id-kw-sm) (write-sm-count out len))
(md-count? len) (do (write-id out id-kw-md) (write-md-count out len))
;; :else (do (write-id out id-kw-lg) (write-lg-count out len)) ; Unrealistic
:else (throw (ex-info "Keyword too long" {:name s})))
:else (truss/ex-info! "Keyword too long" {:name s}))
(.write out ba 0 len)))
@ -706,7 +708,7 @@
(sm-count? len) (do (write-id out id-sym-sm) (write-sm-count out len))
(md-count? len) (do (write-id out id-sym-md) (write-md-count out len))
;; :else (do (write-id out id-sym-lg) (write-lg-count out len)) ; Unrealistic
:else (throw (ex-info "Symbol too long" {:name s})))
:else (truss/ex-info! "Symbol too long" {:name s}))
(.write out ba 0 len)))
@ -893,51 +895,45 @@
(-run! (fn [in] (-freeze-with-meta! in out)) s)))))
(defn- write-serializable [^DataOutput out x ^String class-name]
(defn- write-serializable [^DataOutput out x]
(when-debug (println (str "write-serializable: " (type x))))
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
len (alength class-name-ba)]
(when (and (instance? Serializable x) (not (fn? x)))
(let [class-name (.getName (class x))] ; Reflect
(when (freeze-serializable-allowed? class-name)
(let [class-name-ba (.getBytes class-name StandardCharsets/UTF_8)
len (alength class-name-ba)]
(enc/cond
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
:else (throw (ex-info "Serializable class name too long" {:name class-name})))
(enc/cond
(sm-count? len) (do (write-id out id-sz-quarantined-sm) (write-bytes-sm out class-name-ba))
(md-count? len) (do (write-id out id-sz-quarantined-md) (write-bytes-md out class-name-ba))
;; :else (do (write-id out id-sz-quarantined-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
:else (truss/ex-info! "Serializable class name too long" {:name class-name}))
;; Legacy: write object directly to out.
;; (.writeObject (ObjectOutputStream. out) x)
;; Legacy: write object directly to out.
;; (.writeObject (ObjectOutputStream. out) x)
;; Quarantined: write object to ba, then ba to out.
;; We'll have object length during thaw, allowing us to skip readObject.
(let [quarantined-ba (ByteArrayOutputStream.)]
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
(write-bytes out (.toByteArray quarantined-ba)))))
;; Quarantined: write object to ba, then ba to out.
;; We'll have object length during thaw, allowing us to skip readObject.
(let [quarantined-ba (ByteArrayOutputStream.)]
(.writeObject (ObjectOutputStream. (DataOutputStream. quarantined-ba)) x)
(write-bytes out (.toByteArray quarantined-ba)))
true)))))
(defn- write-readable [^DataOutput out x]
(when-debug (println (str "write-readable: " (type x))))
(let [edn (enc/pr-edn x)
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
len (alength edn-ba)]
(enc/cond
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))))
(defn try-write-serializable [out x]
(when (and (instance? Serializable x) (not (fn? x)))
(try
(let [class-name (.getName (class x))] ; Reflect
(when (freeze-serializable-allowed? class-name)
(write-serializable out x class-name)
true))
(catch Throwable _ nil))))
(defn try-write-readable [out x]
(when (impl/seems-readable? x)
(try
(write-readable out x)
true
(catch Throwable _ nil))))
(let [edn (enc/pr-edn x)
edn-ba (.getBytes ^String edn StandardCharsets/UTF_8)
len (alength edn-ba)]
(enc/cond
(sm-count? len) (do (write-id out id-reader-sm) (write-bytes-sm out edn-ba))
(md-count? len) (do (write-id out id-reader-md) (write-bytes-md out edn-ba))
:else (do (write-id out id-reader-lg) (write-bytes-lg out edn-ba)))
true)))
(defn ^:deprecated try-write-serializable [out x] (truss/catching :all (write-serializable out x)))
(defn ^:deprecated try-write-readable [out x] (truss/catching :all (write-readable out x)))
(defn- try-pr-edn [x]
(try
@ -955,13 +951,6 @@
:content (try-pr-edn x)}}
out))
(defn throw-unfreezable [x]
(let [t (type x)]
(throw
(ex-info (str "Unfreezable type: " t)
{:type t
:as-str (try-pr-edn x)}))))
;; Public `-freeze-with-meta!` with different arg order
(defn freeze-to-out!
"Serializes arg (any Clojure data type) to a DataOutput.
@ -1040,7 +1029,7 @@
(when first-occurance? (-freeze-with-meta! x-val out)))
:else
;; (throw (ex-info "Max cache size exceeded" {:idx idx}))
;; (truss/ex-info! "Max cache size exceeded" {:idx idx})
(-freeze-with-meta! x-val out) ; Just freeze uncached
))
@ -1057,7 +1046,7 @@
(vswap! cache_ assoc idx x)
x)
v))
(throw (ex-info "Can't thaw without cache available. See `with-cache`." {}))))))
(truss/ex-info! "Can't thaw without cache available. See `with-cache`." {})))))
(comment
(thaw (freeze [(cache "foo") (cache "foo") (cache "foo")]))
@ -1116,12 +1105,10 @@
(freezer (Class/forName "[Ljava.lang.Object;") nil true (write-array-lg out x (alength ^"[Ljava.lang.Object;" x) id-object-array-lg))
(when (impl/target-release>= 350)
(freezer (Class/forName "[J") nil true (write-array-lg out x (alength ^"[J" x) id-long-array-lg))
(freezer (Class/forName "[I") nil true (write-array-lg out x (alength ^"[I" x) id-int-array-lg))
(freezer (Class/forName "[D") nil true (write-array-lg out x (alength ^"[D" x) id-double-array-lg))
(freezer (Class/forName "[J") nil true (write-array-lg out x (alength ^"[J" x) id-long-array-lg))
(freezer (Class/forName "[F") nil true (write-array-lg out x (alength ^"[F" x) id-float-array-lg))
(freezer (Class/forName "[D") nil true (write-array-lg out x (alength ^"[D" x) id-double-array-lg))
(freezer (Class/forName "[Ljava.lang.String;") nil true (write-array-lg out x (alength ^"[Ljava.lang.String;" x) id-string-array-lg)))
(freezer PersistentQueue nil true (write-counted-coll out id-queue-lg x))
@ -1141,7 +1128,7 @@
(sm-count? len) (do (write-id out id-record-sm) (write-bytes-sm out class-name-ba))
(md-count? len) (do (write-id out id-record-md) (write-bytes-md out class-name-ba))
;; :else (do (write-id out id-record-lg) (write-bytes-md out class-name-ba)) ; Unrealistic
:else (throw (ex-info "Record class name too long" {:name class-name})))
:else (truss/ex-info! "Record class name too long" {:name class-name}))
(-freeze-without-meta! (into {} x) out)))
@ -1196,13 +1183,20 @@
(write-unfreezable out x)))
;; Without ff
(or
(try-write-serializable out x)
(try-write-readable out x)
(enc/cond
:let [[r1 e1] (try [(write-serializable out x)] (catch Throwable t [nil t]))], r1 r1
:let [[r2 e2] (try [(write-readable out x)] (catch Throwable t [nil t]))], r2 r2
(when-let [fff *final-freeze-fallback*] (fff out x) true) ; Deprecated
(throw-unfreezable x)))))
:if-let [fff *final-freeze-fallback*] (fff out x) ; Deprecated
:else
(let [t (type x)]
(truss/ex-info! (str "Failed to freeze type: " t)
(enc/assoc-some
{:type t
:as-str (try-pr-edn x)}
{:serializable-error e1
:readable-error e2})
(or e1 e2)))))))
;;;;
@ -1216,9 +1210,8 @@
(defn- wrap-header [data-ba head-meta]
(if-let [head-ba (get-head-ba head-meta)]
(enc/ba-concat head-ba data-ba)
(throw
(ex-info (str "Unrecognized header meta: " head-meta)
{:head-meta head-meta}))))
(truss/ex-info! (str "Unrecognized header meta: " head-meta)
{:head-meta head-meta})))
(comment (wrap-header (.getBytes "foo") {:compressor-id :lz4
:encryptor-id nil}))
@ -1352,7 +1345,7 @@
id-byte-array-md (read-bytes in (read-md-count in))
id-byte-array-lg (read-bytes in (read-lg-count in)))))
(defmacro ^:private read-array [in thaw-type array array-type]
(defmacro ^:private read-array [in thaw-type array-type array]
(let [thawed-sym (with-meta 'thawed-sym {:tag thaw-type})
array-sym (with-meta 'array-sym {:tag array-type})]
`(let [~array-sym ~array]
@ -1381,7 +1374,7 @@
(defmacro ^:private editable? [coll] `(instance? clojure.lang.IEditableCollection ~coll))
(defn- xform* [xform] (enc/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
(defn- xform* [xform] (truss/catching-xform {:error/msg "Error thrown via `*thaw-xform*`"} xform))
(let [rf! (fn rf! ([x] (persistent! x)) ([acc x] (conj! acc x)))
rf* (fn rf* ([x] x) ([acc x] (conj acc x)))]
@ -1418,14 +1411,12 @@
(try
(custom-reader in)
(catch Exception e
(throw
(ex-info
(str "Reader exception for custom type id: " type-id)
{:type-id type-id, :prefixed? prefixed?} e))))
(throw
(ex-info
(str "No reader provided for custom type id: " type-id)
{:type-id type-id, :prefixed? prefixed?}))))
(truss/ex-info!
(str "Reader exception for custom type id: " type-id)
{:type-id type-id, :prefixed? prefixed?} e)))
(truss/ex-info!
(str "No reader provided for custom type id: " type-id)
{:type-id type-id, :prefixed? prefixed?})))
(defn- read-edn [edn]
(try
@ -1502,9 +1493,9 @@
[^DataInput in class-name]
(if (thaw-serializable-allowed? class-name)
(read-object in class-name)
(throw ; No way to skip bytes, so best we can do is throw
(ex-info "Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
{:class-name class-name}))))
(truss/ex-info! ; No way to skip bytes, so best we can do is throw
"Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed. This is a security feature. See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!"
{:class-name class-name})))
(defn- read-record [in class-name]
(let [content (thaw-from-in! in)]
@ -1607,14 +1598,14 @@
id-byte-array-md (read-bytes in (read-md-count in))
id-byte-array-lg (read-bytes in (read-lg-count in))
id-long-array-lg (read-array in long (long-array (read-lg-count in)) "[J")
id-int-array-lg (read-array in int (int-array (read-lg-count in)) "[I")
id-long-array-lg (read-array in long "[J" (long-array (read-lg-count in)))
id-int-array-lg (read-array in int "[I" (int-array (read-lg-count in)))
id-double-array-lg (read-array in double (double-array (read-lg-count in)) "[D")
id-float-array-lg (read-array in float (float-array (read-lg-count in)) "[F")
id-double-array-lg (read-array in double "[D" (double-array (read-lg-count in)))
id-float-array-lg (read-array in float "[F" (float-array (read-lg-count in)))
id-string-array-lg (read-array in String (make-array String (read-lg-count in)) "[Ljava.lang.String;")
id-object-array-lg (read-array in Object (object-array (read-lg-count in)) "[Ljava.lang.Object;")
id-string-array-lg (read-array in String "[Ljava.lang.String;" (make-array String (read-lg-count in)))
id-object-array-lg (read-array in Object "[Ljava.lang.Object;" (object-array (read-lg-count in)))
id-str-0 ""
id-str-sm* (read-str in (read-sm-count* in))
@ -1757,15 +1748,13 @@
(if (neg? type-id)
(read-custom! in nil type-id) ; Unprefixed custom type
(throw
(ex-info
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
{:type-id type-id}))))
(truss/ex-info!
(str "Unrecognized type id (" type-id "). Data frozen with newer Nippy version?")
{:type-id type-id})))
(catch Throwable t
(throw
(ex-info (str "Thaw failed against type-id: " type-id)
{:type-id type-id} t))))))
(truss/ex-info! (str "Thaw failed against type-id: " type-id)
{:type-id type-id} t)))))
(let [head-sig head-sig] ; Not ^:const
(defn- try-parse-header [^bytes ba]
@ -1785,20 +1774,20 @@
:lzma2 lzma2-compressor
:lz4 lz4-compressor
:zstd zstd-compressor
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
:else (throw (ex-info ":auto not supported for non-standard compressors." {}))
(do (throw (ex-info (str "Unrecognized :auto compressor id: " compressor-id)
{:compressor-id compressor-id})))))
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
:else (truss/ex-info! ":auto not supported for non-standard compressors." {})
(do (truss/ex-info! (str "Unrecognized :auto compressor id: " compressor-id)
{:compressor-id compressor-id}))))
(defn- get-auto-encryptor [encryptor-id]
(case encryptor-id
nil nil
:aes128-gcm-sha512 aes128-gcm-encryptor
:aes128-cbc-sha512 aes128-cbc-encryptor
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
:else (throw (ex-info ":auto not supported for non-standard encryptors." {}))
(do (throw (ex-info (str "Unrecognized :auto encryptor id: " encryptor-id)
{:encryptor-id encryptor-id})))))
:no-header (truss/ex-info! ":auto not supported on headerless data." {})
:else (truss/ex-info! ":auto not supported for non-standard encryptors." {})
(do (truss/ex-info! (str "Unrecognized :auto encryptor id: " encryptor-id)
{:encryptor-id encryptor-id}))))
(def ^:private err-msg-unknown-thaw-failure "Possible decryption/decompression error, unfrozen/damaged data, etc.")
(def ^:private err-msg-unrecognized-header "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?")
@ -1845,24 +1834,23 @@
(fn ex
([ msg] (ex nil msg))
([e msg]
(throw
(ex-info (str "Thaw failed. " msg)
{:opts
(assoc opts
:compressor compressor
:encryptor encryptor)
(truss/ex-info! (str "Thaw failed. " msg)
{:opts
(assoc opts
:compressor compressor
:encryptor encryptor)
:bindings
(enc/assoc-some {}
'*freeze-fallback* *freeze-fallback*
'*final-freeze-fallback* *final-freeze-fallback*
'*auto-freeze-compressor* *auto-freeze-compressor*
'*custom-readers* *custom-readers*
'*incl-metadata?* *incl-metadata?*
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
'*thaw-xform* *thaw-xform*)}
:bindings
(enc/assoc-some {}
'*freeze-fallback* *freeze-fallback*
'*final-freeze-fallback* *final-freeze-fallback*
'*auto-freeze-compressor* *auto-freeze-compressor*
'*custom-readers* *custom-readers*
'*incl-metadata?* *incl-metadata?*
'*thaw-serializable-allowlist* *thaw-serializable-allowlist*
'*thaw-xform* *thaw-xform*)}
e))))
e)))
thaw-data
(fn [data-ba compressor-id encryptor-id ex-fn]
@ -2019,7 +2007,7 @@
[{:keys [comparable?] :as opts}]
(let [rng (java.util.Random. 123456) ; Seeded for determinism
rand-nth (fn [coll] (nth coll (.nextInt rng (count coll))))
all
base
{:nil nil
:true true
:false false
@ -2056,7 +2044,6 @@
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{[] ()}}] #{:a :b}}
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5 "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"] {} #{} [] ()]]]]]
:regex #"^(https?:)?//(www\?|\?)?"
:sorted-set (sorted-set 1 2 3 4 5)
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)
:lazy-seq-empty (map identity ())
@ -2064,25 +2051,10 @@
:queue (into clojure.lang.PersistentQueue/EMPTY [:a :b :c :d :e :f :g])
:queue-empty clojure.lang.PersistentQueue/EMPTY
:uuid (java.util.UUID. 7232453380187312026 -7067939076204274491)
:uri (java.net.URI. "https://clojure.org")
:defrecord (StressRecord. "data")
:deftype (StressType. "data")
:bytes (byte-array [(byte 1) (byte 2) (byte 3)])
:objects (object-array [1 "two" {:data "data"}])
;; TODO (target-release>= 350)
;; :byte-array (byte-array [(byte 1) (byte 2) (byte 3) (byte 4)])
;; :long-array (long-array [1 2 3 4])
;; :int-array (int-array [1 2 3 4])
;; :double-array (double-array [1.5 2.5 3.5 4.5])
;; :float-array (float-array [1.5 2.5 3.5 4.5])
;; :object-array (object-array [1 "two" {:data "data"}])
;; :string-array (into-array String ["a" "b" "c"])
:uuid (java.util.UUID. 7232453380187312026 -7067939076204274491)
:uri (java.net.URI. "https://clojure.org")
:defrecord (StressRecord. "data")
:deftype (StressType. "data")
:util-date (java.util.Date. 1577884455500)
:sql-date (java.sql.Date. 1577884455500)
@ -2090,10 +2062,6 @@
:duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) ::skip)
:period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) ::skip)
:throwable (Throwable. "Msg")
:exception (Exception. "Msg")
:ex-info (ex-info "Msg" {:data "data"})
:many-longs (vec (repeatedly 512 #(rand-nth (range 10))))
:many-doubles (vec (repeatedly 512 #(double (rand-nth (range 10)))))
:many-strings (vec (repeatedly 512 #(rand-nth ["foo" "bar" "baz" "qux"])))
@ -2103,8 +2071,24 @@
(rand-nth ["foo" "bar" "baz" "qux" ]))))}]
(if comparable?
(dissoc all :bytes :objects :throwable :exception :ex-info :regex)
(do all))))
base
(assoc base
:non-comparable
{:regex #"^(https?:)?//(www\?|\?)?"
:throwable (Throwable. "Msg")
:exception (Exception. "Msg")
:ex-info (ex-info "Msg" {:data "data"})
:arrays
{:boolean (boolean-array (mapv even? (range 32)))
:byte (byte-array (mapv byte (range 32)))
:short (short-array (mapv short (range 32)))
:int (int-array (mapv int (range 32)))
:long (long-array (mapv long (range 32)))
:float (float-array (mapv float (range 32)))
:double (double-array (mapv double (range 32)))
:char (char-array (mapv char (range 32)))
:str (into-array String (mapv str (range 32)))
:object (object-array (mapv vector (range 32)))}}))))
(comment
[(= (stress-data {:comparable? true}) (stress-data {:comparable? true}))
@ -2247,5 +2231,5 @@
(alter-var-root *thaw-serializable-allowlist* f) and/or
(alter-var-root *freeze-serializable-allow-list* f) instead."
[f]
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (enc/have set? old))))
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (enc/have set? old))))))
(alter-var-root *freeze-serializable-allowlist* (fn [old] (f (truss/have set? old))))
(alter-var-root *thaw-serializable-allowlist* (fn [old] (f (truss/have set? old))))))

View file

@ -1,6 +1,8 @@
(ns ^:no-doc taoensso.nippy.compression
"Private, implementation detail."
(:require [taoensso.encore :as enc])
(:require
[taoensso.truss :as truss]
[taoensso.encore :as enc])
(:import
[java.nio ByteBuffer]
[java.io
@ -156,7 +158,7 @@
(.read xzs ba 0 len-decomp)
(if (== -1 (.read xzs)) ; Good practice as extra safety measure
nil
(throw (ex-info "LZMA2 Decompress failed: corrupt data?" {:ba ba})))
(truss/ex-info! "LZMA2 Decompress failed: corrupt data?" {:ba ba}))
ba)))
;;;; Public API

View file

@ -2,7 +2,9 @@
"Low-level crypto utils.
Private & alpha, very likely to change!"
(:refer-clojure :exclude [rand-nth])
(:require [taoensso.encore :as enc]))
(:require
[taoensso.truss :as truss]
[taoensso.encore :as enc]))
;; Note that AES128 may be preferable to AES256 due to known attack
;; vectors specific to AES256, Ref. <https://goo.gl/qU4CCV>
@ -45,7 +47,7 @@
(defn take-ba ^bytes [n ^bytes ba] (java.util.Arrays/copyOf ba ^int n)) ; Pads if ba too small
(defn utf8->ba ^bytes [^String s] (.getBytes s "UTF-8"))
(defn- add-salt ^bytes [?salt-ba ba] (if ?salt-ba (enc/ba-concat ?salt-ba ba) ba))
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (enc/have enc/bytes? utf8-or-ba)))
(defn pwd-as-ba ^bytes [utf8-or-ba] (if (string? utf8-or-ba) (utf8->ba utf8-or-ba) (truss/have enc/bytes? utf8-or-ba)))
(comment (seq (pwd-as-ba "foo")))

View file

@ -1,6 +1,7 @@
(ns ^:no-doc taoensso.nippy.encryption
"Private, implementation detail."
(:require
[taoensso.truss :as truss]
[taoensso.encore :as enc]
[taoensso.nippy.crypto :as crypto]))
@ -14,11 +15,11 @@
(decrypt ^bytes [encryptor pwd ba]))
(defn- throw-destructure-ex [typed-password]
(throw (ex-info
(str "Expected password form: "
"[<#{:salted :cached}> <password-string>].\n "
"See `aes128-encryptor` docstring for details!")
{:typed-password typed-password})))
(truss/ex-info!
(str "Expected password form: "
"[<#{:salted :cached}> <password-string>].\n "
"See `aes128-encryptor` docstring for details!")
{:typed-password typed-password}))
(defn- destructure-typed-pwd [typed-password]
(if (vector? typed-password)

View file

@ -2,6 +2,7 @@
"Private, implementation detail."
(:require
[clojure.string :as str]
[taoensso.truss :as truss]
[taoensso.encore :as enc]))
;;;; Fallback type tests
@ -63,7 +64,7 @@
(when x
(if (string? x)
(if (= x "") #{} (set (mapv str/trim (str/split x #"[,:]"))))
(enc/have set? x))))
(truss/have set? x))))
(comment
(mapv classname-set [nil #{"foo"} "" "foo, bar:baz"])
@ -151,6 +152,29 @@
See that function's docstring for more info."
[] (trim nmax (state_))))
(comment
(count (get-recorded-serializable-classes))
(enc/reduce-n
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
nil 0 1e5))
(let [compile
(enc/fmemoize
(fn [x]
(if (allow-and-record? x)
allow-and-record-any-serializable-class-unsafe
(enc/name-filter x))))
fn? fn?
conform?
(fn [x cn]
(if (fn? x)
(x cn) ; Intentionally uncached, can be handy
((compile x) cn)))]
(defn serializable-allowed? [allow-list class-name]
(conform? allow-list class-name)))
;;;; Release targeting
(comment
@ -159,8 +183,8 @@
;; To help support release targeting, we track new type ids added over time
(let [id-history ; {<release> #{type-ids}}
{350 ; v3.5.0 (YYYY-MM-DD), added 5x
;; #{string-array-lg long-array-lg int-array-lg double-array-lg float-array-lg}
{350 ; v3.5.0 (2025-04-15), added 5x
;; #{int-array-lg long-array-lg float-array-lg double-array-lg string-array-lg}
#{0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
@ -247,28 +271,3 @@
[min-release] (target>= min-release)))
(comment (macroexpand '(target-release>= 340)))
;;;
(comment
(count (get-recorded-serializable-classes))
(enc/reduce-n
(fn [_ n] (allow-and-record-any-serializable-class-unsafe (str n)))
nil 0 1e5))
(let [compile
(enc/fmemoize
(fn [x]
(if (allow-and-record? x)
allow-and-record-any-serializable-class-unsafe
(enc/name-filter x))))
fn? fn?
conform?
(fn [x cn]
(if (fn? x)
(x cn) ; Intentionally uncached, can be handy
((compile x) cn)))]
(defn serializable-allowed? [allow-list class-name]
(conform? allow-list class-name)))

View file

@ -1,10 +1,8 @@
(ns taoensso.nippy.tools
"Utils for community tools that want to add user-configurable Nippy support.
Used by Carmine, Faraday, etc."
(:refer-clojure :exclude [binding])
(:require
[taoensso.encore :as enc :refer [binding]]
[taoensso.encore :as enc]
[taoensso.nippy :as nippy]))
(def ^:dynamic *freeze-opts* nil)

View file

@ -4,8 +4,11 @@
[clojure.test.check :as tc]
[clojure.test.check.generators :as tc-gens]
[clojure.test.check.properties :as tc-props]
[taoensso.truss :as truss :refer [throws?]]
[taoensso.encore :as enc :refer [ba=]]
[taoensso.nippy :as nippy :refer [freeze thaw]]
[taoensso.nippy.impl :as impl]
[taoensso.nippy.tools :as tools]
[taoensso.nippy.compression :as compr]
[taoensso.nippy.crypto :as crypto]
[taoensso.nippy-benchmarks :as benchmarks]))
@ -53,10 +56,6 @@
#(freeze % {:password [:salted "p"]}))
test-data)))
(let [d (nippy/stress-data {})]
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
(is (= test-data ((comp #(thaw % {:compressor nippy/lzma2-compressor})
#(freeze % {:compressor nippy/lzma2-compressor}))
test-data)))
@ -75,9 +74,9 @@
#(freeze % {:compressor nippy/zstd-compressor}))
test-data)))
(is (enc/throws? Exception (thaw (freeze test-data {:password "malformed"}))))
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
(is (enc/throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
(is (throws? Exception (thaw (freeze test-data {:password "malformed"}))))
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
(is (throws? Exception (thaw (freeze test-data {:password [:salted "p"]}))))
(is
(= "payload"
@ -94,14 +93,23 @@
(let [n range-uint+] (= (thaw (freeze n)) n))
(let [n (- range-uint+)] (= (thaw (freeze n)) n))]))
(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (fn []))))
(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (fn []))))
(testing "Clojure v1.10+ metadata protocol extensions"
[(is (enc/throws? :ex-info "Unfreezable type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
[(is (throws? :ex-info "Failed to freeze type" (nippy/freeze (with-meta [] {:a :A, 'b (fn [])}))))
(is (= {:a :A} (meta (nippy/thaw (nippy/freeze (with-meta [] {:a :A, 'b/c (fn [])}))))))
(is (= nil (meta (nippy/thaw (nippy/freeze (with-meta [] { 'b/c (fn [])})))))
"Don't attach empty metadata")])
(let [d (nippy/stress-data {})]
[(is (= (vec (:bytes d)) ((comp vec thaw freeze) (:bytes d))))
(is (= (vec (:objects d)) ((comp vec thaw freeze) (:objects d))))])
(testing "Arrays"
(binding [nippy/*thaw-serializable-allowlist* nippy/default-freeze-serializable-allowlist]
(mapv (fn [[k aval]] (is (= (vec aval) (-> aval nippy/freeze nippy/thaw vec)) (name k)))
(get-in (nippy/stress-data {}) [:non-comparable :arrays]))))
(is (gen-test 1600 [gen-data] (= gen-data (thaw (freeze gen-data)))) "Generative")])
;;;; Custom types & records
@ -112,7 +120,7 @@
(deftest _types
[(testing "Extend to custom type"
[(is
(enc/throws? Exception ; No thaw extension yet
(throws? Exception ; No thaw extension yet
(do
(alter-var-root #'nippy/*custom-readers* (constantly {}))
(nippy/extend-freeze MyType 1 [x s]
@ -323,7 +331,7 @@
[(is (= nippy/*thaw-serializable-allowlist* #{"base.1" "base.2" "add.1" "add.2"})
"JVM properties override initial allowlist values")
(is (enc/throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
(is (throws? Exception (nippy/freeze sem {:serializable-allowlist #{}}))
"Can't freeze Serializable objects unless approved by allowlist")
(is (sem?
@ -435,8 +443,8 @@
(is (= (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze []))) []) "rf not run on empty colls")
(let [ex (enc/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
(is (= (-> ex enc/ex-cause enc/ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
(let [ex (truss/throws :default (binding [nippy/*thaw-xform* (map (fn [x] (/ 1 0)))] (thaw (freeze [:a :b]))))]
(is (= (-> ex ex-cause ex-cause ex-data :call) '(rf acc in)) "Error thrown via `*thaw-xform*`"))])
;;;; Compressors
@ -453,7 +461,7 @@
(print ".") (flush)
(dotimes [_ 1000]
(is
(nil? (enc/catching (compr/decompress c (crypto/rand-bytes 1024))))
(nil? (truss/catching :all (compr/decompress c (crypto/rand-bytes 1024))))
"Decompression never crashes JVM, even against invalid data")))
(println)))