luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'dev'

Browse source
This commit is contained in:
Peter Taoussanis 2014-11-27 17:43:32 +07:00
commit a47ca82ce1
5 changed files with 43 additions and 24 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
CHANGELOG.md
View file

@ -1,5 +1,14 @@
> This project uses [Break Versioning](https://github.com/ptaoussanis/encore/blob/master/BREAK-VERSIONING.md) as of **Aug 16, 2014**.
## v2.7.1 / 2014 Nov 27
> This is a **minor maintenance release** & should be a safe upgrade for users of v2.7.0/RC-1.
* **CHANGE**: Improve some error messages by exposing trapped throwables when possible (@kul).
* **FIX**: Nippy v1 thaw compatibility was broken in some cases.
* Bumped dependencies.
## v2.7.0 / 2014 Oct 6
No changes from `v2.7.0-RC1`.

2
README.md
View file

@ -1,7 +1,7 @@
**[API docs][]** | **[CHANGELOG][]** | [other Clojure libs][] | [Twitter][] | [contact/contrib](#contact--contributing) | current [Break Version][]:
```clojure
[com.taoensso/nippy "2.7.0"] ; Please see CHANGELOG for details
[com.taoensso/nippy "2.7.1"] ; Please see CHANGELOG for details
```
v2.7 is a major, **mostly backwards-compatible** release focused on improved performance and a new default compression scheme (LZ4). See the [CHANGELOG][] for details. Thanks to [mpenet](https://github.com/mpenet) for his work on the LZ4 support!

14
project.clj
View file

@ -1,4 +1,4 @@
(defproject com.taoensso/nippy "2.7.0"
(defproject com.taoensso/nippy "2.7.1"
:author "Peter Taoussanis <https://www.taoensso.com>"
:description "Clojure serialization library"
:url "https://github.com/ptaoussanis/nippy"
@ -12,11 +12,11 @@
:dependencies
[[org.clojure/clojure "1.4.0"]
[org.clojure/tools.reader "0.8.9"]
[com.taoensso/encore "1.11.2"]
[org.clojure/tools.reader "0.8.12"]
[com.taoensso/encore "1.16.2"]
[org.iq80.snappy/snappy "0.3"]
[org.tukaani/xz "1.5"]
[net.jpountz.lz4/lz4 "1.2.0"]]
[net.jpountz.lz4/lz4 "1.3"]]
:profiles
{;; :default [:base :system :user :provided :dev]
@ -24,11 +24,11 @@
:1.5 {:dependencies [[org.clojure/clojure "1.5.1"]]}
:1.6 {:dependencies [[org.clojure/clojure "1.6.0"]]}
:test {:jvm-opts ["-Xms1024m" "-Xmx2048m"]
:dependencies [[expectations "2.0.12"]
[org.clojure/test.check "0.5.9"]
:dependencies [[expectations "2.0.13"]
[org.clojure/test.check "0.6.1"]
;; [com.cemerick/double-check "0.5.7"]
[org.clojure/data.fressian "0.2.0"]
[org.xerial.snappy/snappy-java "1.1.1.3"]]}
[org.xerial.snappy/snappy-java "1.1.1.6"]]}
:dev [:1.6 :test
{:plugins
[[lein-pprint "1.1.1"]

39
src/taoensso/nippy.clj
View file

@ -436,18 +436,20 @@
id-reader
(let [edn (read-utf8 in)]
(try (edn/read-string {:readers *data-readers*} edn)
(catch Exception _ {:nippy/unthawable edn
:type :reader})))
(catch Exception e {:nippy/unthawable edn
:type :reader
:throwable e})))
id-serializable
(let [class-name (read-utf8 in)]
(try (let [;; .readObject _before_ Class/forName: it'll always read
;; all data before throwing
object (.readObject (ObjectInputStream. in))
class ^Class (Class/forName class-name)]
^Class class (Class/forName class-name)]
(cast class object))
(catch Exception _ {:nippy/unthawable class-name
:type :serializable})))
(catch Exception e {:nippy/unthawable class-name
:type :serializable
:throwable e})))
id-bytes (read-bytes in)
id-nil nil
@ -551,7 +553,7 @@
nil nil
:aes128-sha512 aes128-encryptor
:no-header (throw (ex-info ":auto not supported on headerless data." {}))
:else (throw (ex-info ":auto not supported for non-standard encryptors."))
:else (throw (ex-info ":auto not supported for non-standard encryptors." {}))
(throw (ex-info (format "Unrecognized :auto encryptor id: %s" encryptor-id)
{:encryptor-id encryptor-id}))))
@ -563,10 +565,13 @@
Options include:
:compressor - An ICompressor, :auto (requires Nippy header), or nil.
:encryptor - An IEncryptor, :auto (requires Nippy header), or nil."
[^bytes ba & [{:keys [compressor encryptor password]
:or {compressor :auto
encryptor :auto}
:as opts}]]
[^bytes ba
& [{:keys [compressor encryptor password v1-compatibility?]
:or {compressor :auto
encryptor :auto
v1-compatibility? true ; Recommend disabling when possible
}
:as opts}]]
(assert (not (contains? opts :headerless-meta))
":headerless-meta `thaw` option removed as of Nippy v2.7.")
@ -596,11 +601,15 @@
(catch Exception e
(ex "Decryption/decompression failure, or data unfrozen/damaged.")))))
thaw-nippy-v1-data ; A little hackish, but necessary
;; This is hackish and can actually currently result in JVM core dumps
;; due to buggy Snappy behaviour, Ref. http://goo.gl/mh7Rpy.
thaw-nippy-v1-data
(fn [data-ba]
(try (thaw-data data-ba :snappy nil)
(catch Exception _
(thaw-data data-ba nil nil))))]
(if-not v1-compatibility?
(throw (Exception. "v1 compatibility disabled"))
(try (thaw-data data-ba :snappy nil)
(catch Exception _
(thaw-data data-ba nil nil)))))]
(if-let [[data-ba {:keys [compressor-id encryptor-id unrecognized-meta?]
:as head-meta}] (try-parse-header ba)]
@ -609,7 +618,7 @@
;; unlikely that this is a fluke and data is actually headerless):
(try (thaw-data data-ba compressor-id encryptor-id)
(catch Exception e
(try (thaw-nippy-v1-data)
(try (thaw-nippy-v1-data data-ba)
(catch Exception _
(if unrecognized-meta?
(ex "Unrecognized (but apparently well-formed) header. Data frozen with newer Nippy version?"

3
src/taoensso/nippy/compression.clj
View file

@ -77,7 +77,8 @@
(let [len-decomp (alength ^bytes ba)
max-len-comp (.maxCompressedLength compressor len-decomp)
ba-comp* (byte-array max-len-comp) ; Over-sized
len-comp (.compress compressor ba 0 len-decomp ba-comp* 0 max-len-comp)
len-comp (.compress compressor ^bytes ba 0 len-decomp
ba-comp* 0 max-len-comp)
;;
baos (ByteArrayOutputStream. (+ len-comp 4))
dos (DataOutputStream. baos)]