luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
700 commits 2 branches 109 tags 2 MiB
Commit graph

384 commits

Author SHA1 Message Date
Peter Taoussanis
2028f80854 Experimental caching impl. 2016-04-14 12:16:51 +07:00
Peter Taoussanis
b623b4a8cc NB *BREAKING*: refactor type defs, variable-sized types, etc. 
Changes incl:
  - Hid a bunch of undocumented impl. details
  - A number of performance optimizations
 2016-04-14 12:16:27 +07:00
Peter Taoussanis
3f43542adb Tools housekeeping 2016-04-13 11:13:01 +07:00
Peter Taoussanis
892937eb34 [#80] Clarify docstrings for low-level freeze/thaw utils 2016-04-13 11:13:01 +07:00
Peter Taoussanis
d327f0ff38 Types: add dedicated 2 and 3 tuple type 2016-04-13 11:13:01 +07:00
Peter Taoussanis
7adad2240c Perf: optimize coll freezing via new enc/reduce-n 
- Take advantage of clojure.lang.LongRange
  - Avoid unnecessary temp `[k v]` (map entry) constructions
 2016-04-13 11:13:01 +07:00
Peter Taoussanis
28765ac501 Bump deps 2016-01-23 11:10:17 +07:00
Peter Taoussanis
c483e157bd Encryption ns housekeeping 2015-12-01 15:32:22 +07:00
Peter Taoussanis
643d762bbe Fix :auto encryption unit tests 2015-12-01 15:27:26 +07:00
Peter Taoussanis
f59f2f33cb NB fix min-val int-as-long 2015-12-01 14:45:24 +07:00
Peter Taoussanis
4df5446c5b Update benchmarks 2015-10-06 17:56:15 +07:00
Peter Taoussanis
3479ddad00 Clean up thaw fallback behaviour, decrease number of fallback cases 2015-10-06 17:56:15 +07:00
Peter Taoussanis
037cb14739 Misc hk 2015-10-06 15:57:55 +07:00
Peter Taoussanis
0905b96ca6 NB: Refactor thaw v1 compatibility support 2015-10-06 14:39:34 +07:00
Peter Taoussanis
9c8adfe513 **NB BREAKING**: change default :v1-compatibility? thaw option 
Motivation for changing this default:

v1 compatibility requires that in the event of a thaw failure, a fallback
attempt is made using v1 options. This must include an attempt at Snappy
decompression.

But the version of Snappy we're using has a major bug that can segfault +
crash the JVM when attempted against non-Snappy data:
https://github.com/dain/snappy/issues/20

I'd switch to an alternative Snappy implementation, but the only other
implementation I'm aware of uses JNI which can introduce troublesome
compatibility issues even for people who don't want the Snappy support.

Had hoped that the Snappy bug would eventually get fixed, but that's
looking unlikely.

Nippy v2 was released on July 22nd 2013 (2 years, 2 months ago) - so
am hoping that the majority of lib users will no longer have a need
for v1 data thaw support at this point.

For those that do, they can re-enable v1 thaw support with this flag.

If a better alternative solution ever presents (e.g. the Snappy bug
is fixed, an alternative implementation turns up, or we write a util
to reliably identify Snappy compressed data) - we can re-enable this
flag by default.
 2015-10-06 13:12:29 +07:00
Peter Taoussanis
cf38d6f111 Fix final-freeze-fallback arg order 2015-10-06 13:07:42 +07:00
Peter Taoussanis
2df9cb80d6 Add small-bytes type 2015-10-06 13:07:42 +07:00
Peter Taoussanis
c7c0c6fe54 Stop documenting :skip-header? option 
It's almost entirely useless now, and dangerous: folks who absolutely know what
they're doing can keep using it, but don't broadcast its existance.
 2015-10-06 13:07:41 +07:00
Peter Taoussanis
c5901730ea Update ba inspector 2015-10-06 13:07:41 +07:00
Peter Taoussanis
7faaf48ee7 Deprecate Compressable-LZMA2 (was anyway marked as experimental) 2015-10-06 13:07:41 +07:00
Peter Taoussanis
7072f73952 Misc hk 2015-10-06 13:07:41 +07:00
Peter Taoussanis
d61fb06f3b Primitive ided-long checks 2015-09-30 12:29:05 +07:00
Peter Taoussanis
2ebd8ce2ac Fix id typing 2015-09-30 12:29:05 +07:00
Peter Taoussanis
998dabc195 NB: refactor freezing utils for easier use by libs + custom extensions, etc. 2015-09-30 12:29:05 +07:00
Peter Taoussanis
15f0de1658 Simplify stream thaw API, switch from macros->fns 2015-09-30 12:29:04 +07:00
Peter Taoussanis
50ffb78c22 Refer rename: encore->enc 2015-09-30 12:29:04 +07:00
Peter Taoussanis
734e88b20c defonce on all dynamic vars (allow alter-var-root) 2015-09-30 12:29:04 +07:00
Peter Taoussanis
f67f9da64e Remove alpha status on final-freeze-fallback 2015-09-30 12:29:04 +07:00
Peter Taoussanis
9c1e8751c4 Simplify stream freeze API, switch from macros->fns 2015-09-30 12:29:04 +07:00
Peter Taoussanis
da77b3d582 NB: Remove (long-deprecated) freezing legacy mode 2015-09-30 12:29:04 +07:00
Peter Taoussanis
89c9328596 Experimental optimization: zero-copy freeze mode 2015-09-30 12:29:04 +07:00
Peter Taoussanis
fa17eb3a78 Update benchmarks 2015-09-30 12:29:04 +07:00
Peter Taoussanis
327a800d80 Experimental: optimize common case of small maps, sets, vectors 2015-09-30 12:29:04 +07:00
Peter Taoussanis
b298d690c7 Misc hk, reorganize type ids 2015-09-30 12:29:04 +07:00
Peter Taoussanis
885f192f6b Micro optimization: drop unnecessary double kvs count 
Had a vestigial count doubling from an historical implementation
that constructed hash-maps using `(apply hash-map ...)`
 2015-09-30 12:20:53 +07:00
Peter Taoussanis
4765a32e4e Optimize compact long freezer 2015-09-30 11:57:47 +07:00
Peter Taoussanis
1506747e42 Tune buffer size, freeze compressor selector 2015-09-30 11:57:47 +07:00
Peter Taoussanis
9c33f4f5ac Update benchmarks 2015-09-30 11:57:47 +07:00
Peter Taoussanis
ea9286dc90 Micro optimization: kv run is faster still 2015-09-30 11:57:47 +07:00
Peter Taoussanis
1ae8e6c389 Micro optimization: destructure faster than explicit calls here 2015-09-30 11:57:47 +07:00
Peter Taoussanis
956ce7df7e Micro optimization: read-bytes expansion 2015-09-30 11:57:46 +07:00
Peter Taoussanis
7ae954a229 Micro optimization: remove & args 2015-09-30 11:57:46 +07:00
Peter Taoussanis
2f27666d05 Rename deprecated type ids 
Making room for >1 deprecated id per type
 2015-09-30 11:57:46 +07:00
Peter Taoussanis
bbbc12ce30 Misc hk 2015-09-30 11:57:46 +07:00
Peter Taoussanis
db375e7686 Perf: anon fn is faster here 2015-09-30 11:57:46 +07:00
Peter Taoussanis
cea505484a Note re double vestigial kvs length 2015-09-30 11:57:46 +07:00
Peter Taoussanis
36abe07f2b Revert "Experimental/perf: kvs work directly against MapEntry" 
This reverts commit e150775cfe82f8206ddc88034417421e200851fa.
 2015-09-30 11:57:45 +07:00
Peter Taoussanis
aa9a18088f Experimental/perf: kvs work directly against MapEntry 2015-09-30 11:57:45 +07:00
Peter Taoussanis
9d4db3106e Perf: create Ratio's directly 2015-09-30 11:57:45 +07:00
Peter Taoussanis
41d3dcd467 *default-freeze-compressor-selector* should be public 2015-09-17 11:11:06 +07:00
Peter Taoussanis
a3847a4818 Misc housekeeping 2015-09-17 11:11:06 +07:00
Peter Taoussanis
d57788125a Update benchmarks 2015-09-17 11:11:06 +07:00
Peter Taoussanis
e403c17417 Housekeeping for 0a35b8c 2015-09-17 11:11:06 +07:00
Karsten Schmidt
3f9fe327e0 [#70] move small? check outside write-bytes macro body, replace if-not's w/ if's (@postspectacular) 2015-09-17 11:11:06 +07:00
Peter Taoussanis
1ba3c38ab2 Update benchmarks 2015-09-14 20:12:11 +07:00
Peter Taoussanis
56b33e23f0 Perf: fix boxed math on long compression 2015-09-14 20:12:11 +07:00
Peter Taoussanis
da671cbba4 Housekeeping 2015-09-14 17:08:23 +07:00
Peter Taoussanis
6b4e1341ee Realign type ids 2015-09-14 17:08:23 +07:00
Peter Taoussanis
11545690c8 NB switch to encore edn reader/writer 2015-09-14 17:08:23 +07:00
Peter Taoussanis
e5ccd25848 Fix custom-readers var declaration 2015-09-14 17:08:23 +07:00
Peter Taoussanis
5a623870fe Warn about thaw-id reset at runtime rather than expansion time 2015-09-14 17:08:23 +07:00
Peter Taoussanis
82294f54f9 Switch to dynamic custom-readers 2015-09-14 17:08:23 +07:00
Peter Taoussanis
a005a9d7fa Switch to dynamic default-freeze-compressor-selector 2015-09-14 17:08:23 +07:00
Peter Taoussanis
b42aff9cc9 [#68] NB hotfix: encryption thread safety 2015-09-14 16:58:02 +07:00
Peter Taoussanis
c5f4ef74fd Update benchmarks 2015-06-04 18:55:33 +07:00
Peter Taoussanis
2b19f8eab5 Refactor reading of possibly-unthawable types 2015-05-29 14:20:14 +07:00
Peter Taoussanis
53714cc192 More robust record deserialization 2015-05-29 14:13:35 +07:00
Peter Taoussanis
7753eefd2e Switch to encore v1.28.0, doseq->backport-run! (better perf) 2015-05-06 17:37:12 +07:00
Peter Taoussanis
d5fea13bfb Misc housekeeping 2015-04-29 11:47:39 +07:00
Peter Taoussanis
fb2154fb01 Update benchmarks 2015-04-29 11:47:39 +07:00
Peter Taoussanis
89e709d5db Boxed math optimizations 2015-04-18 14:13:29 +07:00
Peter Taoussanis
0e691cbbcf Experimental optimization: allow smart, automatic compressor selection 2015-04-18 14:13:29 +07:00
Peter Taoussanis
e991ec7899 Add encore version check 2015-02-18 17:33:37 +07:00
Peter Taoussanis
df1d83275d [#63] Fix missing thaw exception cause (@cespare) 2015-02-18 17:33:37 +07:00
Peter Taoussanis
af293fec7b [#59] freezable? should return true for clojure.lang.PersistentVector (@chairmanwow) 2015-01-13 13:32:37 +07:00
Peter Taoussanis
26c01c799a Add option to disable v1 thaw compatibility 2014-11-27 17:19:02 +07:00
Peter Taoussanis
f1b4c7ff88 Add type-hint for new LZ4 dep 2014-11-27 15:43:31 +07:00
Peter Taoussanis
e142063201 [#54] Fix bad v1 data thaw call (@jafingerhut) 2014-11-10 11:17:28 +07:00
kul
e006fbcbb0 Expose exception to user as :throwable (@kul) 2014-11-04 12:33:19 +07:00
Peter Taoussanis
152fec95e0 Fix ex-info call args (@kul) 2014-11-04 12:31:41 +07:00
Peter Taoussanis
4d1a374c8f Typehint typo 2014-11-04 12:31:41 +07:00
Peter Taoussanis
7925982939 Print warning when replacing a custom type reader 2014-08-27 19:20:23 +07:00
Peter Taoussanis
13cdf06a60 Make coerce-custom-type-id private 2014-08-27 19:20:23 +07:00
Peter Taoussanis
9e60939848 EXPERIMENTAL: Support keyword-id extensions (#50) 2014-07-06 13:25:08 +07:00
Peter Taoussanis
70dd8f637c Lock-free memoized type test 2014-04-29 23:10:26 +07:00
Peter Taoussanis
06cf8aefd6 Update benchmarks 2014-04-29 23:10:26 +07:00
Peter Taoussanis
20b1c2b1d2 Encode compression type in Nippy header, major refactor/housekeeping 
Housekeeping includes:
  * Importing useful encryption+compression stuff into primary ns
    for lib consumers.
  * Promoting a number of things from Alpha status.
  * Exceptions are now all `ex-info`s.
  * Simplification of `thaw` API: Nippy v1 support is now automatic
    & configuration-free (performance impact in most cases is negligible).
 2014-04-29 23:10:26 +07:00
Peter Taoussanis
b7a454a9c8 EXPERIMENTAL: Make LZ4 the default Nippy compressor (back-compatible for header'ed data) 2014-04-29 23:10:26 +07:00
Peter Taoussanis
5b9358acf9 LZ4: use fast decompressor 2014-04-29 23:10:26 +07:00
Peter Taoussanis
a6aba2c92a Add experimental LZ4 compressors 2014-04-29 23:10:26 +07:00
Peter Taoussanis
3c8661a094 [#48] Fix freeze/thaw identity on empty lazy seqs 2014-04-29 23:08:00 +07:00
Peter Taoussanis
7d84099074 IMPORTANT FIX: small strings weren't getting a proper UTF-8 encoding (xkihzew) 
`(.getBytes <string>)` was being used here instead of
`(.getBytes <string> "UTF-8")` as is correct and done elsewhere.

This means that small UTF-8 strings may have been incorrectly stored
in environments where UTF-8 is not the default JVM character encoding.

Bug was introduced in Nippy v2.6.0, released 9 days ago (2014 Mar 30).

*********************************************************************
Please check for possible errors in Unicode text written using Nippy
v2.6.0 if your JVM uses an alternative character encoding by default
*********************************************************************

Really sorry about this, not sure how this slipped in.

Thanks to @xkihzew for the bug report.
 2014-04-08 17:04:53 +07:00
Peter Taoussanis
6058ac7b54 Fix freezable? fn name typo (mbossenbroek) 2014-03-18 14:23:15 +07:00
Peter Taoussanis
54389cc3a0 NB migrate utils stuff to encore lib 2014-02-23 19:15:55 +07:00
Peter Taoussanis
107cc53f65 Check freezable? against atom type 2014-02-16 19:09:02 +07:00
Peter Taoussanis
42b366a432 Experimental freezable? fn 2014-02-16 18:49:45 +07:00
Peter Taoussanis
0b56746a61 Mod (experimental) Compressable-LZMA2 to use new :skip-header? mode, add :headerless-meta assertion 2014-02-09 18:29:16 +07:00
Peter Taoussanis
e2847f348f [#38]: Distinguish between BigInt/BigInteger on thawing (mlacorte) 2014-01-30 16:16:50 +07:00
Peter Taoussanis
3f982e78cd Make public some macros useful for custom extensions, add experimental compact-long type 2014-01-23 23:13:21 +07:00
Peter Taoussanis
cfa0969bea Optimize some small, common types 2014-01-22 16:50:53 +07:00
Peter Taoussanis
1edde2d53b Revert "New compact byte representation, backwards-compatible" 
This reverts commit 93ce024f58.
 2014-01-22 15:38:32 +07:00
Peter Taoussanis
93ce024f58 New compact byte representation, backwards-compatible 2014-01-22 15:38:22 +07:00
Peter Taoussanis
1d2daf206b Clean up new DataInput/Output API 2014-01-22 15:01:12 +07:00
Peter Taoussanis
87fcd3a9c6 Generalise to allow serialising from/to any DataInput/DataOutput. (cmf) 2014-01-22 14:56:07 +07:00
Peter Taoussanis
b413adc62a Make id defs easily IDE-collapsable 2014-01-22 14:56:07 +07:00
Peter Taoussanis
5690a9611e Improve support for headerless freezing (docs, error msgs, etc.) 2014-01-22 14:01:27 +07:00
Peter Taoussanis
d8bbbbdb32 Add Fressian benchmark 2014-01-21 14:55:51 +07:00
Peter Taoussanis
42f45a0a06 Bench :fast before :encrypted 2014-01-21 14:55:50 +07:00
Peter Taoussanis
42950ec24a Add stress-data-comparable, stress-data-benchable 2014-01-21 14:55:50 +07:00
Peter Taoussanis
7e1b1bb6b4 Deprecate :legacy-mode, add :skip-header? mode 2014-01-21 14:55:50 +07:00
Peter Taoussanis
32e4e82091 BREAKING: Clean up (experimental) Compressable-LZMA2 2014-01-21 12:56:36 +07:00
Peter Taoussanis
c11d4e36a8 hk 2014-01-21 12:47:46 +07:00
Peter Taoussanis
d112ecc395 Revert "Experimental: write-compact-int for _all_ int-like writing" 
This reverts commit 93afb3820e.
 2014-01-21 12:46:35 +07:00
Peter Taoussanis
93afb3820e Experimental: write-compact-int for _all_ int-like writing 2014-01-21 12:46:23 +07:00
Peter Taoussanis
a5c2f39865 Revert "wip: experimenting with common number optimizations" 
This reverts commit 307a265447.
 2014-01-20 15:12:18 +07:00
Peter Taoussanis
307a265447 wip: experimenting with common number optimizations 2014-01-20 14:44:32 +07:00
Peter Taoussanis
f8239531da v2.5.2 2013-12-07 16:49:34 +07:00
Peter Taoussanis
5ea4af7356 Further work on freezer fallback reliability 2013-12-07 16:44:53 +07:00
Peter Taoussanis
3093eae322 Experimental: add freeze-fallback-as-str fn 2013-12-03 16:46:30 +07:00
Peter Taoussanis
43c85059d7 Throw on freeze (rather than thaw) when trying to freeze an unreadable object with pr-str 2013-12-03 16:22:52 +07:00
Peter Taoussanis
1a7812522c Experimental: add inspect-ba util for inspecting possibly-frozen data 2013-12-03 16:22:51 +07:00
Peter Taoussanis
730b8774c4 Utils: housekeeping, update bench, memoized 2013-11-08 21:45:28 +07:00
Peter Taoussanis
0fd9275e37 Experimental compressable-lzma2: fix thaw bug 2013-11-07 12:47:36 +07:00
Peter Taoussanis
da1622f271 Experimental: add Compressable-LZMA2 type & (replaceable) de/serializer 2013-11-06 13:58:55 +07:00
Peter Taoussanis
7e7975910a tools: fix thaw arg destructuring & opt merging 2013-11-05 22:41:51 +07:00
Peter Taoussanis
84e336fe65 Make byte transform on seqs 1-to-1, re-add ISeq as a fallback implementation 2013-10-31 13:29:34 +07:00
Peter Taoussanis
aa78ad6a0b Rename test type: :coll->:lazy-seq 2013-10-31 13:16:26 +07:00
Peter Taoussanis
b1dab1dc4b Add debug-mode flag 2013-10-31 13:15:22 +07:00
Peter Taoussanis
8f8ab99e95 Add some Serializable tests (exceptions) 2013-10-24 13:55:47 +07:00
Peter Taoussanis
9448d2b3ce Add Java's Serializable as a pre-Reader fallback mechanism 2013-10-24 13:55:47 +07:00
Peter Taoussanis
c58bf97af7 Move Record test into stress data 2013-10-24 13:55:47 +07:00
Peter Taoussanis
5c48ba8e26 Reduce amount of (unnecessary) freeze procotol extensions to interfaces 
These can cause confusing, inconsistent behaviour when the protocol dispatch
needs to choose between multiple implementations:
Ref. https://groups.google.com/forum/#!topic/clojure-dev/-zoLA78--Mo

The approach as of this commit is more robust and allows more flexibility in
extension.
 2013-10-24 13:55:47 +07:00
Peter Taoussanis
75dd0d9040 Remove tagged literal labels for uuid & date (both have been baked-in for some time) 2013-10-24 13:55:47 +07:00
Peter Taoussanis
a122e62fe0 Reposition freeze-to-stream! fn for clarity 2013-10-24 13:55:47 +07:00
Peter Taoussanis
26f32c7a95 Housekeeping 2013-10-24 13:34:38 +07:00
Peter Taoussanis
33eee786be Lower default LZMA2 compression level -> 0 (much better perf @ ~same ratio) 2013-10-19 18:17:47 +07:00
Peter Taoussanis
3879cf4d62 Add experimental LZMA2 compressor for higher compression ratios than Snappy 2013-10-19 15:37:38 +07:00
Peter Taoussanis
512aca32aa Housekeeping 2013-10-19 13:51:58 +07:00
Peter Taoussanis
f6d5ebd862 Revert "add fast-path encoding for long, doubles, keywords, and strings" 
This reverts commit a92c493375.

As per https://github.com/ptaoussanis/nippy/pull/31, tests appear to be
inconclusive about the effetcs of this commit. Candidate for future
reconsideration.
 2013-10-17 17:44:54 +07:00
Peter Taoussanis
ae0a2e7907 Update benchmarks 2013-10-12 16:45:19 +07:00
Zach Tellman
a92c493375 add fast-path encoding for long, doubles, keywords, and strings 2013-10-12 16:45:19 +07:00
Zach Tellman
5492c1ea0f don't iterate twice over uncounted seqs, and use explicit key/val accessors for kv-collections, appears to give ~25% improvement in 'freeze' 2013-10-12 16:45:19 +07:00
Peter Taoussanis
1b27c1be7a v2.2.0 2013-10-03 21:21:59 +07:00
Peter Taoussanis
49fbbf270f Deps 2013-09-25 16:11:22 +07:00
Peter Taoussanis
6cce8a2857 Housekeeping: docstring typo 2013-08-08 14:22:46 +07:00
Peter Taoussanis
2061dd0868 Add & _ arg for thaw-from-stream! 2013-08-07 18:43:04 +07:00
Peter Taoussanis
d15a12af76 Update benchmarks 2013-08-07 18:29:36 +07:00
Peter Taoussanis
08f2cb1273 Use *data-readers* value for edn/read-string 2013-08-07 17:52:58 +07:00
Peter Taoussanis
d3120f0246 Remove :print-dup? option (no longer useful since switch to edn/read-string) 2013-08-07 17:52:00 +07:00
Peter Taoussanis
efa56eca73 Use tools.reader.edn in benchmarks 2013-08-07 17:37:22 +07:00
Peter Taoussanis
05907403dc Revert 8e4cc07 (unnecessary, slower) 2013-08-07 17:37:22 +07:00
Peter Taoussanis
3addeed195 Swap IPersistentMap -> APersistentMap 2013-08-07 17:37:22 +07:00
Peter Taoussanis
8e1d649bcb Swap IPersistentCollection->ISeq (the former seems to interfere with IRecord dispatch) 2013-08-07 17:37:22 +07:00
Peter Taoussanis
a76734a25a Housekeeping, add test for records with custom extension 2013-08-07 17:30:13 +07:00
James Reeves
5442216343 Add a fast serialization for UUID objects 2013-08-07 10:21:10 +01:00
James Reeves
8b13ff68b9 Add a fast serialization for Date objects 2013-08-07 10:19:53 +01:00
Peter Taoussanis
e48ccb4c45 Merge pull request #26 from weavejester/fast-records 
Fast serialization for records
 2013-08-06 23:54:33 -07:00
James Reeves
6ab00df42c Add fast serialization for records 2013-08-06 17:56:43 +01:00
James Reeves
6caff3503e Remove read-eval option made unnecessary by EDN reader 2013-08-06 16:59:04 +01:00
James Reeves
6024a9cc31 Replace unsafe Clojure reader with safe EDN reader 2013-08-06 16:45:30 +01:00
Peter Taoussanis
c69bb0ec5f Perf: only set bindings when necessary 2013-08-02 21:28:51 +07:00
Peter Taoussanis
4071d0f3ec Switch to simpler extend-freeze, extend-thaw custom type API (ALPHA) 2013-08-02 21:28:51 +07:00
Peter Taoussanis
c2a964932c Add custom-freezer macro for easier Freezable extension 2013-08-02 15:04:11 +07:00
Peter Taoussanis
99091b0a32 Add support for custom thaw readers 2013-07-29 16:43:29 +07:00
Peter Taoussanis
8e4cc072e3 Move Reader fallback out of Freezable protocol to make protocol extensible 2013-07-29 15:58:51 +07:00
Peter Taoussanis
69611657c7 Expose low-level fns: freeze-to-stream!, thaw-from-stream! 2013-07-29 15:57:36 +07:00
Peter Taoussanis
a8b1686a54 Make head-meta ^:const 2013-07-25 15:41:13 +07:00
Peter Taoussanis
93ef118e80 Housekeeping 2013-07-06 15:55:59 +07:00
Peter Taoussanis
4adb7fa376 Housekeeping 2013-06-24 15:22:54 +07:00
Peter Taoussanis
8a33f15801 Temporarily disable grease dep 2013-06-22 18:38:16 +07:00
Max Penet
fb09ebb89b try grease 0.2.1 2013-06-18 18:32:54 +02:00
Peter Taoussanis
ac3901f626 Reverse use of fast io-streams 
Seemed to be causing issues in multi-JVM auto-tests. The added perf. benefit was
minor for Nippy's use case (~2-3%) - so this isn't an urgent change. Will
re-examine again later.
 2013-06-18 14:08:22 +07:00
Peter Taoussanis
f7a96bd37f Housekeeping 2013-06-18 13:48:34 +07:00
Peter Taoussanis
9a2b0a068b thaw API should throw on first non-legacy error 2013-06-18 09:49:42 +07:00
Peter Taoussanis
4326da0ae5 Add tooling API 2013-06-17 23:20:17 +07:00
Peter Taoussanis
c8c0b99ae2 Update benchmarks (2.0.0-alpha6 w/fast io-streams) 2013-06-17 15:12:18 +07:00
Peter Taoussanis
3e6b12ac11 Housekeeping for 67633d5 2013-06-17 13:54:45 +07:00
Max Penet
67633d5fd6 try non sync version of byte-array iostreams 2013-06-16 22:27:22 +02:00
Peter Taoussanis
559c73abef Fix core dump issue (important), simplify API 
PROBLEM: :legacy-mode :auto/true thawing was resulting in JVM core
dumps when attempting to use Snappy to decompress encrypted data.

CAUSE: The org.iq80.snappy implementation seems to choke on the
random IV byte data being generated by the AES128 encrypter. This
may or may not be a bug (still awaiting feedback from lib's authors).

SOLUTION: We're only susceptible to this issue when trying to
decompress data that is: a) encrypted, b) being thawed in legacy mode.
In particular, we're _not_ susceptible to this issue when thawing
in non-legacy mode because in that case we have a header explicitly
warning us that the data is encrypted.

An obvious work-around, therefore, is just to disable decryption when
attempting to thaw legacy-mode data. In practice this isn't a problem
because older versions of Nippy didn't support encryption anyway.
 2013-06-16 19:33:23 +07:00
Peter Taoussanis
e351fdfc43 Rename default compressor, encryptor 2013-06-16 17:55:20 +07:00
Peter Taoussanis
a2376184dc Auto AES128 block size 2013-06-16 16:20:55 +07:00
Peter Taoussanis
5877dc4df0 Update benchmarks (2.0.0-alpha6) 2013-06-16 14:06:36 +07:00
Peter Taoussanis
4ec950417c Rewrite all stream helpers as macros (performance) 2013-06-16 13:23:32 +07:00
Peter Taoussanis
26ad8dd252 Reorganize low-level stream fns (prepping for perf work) 2013-06-16 13:23:32 +07:00
Peter Taoussanis
c5d039b183 Make coll-thaw, coll-thaw-kvs macros (performance) 2013-06-16 13:23:32 +07:00
Peter Taoussanis
9c207cd6af Housekeeping 2013-06-16 13:23:32 +07:00
Max Penet
63765c8510 move thaw exception handler outside of thaw (it's recreated everytime the function gets called now) 2013-06-16 11:58:58 +07:00
Max Penet
e4cde95d54 repeatedly-into as a macro 2013-06-16 11:47:56 +07:00
Peter Taoussanis
823f2c3c46 Housekeeping 2013-06-15 20:41:43 +07:00
Peter Taoussanis
c42457a48c Housekeeping, new keyword implementation (mpenet) 2013-06-14 17:52:25 +07:00
Peter Taoussanis
f706a51a4d Rework v2 header format 2013-06-14 17:49:53 +07:00
Peter Taoussanis
15dd24ac06 Add :auto legacy mode for _full_, transparent backwards-compatibility 2013-06-13 19:01:57 +07:00
Peter Taoussanis
4bc6dde12b Update benchmarks (2.0.0-alpha1) 2013-06-13 16:41:54 +07:00
Peter Taoussanis
8d48ec9d75 NB: Simpler, more flexible API (backwards-compatible) 2013-06-13 16:41:54 +07:00
Peter Taoussanis
284d11c660 Prep for pluggable compression+encryption 2013-06-13 16:41:54 +07:00
Peter Taoussanis
5a398efd9f Rename repeatedly* -> repeatedly-into 2013-06-13 16:41:54 +07:00
Peter Taoussanis
bfc16ce3ab Optimize collection creation (much faster) 2013-06-13 16:41:54 +07:00
Peter Taoussanis
613c77b8a1 Further prep for new API, mark freeze-to-bytes and thaw-from-bytes as deprecated 2013-06-13 11:45:45 +07:00
Peter Taoussanis
9734e882bb Swap repeatedly -> utils/repeatedly* (faster) 2013-06-13 11:45:45 +07:00
Peter Taoussanis
7705c42142 Housekeeping prep for new API 2013-06-13 11:45:45 +07:00