luciano/nippy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
573 commits 2 branches 109 tags 2 MiB
Commit graph

310 commits

Author SHA1 Message Date
Peter Taoussanis
15f0de1658 Simplify stream thaw API, switch from macros->fns 2015-09-30 12:29:04 +07:00
Peter Taoussanis
50ffb78c22 Refer rename: encore->enc 2015-09-30 12:29:04 +07:00
Peter Taoussanis
734e88b20c defonce on all dynamic vars (allow alter-var-root) 2015-09-30 12:29:04 +07:00
Peter Taoussanis
f67f9da64e Remove alpha status on final-freeze-fallback 2015-09-30 12:29:04 +07:00
Peter Taoussanis
9c1e8751c4 Simplify stream freeze API, switch from macros->fns 2015-09-30 12:29:04 +07:00
Peter Taoussanis
da77b3d582 NB: Remove (long-deprecated) freezing legacy mode 2015-09-30 12:29:04 +07:00
Peter Taoussanis
89c9328596 Experimental optimization: zero-copy freeze mode 2015-09-30 12:29:04 +07:00
Peter Taoussanis
fa17eb3a78 Update benchmarks 2015-09-30 12:29:04 +07:00
Peter Taoussanis
327a800d80 Experimental: optimize common case of small maps, sets, vectors 2015-09-30 12:29:04 +07:00
Peter Taoussanis
b298d690c7 Misc hk, reorganize type ids 2015-09-30 12:29:04 +07:00
Peter Taoussanis
885f192f6b Micro optimization: drop unnecessary double kvs count 
Had a vestigial count doubling from an historical implementation
that constructed hash-maps using `(apply hash-map ...)`
 2015-09-30 12:20:53 +07:00
Peter Taoussanis
4765a32e4e Optimize compact long freezer 2015-09-30 11:57:47 +07:00
Peter Taoussanis
1506747e42 Tune buffer size, freeze compressor selector 2015-09-30 11:57:47 +07:00
Peter Taoussanis
9c33f4f5ac Update benchmarks 2015-09-30 11:57:47 +07:00
Peter Taoussanis
ea9286dc90 Micro optimization: kv run is faster still 2015-09-30 11:57:47 +07:00
Peter Taoussanis
1ae8e6c389 Micro optimization: destructure faster than explicit calls here 2015-09-30 11:57:47 +07:00
Peter Taoussanis
956ce7df7e Micro optimization: read-bytes expansion 2015-09-30 11:57:46 +07:00
Peter Taoussanis
7ae954a229 Micro optimization: remove & args 2015-09-30 11:57:46 +07:00
Peter Taoussanis
2f27666d05 Rename deprecated type ids 
Making room for >1 deprecated id per type
 2015-09-30 11:57:46 +07:00
Peter Taoussanis
bbbc12ce30 Misc hk 2015-09-30 11:57:46 +07:00
Peter Taoussanis
db375e7686 Perf: anon fn is faster here 2015-09-30 11:57:46 +07:00
Peter Taoussanis
cea505484a Note re double vestigial kvs length 2015-09-30 11:57:46 +07:00
Peter Taoussanis
36abe07f2b Revert "Experimental/perf: kvs work directly against MapEntry" 
This reverts commit e150775cfe82f8206ddc88034417421e200851fa.
 2015-09-30 11:57:45 +07:00
Peter Taoussanis
aa9a18088f Experimental/perf: kvs work directly against MapEntry 2015-09-30 11:57:45 +07:00
Peter Taoussanis
9d4db3106e Perf: create Ratio's directly 2015-09-30 11:57:45 +07:00
Peter Taoussanis
41d3dcd467 *default-freeze-compressor-selector* should be public 2015-09-17 11:11:06 +07:00
Peter Taoussanis
a3847a4818 Misc housekeeping 2015-09-17 11:11:06 +07:00
Peter Taoussanis
d57788125a Update benchmarks 2015-09-17 11:11:06 +07:00
Peter Taoussanis
e403c17417 Housekeeping for 0a35b8c 2015-09-17 11:11:06 +07:00
Karsten Schmidt
3f9fe327e0 [#70] move small? check outside write-bytes macro body, replace if-not's w/ if's (@postspectacular) 2015-09-17 11:11:06 +07:00
Peter Taoussanis
1ba3c38ab2 Update benchmarks 2015-09-14 20:12:11 +07:00
Peter Taoussanis
56b33e23f0 Perf: fix boxed math on long compression 2015-09-14 20:12:11 +07:00
Peter Taoussanis
da671cbba4 Housekeeping 2015-09-14 17:08:23 +07:00
Peter Taoussanis
6b4e1341ee Realign type ids 2015-09-14 17:08:23 +07:00
Peter Taoussanis
11545690c8 NB switch to encore edn reader/writer 2015-09-14 17:08:23 +07:00
Peter Taoussanis
e5ccd25848 Fix custom-readers var declaration 2015-09-14 17:08:23 +07:00
Peter Taoussanis
5a623870fe Warn about thaw-id reset at runtime rather than expansion time 2015-09-14 17:08:23 +07:00
Peter Taoussanis
82294f54f9 Switch to dynamic custom-readers 2015-09-14 17:08:23 +07:00
Peter Taoussanis
a005a9d7fa Switch to dynamic default-freeze-compressor-selector 2015-09-14 17:08:23 +07:00
Peter Taoussanis
b42aff9cc9 [#68] NB hotfix: encryption thread safety 2015-09-14 16:58:02 +07:00
Peter Taoussanis
c5f4ef74fd Update benchmarks 2015-06-04 18:55:33 +07:00
Peter Taoussanis
2b19f8eab5 Refactor reading of possibly-unthawable types 2015-05-29 14:20:14 +07:00
Peter Taoussanis
53714cc192 More robust record deserialization 2015-05-29 14:13:35 +07:00
Peter Taoussanis
7753eefd2e Switch to encore v1.28.0, doseq->backport-run! (better perf) 2015-05-06 17:37:12 +07:00
Peter Taoussanis
d5fea13bfb Misc housekeeping 2015-04-29 11:47:39 +07:00
Peter Taoussanis
fb2154fb01 Update benchmarks 2015-04-29 11:47:39 +07:00
Peter Taoussanis
89e709d5db Boxed math optimizations 2015-04-18 14:13:29 +07:00
Peter Taoussanis
0e691cbbcf Experimental optimization: allow smart, automatic compressor selection 2015-04-18 14:13:29 +07:00
Peter Taoussanis
e991ec7899 Add encore version check 2015-02-18 17:33:37 +07:00
Peter Taoussanis
df1d83275d [#63] Fix missing thaw exception cause (@cespare) 2015-02-18 17:33:37 +07:00
Peter Taoussanis
af293fec7b [#59] freezable? should return true for clojure.lang.PersistentVector (@chairmanwow) 2015-01-13 13:32:37 +07:00
Peter Taoussanis
26c01c799a Add option to disable v1 thaw compatibility 2014-11-27 17:19:02 +07:00
Peter Taoussanis
f1b4c7ff88 Add type-hint for new LZ4 dep 2014-11-27 15:43:31 +07:00
Peter Taoussanis
e142063201 [#54] Fix bad v1 data thaw call (@jafingerhut) 2014-11-10 11:17:28 +07:00
kul
e006fbcbb0 Expose exception to user as :throwable (@kul) 2014-11-04 12:33:19 +07:00
Peter Taoussanis
152fec95e0 Fix ex-info call args (@kul) 2014-11-04 12:31:41 +07:00
Peter Taoussanis
4d1a374c8f Typehint typo 2014-11-04 12:31:41 +07:00
Peter Taoussanis
7925982939 Print warning when replacing a custom type reader 2014-08-27 19:20:23 +07:00
Peter Taoussanis
13cdf06a60 Make coerce-custom-type-id private 2014-08-27 19:20:23 +07:00
Peter Taoussanis
9e60939848 EXPERIMENTAL: Support keyword-id extensions (#50) 2014-07-06 13:25:08 +07:00
Peter Taoussanis
70dd8f637c Lock-free memoized type test 2014-04-29 23:10:26 +07:00
Peter Taoussanis
06cf8aefd6 Update benchmarks 2014-04-29 23:10:26 +07:00
Peter Taoussanis
20b1c2b1d2 Encode compression type in Nippy header, major refactor/housekeeping 
Housekeeping includes:
  * Importing useful encryption+compression stuff into primary ns
    for lib consumers.
  * Promoting a number of things from Alpha status.
  * Exceptions are now all `ex-info`s.
  * Simplification of `thaw` API: Nippy v1 support is now automatic
    & configuration-free (performance impact in most cases is negligible).
 2014-04-29 23:10:26 +07:00
Peter Taoussanis
b7a454a9c8 EXPERIMENTAL: Make LZ4 the default Nippy compressor (back-compatible for header'ed data) 2014-04-29 23:10:26 +07:00
Peter Taoussanis
5b9358acf9 LZ4: use fast decompressor 2014-04-29 23:10:26 +07:00
Peter Taoussanis
a6aba2c92a Add experimental LZ4 compressors 2014-04-29 23:10:26 +07:00
Peter Taoussanis
3c8661a094 [#48] Fix freeze/thaw identity on empty lazy seqs 2014-04-29 23:08:00 +07:00
Peter Taoussanis
7d84099074 IMPORTANT FIX: small strings weren't getting a proper UTF-8 encoding (xkihzew) 
`(.getBytes <string>)` was being used here instead of
`(.getBytes <string> "UTF-8")` as is correct and done elsewhere.

This means that small UTF-8 strings may have been incorrectly stored
in environments where UTF-8 is not the default JVM character encoding.

Bug was introduced in Nippy v2.6.0, released 9 days ago (2014 Mar 30).

*********************************************************************
Please check for possible errors in Unicode text written using Nippy
v2.6.0 if your JVM uses an alternative character encoding by default
*********************************************************************

Really sorry about this, not sure how this slipped in.

Thanks to @xkihzew for the bug report.
 2014-04-08 17:04:53 +07:00
Peter Taoussanis
6058ac7b54 Fix freezable? fn name typo (mbossenbroek) 2014-03-18 14:23:15 +07:00
Peter Taoussanis
54389cc3a0 NB migrate utils stuff to encore lib 2014-02-23 19:15:55 +07:00
Peter Taoussanis
107cc53f65 Check freezable? against atom type 2014-02-16 19:09:02 +07:00
Peter Taoussanis
42b366a432 Experimental freezable? fn 2014-02-16 18:49:45 +07:00
Peter Taoussanis
0b56746a61 Mod (experimental) Compressable-LZMA2 to use new :skip-header? mode, add :headerless-meta assertion 2014-02-09 18:29:16 +07:00
Peter Taoussanis
e2847f348f [#38]: Distinguish between BigInt/BigInteger on thawing (mlacorte) 2014-01-30 16:16:50 +07:00
Peter Taoussanis
3f982e78cd Make public some macros useful for custom extensions, add experimental compact-long type 2014-01-23 23:13:21 +07:00
Peter Taoussanis
cfa0969bea Optimize some small, common types 2014-01-22 16:50:53 +07:00
Peter Taoussanis
1edde2d53b Revert "New compact byte representation, backwards-compatible" 
This reverts commit 93ce024f58.
 2014-01-22 15:38:32 +07:00
Peter Taoussanis
93ce024f58 New compact byte representation, backwards-compatible 2014-01-22 15:38:22 +07:00
Peter Taoussanis
1d2daf206b Clean up new DataInput/Output API 2014-01-22 15:01:12 +07:00
Peter Taoussanis
87fcd3a9c6 Generalise to allow serialising from/to any DataInput/DataOutput. (cmf) 2014-01-22 14:56:07 +07:00
Peter Taoussanis
b413adc62a Make id defs easily IDE-collapsable 2014-01-22 14:56:07 +07:00
Peter Taoussanis
5690a9611e Improve support for headerless freezing (docs, error msgs, etc.) 2014-01-22 14:01:27 +07:00
Peter Taoussanis
d8bbbbdb32 Add Fressian benchmark 2014-01-21 14:55:51 +07:00
Peter Taoussanis
42f45a0a06 Bench :fast before :encrypted 2014-01-21 14:55:50 +07:00
Peter Taoussanis
42950ec24a Add stress-data-comparable, stress-data-benchable 2014-01-21 14:55:50 +07:00
Peter Taoussanis
7e1b1bb6b4 Deprecate :legacy-mode, add :skip-header? mode 2014-01-21 14:55:50 +07:00
Peter Taoussanis
32e4e82091 BREAKING: Clean up (experimental) Compressable-LZMA2 2014-01-21 12:56:36 +07:00
Peter Taoussanis
c11d4e36a8 hk 2014-01-21 12:47:46 +07:00
Peter Taoussanis
d112ecc395 Revert "Experimental: write-compact-int for _all_ int-like writing" 
This reverts commit 93afb3820e.
 2014-01-21 12:46:35 +07:00
Peter Taoussanis
93afb3820e Experimental: write-compact-int for _all_ int-like writing 2014-01-21 12:46:23 +07:00
Peter Taoussanis
a5c2f39865 Revert "wip: experimenting with common number optimizations" 
This reverts commit 307a265447.
 2014-01-20 15:12:18 +07:00
Peter Taoussanis
307a265447 wip: experimenting with common number optimizations 2014-01-20 14:44:32 +07:00
Peter Taoussanis
f8239531da v2.5.2 2013-12-07 16:49:34 +07:00
Peter Taoussanis
5ea4af7356 Further work on freezer fallback reliability 2013-12-07 16:44:53 +07:00
Peter Taoussanis
3093eae322 Experimental: add freeze-fallback-as-str fn 2013-12-03 16:46:30 +07:00
Peter Taoussanis
43c85059d7 Throw on freeze (rather than thaw) when trying to freeze an unreadable object with pr-str 2013-12-03 16:22:52 +07:00
Peter Taoussanis
1a7812522c Experimental: add inspect-ba util for inspecting possibly-frozen data 2013-12-03 16:22:51 +07:00
Peter Taoussanis
730b8774c4 Utils: housekeeping, update bench, memoized 2013-11-08 21:45:28 +07:00
Peter Taoussanis
0fd9275e37 Experimental compressable-lzma2: fix thaw bug 2013-11-07 12:47:36 +07:00
Peter Taoussanis
da1622f271 Experimental: add Compressable-LZMA2 type & (replaceable) de/serializer 2013-11-06 13:58:55 +07:00
Peter Taoussanis
7e7975910a tools: fix thaw arg destructuring & opt merging 2013-11-05 22:41:51 +07:00
Peter Taoussanis
84e336fe65 Make byte transform on seqs 1-to-1, re-add ISeq as a fallback implementation 2013-10-31 13:29:34 +07:00
Peter Taoussanis
aa78ad6a0b Rename test type: :coll->:lazy-seq 2013-10-31 13:16:26 +07:00
Peter Taoussanis
b1dab1dc4b Add debug-mode flag 2013-10-31 13:15:22 +07:00
Peter Taoussanis
8f8ab99e95 Add some Serializable tests (exceptions) 2013-10-24 13:55:47 +07:00
Peter Taoussanis
9448d2b3ce Add Java's Serializable as a pre-Reader fallback mechanism 2013-10-24 13:55:47 +07:00
Peter Taoussanis
c58bf97af7 Move Record test into stress data 2013-10-24 13:55:47 +07:00
Peter Taoussanis
5c48ba8e26 Reduce amount of (unnecessary) freeze procotol extensions to interfaces 
These can cause confusing, inconsistent behaviour when the protocol dispatch
needs to choose between multiple implementations:
Ref. https://groups.google.com/forum/#!topic/clojure-dev/-zoLA78--Mo

The approach as of this commit is more robust and allows more flexibility in
extension.
 2013-10-24 13:55:47 +07:00
Peter Taoussanis
75dd0d9040 Remove tagged literal labels for uuid & date (both have been baked-in for some time) 2013-10-24 13:55:47 +07:00
Peter Taoussanis
a122e62fe0 Reposition freeze-to-stream! fn for clarity 2013-10-24 13:55:47 +07:00
Peter Taoussanis
26f32c7a95 Housekeeping 2013-10-24 13:34:38 +07:00
Peter Taoussanis
33eee786be Lower default LZMA2 compression level -> 0 (much better perf @ ~same ratio) 2013-10-19 18:17:47 +07:00
Peter Taoussanis
3879cf4d62 Add experimental LZMA2 compressor for higher compression ratios than Snappy 2013-10-19 15:37:38 +07:00
Peter Taoussanis
512aca32aa Housekeeping 2013-10-19 13:51:58 +07:00
Peter Taoussanis
f6d5ebd862 Revert "add fast-path encoding for long, doubles, keywords, and strings" 
This reverts commit a92c493375.

As per https://github.com/ptaoussanis/nippy/pull/31, tests appear to be
inconclusive about the effetcs of this commit. Candidate for future
reconsideration.
 2013-10-17 17:44:54 +07:00
Peter Taoussanis
ae0a2e7907 Update benchmarks 2013-10-12 16:45:19 +07:00
Zach Tellman
a92c493375 add fast-path encoding for long, doubles, keywords, and strings 2013-10-12 16:45:19 +07:00
Zach Tellman
5492c1ea0f don't iterate twice over uncounted seqs, and use explicit key/val accessors for kv-collections, appears to give ~25% improvement in 'freeze' 2013-10-12 16:45:19 +07:00
Peter Taoussanis
1b27c1be7a v2.2.0 2013-10-03 21:21:59 +07:00
Peter Taoussanis
49fbbf270f Deps 2013-09-25 16:11:22 +07:00
Peter Taoussanis
6cce8a2857 Housekeeping: docstring typo 2013-08-08 14:22:46 +07:00
Peter Taoussanis
2061dd0868 Add & _ arg for thaw-from-stream! 2013-08-07 18:43:04 +07:00
Peter Taoussanis
d15a12af76 Update benchmarks 2013-08-07 18:29:36 +07:00
Peter Taoussanis
08f2cb1273 Use *data-readers* value for edn/read-string 2013-08-07 17:52:58 +07:00
Peter Taoussanis
d3120f0246 Remove :print-dup? option (no longer useful since switch to edn/read-string) 2013-08-07 17:52:00 +07:00
Peter Taoussanis
efa56eca73 Use tools.reader.edn in benchmarks 2013-08-07 17:37:22 +07:00
Peter Taoussanis
05907403dc Revert 8e4cc07 (unnecessary, slower) 2013-08-07 17:37:22 +07:00
Peter Taoussanis
3addeed195 Swap IPersistentMap -> APersistentMap 2013-08-07 17:37:22 +07:00
Peter Taoussanis
8e1d649bcb Swap IPersistentCollection->ISeq (the former seems to interfere with IRecord dispatch) 2013-08-07 17:37:22 +07:00
Peter Taoussanis
a76734a25a Housekeeping, add test for records with custom extension 2013-08-07 17:30:13 +07:00
James Reeves
5442216343 Add a fast serialization for UUID objects 2013-08-07 10:21:10 +01:00
James Reeves
8b13ff68b9 Add a fast serialization for Date objects 2013-08-07 10:19:53 +01:00
Peter Taoussanis
e48ccb4c45 Merge pull request #26 from weavejester/fast-records 
Fast serialization for records
 2013-08-06 23:54:33 -07:00
James Reeves
6ab00df42c Add fast serialization for records 2013-08-06 17:56:43 +01:00
James Reeves
6caff3503e Remove read-eval option made unnecessary by EDN reader 2013-08-06 16:59:04 +01:00
James Reeves
6024a9cc31 Replace unsafe Clojure reader with safe EDN reader 2013-08-06 16:45:30 +01:00
Peter Taoussanis
c69bb0ec5f Perf: only set bindings when necessary 2013-08-02 21:28:51 +07:00
Peter Taoussanis
4071d0f3ec Switch to simpler extend-freeze, extend-thaw custom type API (ALPHA) 2013-08-02 21:28:51 +07:00
Peter Taoussanis
c2a964932c Add custom-freezer macro for easier Freezable extension 2013-08-02 15:04:11 +07:00
Peter Taoussanis
99091b0a32 Add support for custom thaw readers 2013-07-29 16:43:29 +07:00
Peter Taoussanis
8e4cc072e3 Move Reader fallback out of Freezable protocol to make protocol extensible 2013-07-29 15:58:51 +07:00
Peter Taoussanis
69611657c7 Expose low-level fns: freeze-to-stream!, thaw-from-stream! 2013-07-29 15:57:36 +07:00
Peter Taoussanis
a8b1686a54 Make head-meta ^:const 2013-07-25 15:41:13 +07:00
Peter Taoussanis
93ef118e80 Housekeeping 2013-07-06 15:55:59 +07:00
Peter Taoussanis
4adb7fa376 Housekeeping 2013-06-24 15:22:54 +07:00
Peter Taoussanis
8a33f15801 Temporarily disable grease dep 2013-06-22 18:38:16 +07:00
Max Penet
fb09ebb89b try grease 0.2.1 2013-06-18 18:32:54 +02:00
Peter Taoussanis
ac3901f626 Reverse use of fast io-streams 
Seemed to be causing issues in multi-JVM auto-tests. The added perf. benefit was
minor for Nippy's use case (~2-3%) - so this isn't an urgent change. Will
re-examine again later.
 2013-06-18 14:08:22 +07:00
Peter Taoussanis
f7a96bd37f Housekeeping 2013-06-18 13:48:34 +07:00
Peter Taoussanis
9a2b0a068b thaw API should throw on first non-legacy error 2013-06-18 09:49:42 +07:00