Compare commits

..

89 commits
cljc ... master

Author SHA1 Message Date
Imre Kószó
78076f8cd0 update build script
- add scm block to pom so it links to the repo
- add license as it is now required by clojars
- fix paths in build alias
- bump tools.build
2024-01-22 13:16:55 +01:00
Imre Kószó
18de93c8fe cut 0.19.6 2024-01-22 11:43:53 +01:00
Imre Kószó
7ae8bed997 fix MapEntry not found error in clj 2024-01-22 11:43:53 +01:00
Christophe Grand
2079b74271 tidying for released 2023-07-13 11:21:09 +02:00
Baptiste Dupuch
3257561937 Bug fixing 2023-07-13 11:21:09 +02:00
Baptiste Dupuch
c6f4b7d041 cljd support 2023-07-13 11:21:09 +02:00
Baptiste Dupuch
1e23966d5e rfs done for cljd 2023-07-13 11:21:09 +02:00
Christophe Grand
66f3c1ac59
Merge pull request #51 from The-Alchemist/patch-1
removing Travis CI build status from README.md
2023-07-13 10:23:14 +02:00
The Alchemist
a814a220d0
removing Travis CI build status from README.md
Appears to be a dead link.
2023-07-09 14:59:58 -04:00
Imre Kószó
550dbc150a Cut version 0.19.4 2022-08-05 21:46:04 +02:00
Imre Kószó
9f61ac1747 Fix releases link in readme 2022-08-05 21:42:55 +02:00
Imre Kószó
bbdf1633ce Update readme to always refer to latest release 2022-08-05 21:38:27 +02:00
Imre Kószó
d6edde7704 Fix #40 x/sort & x/sort-by in cljs 2022-08-05 21:34:40 +02:00
Imre Kószó
1c209146bf fix cljs compilation broken in 0.19.3 2022-08-05 20:45:21 +02:00
Imre Kószó
bf6e6c4413 Fix imbalanced code block in readme 2022-08-02 13:37:50 +02:00
Imre Kószó
52cbfd29fc Extend dependency info 2022-08-02 13:36:49 +02:00
Imre Kószó
c4b8129c1b Update dependency coordinates 2022-08-02 13:32:14 +02:00
Imre Kószó
f4ebaea156 Cut version 0.19.3 2022-08-01 15:20:25 +02:00
Imre Kószó
516b8b6306
Merge pull request #48 from imrekoszo/fixes-2022-07
Fix various bugs, add deps.edn, support babashka
2022-08-01 15:09:53 +02:00
Imre Kószó
6c5d75c65e Update changelog 2022-08-01 15:00:31 +02:00
Imre Kószó
8191719956 Merge branch 'intoperf' into fixes-2022 2022-07-31 15:49:15 +02:00
Imre Kószó
be68293056 Merge branch 'issue-44' into fixes-2022 2022-07-31 15:49:08 +02:00
Imre Kószó
e8807764b1 Merge branch 'test-nits' into fixes-2022 2022-07-31 15:47:30 +02:00
Imre Kószó
5872fa5147 Merge remote-tracking branch 'guv/reflection-in-io' into fixes-2022 2022-07-31 15:46:36 +02:00
Imre Kószó
e8c8e1a37d Merge branch 'tools-deps' into fixes-2022 2022-07-31 15:46:05 +02:00
Imre Kószó
4391e6cc91 Add deps.edn & dev/test tooling 2022-07-31 15:38:07 +02:00
Imre Kószó
86d9ef14f1 Fix some reflection warnings seen in tests 2022-07-31 15:32:37 +02:00
Imre Kószó
891a7f98ba Fix xforms-test shadowing clojure.core vars
Get rid of the following warnings:
WARNING: reductions already refers to: #'clojure.core/reductions in namespace: net.cgrand.xforms-test, being replaced by: #'net.cgrand.xforms-test/reductions
WARNING: partition already refers to: #'clojure.core/partition in namespace: net.cgrand.xforms-test, being replaced by: #'net.cgrand.xforms-test/partition
2022-07-31 15:32:37 +02:00
Imre Kószó
10c58a0a81 Fix x/into perf problem when from is a small map 2022-07-31 15:05:46 +02:00
Imre Kószó
150a5712de Fix #44 (destructuring-pair? mistake) 2022-07-31 14:51:36 +02:00
Michiel Borkent
283cb13e58 Add compatibility with babashka 2021-11-24 12:06:29 +01:00
Gunnar Völkel
e9216a7d71 Fixes reflection usage in xforms.io. 2020-05-27 10:40:09 +02:00
Christophe Grand
62375212a8 Pushing 0.19.2 because 0.19.1 on clojars is badly built 2019-10-02 10:55:05 +02:00
Christophe Grand
9808cc8555 fix #31 -- but should be fixed on the CLJS side 2019-09-09 12:55:03 +02:00
Christophe Grand
00fae8ede4 Fix example
wax failing on duplicate values
2019-02-28 15:30:20 +01:00
Christophe Grand
3ab242de11
Fix typo thanks to jjttjj on Clojurians Slack for reporting it 2019-02-27 16:29:18 +01:00
Christophe Grand
e9361072c1 0.19.0 add x/time 2018-11-14 14:06:06 +01:00
Christophe Grand
9e542d9d00
Merge pull request #28 from zmthy/rfs-reflection-warnings
Remove reflection warnings from min/max rfs
2018-08-30 08:15:56 +00:00
Timothy Jones
521e08b547
Remove reflection warnings from min/max rfs 2018-08-29 14:18:03 -04:00
Christophe Grand
545b97ddf8 min (resp. max) now treats s as their absolute maximum (resp. minimum), so rf/min and rf/max are now associative and practically s are ignored. 2018-06-15 18:15:50 +02:00
Christophe Grand
0410484f90 v0.18.1: new avg semantics on empty inputs (emits nil) 2018-06-13 23:08:05 +02:00
Christophe Grand
445419effb v0.18.0 2018-06-13 23:02:12 +02:00
Christophe Grand
bae87d1b50 x/avg and rfs/avg return nil when no input 2018-06-13 23:02:12 +02:00
Christophe Grand
c65e766395
Merge pull request #23 from zmthy/deque-reflection-warnings
Resolve reflection warnings on ArrayDeque
2018-04-20 15:47:23 +02:00
Christophe Grand
d2533ae7fc
Merge pull request #22 from jgrodziski/master
Add a Troubleshooting section to README.md because of a bug I encountered
2018-04-20 15:47:02 +02:00
Timothy Jones
284203079a
Resolve reflection warnings on ArrayDeque 2018-04-20 11:02:43 +12:00
Christophe Grand
f2165ba932 xio/sh produces reducible collections 2018-04-16 23:08:27 +02:00
Jérémie Grodziski
34427101d7
Add a Troubleshooting section
I stumbled upon a strange interaction and bug when using xforms in a [clojurescript + figwheel + emacs / cider nrepl] dev environment. Adding xforms triggers a bug with the REPL evaluation result, adding the nrepl-middleware to figwheel solved the issue (dev env was correctly working before). I submit this to keep track of the issue if other people stumble upon it in the future.
2018-03-16 16:14:33 +00:00
Christophe Grand
00e19651ed
Can't type, blame the flu. 2018-01-25 12:13:04 +01:00
Christophe Grand
7c048e11bb 0.16.0: 2-arg x/str is now a string-producing transducing context. Adds x/wrap.
Also change documentation to use the term aggregator for 1-item-out transducers.
2018-01-25 10:41:36 +01:00
Christophe Grand
db0555c358 make xforms a npm package 2017-11-17 09:30:27 +01:00
Christophe Grand
82c7c1fff2 v0.15 stream.Transformer 2017-11-16 18:53:46 +01:00
Christophe Grand
c52dc4c873 v0.14: add sort & sort-by 2017-11-16 16:58:34 +01:00
Christophe Grand
0515b711df 0.13.0 x/without, the opposite of x/into: dissoc/disj'ing instead of conj'ing. 2017-10-26 18:11:19 +02:00
Christophe Grand
37d4732117 Merge pull request #18 from jstokes/patch-1
Remove space in xforms dependency coordinates
2017-10-21 16:44:42 -05:00
Jeff Stokes
00b9e93e6e Remove space in xforms dependency coordinates 2017-10-21 15:43:35 -04:00
Christophe Grand
023ca6043e Fix issue with iterator
When there was only 1-item produced by an aggregating xform .hasNext was returning false.
2017-10-19 11:38:19 -05:00
Christophe Grand
da53490f20 v0.12.1 2017-10-18 19:38:06 -05:00
Christophe Grand
5017069c7a docstring fix 2017-10-18 19:34:22 -05:00
Christophe Grand
6145006946 Fix non-completion bug in iterator 2017-10-18 19:33:57 -05:00
Christophe Grand
51b61f3889 Add the iterator transducing context 2017-10-18 14:39:50 -05:00
Christophe Grand
b96d9d7994 Make kvrf smarter: can infer arity 2 from and 3 from 2 2017-10-18 14:39:15 -05:00
Christophe Grand
899154c0df xforms 0.11.0, with xio/sh to use any shell process as a transducer 2017-10-05 13:23:40 +02:00
Christophe Grand
89d384ce74 Make 3+-arg arities of edn-out and lines-out to be transducing contexts. 2017-10-05 10:31:53 +02:00
Christophe Grand
ea7a3e699d 0.10.1 fix edn-out and add documentation on edn-in 2017-10-04 15:58:50 +02:00
Christophe Grand
809f8f709b xforms 0.10.0: new io namespace, some as a transducing context (and a rf) too 2017-10-04 15:46:25 +02:00
Christophe Grand
8f04ad0748 Make x/for to unroll some reductions
When an expression in collection position in `x/for` is a collection literal with less than 4 items (or tagged with `^:unroll`) then the collection is not allocated and the reduction over it is unrolled.
2017-09-19 17:26:11 +02:00
Christophe Grand
23feac44dc update CI conf 2017-09-12 22:44:54 +02:00
Christophe Grand
8f9b954a2b 0.9.4: add x/into-by-key, shorthand for (comp (x/by-key ..) (x/into ..)) 2017-09-12 16:29:42 +02:00
Christophe Grand
81d0f0c171 Merge pull request #15 from maacl/patch-1
Clarification regarding use of 1-item transducers
2017-04-28 12:43:29 +00:00
Martin Clausen
568f9fe39d Clarification regarding use of 1-item transducers 2017-04-28 13:25:00 +02:00
Christophe Grand
3bb82ad06d 0.9.3 2017-04-04 09:37:03 +02:00
Christophe Grand
5025e583f2 Fix #14: no call to downstream complete with padded partition when there's no incomplete partition 2017-04-04 09:36:35 +02:00
Christophe Grand
e327d1899b Add failing test for #14 2017-04-04 09:35:09 +02:00
Christophe Grand
362febaf5f for was not emitting kv pairs in cljs because of extra compiler metadata, fix #13 2017-03-28 16:25:35 +02:00
Christophe Grand
5ea2d1b3d3 x/str was documented but not in the repo 2017-02-25 03:44:49 +01:00
Christophe Grand
64eb4c6158 v0.9.0 add take-last and drop-last 2017-02-02 13:16:51 +01:00
Christophe Grand
8a9d383198 v0.8.3: fix #10 (and another bug in padded partition) and update doc 2017-01-26 13:50:10 +01:00
Christophe Grand
d5e5aa3670 0.8.2 identical to 0.8.1 but 0.8.1 on clojars is a dirty build 2017-01-17 12:35:54 +01:00
Christophe Grand
6d2d448e34 0.8.1 2017-01-09 16:41:24 +01:00
Christophe Grand
320668e7b5 fix reader conditional, close #8 2017-01-09 16:41:00 +01:00
Christophe Grand
6047563033 using Welford formula for sd 2017-01-09 16:38:55 +01:00
Christophe Grand
b130b6b6eb v0.8.0 2017-01-05 15:41:39 +01:00
Christophe Grand
d1f721883a Add x/sd and rf/sd for standard deviation 2017-01-05 15:41:15 +01:00
Christophe Grand
d4f0280bb5 use macros/replace 2017-01-05 15:40:46 +01:00
Christophe Grand
0422678643 fix typo in docstring 2017-01-05 15:38:40 +01:00
Christophe Grand
481a5a3b1e 0.7.2 2016-12-19 14:19:50 +01:00
Christophe Grand
e1d2d6c848 fix port bug.perf issue in cljs: transient code path was never taken. 2016-12-19 14:18:01 +01:00
Christophe Grand
3de578463f 0.7.1 2016-12-16 16:28:43 +01:00
16 changed files with 1221 additions and 224 deletions

25
.gitignore vendored Normal file
View file

@ -0,0 +1,25 @@
# Created by https://www.toptal.com/developers/gitignore/api/clojure
# Edit at https://www.toptal.com/developers/gitignore?templates=clojure
### Clojure ###
pom.xml
pom.xml.asc
*.jar
*.class
/lib/
/classes/
/bin/
/node_modules/
/.lumo_cache/
/target/
/checkouts/
.lein-deps-sum
.lein-repl-history
.lein-plugins/
.lein-failures
.nrepl-port
.cpcache/
.clj-kondo/*/
/cljs-test-runner-out/
# End of https://www.toptal.com/developers/gitignore/api/clojure

View file

@ -1,6 +1,6 @@
language: clojure
lein: lein2
script: lein2 do test
lein: lein
script: lein test
jdk:
- openjdk6
- openjdk7

160
README.md
View file

@ -1,29 +1,44 @@
# xforms
More transducers and reducing functions for Clojure!
More transducers and reducing functions for Clojure(script)!
[![Build Status](https://travis-ci.org/cgrand/xforms.png?branch=master)](https://travis-ci.org/cgrand/xforms)
*Transducers* can be classified in three groups: regular ones, higher-order ones
(which accept other transducers as arguments) and aggregators (transducers which emit only 1 item out no matter how many went in).
Aggregators generally only make sense in the context of a higher-order transducer.
*Transducers* (in `net.cgrand.xforms`) can be classified in three groups: regular ones, higher-order ones
(which accept other transducers as arguments) and 1-item ones which emit only 1 item out no matter how many went in.
They generally only make sense in the context of a higher-order transducer.
In `net.cgrand.xforms`:
* regular ones: `partition` (1 arg), `reductions`, `for`, `take-last`, `drop-last`, `sort`, `sort-by`, `wrap`, `window` and `window-by-time`
* higher-order ones: `by-key`, `into-by-key`, `multiplex`, `transjuxt`, `partition` (2+ args), `time`
* aggregators: `reduce`, `into`, `without`, `transjuxt`, `last`, `count`, `avg`, `sd`, `min`, `minimum`, `max`, `maximum`, `str`
In `net.cgrand.xforms.io`:
* `sh` to use any process as a reducible collection (of stdout lines) or as a transducers (input as stdin lines, stdout lines as output).
* regular ones: `partition` (1 arg), `reductions`, `for`, `window` and `window-by-time`
* higher-order ones: `by-key`, `multiplex`, `transjuxt`, `partition` (2+ args)
* 1-item ones: `reduce`, `into`, `last`, `count`, `avg`, `min`, `minimum`, `max`, `maximum`, `str`
*Reducing functions* (in `net.cgrand.xforms.rfs`): `min`, `minimum`, `max`, `maximum`, `str`, `str!`, `avg`, `juxt` and `last`.
*Reducing functions*
Transducing contexts: `transjuxt` (for performing several transductions in a single pass), `into`, `count`.
* in `net.cgrand.xforms.rfs`: `min`, `minimum`, `max`, `maximum`, `str`, `str!`, `avg`, `sd`, `last` and `some`.
* in `net.cgrand.xforms.io`: `line-out` and `edn-out`.
(in `net.cgrand.xforms`)
*Transducing contexts*:
* in `net.cgrand.xforms`: `transjuxt` (for performing several transductions in a single pass), `iterator` (clojure only), `into`, `without`, `count`, `str` (2 args) and `some`.
* in `net.cgrand.xforms.io`: `line-out` (3+ args) and `edn-out` (3+ args).
* in `net.cgrand.xforms.nodejs.stream`: `transformer`.
*Reducible views* (in `net.cgrand.xforms.io`): `lines-in` and `edn-in`.
**Note:** it should always be safe to update to the latest xforms version; short of bugfixes, breaking changes are avoided.
## Add as a dependency
For specific coordinates see the [Releases](https://github.com/cgrand/xforms/releases) page.
## Usage
Add this dependency to your project:
```clj
[net.cgrand/xforms "0.7.0"]
```
```clj
=> (require '[net.cgrand.xforms :as x])
```
@ -103,19 +118,23 @@ Padding is achieved as usual:
;; avg of last 4 items
=> (sequence
(x/window 4 x/avg #(x/avg %1 %2 -1))
(x/window 4 rf/avg #(rf/avg %1 %2 -1))
nums)
(11 19/2 17 77/4 18 37/2 79/4 77/4)
;; min of last 3 items
=> (sequence
(x/window 3
(fn
([] (sorted-set))
([s] (first s))
([s x] (conj s x)))
disj)
nums)
(x/window 3
(fn
([] (sorted-map))
([m] (key (first m)))
([m x] (update m x (fnil inc 0))))
(fn [m x]
(let [n (dec (m x))]
(if (zero? n)
(dissoc m x)
(assoc m x (dec n))))))
nums)
(11 8 8 8 6 6 6 10)
```
@ -127,16 +146,16 @@ It's worth noting that all transformed outputs are subsequently interleaved. See
```clj
=> (sequence (x/partition 2 1 identity) (range 8))
(0 1 1 2 2 3 3 4 4 5 5 6 6 7 7)
(0 1 1 2 2 3 3 4 4 5 5 6 6 7)
=> (sequence (x/by-key odd? identity) (range 8))
([false 0] [true 1] [false 2] [true 3] [false 4] [true 5] [false 6] [true 7])
```
That's why most of the time the last stage of the sub-transducer will be a `x/reduce` or a `x/into`:
That's why most of the time the last stage of the sub-transducer will be an aggregator like `x/reduce` or `x/into`:
```clj
=> (sequence (x/partition 2 1 (x/into [])) (range 8))
([0 1] [1 2] [2 3] [3 4] [4 5] [5 6] [6 7] [7])
([0 1] [1 2] [2 3] [3 4] [4 5] [5 6] [6 7])
=> (sequence (x/by-key odd? (x/into [])) (range 8))
([false [0 2 4 6]] [true [1 3 5 7]])
```
@ -209,6 +228,69 @@ Evaluation count : 24 in 6 samples of 4 calls.
## Changelog
### 0.19.6
* Fix regression in 0.19.5 #54
### 0.19.5
* Support ClojureDart
### 0.19.4
* Fix ClojureScript compilation broken in `0.19.3` #49
* Fix `x/sort` and `x/sort-by` for ClojureScript #40
### 0.19.3
* Add `deps.edn` to enable usage as a [git library](https://clojure.org/guides/deps_and_cli#_using_git_libraries)
* Bump `macrovich` to make Clojure and ClojureScript provided dependencies #34
* Fix reflection warnings in `xforms.io` #35 #36
* Add compatibility with [babashka](https://github.com/babashka/babashka) #42
* Fix `x/destructuring-pair?` #44 #45
* Fix `x/into` performance hit with small maps #46 #47
* Fix reflection and shadowing warnings in tests
### 0.19.2
* Fix infinity symbol causing issues with ClojureScript #31
### 0.19.0
`time` allows to measure time spent in one transducer (excluding time spent downstream).
```clj
=> (time ; good old Clojure time
(count (into [] (comp
(x/time "mapinc" (map inc))
(x/time "filterodd" (filter odd?))) (range 1e6))))
filterodd: 61.771738 msecs
mapinc: 143.895317 msecs
"Elapsed time: 438.34291 msecs"
500000
```
First argument can be a function that gets passed the time (in ms),
this allows for example to log time instead of printing it.
### 0.9.5
* Short (up to 4) literal collections (or literal collections with `:unroll` metadata) in collection positions in `x/for` are unrolled.
This means that the collection is not allocated.
If it's a collection of pairs (e.g. maps), pairs themselves won't be allocated.
### 0.9.4
* Add `x/into-by-key` short hand
### 0.7.2
* Fix transients perf issue in Clojurescript
### 0.7.1
* Works with Clojurescript (even self-hosted).
### 0.7.0
* Added 2-arg arity to `x/count` where it acts as a transducing context e.g. `(x/count (filter odd?) (range 10))`
@ -219,6 +301,30 @@ Evaluation count : 24 in 6 samples of 4 calls.
* Added `x/reductions`
* Now if the first collection expression in `x/for` is not a placeholder then `x/for` works like `x/for` but returns an eduction and performs all iterations using reduce.
## Troubleshooting xforms in a Clojurescript dev environment
If you use xforms with Clojurescript and the Emacs editor to start your figwheel REPL be sure to include the `cider.nrepl/cider-middleware` to your figwheel's nrepl-middleware.
```
:figwheel {...
:nrepl-middleware [cider.nrepl/cider-middleware;;<= that middleware
refactor-nrepl.middleware/wrap-refactor
cemerick.piggieback/wrap-cljs-repl]
...}
```
Otherwise a strange interaction occurs and every results from your REPL evaluation would be returned as a String. Eg.:
```
cljs.user> 1
"1"
cljs.user>
```
instead of:
```
cljs.user> 1
1
cljs.user>
```
## License
Copyright © 2015-2016 Christophe Grand

56
bb.edn Normal file
View file

@ -0,0 +1,56 @@
{:deps {local/deps {:local/root "."}}
:paths ["src" "test"]
:tasks
{:requires ([clojure.string :as str])
:init
(do
(defn kaocha [alias args]
(apply shell "bin/kaocha" alias args))
(defn test-cljs [alias args]
(apply clojure (str/join ["-M:test:cljs-test-runner" alias]) args)))
test-clj-9
{:task (kaocha :clj-1-9 *command-line-args*)}
test-clj-10
{:task (kaocha :clj-1-10 *command-line-args*)}
test-clj-11
{:task (kaocha :clj-1-11 *command-line-args*)}
test-clj
{:depends [test-clj-9 test-clj-10 test-clj-11]}
test-cljs-9
{:task (test-cljs :clj-1-9 *command-line-args*)}
test-cljs-10
{:task (test-cljs :clj-1-10 *command-line-args*)}
test-cljs-11
{:task (test-cljs :clj-1-11 *command-line-args*)}
test-cljs
{:depends [#_test-cljs-9 test-cljs-10 test-cljs-11]}
test-bb
{:requires ([clojure.test :as t]
[net.cgrand.xforms-test])
:task (t/run-tests 'net.cgrand.xforms-test)}
test-all
{:depends [test-bb test-clj test-cljs]}
perf-bb
{:requires ([net.cgrand.xforms :as x])
:task
(let [n 10000
m (zipmap (range 100) (range))
mapping (map (fn [[k v]] [k (inc v)]))
xforing (x/for [[k v] _] [k (inc v)])]
(time (dotimes [_ n] (into {} mapping m)))
(time (dotimes [_ n] (into {} xforing m)))
(time (dotimes [_ n] (x/into {} xforing m))))}}}

3
bin/kaocha Executable file
View file

@ -0,0 +1,3 @@
#!/usr/bin/env bash
set -x
clojure -Srepro -M:kaocha:test"$1" "${@:2}"

46
build.clj Normal file
View file

@ -0,0 +1,46 @@
(ns build
(:require [clojure.tools.build.api :as b]
[clojure.java.shell :as sh]))
(def lib 'net.cgrand/xforms)
(def version "0.19.6" #_(format "0.0.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def basis (b/create-basis {:project "deps.edn"}))
(def jar-file (format "target/%s-%s.jar" (name lib) version))
(def scm {:connection "scm:git:git://github.com/cgrand/xforms.git"
:developerConnection "scm:git:git://github.com/cgrand/xforms.git"
:url "https://github.com/cgrand/xforms"})
(def extra-pom-data
[[:licenses
[:license
[:name "Eclipse Public License 1.0"]
[:url "https://opensource.org/license/epl-1-0/"]
[:distribution "repo"]]
[:license
[:name "Eclipse Public License 2.0"]
[:url "https://opensource.org/license/epl-2-0/"]
[:distribution "repo"]]]])
(defn clean [_]
(b/delete {:path "target"}))
(defn jar [_]
(b/write-pom {:class-dir class-dir
:lib lib
:version version
:basis basis
:src-dirs ["src"]
:scm (assoc scm :tag (str "v" version))
:pom-data extra-pom-data})
(b/copy-dir {:src-dirs ["src" "resources"]
:target-dir class-dir})
(b/jar {:class-dir class-dir
:jar-file jar-file}))
(defn clojars [_]
(sh/sh
"mvn" "deploy:deploy-file" (str "-Dfile=" jar-file)
;target/classes/META-INF/maven/net.cgrand/xforms/pom.xml
(format "-DpomFile=%s/META-INF/maven/%s/%s/pom.xml"
class-dir (namespace lib) (name lib))
"-DrepositoryId=clojars" "-Durl=https://clojars.org/repo/"))

45
deps.edn Normal file
View file

@ -0,0 +1,45 @@
{:deps {net.cgrand/macrovich {:mvn/version "0.2.2"}}
:paths ["src"]
:aliases
{:dev
{:extra-paths ["dev"]}
:cljd
{:extra-deps
{tensegritics/clojuredart
{:git/url "https://github.com/tensegritics/ClojureDart.git"
:sha "ae1b485e84ccc35b122f776dfc7cc62198274701"}}}
:clj-1-9
{:extra-deps
{org.clojure/clojure {:mvn/version "1.9.0"}
org.clojure/clojurescript {:mvn/version "1.9.293"}}}
:clj-1-10
{:extra-deps
{org.clojure/clojure {:mvn/version "1.10.3"}
org.clojure/clojurescript {:mvn/version "1.10.914"}}}
:clj-1-11
{:extra-deps
{org.clojure/clojure {:mvn/version "1.11.1"}
org.clojure/clojurescript {:mvn/version "1.11.60"}}}
:test
{:extra-paths ["test"]}
:kaocha
{:extra-paths ["test"]
:extra-deps {lambdaisland/kaocha {:mvn/version "1.69.1069"}}
:main-opts ["-m" "kaocha.runner"]}
:cljs-test-runner
{:extra-paths ["test"]
:extra-deps {olical/cljs-test-runner {:mvn/version "3.8.0"}}
:main-opts ["-m" "cljs-test-runner.main"]}
:build
{:paths ["."]
:deps {io.github.clojure/tools.build {:git/tag "v0.9.6" :git/sha "8e78bcc"}}
:ns-default build}}}

3
dev/user.clj Normal file
View file

@ -0,0 +1,3 @@
(ns user)
(set! *warn-on-reflection* true)

21
package.json Normal file
View file

@ -0,0 +1,21 @@
{
"name": "xforms",
"version": "0.16.0",
"description": "Extra transducers for Clojurescript",
"repository": "https://github.com/cgrand/xforms.git",
"author": "Christophe Grand <christophe@cgrand.net>",
"license": "EPL-1.0",
"directories": {
"lib": "src",
"cache": "./lumo-cache"
},
"keywords": [
"cljs",
"cljc",
"self-host",
"transducer"
],
"dependencies": {
"macrovich": "^0.2.1-SNAPSHOT"
}
}

View file

@ -1,8 +0,0 @@
(defproject net.cgrand/xforms "0.7.0"
:description "Extra transducers for Clojure"
#_#_:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.8.0"]
[org.clojure/clojurescript "1.9.293"]
[net.cgrand/macrovich "0.1.0"]])

View file

@ -4,14 +4,43 @@
#?(:cljs (:require-macros
[net.cgrand.macrovich :as macros]
[net.cgrand.xforms :refer [for kvrf let-complete]])
:clj (:require [net.cgrand.macrovich :as macros]))
(:refer-clojure :exclude [reduce reductions into count for partition str last keys vals min max])
(:require [#?(:clj clojure.core :cljs cljs.core) :as core]
[net.cgrand.xforms.rfs :as rf])
:default (:require [net.cgrand.macrovich :as macros]))
(:refer-clojure :exclude [some reduce reductions into count for partition
str last keys vals min max drop-last take-last
sort sort-by time #?@(:bb [] :cljd/clj-host [] :clj [satisfies?])])
(:require [#?(:cljd cljd.core :clj clojure.core :cljs cljs.core) :as core]
[net.cgrand.xforms.rfs :as rf]
#?@(:cljd [["dart:collection" :as dart:coll]] :clj [[clojure.core.protocols]] :cljs []))
#?(:cljd/clj-host
; customize the clj/jvm ns used for macroexpansion
(:host-ns (:require [clojure.core :as core]
[net.cgrand.macrovich :as macros])))
#?(:cljs (:import [goog.structs Queue])))
(defn- ^:macro-support pair? [x] (and (vector? x) (= 2 (core/count x))))
(def ^:macro-support destructuring-pair?
(let [kw-or-& #(or (keyword? %) (= '& %))]
(fn [x]
(and (pair? x)
(not (kw-or-& (first x)))))))
(macros/deftime
(defn- ^:macro-support no-user-meta? [x]
(= {} (dissoc (or (meta x) {}) :file :line :column :end-line :end-column)))
(defmacro unreduced->
"Thread first while threaded value is not reduced.
Doesn't unreduce the final value."
([x] x)
([x expr & exprs]
`(let [x# ~x]
(if (reduced? x#)
x#
(unreduced-> (-> x# ~expr) ~@exprs)))))
(defmacro for
"Like clojure.core/for with the first expression being replaced by % (or _). Returns a transducer.
When the first expression is not % (or _) returns an eduction."
@ -19,58 +48,84 @@
(if-not (and (symbol? %or_) (#{"%" "_"} (name %or_)))
`(eduction (for [~binding ~'% ~@seq-exprs] ~body-expr) ~%or_)
(let [rf (gensym 'rf)
acc (gensym 'acc)
pair? #(and (vector? %) (= 2 (core/count %)))
destructuring-pair? (every-pred pair?
#(not-any? (some-fn keyword? #{'&}) %))
rpairs (core/partition 2 (rseq (vec seq-exprs)))
build (fn [init]
(core/reduce (fn [body [expr binding]]
(case binding
:let `(let ~expr ~body)
:when `(if ~expr ~body ~acc)
:while `(if ~expr ~body (reduced ~acc))
(if (destructuring-pair? binding)
`(let [expr# ~expr]
(if (and (map? expr#) (kvreducible? expr#))
(core/reduce-kv (fn [~acc ~@binding] ~body) ~acc expr#)
(core/reduce (fn [~acc ~binding] ~body) ~acc expr#)))
`(core/reduce (fn [~acc ~binding] ~body) ~acc ~expr))))
init rpairs))
nested-reduceds (core/for [[expr binding] rpairs
:when (not (keyword? binding))]
`reduced)
body (build `(let [acc# (~rf ~acc ~@(if (and (pair? body-expr) (nil? (meta body-expr)))
body-expr
[body-expr]))]
(if (reduced? acc#)
(-> acc# ~@nested-reduceds)
acc#)))]
`(fn [~rf]
(let [~rf (ensure-kvrf ~rf)]
(kvrf
([] (~rf))
([~acc] (~rf ~acc))
([~acc ~binding] ~body)
~(if (destructuring-pair? binding)
`([~acc ~@binding] ~body)
`([~acc k# v#]
(let [~binding (net.cgrand.macrovich/case :clj (clojure.lang.MapEntry. k# v#) :cljs [k# v#])] ~body)))))))))
acc (gensym 'acc)
rpairs (core/partition 2 (rseq (vec seq-exprs)))
build (fn [init]
(core/reduce (fn [body [expr binding]]
(case binding
:let `(let ~expr ~body)
:when `(if ~expr ~body ~acc)
:while `(if ~expr ~body (reduced ~acc))
(if (and (coll? expr) (not (seq? expr))
(or (<= (core/count expr) 4) (:unroll (meta expr))))
(let [body-rf (gensym 'body-rf)]
(if (and (destructuring-pair? binding) (every? vector? expr))
`(let [~body-rf (fn [~acc ~@binding] ~body)]
(unreduced (unreduced-> ~acc
~@(map (fn [[k v]] `(~body-rf ~k ~v)) expr))))
`(let [~body-rf (fn [~acc ~binding] ~body)]
(unreduced (unreduced-> ~acc
~@(map (fn [v] `(~body-rf ~v)) expr))))))
(if (destructuring-pair? binding)
`(let [expr# ~expr]
(if (and (map? expr#) (kvreducible? expr#))
(core/reduce-kv (fn [~acc ~@binding] ~body) ~acc expr#)
(core/reduce (fn [~acc ~binding] ~body) ~acc expr#)))
`(core/reduce (fn [~acc ~binding] ~body) ~acc ~expr)))))
init rpairs))
nested-reduceds (core/for [[expr binding] rpairs
:when (not (keyword? binding))]
`reduced)
body (build `(let [acc# (~rf ~acc ~@(if (and (pair? body-expr) (no-user-meta? body-expr))
body-expr
[body-expr]))]
(if (reduced? acc#)
(-> acc# ~@nested-reduceds)
acc#)))]
`(fn [~rf]
(let [~rf (ensure-kvrf ~rf)]
(kvrf
([] (~rf))
([~acc] (~rf ~acc))
([~acc ~binding] ~body)))))))
(defn- ^:macro-support arity [[arglist & body :as fn-body]]
(let [[fixargs varargs] (split-with (complement #{'&}) arglist)]
(if (seq varargs) (zipmap (range (core/count fixargs) 4) (repeat fn-body)))
{(core/count fixargs) fn-body}))
(defmacro kvrf [name? & fn-bodies]
(let [name (if (symbol? name?) name? (gensym '_))
fn-bodies (if (symbol? name?) fn-bodies (cons name? fn-bodies))
fn-bodies (if (vector? (first fn-bodies)) (list fn-bodies) fn-bodies)]
fn-bodies (if (vector? (first fn-bodies)) (list fn-bodies) fn-bodies)
arities (core/into {} (mapcat arity) fn-bodies)
_ (when-not (core/some arities [2 3]) (throw (ex-info "Either arity 2 or 3 should be defined in kvrf." {:form &form})))
fn-bodies (cond-> fn-bodies
(not (arities 3)) (conj (let [[[acc arg] & body] (arities 2)]
(if (destructuring-pair? arg)
(let [[karg varg] arg]
`([~acc ~karg ~varg] ~@body))
(let [k (gensym "k__")
v (gensym "v__")
arg-value (macros/case
:clj `(clojure.lang.MapEntry. ~k ~v)
:cljs [k v]
:cljd `(MapEntry ~k ~v))]
`([~acc ~k ~v] (let [~arg ~arg-value] ~@body))))))
(not (arities 2)) (conj (let [[[acc karg varg] & body] (arities 3)]
`([~acc [~karg ~varg]] ~@body))))]
`(reify
~@(macros/case :clj '[clojure.lang.Fn])
#?@(:bb [] ;; babashka currently only supports reify with one Java interface at a time
:default [~@(macros/case :cljd '[cljd.core/Fn] :clj '[clojure.lang.Fn])])
KvRfable
(some-kvrf [this#] this#)
~(macros/case :cljs `core/IFn :clj 'clojure.lang.IFn)
(~'some-kvrf [this#] this#)
~(macros/case :cljs `core/IFn :clj 'clojure.lang.IFn :cljd 'cljd.core/IFn)
~@(core/for [[args & body] fn-bodies]
(let [nohint-args (map (fn [arg] (if (:tag (meta arg)) (gensym 'arg) arg)) args)
rebind (mapcat (fn [arg nohint]
(when-not (= arg nohint) [arg nohint])) args nohint-args)]
`(~(macros/case :cljs `core/-invoke :clj 'invoke) [~name ~@nohint-args] (let [~@rebind] ~@body)))))))
`(~(macros/case :cljd '-invoke :cljs `core/-invoke :clj 'invoke)
[~name ~@nohint-args] ~@(if (seq rebind) [`(let [~@rebind] ~@body)] body)))))))
(defmacro ^:private let-complete [[binding volatile] & body]
`(let [v# @~volatile]
@ -87,20 +142,54 @@
(macros/usetime
(defn kvreducible? [coll]
(satisfies? #?(:clj clojure.core.protocols/IKVReduce :cljs IKVReduce) coll))
;; Workaround clojure.core/satisfies? being slow in Clojure
;; see https://ask.clojure.org/index.php/3304/make-satisfies-as-fast-as-a-protocol-method-call
#?(:bb nil
:cljd nil
:clj
(defn fast-satisfies?-fn
"Ported from https://github.com/clj-commons/manifold/blob/37658e91f836047a630586a909a2e22debfbbfc6/src/manifold/utils.clj#L77-L89"
[protocol-var]
(let [^java.util.concurrent.ConcurrentHashMap classes
(java.util.concurrent.ConcurrentHashMap.)]
(add-watch protocol-var ::memoization (fn [& _] (.clear classes)))
(fn [x]
(let [cls (class x)
val (.get classes cls)]
(if (nil? val)
(let [val (core/satisfies? @protocol-var x)]
(.put classes cls val)
val)
val))))))
#?(:cljs
(defn kvreducible? [coll]
(satisfies? IKVReduce coll))
:cljd
(defn kvreducible? [coll]
(satisfies? cljd.core/IKVReduce coll))
:clj
(let [satisfies-ikvreduce? #?(:bb #(satisfies? clojure.core.protocols/IKVReduce %)
:default (fast-satisfies?-fn #'clojure.core.protocols/IKVReduce))]
(if (satisfies-ikvreduce? (Object.))
(defn kvreducible?
"Clojure 1.11 makes everything satisfy IKVReduce, so we can short-circuit"
[_] true)
(defn kvreducible? [coll] (satisfies-ikvreduce? coll)))))
(extend-protocol KvRfable
#?(:clj Object :cljs default) (some-kvrf [_] nil)
nil (some-kvrf [_] nil))
#?(:cljd fallback :clj Object :cljs default) (some-kvrf [_] nil)
#?@(:clj [nil (some-kvrf [_] nil)]))
(defn ensure-kvrf [rf]
(or (some-kvrf rf)
(kvrf
([] (rf))
([acc] (rf acc))
([acc x] (rf acc x))
([acc k v] (rf acc #?(:clj (clojure.lang.MapEntry. k v) :cljs [k v]))))))
([acc x] (rf acc x)))))
(defn reduce
"A transducer that reduces a collection to a 1-item collection consisting of only the reduced result.
@ -126,7 +215,9 @@
(defn- into-rf [to]
(cond
(instance? #?(:clj clojure.lang.IEditableCollection :cljs IEditableCollection) to)
#?(:cljd (satisfies? cljd.core/IEditableCollection to)
:clj (instance? clojure.lang.IEditableCollection to)
:cljs (satisfies? IEditableCollection to))
(if (map? to)
(kvrf
([] (transient to))
@ -161,6 +252,43 @@
(rf (core/reduce-kv rf (rf) from))
(rf (core/reduce rf (rf) from))))))
(defn- without-rf [from]
(cond
#?(:cljd (satisfies? cljd.core/IEditableCollection from)
:clj (instance? clojure.lang.IEditableCollection from)
:cljs (satisfies? IEditableCollection from))
(if (map? from)
(fn
([] (transient from))
([acc] (persistent! acc))
([acc x] (dissoc! acc x)))
(fn
([] (transient from))
([acc] (persistent! acc))
([acc x] (disj! acc x))))
(map? from)
(fn
([] from)
([acc] acc)
([acc x] (dissoc acc x)))
:else
(fn
([] from)
([acc] acc)
([acc x] (disj acc x)))))
(defn without
"The opposite of x/into: dissociate or disjoin from the target."
([target]
(reduce (without-rf target)))
([target keys]
(without target identity keys))
([target xform keys]
(let [rf (xform (without-rf target))]
(if-let [rf (and (map? keys) (kvreducible? keys) (some-kvrf rf))]
(rf (core/reduce-kv rf (rf) keys))
(rf (core/reduce rf (rf) keys))))))
(defn minimum
([comparator]
(minimum comparator nil))
@ -177,21 +305,48 @@
(def max (reduce rf/max))
(defn str
"When used as a value, it's an aggregating transducer that concatenates input values
into a single output value.
When used as a function of two args (xform and coll) it's a transducing context that
concatenates all values in a string."
{:arglists '([xform coll])}
([rf] ((reduce rf/str) rf))
([xform coll]
(transduce xform rf/str coll)))
(defn wrap
"Transducer. Adds open as the first item, and close as the last. Optionally inserts delim between each input item."
([open close]
(fn [rf]
(let [vrf (volatile! nil)]
(vreset! vrf
(fn [acc x]
(let [acc (rf acc open)]
(vreset! vrf rf)
(if (reduced? acc)
acc
(rf acc x)))))
(fn
([] (rf))
([acc] (rf (unreduced (rf acc close))))
([acc x] (@vrf acc x))))))
([open close delim]
(comp (interpose delim) (wrap open close))))
(defn vals [rf]
(kvrf
([] (rf))
([acc] (rf acc))
([acc kv] (rf acc (val kv)))
([acc k v] (rf acc v))))
(defn keys [rf]
(kvrf
([] (rf))
([acc] (rf acc))
([acc kv] (rf acc (key kv)))
([acc k v] (rf acc k))))
;; for both map entries and vectors
;; for both map entries and vectors
(defn- key' [kv] (nth kv 0))
(defn- val' [kv] (nth kv 1))
@ -238,8 +393,6 @@
(kvrf self
([] (rf))
([acc] (let-complete [m m] (rf (core/reduce (fn [acc krf] (krf acc)) acc (core/vals (persistent! m))))))
([acc x]
(self acc (key' x) (val' x)))
([acc k v]
(let [krf (or (get @m k) (doto (xform (make-rf k)) (->> (vswap! m assoc! k))))
acc (krf acc v)]
@ -269,68 +422,153 @@
(do
(vswap! m assoc! k nop-rf)
(krf @acc)))
acc)))
([acc k v] (self acc #?(:clj (clojure.lang.MapEntry. k v) :cljs [k v])))))))))))
acc)))))))))))
(defn partition
"Returns a partitioning transducer. Each partition is independently transformed using the xform transducer."
([n]
(partition n n (into [])))
([n step-or-xform]
(if (fn? step-or-xform)
(partition n n step-or-xform)
(partition n step-or-xform (into []))))
([n step pad-or-xform]
(if (fn? pad-or-xform)
(let [xform pad-or-xform]
(fn [rf]
(let [mxrf (multiplexable rf)
dq #?(:clj (java.util.ArrayDeque. n) :cljs (Queue.))
barrier (volatile! n)
xform (comp (map #(if (identical? dq %) nil %)) xform)]
(fn
([] (rf))
([acc] (.clear dq) (rf acc))
([acc x]
(let [b (vswap! barrier dec)]
(when (< b n) (#?(:clj .add :cljs .enqueue) dq (if (nil? x) dq x)))
(if (zero? b)
; this transduce may return a reduced because of mxrf wrapping reduceds coming from rf
(let [acc (transduce xform mxrf acc #?(:clj dq :cljs (.getValues dq)))]
(dotimes [_ (core/min n step)] (#?(:clj .poll :cljs .dequeue) dq))
(vswap! barrier + step)
acc)
acc)))))))
(partition n step pad-or-xform (into []))))
([n step pad xform]
(defn into-by-key
"A shorthand for the common case (comp (x/by-key ...) (x/into coll))."
[coll & by-key-args]
(comp (apply by-key by-key-args) (into coll)))
(macros/replace
[#?(:cljd {(java.util.ArrayDeque. n) (dart:coll/Queue)
.add .add
.poll .removeFirst
.size .-length})
#?(:cljs {(java.util.ArrayDeque. n) (Queue.)
.add .enqueue
.poll .dequeue
.size .getCount})
#?(:clj {(.getValues dq) dq})]
(defn partition
"Returns a partitioning transducer. Each partition is independently transformed using the xform transducer."
([n]
(partition n n (into [])))
([n step-or-xform]
(if (fn? step-or-xform)
(partition n n step-or-xform)
(partition n step-or-xform (into []))))
([#?(:cljd ^int n :default ^long n) step pad-or-xform]
(if (fn? pad-or-xform)
(let [xform pad-or-xform]
(fn [rf]
(let [mxrf (multiplexable rf)
dq (java.util.ArrayDeque. n)
barrier (volatile! n)
xform (comp (map #(if (identical? dq %) nil %)) xform)]
(fn
([] (rf))
([acc] (.clear dq) (rf acc))
([acc x]
(let [b (vswap! barrier dec)]
(when (< b n) (.add dq (if (nil? x) dq x)))
(if (zero? b)
; this transduce may return a reduced because of mxrf wrapping reduceds coming from rf
(let [acc (transduce xform mxrf acc (.getValues dq))]
(dotimes [_ (core/min n step)] (.poll dq))
(vswap! barrier + step)
acc)
acc)))))))
(partition n step pad-or-xform (into []))))
([#?(:cljd ^int n :default ^long n) step pad xform]
(fn [rf]
(let [mxrf (multiplexable rf)
dq (java.util.ArrayDeque. n)
barrier (volatile! n)
xform (comp (map #(if (identical? dq %) nil %)) xform)]
(fn
([] (rf))
([acc] (if (< @barrier n)
(let [xform (comp cat (take n) xform)
; don't use mxrf for completion: we want completion and don't want reduced-wrapping
acc (transduce xform rf acc [(.getValues dq) pad])]
(vreset! barrier n)
(.clear dq)
acc)
(rf acc)))
([acc x]
(let [b (vswap! barrier dec)]
(when (< b n) (.add dq (if (nil? x) dq x)))
(if (zero? b)
; this transduce may return a reduced because of mxrf wrapping reduceds coming from rf
(let [acc (core/transduce xform mxrf acc (.getValues dq))]
(dotimes [_ (core/min n step)] (.poll dq))
(vswap! barrier + step)
acc)
acc))))))))
#_(defn zip [xform1 xform2]
(fn [rf]
(let )))
(defn take-last [#?(:cljd ^int n :default ^long n)]
(fn [rf]
(let [mxrf (multiplexable rf)
dq #?(:clj (java.util.ArrayDeque. n) :cljs (Queue.))
barrier (volatile! n)
xform (comp (map #(if (identical? dq %) nil %)) xform)]
(let [dq (java.util.ArrayDeque. n)]
(fn
([] (rf))
([acc] (if (< @barrier n)
(let [xform (comp cat (take n) xform)
; don't use mxrf for completion: we want completion and don't want reduced-wrapping
acc (transduce xform rf acc [(.getValues dq) pad])]
(vreset! @barrier n)
(.clear dq)
acc)
acc))
([acc] (transduce (map #(if (identical? dq %) nil %)) rf acc (.getValues dq)))
([acc x]
(let [b (vswap! barrier dec)]
(when (< b n) (#?(:clj .add :cljs .enqueue) dq (if (nil? x) dq x)))
(if (zero? b)
; this transduce may return a reduced because of mxrf wrapping reduceds coming from rf
(let [acc (transduce xform mxrf acc #?(:clj dq :cljs (.getValues dq)))]
(dotimes [_ (min n step)] (#?(:clj .poll :cljs .dequeue) dq))
(vswap! barrier + step)
acc)
acc))))))))
(.add dq (if (nil? x) dq x))
(when (< n (.size dq)) (.poll dq))
acc)))))
(defn drop-last
([] (drop-last 1))
([#?(:cljd ^int n :default ^long n)]
(fn [rf]
(let [dq (java.util.ArrayDeque. n)
xform (map #(if (identical? dq %) nil %))
rf (xform rf)]
(fn
([] (rf))
([acc] (rf acc))
([acc x]
(.add dq (if (nil? x) dq x))
(if (< n (.size dq))
(rf acc (.poll dq))
acc)))))))
)
#?(:cljs
(defn ^:private fn->comparator
"Given a fn that might be boolean valued or a comparator,
return a fn that is a comparator.
Copied from cljs.core: https://github.com/clojure/clojurescript/blob/95c5cf384a128503b072b7b1916af1a1d5c8871c/src/main/cljs/cljs/core.cljs#L2459-L2471"
[f]
(if (= f compare)
compare
(fn [x y]
(let [r (f x y)]
(if (number? r)
r
(if r
-1
(if (f y x) 1 0))))))))
(defn sort
([] (sort compare))
([cmp]
(fn [rf]
(let [buf #?(:cljd #dart [] :clj (java.util.ArrayList.) :cljs #js [])]
(fn
([] (rf))
([acc] (rf (core/reduce rf acc (doto buf #?(:cljd (.sort (dart-comparator cmp))
:clj (java.util.Collections/sort cmp)
:cljs (.sort (fn->comparator cmp)))))))
([acc x] (#?(:cljd .add :clj .add :cljs .push) buf x) acc))))))
(defn sort-by
([kfn] (sort-by kfn compare))
([kfn cmp]
(sort (fn [a b]
#?(:cljd (cmp (kfn a) (kfn b))
:clj (.compare ^java.util.Comparator cmp (kfn a) (kfn b))
:cljs (cmp (kfn a) (kfn b)))))))
(defn reductions
"Transient version of reductions. There's a difference in behavior when init is not provided: (f) is used.
"Transducer version of reductions. There's a difference in behavior when init is not provided: (f) is used.
So x/reductions works like x/reduce or transduce, not like reduce and reductions when no init and 1-item input."
([f] (reductions f (f)))
([f init]
@ -354,6 +592,7 @@
(rf acc curr))))))))))
(def avg (reduce rf/avg))
(def sd (reduce rf/sd))
(defn window
"Returns a transducer which computes an accumulator over the last n items
@ -368,7 +607,7 @@
If you don't want to see the accumulator until the window is full then you need to
use (drop (dec n)) to remove them.
If you don't have an inverse function, consider using partition and reduce:
If you don't have an inverse function, consider using partition and reduce:
(x/partition 4 (x/reduce rf))"
[n f invf]
(fn [rf]
@ -391,10 +630,44 @@
(vreset! vi (let [i (inc i)] (if (= n i) 0 i)))
(rf acc (f (vreset! vwacc (f (invf wacc x') x))))))))))))
#?(:clj
#?(:cljd nil
:clj
(defn iterator
"Iterator transducing context, returns an iterator on the transformed data.
Equivalent to (.iterator (eduction xform (iterator-seq src-iterator))) except there's is no buffering on values (as in iterator-seq),
This buffering may cause problems when mutable objects are returned by the src-iterator."
^java.util.Iterator [xform ^java.util.Iterator src-iterator]
(let [NULL (Object.)
dq (java.util.ArrayDeque. 32)
rf (xform (fn ([acc] acc) ([acc x] (.push dq (if (some? x) x NULL)) acc)))
vopen (volatile! true)
ensure-next #(or (some? (.peek dq))
(and @vopen
(if (.hasNext src-iterator)
(let [acc (rf nil (.next src-iterator))]
(when (reduced? acc)
(rf nil)
(vreset! vopen false))
(recur))
(do
(rf nil)
(vreset! vopen false)
(recur)))))]
(reify java.util.Iterator
(hasNext [_]
(ensure-next))
(next [_]
(if (ensure-next)
(let [x (.poll dq)]
(if (identical? NULL x) nil x))
(throw (java.util.NoSuchElementException.))))))))
#?(:cljd nil
:clj
(defn window-by-time
"Returns a transducer which computes a windowed accumulator over chronologically sorted items.
"ALPHA
Returns a transducer which computes a windowed accumulator over chronologically sorted items.
timef is a function from one item to its scaled timestamp (as a double). The window length is always 1.0
so timef must normalize timestamps. For example if timestamps are in seconds (and under the :ts key),
to get a 1-hour window you have to use (fn [x] (/ (:ts x) 3600.0)) as timef.
@ -403,8 +676,8 @@
f and invf work like in #'window."
([timef n f]
(window-by-time timef n
(fn
(window-by-time timef n
(fn
([] clojure.lang.PersistentQueue/EMPTY)
([q] (f (core/reduce f (f) q)))
([q x] (conj q x)))
@ -456,11 +729,11 @@
"Count the number of items. Either used directly as a transducer or invoked with two args
as a transducing context."
([rf]
(let [n #?(:clj (java.util.concurrent.atomic.AtomicLong.) :cljs (atom 0))]
(let [n #?(:cljd (volatile! 0) :clj (java.util.concurrent.atomic.AtomicLong.) :cljs (volatile! 0))]
(fn
([] (rf))
([acc] (rf (unreduced (rf acc #?(:clj (.get n) :cljs @n)))))
([acc _] #?(:clj (.incrementAndGet n) :cljs (swap! n inc)) acc))))
([acc] (rf (unreduced (rf acc #?(:cljd @n :clj (.get n) :cljs @n)))))
([acc _] #?(:cljd (vswap! n inc) :clj (.incrementAndGet n) :cljs (vswap! n inc)) acc))))
([xform coll]
(transduce (comp xform count) rf/last coll)))
@ -522,13 +795,18 @@
(def last (reduce rf/last))
(defn some
"Process coll through the specified xform and returns the first local true value."
[xform coll]
(transduce xform rf/some nil coll))
(defn transjuxt
"Performs several transductions over coll at once. xforms-map can be a map or a sequential collection.
When xforms-map is a map, returns a map with the same keyset as xforms-map.
When xforms-map is a sequential collection returns a vector of same length as xforms-map.
Returns a transducer when coll is omitted."
([xforms-map]
(let [collect-xform (if (map? xforms-map)
(let [collect-xform (if (map? xforms-map)
(into {})
(reduce (kvrf
([] (core/reduce (fn [v _] (conj! v nil))
@ -542,4 +820,72 @@
([xforms-map coll]
(transduce (transjuxt xforms-map) rf/last coll)))
(macros/replace
[#?(:cljs {(java.util.concurrent.atomic.AtomicLong.) (atom 0)
(System/nanoTime) (system-time)
(.addAndGet at (- t (System/nanoTime))) (swap! at + (- t (system-time)))
(.addAndGet at (- (System/nanoTime) t)) (swap! at + (- (system-time) t))
.size .getCount})]
#?(:cljd nil
:default
(defn time
"Measures the time spent in this transformation and prints the measured time.
tag-or-f may be either a function of 1 argument (measured time in ms) in which case
this function will be called instead of printing, or tag-or-f will be print before the measured time."
([xform] (time "Elapsed time" xform))
([tag-or-f xform]
(let [pt (if (fn? tag-or-f)
tag-or-f
#(println (core/str tag-or-f ": " % " msecs")))]
(fn [rf]
(let [at (java.util.concurrent.atomic.AtomicLong.)
rf
(fn
([] (rf))
([acc] (let [t (System/nanoTime)
r (rf acc)]
(.addAndGet at (- t (System/nanoTime)))
r))
([acc x]
(let [t (System/nanoTime)
r (rf acc x)]
(.addAndGet at (- t (System/nanoTime)))
r)))
rf (xform rf)]
(fn
([] (rf))
([acc]
(let [t (System/nanoTime)
r (rf acc)
total (.addAndGet at (- (System/nanoTime) t))]
(pt #?(:clj (* total 1e-6) :cljs total))
r))
([acc x]
(let [t (System/nanoTime)
r (rf acc x)]
(.addAndGet at (- (System/nanoTime) t))
r))))))))))
#_(defn rollup
"Roll-up input data along the provided dimensions (which are functions of one input item),
Values of interest are extracted from items using the valfn function and are then summarized
by summary-fn (a reducing function over values returned by valfn or summaries).
Each level of rollup is a map with two keys: :summary and :details."
([dimensions valfn summary-fn]
(let [[dim & dims] (reverse dimensions)]
(core/reduce
(fn [xform dim]
(comp
(by-key dim xform)
(transjuxt
{:detail (into {})
:summary (comp vals (map :summary) (reduce summary-fn))})))
(comp (by-key dim (map valfn))
(transjuxt
{:detail (into {})
:summary (comp vals (reduce summary-fn))}))
dims)))
([dimensions valfn summary-fn coll]
(into {} (rollup dimensions valfn summary-fn) coll)))
)

View file

@ -0,0 +1,206 @@
(ns net.cgrand.xforms.io
(:require [clojure.java.io :as io]
[clojure.java.shell :as sh]
[clojure.edn :as edn])
(:import (java.io Reader BufferedReader IOException InputStream OutputStream BufferedWriter Writer PushbackReader InputStreamReader OutputStreamWriter Closeable)
(java.util Arrays List)
(java.util.concurrent ArrayBlockingQueue)
(java.lang ProcessBuilder$Redirect)
(clojure.lang IFn Fn IReduce)))
(defn keep-opts [m like]
(let [ns (namespace like)]
(into {}
(keep (fn [[k v]]
(when (= ns (or (namespace k) ns))
[(keyword (name k)) v])))
m)))
(defn lines-in
"Returns a reducible view over the provided input.
Input is read line by line. Coercion of the input is done by io/reader (and opts are passed to it).
Input is automatically closed upon completion or error."
[in & opts]
(let [no-init (Object.)]
(reify IReduce
(reduce [self f] (.reduce self f no-init))
(reduce [self f init]
(with-open [^Reader rdr (apply io/reader in opts)]
(let [^BufferedReader rdr (cond-> rdr (not (instance? BufferedReader rdr)) (BufferedReader.))
init (if (identical? init no-init)
(or (.readLine rdr) (f))
init)]
(loop [state init]
(if-some [line (.readLine rdr)]
(let [state (f state line)]
(if (reduced? state)
(unreduced state)
(recur state)))
state))))))))
(defn lines-out
"1-2 args: reducing function that writes values serialized to its accumulator (a java.io.BufferedWriter).
3+ args: transducing context that writes transformed values to the specified output. The output is
coerced to a BufferedWriter by passing out and opts to clojure.java.io/writer. The output is automatically closed.
Returns the writer."
([w] w)
([^BufferedWriter w line]
(doto w
(.write (str line))
(.newLine)))
([out xform coll & opts]
(with-open [^Writer w (apply io/writer out opts)]
(transduce xform lines-out w coll))))
(defn edn-in
"Returns a reducible view over the provided input.
Input is read form by form. Coercion of the input is done by io/reader.
Input is automatically closed upon completion or error.
Unqualified options are passed to both edn/read and io/writer, options qualified by clojure.java.io
are only passed (once dequalified) to io/writer, options qualified by clojure.edn are only passed to
edn/read"
[in & {:as opts}]
(let [no-init (Object.)]
(reify IReduce
(reduce [self f] (.reduce self f no-init))
(reduce [self f init]
(with-open [^Reader rdr (apply io/reader in (mapcat seq (keep-opts opts ::io/opts)))]
(let [^BufferedReader rdr (cond-> rdr (not (instance? PushbackReader rdr)) PushbackReader.)
opts (assoc (keep-opts opts ::edn/opts) :eof no-init)
init (if (identical? init no-init)
(let [form (edn/read opts rdr)]
(if (identical? no-init form)
(f)
form))
init)]
(loop [state init]
(let [form (edn/read opts rdr)]
(if (identical? no-init form)
state
(let [state (f state form)]
(if (reduced? state)
(unreduced state)
(recur state))))))))))))
(defn edn-out
"1-2 args: reducing function that writes values serialized as EDN to its accumulator (a java.io.Writer).
3+ args: transducing context that writes transformed values to the specified output. The output is
coerced to a Writer by passing out and opts to clojure.java.io/writer. The output is automatically closed.
Returns the writer."
([w] w)
([^Writer w x]
(binding [*out* w
*print-length* nil
*print-level* nil
*print-dup* false
*print-meta* false
*print-readably* true]
(prn x)
w))
([out xform coll & opts]
(with-open [^Writer w (apply io/writer out opts)]
(transduce xform edn-out w coll))))
(defn- stream-spec [x]
(into {:mode :lines :enc "UTF-8"}
(cond (map? x) x (string? x) {:enc x} (keyword? x) {:mode x})))
(defn sh
"Transducer or reducible view (in this case assumes empty stdin).
Spawns a process (program cmd with optional arguments arg1 ... argN) and pipes data through it.
Options may be:
* :env, an environment variables map, it will be merged with clojure.java.shell/*sh-env* and JVM environment (in decreasing precedence order),
* :dir, the current dir (defaults to clojure.java.shell/*sh-dir* or JVM current dir),
* :in and :out which are maps with keys :mode (:lines (default), :text or :bytes) and :enc (defaults to \"UTF-8\");
encoding applies only for modes :lines or :text; shorthands exist: a single keyword is equivalent to {:mode k :enc \"UTF-8\"},
a single string is equivalent to {:mode :lines, :enc s}.
In :bytes mode, values are bytes array.
In :lines mode, values are strings representing lines without line delimiters.
In :text mode, values are strings."
{:arglists '([cmd arg1 ... argN & opts])}
[& args]
(reify
IReduce
(reduce [self rf]
(reduce rf (eduction self nil))) ; quick way to handle no init
(reduce [self rf init]
(let [xf (self rf)]
(xf init)))
Fn
IFn
(invoke [_ rf]
(let [[cmd [& {:as opts :keys [env in out dir] :or {dir sh/*sh-dir*}}]] (split-with string? args)
env (into (or sh/*sh-env* {}) env)
env (into {} (for [[k v] env] [(name k) (str v)]))
proc (-> ^List (map str cmd) ProcessBuilder.
(.redirectError ProcessBuilder$Redirect/INHERIT)
(doto (-> .environment (.putAll env)))
(.directory (io/as-file dir))
.start)
EOS (Object.)
q (ArrayBlockingQueue. 16)
drain (fn [acc]
(loop [acc acc]
(if-some [x (.poll q)]
(let [acc (if (identical? EOS x) (reduced acc) (rf acc x))]
(if (reduced? acc)
(do
(.destroy proc)
acc)
(recur acc)))
acc)))
in (stream-spec in)
out (stream-spec out)
^Closeable stdin (cond-> (.getOutputStream proc) (#{:lines :text} (:mode in)) (-> (OutputStreamWriter. ^String (:enc in)) BufferedWriter.))
stdout (cond-> (.getInputStream proc) (#{:lines :text} (:mode out)) (-> (InputStreamReader. ^String (:enc out)) BufferedReader.))
write!
(case (:mode in)
:lines
(fn [x]
(doto ^BufferedWriter stdin
(.write (str x))
.newLine))
:text
(fn [x]
(.write ^BufferedWriter stdin (str x)))
:bytes
(fn [^bytes x]
(.write ^OutputStream stdin x)))]
(-> (case (:mode out)
:lines
#(loop []
(if-some [s (.readLine ^BufferedReader stdout)]
(do (.put q s) (recur))
(.put q EOS)))
:text
#(let [buf (char-array 1024)]
(loop []
(let [n (.read ^BufferedReader stdout buf)]
(if (neg? n)
(.put q EOS)
(do (.put q (String. buf 0 n)) (recur))))))
:bytes
#(let [buf (byte-array 1024)]
(loop []
(let [n (.read ^InputStream stdout buf)]
(if (neg? n)
(.put q EOS)
(do (.put q (Arrays/copyOf buf n)) (recur)))))))
Thread. .start)
(fn
([] (rf))
([acc]
(.close stdin)
(loop [acc acc]
(let [acc (drain acc)]
(if (reduced? acc)
(rf (unreduced acc))
(recur acc)))))
([acc x]
(let [acc (drain acc)]
(try
(when-not (reduced? acc)
(write! x))
acc
(catch IOException e
(ensure-reduced acc))))))))))

View file

@ -0,0 +1,31 @@
(ns net.cgrand.xforms.nodejs.stream)
(def ^:private Transform (.-Transform (js/require "stream")))
(defn transformer
"Returns a stream.Transform object that performs the specified transduction.
options is a js object as per stream.Transform -- however :readableObjectMode and :writableObjectMode are set to true by default."
([xform] (transformer #js {} xform))
([options xform]
(let [xrf (xform (fn
([transform] (doto transform .end))
([transform x]
(when-not (.push transform x)
(throw (js/Error. "Transformer's internal buffer is full, try passing a larger :highWaterMark option.")))
transform)))]
(specify! (Transform. (.assign js/Object #js {:readableObjectMode true
:writableObjectMode true} options))
Object
(_transform [this x _ cb]
(try
(when (reduced? (xrf this x))
(.push this nil))
(cb)
(catch :default err
(cb err))))
(_flush [this cb]
(try
(xrf this)
(cb)
(catch :default err
(cb err))))))))

View file

@ -1,18 +1,21 @@
(ns net.cgrand.xforms.rfs
{:author "Christophe Grand"}
(:refer-clojure :exclude [str last min max])
(:refer-clojure :exclude [str last min max some])
#?(:cljs (:require-macros
[net.cgrand.macrovich :as macros]
[net.cgrand.xforms.rfs :refer [or-instance?]])
:clj (:require [net.cgrand.macrovich :as macros]))
(:require [#?(:clj clojure.core :cljs cljs.core) :as core])
#?(:cljd (:require ["dart:math" :as Math]))
#?(:cljs (:import [goog.string StringBuffer])))
(macros/deftime
(defmacro ^:private or-instance? [class x y]
(let [xsym (gensym 'x_)]
`(let [~xsym ~x]
(if (instance? ~class ~xsym) ~(with-meta xsym {:tag class}) ~y)))))
(if #?(:cljd (dart/is? ~xsym ~class)
:default (instance? ~class ~xsym))
~(with-meta xsym {:tag class}) ~y)))))
(declare str!)
@ -26,28 +29,61 @@
r -1
(f b a) 1
:else 0))))
(defn minimum
([comparator]
(minimum comparator nil))
([#?(:clj ^java.util.Comparator comparator :cljs comparator) absolute-maximum]
(fn
([] ::+∞)
([x] (if (#?(:clj identical? :cljs keyword-identical?) ::+∞ x)
absolute-maximum
x))
([a b] (if (or (#?(:clj identical? :cljs keyword-identical?) ::+∞ a) (pos? (#?(:clj .compare :cljs cmp) comparator a b))) b a)))))
([#?(:cljd comparator :clj ^java.util.Comparator comparator :cljs comparator)]
(let [#?@(:cljd [comparator (dart-comparator comparator)] :default [])]
(fn
([] nil)
([x] x)
([a b] (cond
(nil? a) b
(nil? b) a
(pos? #?(:cljd (comparator a b)
:clj (.compare comparator a b)
:cljs (cmp comparator a b))) b
:else a)))))
([#?(:cljd comparator :clj ^java.util.Comparator comparator :cljs comparator) absolute-maximum]
(let [#?@(:cljd [comparator (dart-comparator comparator)] :default [])]
(fn
([] ::+infinity)
([x] (if (#?(:clj identical? :cljs keyword-identical?) ::+infinity x)
absolute-maximum
x))
([a b]
(if (or
(#?(:clj identical? :cljs keyword-identical?) ::+infinity a)
(pos? #?(:cljd (comparator a b)
:clj (.compare comparator a b)
:cljs (cmp comparator a b))))
b a))))))
(defn maximum
([comparator]
(maximum comparator nil))
([#?(:clj ^java.util.Comparator comparator :cljs comparator) absolute-minimum]
(fn
([] ::-∞)
([x] (if (#?(:clj identical? :cljs keyword-identical?) ::-∞ x)
absolute-minimum
x))
([a b] (if (or (#?(:clj identical? :cljs keyword-identical?) ::-∞ a) (neg? (#?(:clj .compare :cljs cmp) comparator a b))) b a)))))
([#?(:cljd comparator :clj ^java.util.Comparator comparator :cljs comparator)]
(let [#?@(:cljd [comparator (dart-comparator comparator)] :default [])]
(fn
([] nil)
([x] x)
([a b] (cond
(nil? a) b
(nil? b) a
(neg? #?(:cljd (comparator a b)
:clj (.compare comparator a b)
:cljs (cmp comparator a b))) b
:else a)))))
([#?(:cljd comparator :clj ^java.util.Comparator comparator :cljs comparator) absolute-minimum]
(let [#?@(:cljd [comparator (dart-comparator comparator)] :default [])]
(fn
([] ::-infinity)
([x] (if (#?(:clj identical? :cljs keyword-identical?) ::-infinity x)
absolute-minimum
x))
([a b]
(if (or (#?(:clj identical? :cljs keyword-identical?) ::-infinity a)
(neg? #?(:cljd (comparator a b)
:clj (.compare comparator a b)
:cljs (cmp comparator a b))))
b a))))))
(def min (minimum compare))
@ -55,11 +91,33 @@
(defn avg
"Reducing fn to compute the arithmetic mean."
([] (transient [0 0]))
([[n sum]] (/ sum n))
([] nil)
([#?(:cljd ^{:tag #/(List? double)} acc :clj ^doubles acc :cljs ^doubles acc)]
(when acc (/ (aget acc 1) (aget acc 0))))
([acc x] (avg acc x 1))
([[n sum :as acc] x w]
(-> acc (assoc! 0 (+ n w)) (assoc! 1 (+ sum (* w x))))))
([#?(:cljd ^{:tag #/(List? double)} acc :clj ^doubles acc :cljs ^doubles acc) x w] ; weighted mean
(let [acc (or acc #?(:cljd (double-array 2) :clj (double-array 2) :cljs #js [0.0 0.0]))]
(doto acc
(aset 0 (+ (aget acc 0) w))
(aset 1 (+ (aget acc 1) (* w x)))))))
(defn sd
"Reducing fn to compute the standard deviation. Returns 0 if no or only one item."
([] #?(:cljd (double-array 3) :clj (double-array 3) :cljs #js [0.0 0.0 0.0]))
([#?(:cljd ^{:tag #/(List double)} a :default ^doubles a)]
(let [s (aget a 0) n (aget a 2)]
(if (< 1 n)
(Math/sqrt (/ s (dec n)))
0.0)))
([#?(:cljd ^{:tag #/(List double)} a :default ^doubles a) x]
(let [s (aget a 0) m (aget a 1) n (aget a 2)
d (- x m)
n (inc n)
m' (+ m (/ d n))]
(doto a
(aset 0 (+ s (* d (- x m'))))
(aset 1 m')
(aset 2 n)))))
(defn last
"Reducing function that returns the last value."
@ -67,11 +125,22 @@
([x] x)
([_ x] x))
(defn some
"Reducing function that returns the first logical true value."
([] nil)
([x] x)
([_ x] (when x (reduced x))))
(defn str!
"Like xforms/str but returns a StringBuilder."
([] (#?(:clj StringBuilder. :cljs StringBuffer.)))
([sb] (or-instance? #?(:clj StringBuilder :cljs StringBuffer) sb (#?(:clj StringBuilder. :cljs StringBuffer.) (core/str sb)))) ; the instance? checks are for compatibility with str in case of seeded reduce/transduce.
([sb x] (.append (or-instance? #?(:clj StringBuilder :cljs StringBuffer) sb (#?(:clj StringBuilder. :cljs StringBuffer.) (core/str sb))) x)))
([] (#?(:cljd StringBuffer :clj StringBuilder. :cljs StringBuffer.)))
([sb] (or-instance? #?(:cljd StringBuffer :clj StringBuilder :cljs StringBuffer) sb
(#?(:cljd StringBuffer :clj StringBuilder. :cljs StringBuffer.) (core/str sb))))
; the instance? checks are for compatibility with str in case of seeded reduce/transduce.
([sb x] (doto (or-instance?
#?(:cljd StringBuffer :clj StringBuilder :cljs StringBuffer) sb
(#?(:cljd StringBuffer :clj StringBuilder. :cljs StringBuffer.) (core/str sb)))
(#?(:cljd .write :default .append) x))))
(def str
"Reducing function to build strings in linear time. Acts as replacement for clojure.core/str in a reduce/transduce call."
@ -86,12 +155,12 @@
([] (mapv #(vector % (volatile! (%))) rfns))
([acc] (mapv (fn [[rf vacc]] (rf (unreduced @vacc))) acc))
([acc x]
(let [some-unreduced (core/reduce (fn [some-unreduced [rf vacc]]
(let [some-unreduced (core/reduce (fn [some-unreduced [rf vacc]]
(when-not (reduced? @vacc) (vswap! vacc rf x) true))
false acc)]
(if some-unreduced acc (reduced acc))))
([acc k v]
(let [some-unreduced (core/reduce (fn [some-unreduced [rf vacc]]
(let [some-unreduced (core/reduce (fn [some-unreduced [rf vacc]]
(when-not (reduced? @vacc) (vswap! vacc rf k v) true))
false acc)]
(if some-unreduced acc (reduced acc)))))))

View file

@ -1,5 +1,6 @@
(ns net.cgrand.xforms-test
(:require [clojure.test :refer [is deftest testing]]
(:refer-clojure :exclude [partition reductions])
(:require [clojure.test :refer [are is deftest testing]]
[net.cgrand.xforms :as x]))
(defn trial
@ -9,7 +10,7 @@
n is the number of calls to rf before it returns a reduced.
accs is a collection of successive return values for rf."
([xform n coll]
(trial xform n (repeatedly #(#?(:clj Object. :clj js/Object.))) coll))
(trial xform n (repeatedly #(#?(:clj Object. :cljs js/Object.))) coll))
([xform n accs coll]
(let [vaccs (volatile! accs)
vstate (volatile! {:n n :acc (first @vaccs) :state :init})
@ -82,38 +83,84 @@
(is (= (into [] (comp (take 3) (x/reductions +)) (range)) [0 0 1 3]))
(is (= (into [] (x/reductions (constantly (reduced 42)) 0) (range)) [0 42])))
#?(:clj
(deftest window-by-time
(is (= (into
[]
(x/window-by-time :ts 4
(fn
([] clojure.lang.PersistentQueue/EMPTY)
([q] (vec q))
([q x] (conj q x)))
(fn [q _] (pop q)))
(map (fn [x] {:ts x}) (concat (range 0 2 0.5) (range 3 5 0.25))))
[[{:ts 0}] ; t = 0
[{:ts 0}] ; t = 0.25
[{:ts 0} {:ts 0.5}] ; t = 0.5
[{:ts 0} {:ts 0.5}] ; t = 0.75
[{:ts 0.5} {:ts 1.0}] ; t = 1.0
[{:ts 0.5} {:ts 1.0}] ; t = 1.25
[{:ts 1.0} {:ts 1.5}] ; t = 1.5
[{:ts 1.0} {:ts 1.5}] ; t = 1.75
[{:ts 1.5}] ; t = 2.0
[{:ts 1.5}] ; t = 2.25
[] ; t = 2.5
[] ; t = 2.75
[{:ts 3}] ; t = 3.0
[{:ts 3} {:ts 3.25}] ; t = 3.25
[{:ts 3} {:ts 3.25} {:ts 3.5}] ; t = 3.5
[{:ts 3} {:ts 3.25} {:ts 3.5} {:ts 3.75}] ; t = 3.75
[{:ts 3.25} {:ts 3.5} {:ts 3.75} {:ts 4.0}] ; t = 4.0
[{:ts 3.5} {:ts 3.75} {:ts 4.0} {:ts 4.25}] ; t = 4.25
[{:ts 3.75} {:ts 4.0} {:ts 4.25} {:ts 4.5}] ; t = 4.5
[{:ts 4.0} {:ts 4.25} {:ts 4.5} {:ts 4.75}]])))) ; t = 4.75
(deftest partition
(is (= (into [] (x/partition 2 1 nil (x/into [])) (range 8))
[[0 1] [1 2] [2 3] [3 4] [4 5] [5 6] [6 7] [7]]))
(is (= (into [] (x/partition 2 1 (x/into [])) (range 8))
[[0 1] [1 2] [2 3] [3 4] [4 5] [5 6] [6 7]]))
(is (= (into [] (comp (x/partition 2 2 nil) (x/into [])) (range 8))
[[[0 1] [2 3] [4 5] [6 7]]])))
(deftest without
(is (= {0 :ok 2 :ok 4 :ok 6 :ok 8 :ok} (x/without (zipmap (range 10) (repeat :ok)) (filter odd?) (range 20))))
(is (= #{0 2 4 6 8 } (x/without (set (range 10)) (filter odd?) (range 20)))))
#?(:bb nil ;; babashka doesn't currently support calling iterator on range type
:clj
(do
(deftest iterator
(is (true? (.hasNext (x/iterator x/count (.iterator ^java.lang.Iterable (range 5))))))
(is (is (= [5] (iterator-seq (x/iterator x/count (.iterator ^java.lang.Iterable (range 5)))))))
(is (= [[0 1] [1 2] [2 3] [3 4] [4]] (iterator-seq (x/iterator (x/partition 2 1 nil) (.iterator ^java.lang.Iterable (range 5)))))))
(deftest window-by-time
(is (= (into
[]
(x/window-by-time :ts 4
(fn
([] clojure.lang.PersistentQueue/EMPTY)
([q] (vec q))
([q x] (conj q x)))
(fn [q _] (pop q)))
(map (fn [x] {:ts x}) (concat (range 0 2 0.5) (range 3 5 0.25))))
[[{:ts 0}] ; t = 0
[{:ts 0}] ; t = 0.25
[{:ts 0} {:ts 0.5}] ; t = 0.5
[{:ts 0} {:ts 0.5}] ; t = 0.75
[{:ts 0.5} {:ts 1.0}] ; t = 1.0
[{:ts 0.5} {:ts 1.0}] ; t = 1.25
[{:ts 1.0} {:ts 1.5}] ; t = 1.5
[{:ts 1.0} {:ts 1.5}] ; t = 1.75
[{:ts 1.5}] ; t = 2.0
[{:ts 1.5}] ; t = 2.25
[] ; t = 2.5
[] ; t = 2.75
[{:ts 3}] ; t = 3.0
[{:ts 3} {:ts 3.25}] ; t = 3.25
[{:ts 3} {:ts 3.25} {:ts 3.5}] ; t = 3.5
[{:ts 3} {:ts 3.25} {:ts 3.5} {:ts 3.75}] ; t = 3.75
[{:ts 3.25} {:ts 3.5} {:ts 3.75} {:ts 4.0}] ; t = 4.0
[{:ts 3.5} {:ts 3.75} {:ts 4.0} {:ts 4.25}] ; t = 4.25
[{:ts 3.75} {:ts 4.0} {:ts 4.25} {:ts 4.5}] ; t = 4.5
[{:ts 4.0} {:ts 4.25} {:ts 4.5} {:ts 4.75}]]))))) ; t = 4.75
(deftest do-not-kvreduce-vectors
(is (= {0 nil 1 nil} (x/into {} (x/for [[k v] %] [k v]) [[0] [1]])))
(is (= {0 nil 1 nil} (x/into {} (x/for [_ % [k v] [[0] [1]]] [k v]) ["a"]))))
(is (= {0 nil 1 nil} (x/into {} (x/for [_ % [k v] [[0] [1]]] [k v]) ["a"]))))
(deftest sorting
(is (= (range 100) (x/into [] (x/sort) (shuffle (range 100)))))
(is (= (range 100) (x/into [] (x/sort <) (shuffle (range 100)))))
(is (= (reverse (range 100)) (x/into [] (x/sort >) (shuffle (range 100)))))
(is (= (sort-by str (range 100)) (x/into [] (x/sort-by str) (shuffle (range 100)))))
(is (= (sort-by str (comp - compare) (range 100)) (x/into [] (x/sort-by str (comp - compare)) (shuffle (range 100)))))
(is (= (sort-by identity > (shuffle (range 100))) (x/into [] (x/sort-by identity >) (shuffle (range 100))))))
(deftest destructuring-pair?
(let [destructuring-pair? #'x/destructuring-pair?]
(are [candidate expected]
(= expected (destructuring-pair? candidate))
'[a b] true
'[a b c] false
'[& foo] false
'[:as foo] false
1 false
'(a b) false
'{foo bar} false
'{foo :bar} false)))
(defmacro wraps-for-with-no-destructuring []
(x/into [] (x/for [x (range 5)] x)))
(deftest for-in-macro
(is (= [0 1 2 3 4] (wraps-for-with-no-destructuring))))

1
tests.edn Normal file
View file

@ -0,0 +1 @@
#kaocha/v1 {}