luciano/babashka
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

v0.0.4

Browse source 
* add -i and -o options

* add support for -> and ->>

* add test for -io

* support normal Clojure reader tags

* add more functions

* add support for and and or

* add moar functions

* doc
This commit is contained in:
Michiel Borkent 2019-08-10 18:50:48 +02:00 committed by GitHub
parent 377680ac4c
commit 0dfc8a16d6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 300 additions and 99 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

58
README.md
View file

@ -26,10 +26,7 @@ If most of your shell script evolves into Clojure, you might want to turn to:
## Status ## Status
Experimental. Breaking changes are expected to happen at this phase. Not all Experimental. Breaking changes are expected to happen at this phase.
Clojure core functions are supported yet, but can be easily
[added](https://github.com/borkdude/babashka/blob/master/src/babashka/interpreter.clj#L10). PRs
welcome.
## Installation ## Installation
@ -51,40 +48,43 @@ You may also download a binary from [Github](https://github.com/borkdude/babashk
... | bb [--raw] [--println] '<Clojure form>' ... | bb [--raw] [--println] '<Clojure form>'
``` ```
There is one special variable, `*in*`, which is the input read from stdin. The There is one special variable, `*in*`, which is the input read from stdin. The
input is read as EDN by default, unless the `--raw` flag is provided. When using input is read as EDN by default, unless the `-i` flag is provided, then the
the `--println` flag, the output is printed using `println` instead of `prn`. input is read as a string split by newlines into a vector.. The output is
printed as EDN by default, unless the `-o` flag is provided, then the output is
printed using `println`. To combine `-i` and `-o` you can use `-io`.
The current version can be printed with `bb --version`. The current version can be printed with `bb --version`.
Currently only the macros `if`, `when`, `and`, `or`, `->` and `->>` are
supported.
Examples: Examples:
``` shellsession ``` shellsession
$ ls | bb --raw '*in*' $ ls | bb -i '*in*'
["LICENSE" "README.md" "bb" "doc" "pom.xml" "project.clj" "reflection.json" "resources" "script" "src" "target" "test"] ["LICENSE" "README.md" "bb" "doc" "pom.xml" "project.clj" "reflection.json" "resources" "script" "src" "target" "test"]
$ ls | bb --raw '(count *in*)' $ ls | bb -i '(count *in*)'
11 12
$ bb '(vec (dedupe *in*))' <<< '[1 1 1 1 2]' $ bb '(vec (dedupe *in*))' <<< '[1 1 1 1 2]'
[1 2] [1 2]
$ bb '(filter :foo *in*)' <<< '[{:foo 1} {:bar 2}]' $ bb '(filterv :foo *in*)' <<< '[{:foo 1} {:bar 2}]'
({:foo 1}) [{:foo 1}]
``` ```
Functions are written using the reader tag `#f`. Currently up to three Anonymous functions literals are allowed with currently up to three positional
arguments are supported. arguments.
``` shellsession ``` shellsession
$ bb '(#f(+ %1 %2 %3) 1 2 *in*)' <<< 3 $ bb '(#(+ %1 %2 %3) 1 2 *in*)' <<< 3
6 6
``` ```
Regexes are written using the reader tag `#r`.
``` shellsession ``` shellsession
$ ls | bb --raw '(filterv #f(re-find #r "reflection" %) *in*)' $ ls | bb -i '(filterv #(re-find #"reflection" %) *in*)'
["reflection.json"] ["reflection.json"]
``` ```
@ -97,7 +97,7 @@ $ cat /tmp/test.txt
3 Babashka 3 Babashka
4 Goodbye 4 Goodbye
$ < /tmp/test.txt bb --raw '(shuffle *in*)' | bb --println '(str/join "\n" *in*)' $ < /tmp/test.txt bb -io '(shuffle *in*)'
3 Babashka 3 Babashka
2 Clojure 2 Clojure
4 Goodbye 4 Goodbye
@ -113,8 +113,8 @@ Clojure is nice
bar bar
when you're nice to clojure when you're nice to clojure
$ < /tmp/test.txt bb --raw '(map-indexed #f[%1 %2] *in*))' | \ $ < /tmp/test.txt bb -i '(map-indexed #(vector %1 %2) *in*))' | \
bb '(keep #f(when (re-find #r"(?i)clojure" (second %)) (first %)) *in*)' bb '(keep #f(when (re-find #"(?i)clojure" (second %)) (first %)) *in*)'
(1 3) (1 3)
``` ```
@ -137,6 +137,22 @@ You will need leiningen and GraalVM.
script/compile script/compile
## Gallery
Here's a gallery of more useful examples. Do you have a useful example? PR
welcome!
### Fetch latest Github release tag
For converting JSON to EDN, see [jet](https://github.com/borkdude/jet).
``` shellsession
$ curl -s https://api.github.com/repos/borkdude/clj-kondo/tags \
| jet --from json --keywordize --to edn \
| bb '(-> *in* first :name (subs 1))'
"2019.07.31-alpha"
```
## License ## License
Copyright © 2019 Michiel Borkent Copyright © 2019 Michiel Borkent

251
src/babashka/interpreter.clj
View file

@ -1,43 +1,152 @@
(ns babashka.interpreter (ns babashka.interpreter
{:no-doc true} {:no-doc true}
(:refer-clojure :exclude [comparator])
(:require [clojure.walk :refer [postwalk]] (:require [clojure.walk :refer [postwalk]]
[clojure.string :as str] [clojure.string :as str]
[clojure.set :as set])) [clojure.set :as set]))
(def syms '(= < <= >= + +' - * / (defn expand->
aget alength apply assoc assoc-in "The -> macro from clojure.core."
bit-set bit-shift-left bit-shift-right bit-xor boolean boolean? booleans boolean-array butlast [[x & forms]]
char char? conj cons contains? count (loop [x x, forms forms]
dec dec' decimal? dedupe dissoc distinct disj drop (if forms
eduction even? every? (let [form (first forms)
get threaded (if (seq? form)
first float? floats fnil (with-meta (concat (list (first form) x)
identity inc int-array iterate (next form))
(meta form))
(list form x))]
(recur threaded (next forms)))
x)))
(defn expand->>
"The ->> macro from clojure.core."
[[x & forms]]
(loop [x x, forms forms]
(if forms
(let [form (first forms)
threaded (if (seq? form)
(with-meta (concat (cons (first form) (next form))
(list x))
(meta form))
(list form x))]
(recur threaded (next forms)))
x)))
(declare interpret)
(defn eval-and
"The and macro from clojure.core."
[in args]
(if (empty? args) true
(let [[x & xs] args
v (interpret x in)]
(if v
(if (empty? xs) v
(eval-and in xs))
v))))
(defn eval-or
"The or macro from clojure.core."
[in args]
(if (empty? args) nil
(let [[x & xs] args
v (interpret x in)]
(if v v
(if (empty? xs) v
(eval-or in xs))))))
(def syms '(= < <= >= + +' - -' * *' / == aget alength apply assoc assoc-in
associative? array-map
bit-and-not bit-set bit-shift-left bit-shift-right bit-xor boolean
boolean? booleans boolean-array bound? butlast byte-array bytes
bigint bit-test bit-and bounded-count bytes? bit-or bit-flip
biginteger bigdec bit-not byte
cat char char? conj cons contains? count cycle comp concat
comparator coll? compare complement char-array constantly
char-escape-string chars completing counted? chunk-rest
char-name-string class chunk-next
dec dec' decimal? dedupe dissoc distinct distinct? disj double
double? drop drop-last drop-while denominator doubles
eduction empty empty? even? every? every-pred ensure-reduced
first float? floats fnil fnext ffirst flatten false? filter
filterv find format frequencies float float-array
get get-in group-by gensym
hash hash-map hash-set hash-unordered-coll
ident? identical? identity inc inc' int-array interleave into
iterate int int? interpose indexed? integer? ints into-array
juxt juxt
filter filterv find frequencies
last line-seq keep keep-indexed key keys keyword keyword?
keep keep-indexed keys
map mapv map-indexed mapcat merge merge-with munge last line-seq long list list? longs list* long-array
name newline not= num
neg? nth nthrest map map? map-indexed map-entry? mapv mapcat max max-key meta merge
odd? merge-with min min-key munge mod make-array
peek pos?
re-seq re-find re-pattern rest reverse name newline nfirst not not= not-every? num neg? neg-int? nth
set? sequential? some? str nthnext nthrest nil? nat-int? number? not-empty not-any? next
take take-last take-nth tree-seq type nnext namespace-munge numerator
unchecked-inc-int unchecked-long unchecked-negate unchecked-remainder-int
unchecked-subtract-int unsigned-bit-shift-right unchecked-float odd? object-array
vals vec vector?
rand-int rand-nth range reduce reduced? remove peek pop pos? pos-int? partial partition partition-all
second set seq seq? shuffle simple-symbol? sort sort-by subs partition-by
set/difference set/join
str/join str/starts-with? str/ends-with? str/split qualified-ident? qualified-symbol? qualified-keyword? quot
zero?))
re-seq re-find re-pattern re-matches rem remove rest repeatedly
reverse rand-int rand-nth range reduce reduce-kv reduced reduced?
reversible? replicate rsubseq reductions rational? rand replace
rseq ratio? rationalize random-sample repeat
set? sequential? select-keys simple-keyword? simple-symbol? some?
string? str
set/difference set/index set/intersection set/join set/map-invert
set/project set/rename set/rename-keys set/select set/subset?
set/superset? set/union
str/blank? str/capitalize str/ends-with? str/escape str/includes?
str/index-of str/join str/last-index-of str/lower-case
str/re-quote-replacement str/replace str/replace-first str/reverse
str/split str/split-lines str/starts-with? str/trim
str/trim-newline str/triml str/trimr str/upper-case
second set seq seq? seque short shuffle
sort sort-by subs symbol symbol? special-symbol? subvec some-fn
some split-at split-with sorted-set subseq sorted-set-by
sorted-map-by sorted-map sorted? simple-ident? sequence seqable?
shorts
take take-last take-nth take-while transduce tree-seq type true?
to-array
update update-in uri? uuid? unchecked-inc-int unchecked-long
unchecked-negate unchecked-remainder-int unchecked-subtract-int
unsigned-bit-shift-right unchecked-float unchecked-add-int
unchecked-double unchecked-multiply-int unchecked-int
unchecked-multiply unchecked-dec-int unchecked-add unreduced
unchecked-divide-int unchecked-subtract unchecked-negate-int
unchecked-inc unchecked-char unchecked-byte unchecked-short
val vals vary-meta vec vector vector?
xml-seq
zipmap zero?))
;; TODO: ;; TODO:
#_(def all-syms #_(def all-syms
'#{when-first while if-not when-let inc' cat StackTraceElement->vec flush take-while vary-meta <= alter -' if-some conj! repeatedly zipmap reset-vals! alter-var-root biginteger remove * re-pattern min pop! chunk-append prn-str with-precision format reversible? shutdown-agents conj bound? transduce lazy-seq *print-length* *file* compare-and-set! *use-context-classloader* await1 let ref-set pop-thread-bindings interleave printf map? -> defstruct *err* assert-same-protocol get doto identity into areduce long double volatile? definline nfirst meta find-protocol-impl bit-and-not *default-data-reader-fn* var? method-sig unchecked-add-int unquote-splicing hash-ordered-coll future reset-meta! Vec cycle fn seque empty? short definterface add-tap filterv hash quot ns-aliases read unchecked-double key longs not= string? uri? aset-double unchecked-multiply-int chunk-rest pcalls *allow-unresolved-vars* remove-all-methods ns-resolve as-> aset-boolean trampoline double? when-not *1 vec *print-meta* when int map-entry? ns-refers rand second vector-of hash-combine > replace int? associative? unchecked-int set-error-handler! inst-ms* keyword? force bound-fn* namespace-munge group-by prn extend unchecked-multiply some->> default-data-readers ->VecSeq even? unchecked-dec Inst tagged-literal? double-array in-ns create-ns re-matcher defn ref bigint extends? promise aset-char rseq ex-cause construct-proxy agent-errors *compile-files* ex-message *math-context* float pr-str concat aset-short set-agent-send-off-executor! ns symbol to-array-2d mod amap pop use VecNode unquote declare dissoc! reductions aset-byte indexed? ref-history-count - assoc! hash-set reduce-kv or cast reset! name ffirst sorted-set counted? byte-array IVecImpl tagged-literal println extend-type macroexpand-1 assoc-in char-name-string bit-test defmethod requiring-resolve EMPTY-NODE time memoize alter-meta! future? zero? simple-keyword? require unchecked-dec-int persistent! nnext add-watch not-every? class? rem agent-error some future-cancelled? memfn neg-int? struct-map drop *data-readers* nth sorted? nil? extend-protocol split-at *e load-reader random-sample cond-> dotimes select-keys bit-and bounded-count update list* reify update-in prefer-method aset-int *clojure-version* ensure-reduced *' instance? with-open mix-collection-hash re-find run! val defonce unchecked-add loaded-libs ->Vec bytes? not with-meta unreduced the-ns record? type identical? unchecked-divide-int ns-name max-key *unchecked-math* defn- *out* file-seq agent ns-map set-validator! ident? defprotocol swap! vals unchecked-subtract tap> *warn-on-reflection* sorted-set-by sync qualified-ident? assert *compile-path* true? release-pending-sends print empty remove-method *in* print-ctor letfn volatile! / read-line reader-conditional? bit-or clear-agent-errors vector proxy-super >= drop-last not-empty distinct partition loop add-classpath bit-flip long-array descendants merge accessor integer? mapv partition-all partition-by numerator object-array with-out-str condp derive load-string special-symbol? ancestors subseq error-handler gensym cond ratio? delay? intern print-simple flatten doubles halt-when with-in-str remove-watch ex-info ifn? some-> nat-int? proxy-name ns-interns all-ns find-protocol-method subvec for binding partial chunked-seq? find-keyword replicate min-key reduced char-escape-string re-matches array-map unchecked-byte with-local-vars ns-imports send-off defmacro every-pred keys rationalize load-file distinct? pos-int? extenders unchecked-short methods odd? ->ArrayChunk float-array *3 alias frequencies read-string proxy rsubseq inc get-method with-redefs uuid? bit-clear filter locking list + split-with aset ->VecNode keyword *ns* destructure *assert* defmulti chars str next hash-map if-let underive ref-max-history Throwable->map false? *print-readably* ints class some-fn case *flush-on-newline* to-array bigdec list? simple-ident? bit-not io! xml-seq VecSeq byte max == *agent* lazy-cat comment parents count supers *fn-loader* ArrayChunk sorted-map-by apply interpose deref assoc rational? transient clojure-version chunk-cons comparator sorted-map send drop-while proxy-call-with-super realized? char-array resolve compare complement *compiler-options* *print-dup* defrecord with-redefs-fn sequence constantly get-proxy-class make-array shorts completing update-proxy unchecked-negate-int hash-unordered-coll repeat unchecked-inc nthnext and create-struct get-validator number? await-for chunk-next print-str not-any? into-array qualified-symbol? init-proxy chunk-buffer seqable? symbol? when-some unchecked-char ->> future-cancel var-get commute coll? get-in fnext denominator bytes gen-and-load-class refer-clojure}) '#{when-first while if-not when-let if-some let as-> when-not some->> or cond-> loop condp cond some-> if-let case and when-some })
(declare var-lookup apply-fn) (declare var-lookup apply-fn)
@ -51,35 +160,59 @@
(define-lookup) (define-lookup)
(defn resolve-symbol [expr]
(let [n (name expr)]
(if (str/starts-with? n "'")
(symbol (subs n 1))
(or (var-lookup expr)
(throw (Exception. (format "Could not resolve symbol: %s." n)))))))
(defn interpret (defn interpret
[expr in] [expr in]
(cond (let [i #(interpret % in)]
(= '*in* expr) in (cond
(symbol? expr) (var-lookup expr) (= '*in* expr) in
(list? expr) (symbol? expr) (resolve-symbol expr)
(if-let [f (first expr)] (map? expr)
(if-let [v (var-lookup f)] (zipmap (map i (keys expr))
(apply-fn v in (rest expr)) (map i (vals expr)))
(cond (or (vector? expr) (set? expr))
(or (= 'if f) (= 'when f)) (into (empty expr) (map i expr))
(let [[_if cond then else] expr] (seq? expr)
(if (interpret cond in) (if-let [f (first expr)]
(interpret then in) (if-let [v (var-lookup f)]
(interpret else in))) (apply-fn v i (rest expr))
;; bb/fn passed as higher order fn, still needs input (case f
(-> f meta :bb/fn) (if when)
(apply-fn (f in) in (rest expr)) (let [[_if cond then else] expr]
(ifn? f) (if (interpret cond in)
(apply-fn f in (rest expr)) (interpret then in)
:else nil)) (interpret else in)))
expr) ->
;; bb/fn passed as higher order fn, still needs input (interpret (expand-> (rest expr)) in)
(-> expr meta :bb/fn) ->>
(expr in) (interpret (expand->> (rest expr)) in)
:else expr)) and
(eval-and in (rest expr))
or
(eval-or in (rest expr))
;; fallback
;; read fn passed as higher order fn, still needs input
(cond (-> f meta ::fn)
(apply-fn (f in) i (rest expr))
(symbol? f)
(apply-fn (resolve-symbol f) i (rest expr))
(ifn? f)
(apply-fn f i (rest expr))
:else nil)))
expr)
;; read fn passed as higher order fn, still needs input
(-> expr meta ::fn)
(expr in)
:else expr)))
(defn read-fn [form] (defn read-fn [form]
^:bb/fn ^::fn
(fn [in] (fn [in]
(fn [& [x y z]] (fn [& [x y z]]
(interpret (postwalk (fn [elt] (interpret (postwalk (fn [elt]
@ -93,11 +226,15 @@
(defn read-regex [form] (defn read-regex [form]
(re-pattern form)) (re-pattern form))
(defn apply-fn [f in args] (defn apply-fn [f i args]
(let [args (mapv #(interpret % in) args)] (let [args (mapv i args)]
(apply f args))) (apply f args)))
;;;; Scratch ;;;; Scratch
(comment (comment
(interpret '(and *in* 3) 1)
(interpret '(and *in* 3 false) 1)
(interpret '(or *in* 3) nil)
(ifn? 'foo)
) )

40
src/babashka/main.clj
View file

@ -19,7 +19,7 @@
opts-map {} opts-map {}
current-opt nil] current-opt nil]
(if-let [opt (first options)] (if-let [opt (first options)]
(if (starts-with? opt "--") (if (starts-with? opt "-")
(recur (rest options) (recur (rest options)
(assoc opts-map opt []) (assoc opts-map opt [])
opt) opt)
@ -28,28 +28,46 @@
current-opt)) current-opt))
opts-map)) opts-map))
version (boolean (get opts "--version")) version (boolean (get opts "--version"))
raw (boolean (get opts "--raw")) raw-in (boolean (or (get opts "--raw")
(get opts "-i")
(get opts "-io")))
raw-out (boolean (or (get opts "-o")
(get opts "-io")))
println? (boolean (get opts "--println"))] println? (boolean (get opts "--println"))]
{:version version {:version version
:raw raw :raw-in raw-in
:raw-out raw-out
:println? println?})) :println? println?}))
(defn -main (defn -main
[& args] [& args]
(let [{:keys [:version :raw :println?]} (parse-opts args)] (let [{:keys [:version :raw-in :raw-out :println?]} (parse-opts args)]
(cond version (cond version
(println (str/trim (slurp (io/resource "BABASHKA_VERSION")))) (println (str/trim (slurp (io/resource "BABASHKA_VERSION"))))
:else :else
(let [expr (last args) (let [expr (last args)
expr (read-edn expr) expr (read-edn (-> expr
(str/replace "#(" "#f(")
(str/replace "#\"" "#r\"")))
in (slurp *in*) in (slurp *in*)
in (if raw ;; _ (prn in)
in (if raw-in
(str/split in #"\n") (str/split in #"\n")
(read-edn (format "[%s]" in))) (read-edn in))
in (if (= 1 (count in)) (first in) in)] ;; _ (prn in)
((if println? println prn) res (try (i/interpret expr in)
(i/interpret expr in)))))) (catch Exception e
(binding [*out* *err*]
(println (.getMessage e)))
(System/exit 1)))]
(if raw-out
(if (coll? res)
(doseq [l res]
(println l))
(println res))
((if println? println? prn) res))))))
;;;; Scratch ;;;; Scratch
(comment) (comment
)

50
test/babashka/main_test.clj
View file

@ -2,29 +2,59 @@
(:require (:require
[clojure.test :as test :refer [deftest is testing]] [clojure.test :as test :refer [deftest is testing]]
[babashka.test-utils :as test-utils] [babashka.test-utils :as test-utils]
[clojure.edn :as edn])) [clojure.edn :as edn]
[clojure.string :as str]))
(defn bb [input & args] (defn bb [input & args]
(edn/read-string (apply test-utils/bb (str input) (map str args)))) (edn/read-string (apply test-utils/bb (str input) (map str args))))
(deftest main-test (deftest main-test
(testing "-io behaves as identity"
(= "foo\nbar\n" (test-utils/bb "foo\nbar\n" "-io" "*in*")))
(testing "if and when" (testing "if and when"
(is (= 1 (bb 0 '(if (zero? *in*) 1 2)))) (is (= 1 (bb 0 '(if (zero? *in*) 1 2))))
(is (= 2 (bb 1 '(if (zero? *in*) 1 2)))) (is (= 2 (bb 1 '(if (zero? *in*) 1 2))))
(is (= 1 (bb 0 '(when (zero? *in*) 1)))) (is (= 1 (bb 0 '(when (zero? *in*) 1))))
(is (nil? (bb 1 '(when (zero? *in*) 1))))) (is (nil? (bb 1 '(when (zero? *in*) 1)))))
(testing "and and or"
(is (= false (bb 0 '(and false true *in*))))
(is (= 0 (bb 0 '(and true true *in*))))
(is (= 1 (bb 1 '(or false false *in*))))
(is (= false (bb false '(or false false *in*))))
(is (= 3 (bb false '(or false false *in* 3)))))
(testing "fn" (testing "fn"
(is (= 2 (bb 1 "(#f(+ 1 %) *in*)"))) (is (= 2 (bb 1 "(#(+ 1 %) *in*)")))
(is (= [1 2 3] (bb nil "(map #f(+ 1 %) [0 1 2])"))) (is (= [1 2 3] (bb 1 "(map #(+ 1 %) [0 1 2])")))
(is (bb 1 "(#f (when (odd? *in*) *in*) 1)"))) (is (bb 1 "(#(when (odd? *in*) *in*) 1)")))
(testing "map" (testing "map"
(is (= [1 2 3] (bb nil '(map inc [0 1 2]))))) (is (= [1 2 3] (bb 1 '(map inc [0 1 2])))))
(testing "keep" (testing "keep"
(is (= [false true false] (bb nil '(keep odd? [0 1 2]))))) (is (= [false true false] (bb 1 '(keep odd? [0 1 2])))))
(testing "..." (testing "->"
(is (= 4 (bb 1 '(-> *in* inc inc (inc))))))
(testing "->>"
(is (= 10 (edn/read-string (test-utils/bb "foo\n\baar\baaaaz" "-i" "(->> *in* (map count) (apply max))")))))
(testing "literals"
(is (= {:a 4
:b {:a 2}
:c [1 1]
:d #{1 2}}
(bb 1 '{:a (+ 1 2 *in*)
:b {:a (inc *in*)}
:c [*in* *in*]
:d #{*in* (inc *in*)}}))))
(testing "shuffle the contents of a file"
(let [in "foo\n Clojure is nice. \nbar\n If you're nice to clojure. "
in-lines (set (str/split in #"\n"))
out (test-utils/bb in
"-io"
(str '(shuffle *in*)))
out-lines (set (str/split out #"\n"))]
(= in-lines out-lines)))
(testing "find occurrences in file by line number"
(is (= '(1 3) (is (= '(1 3)
(-> (->
(bb "foo\n Clojure is nice. \nbar\n If you're nice to clojure. " (bb "foo\n Clojure is nice. \nbar\n If you're nice to clojure. "
"--raw" "-i"
"(map-indexed #f[%1 %2] *in*)") "(map-indexed #(-> [%1 %2]) *in*)")
(bb "(keep #f(when (re-find #r\"(?i)clojure\" (second %)) (first %)) *in*)")))))) (bb "(keep #f(when (re-find #\"(?i)clojure\" (second %)) (first %)) *in*)"))))))