Compare commits

..

3 commits

Author SHA1 Message Date
Wes Morgan
951a6297d1
Merge branch 'master' into feature/local-manifests 2022-10-10 11:39:47 -06:00
Wes Morgan
6d99ecf40c
Update name of manifest load fn 2022-10-10 11:30:12 -06:00
Wes Morgan
189548ca50
Add support for installing pods from local manifests
...as a part of supporting library-bootstrapped pods
2022-10-07 10:24:34 -06:00
14 changed files with 199 additions and 307 deletions

4
.gitignore vendored
View file

@ -3,7 +3,3 @@
.lein-failures .lein-failures
/pom.xml /pom.xml
.lein-repl-history .lein-repl-history
.cache
.clj-kondo/babashka
.clj-kondo/rewrite-clj
src/scratch.clj

View file

@ -2,20 +2,6 @@
## Unreleased ## Unreleased
- [#63](https://github.com/babashka/pods/issues/63): create directory before un-tarring - #49: don't log socket closed exception
- [#59](https://github.com/babashka/pods/issues/59): delete port file on exit
- [#65](https://github.com/babashka/pods/issues/65): fix warnings when defining var with core name in JVM
- [#66](https://github.com/babashka/pods/issues/66): Allow metadata on fn arguments for transit+json
## v0.2.0 ## ...
- [#61](https://github.com/babashka/pods/issues/61): add transit as explicit JVM dependency
- [#60](https://github.com/babashka/pods/issues/60): transform pod reader error into exception of caller
- Switch "out" and "err" messages to print and flush instead of `println` ([@justone](https://github.com/justone))
- Set TCP_NODELAY on transport socket ([@retrogradeorbit](https://github.com/retrogradeorbit))
- Allow env vars OS_NAME & OS_ARCH to override os props ([@cap10morgan](https://github.com/cap10morgan))
- [#49](https://github.com/babashka/pods/issues/49): don't log socket closed exception
## v0.1.0
Initial version

View file

@ -77,7 +77,7 @@ On the JVM:
When calling `load-pod` with a string or vector of strings (or declaring it in your `bb.edn`), When calling `load-pod` with a string or vector of strings (or declaring it in your `bb.edn`),
the pod is looked up on the local file system (either using the PATH, or using an absolute path). the pod is looked up on the local file system (either using the PATH, or using an absolute path).
When it is called with a qualified symbol and a version - like `(load-pod 'org.babashka/aws "0.0.5")` When it is called with a qualified symbol and a version - like `(load-pod 'org.babashka/aws "0.0.5")`
then it will be looked up in and downloaded from the [pod-registry](https://github.com/babashka/pod-registry). You can customize the file system location that `load-pod` will use by setting the `BABASHKA_PODS_DIR` environment variable. then it will be looked up in and downloaded from the [pod-registry](https://github.com/babashka/pod-registry).
By default babashka will search for a pod binary matching your system's OS and arch. If you want to download By default babashka will search for a pod binary matching your system's OS and arch. If you want to download
pods for a different OS / arch (e.g. for deployment to servers), you can set one or both of the following pods for a different OS / arch (e.g. for deployment to servers), you can set one or both of the following
@ -130,7 +130,7 @@ light weight replacement for native interop (JNI, JNA, etc.).
### Examples ### Examples
Beyond the already available pods mentioned above, educational examples of pods Beyond the already available pods mentioned above, eductional examples of pods
can be found [here](examples): can be found [here](examples):
- [pod-lispyclouds-sqlite](examples/pod-lispyclouds-sqlite): a pod that - [pod-lispyclouds-sqlite](examples/pod-lispyclouds-sqlite): a pod that
@ -228,7 +228,7 @@ JSON. It also declares that the pod exposes one namespace,
To encode payloads in EDN use `"edn"` and for Transit JSON use `"transit+json"`. To encode payloads in EDN use `"edn"` and for Transit JSON use `"transit+json"`.
The pod encodes the above map to bencode and writes it to stdout. The pod client The pod encodes the above map to bencode and writes it to stdoud. The pod client
reads this message from the pod's stdout. reads this message from the pod's stdout.
Upon receiving this message, the pod client creates these namespaces and vars. Upon receiving this message, the pod client creates these namespaces and vars.
@ -376,15 +376,11 @@ nil
#### Metadata #### Metadata
**From pod to pod client**
*Fixed Metadata on vars*
Pods may attach metadata to functions and macros by sending data to the pod client Pods may attach metadata to functions and macros by sending data to the pod client
in a `"meta"` field as part of a `"var"` section. The metadata must be an appropriate in a `"meta"` field as part of a `"var"` section. The metadata must be an appropriate
map, encoded as an EDN string. This is only applicable to vars in the pod and will be map, encoded as an EDN string. This is only applicable to vars in the pod and will be
ignored if the var refers to Client-side code, since metadata can already be defined ignored if the var refers to Client-side code, since metadata can already be defined
in those code blocks (see 'Dynamic Metadata' below to enable the encoding of metadata). in those code blocks.
For example, a pod can define a function called `add`: For example, a pod can define a function called `add`:
@ -396,33 +392,6 @@ For example, a pod can define a function called `add`:
"meta" "{:doc \"arithmetic addition of 2 arguments\" :arglists ([a b])}"}]}]} "meta" "{:doc \"arithmetic addition of 2 arguments\" :arglists ([a b])}"}]}]}
``` ```
*Dynamic Metadata*
Pods may send metadata on values returned to the client if metadata encoding is enabled
for the particular transport format used by the pod.
For example, if your pod uses `:transit+json` as its format, you can enable metadata
encoding by adding `:transform transit/write-meta` (or whatever transit is aliased to)
to the optional map passed to `transit/writer`. e.g.:
````clojure
(transit/writer baos :json {:transform transit/write-meta})
````
##### From pod client to pod
Currently sending metadata on arguments passed to a pod function is available only for the
`transit+json` format and can be enabled on a per var basis.
A pod can enable metadata to be read on arguments by sending the "arg-meta" field to "true"
for the var representing that function. For example:
````clojure
{:format :transit+json
:namespaces [{:name "pod.babashka.demo"
:vars [{"name" "round-trip" "arg-meta" "true"}]}]}
````
#### Deferred namespace loading #### Deferred namespace loading
When your pod exposes multiple namespaces that can be used independently from When your pod exposes multiple namespaces that can be used independently from

View file

@ -1,4 +1,4 @@
(defproject babashka/babashka.pods "0.2.0" (defproject babashka/babashka.pods "0.1.0"
:description "babashka pods" :description "babashka pods"
:url "https://github.com/babashka/babashka.pods" :url "https://github.com/babashka/babashka.pods"
:scm {:name "git" :scm {:name "git"
@ -8,8 +8,7 @@
:dependencies [[org.clojure/clojure "1.10.3"] :dependencies [[org.clojure/clojure "1.10.3"]
[nrepl/bencode "1.1.0"] [nrepl/bencode "1.1.0"]
[cheshire "5.10.0"] [cheshire "5.10.0"]
[babashka/fs "0.1.6"] [babashka/fs "0.1.6"]]
[com.cognitect/transit-clj "1.0.329"]]
:deploy-repositories [["clojars" {:url "https://clojars.org/repo" :deploy-repositories [["clojars" {:url "https://clojars.org/repo"
:username :env/clojars_user :username :env/clojars_user
:password :env/clojars_pass :password :env/clojars_pass

View file

@ -1,17 +0,0 @@
#!/usr/bin/env bb
(ns changelog
(:require [clojure.string :as str]))
(let [changelog (slurp "CHANGELOG.md")
replaced (str/replace changelog
#" #(\d+)"
(fn [[_ issue after]]
(format " [#%s](https://github.com/babashka/pods/issues/%s)%s"
issue issue (str after))))
replaced (str/replace replaced
#"@([a-zA-Z0-9-_]+)([, \.)])"
(fn [[_ name after]]
(format "[@%s](https://github.com/%s)%s"
name name after)))]
(spit "CHANGELOG.md" replaced))

View file

@ -28,9 +28,6 @@
(defn bytes->string [^"[B" bytes] (defn bytes->string [^"[B" bytes]
(String. bytes)) (String. bytes))
(defn bytes->boolean [^"[B" bytes]
(= "true" (String. bytes)))
(defn get-string [m k] (defn get-string [m k]
(-> (get m k) (-> (get m k)
bytes->string)) bytes->string))
@ -39,10 +36,6 @@
(some-> (get m k) (some-> (get m k)
bytes->string)) bytes->string))
(defn get-maybe-boolean [m k]
(some-> (get m k)
bytes->boolean))
(defn next-id [] (defn next-id []
(str (java.util.UUID/randomUUID))) (str (java.util.UUID/randomUUID)))
@ -90,12 +83,10 @@
(let [wh (transit/write-handler tag-fn val-fn)] (let [wh (transit/write-handler tag-fn val-fn)]
(swap! transit-default-write-handlers assoc *pod-id* wh))) (swap! transit-default-write-handlers assoc *pod-id* wh)))
(defn transit-json-write (defn transit-json-write [pod-id ^String s]
[pod-id ^String s metadata?]
(with-open [baos (java.io.ByteArrayOutputStream. 4096)] (with-open [baos (java.io.ByteArrayOutputStream. 4096)]
(let [w (transit/writer baos :json (cond-> {:handlers (get @transit-write-handler-maps pod-id) (let [w (transit/writer baos :json {:handlers (get @transit-write-handler-maps pod-id)
:default-handler (get @transit-default-write-handlers pod-id)} :default-handler (get @transit-default-write-handlers pod-id)})]
metadata? (assoc :transform transit/write-meta)))]
(transit/write w s) (transit/write w s)
(str baos)))) (str baos))))
@ -107,7 +98,7 @@
write-fn (case format write-fn (case format
:edn pr-str :edn pr-str
:json cheshire/generate-string :json cheshire/generate-string
:transit+json #(transit-json-write (:pod-id pod) % (:arg-meta opts))) :transit+json #(transit-json-write (:pod-id pod) %))
id (next-id) id (next-id)
chan (if handlers handlers chan (if handlers handlers
(promise)) (promise))
@ -137,12 +128,11 @@
edn/read-string) edn/read-string)
name-sym (if vmeta name-sym (if vmeta
(with-meta name-sym vmeta) (with-meta name-sym vmeta)
name-sym) name-sym)]
metadata? (get-maybe-boolean var "arg-meta")]
[name-sym [name-sym
(or code (or code
(fn [& args] (fn [& args]
(let [res (invoke pod sym args {:async async? :arg-meta metadata?})] (let [res (invoke pod sym args {:async async?})]
res)))])) res)))]))
vars)) vars))
@ -187,17 +177,14 @@
(let [id (get reply "id") (let [id (get reply "id")
id (bytes->string id) id (bytes->string id)
value* (find reply "value") value* (find reply "value")
[exception value] (try (some->> value* value (some-> value*
second second
bytes->string bytes->string
read-fn read-fn)
(vector nil))
(catch Exception e
[e nil]))
status (get reply "status") status (get reply "status")
status (set (map (comp keyword bytes->string) status)) status (set (map (comp keyword bytes->string) status))
error? (or exception (contains? status :error)) error? (contains? status :error)
done? (or error? exception (contains? status :done)) done? (or error? (contains? status :done))
[ex-message ex-data] [ex-message ex-data]
(when error? (when error?
[(or (some-> (get reply "ex-message") [(or (some-> (get reply "ex-message")
@ -215,9 +202,8 @@
:vars (bencode->vars pod name-str v)})) :vars (bencode->vars pod name-str v)}))
chan (get @chans id) chan (get @chans id)
promise? (instance? clojure.lang.IPending chan) promise? (instance? clojure.lang.IPending chan)
exception (or exception exception (when (and promise? error?)
(when (and promise? error?) (ex-info ex-message ex-data))
(ex-info ex-message ex-data)))
;; NOTE: if we need more fine-grained handlers, we will add ;; NOTE: if we need more fine-grained handlers, we will add
;; a :raw handler that will just get the bencode message's raw ;; a :raw handler that will just get the bencode message's raw
;; data ;; data
@ -320,8 +306,7 @@
(catch java.net.SocketException _ nil))) (catch java.net.SocketException _ nil)))
(defn port-file [pid] (defn port-file [pid]
(doto (io/file (str ".babashka-pod-" pid ".port")) (io/file (str ".babashka-pod-" pid ".port")))
(.deleteOnExit)))
(defn read-port [^java.io.File port-file] (defn read-port [^java.io.File port-file]
(loop [] (loop []
@ -365,7 +350,7 @@
(.redirectError pb java.lang.ProcessBuilder$Redirect/INHERIT)) (.redirectError pb java.lang.ProcessBuilder$Redirect/INHERIT))
_ (cond-> (doto (.environment pb) _ (cond-> (doto (.environment pb)
(.put "BABASHKA_POD" "true")) (.put "BABASHKA_POD" "true"))
socket? (.put "BABASHKA_POD_TRANSPORT" "socket")) socket? (.put "BABASHKA_POD_TRANSPORT" "socket"))
p (.start pb) p (.start pb)
port-file (when socket? (port-file (.pid p))) port-file (when socket? (port-file (.pid p)))
socket-port (when socket? (read-port port-file)) socket-port (when socket? (read-port port-file))

View file

@ -91,10 +91,7 @@
^"[Ljava.nio.file.CopyOption;" ^"[Ljava.nio.file.CopyOption;"
(into-array (into-array
[java.nio.file.StandardCopyOption/REPLACE_EXISTING]))) [java.nio.file.StandardCopyOption/REPLACE_EXISTING])))
(.mkdirs destination-dir) (sh "tar" "xf" (.getPath tmp-file) "--directory" (.getPath destination-dir))
(let [res (sh "tar" "xf" (.getPath tmp-file) "--directory" (.getPath destination-dir))]
(when-not (zero? (:exit res))
(throw (ex-info (:err res) res))))
(.delete tmp-file))) (.delete tmp-file)))
(defn make-executable [dest-dir executables verbose?] (defn make-executable [dest-dir executables verbose?]
@ -134,9 +131,13 @@
"https://raw.githubusercontent.com/babashka/pod-registry/master/manifests/%s/%s/manifest.edn" "https://raw.githubusercontent.com/babashka/pod-registry/master/manifests/%s/%s/manifest.edn"
qsym version)) qsym version))
(defn pod-manifest-file
[qsym version]
(io/file @pods-repo-dir (str qsym) (str version) "manifest.edn"))
(defn pod-manifest (defn pod-manifest
[qsym version force?] [qsym version force?]
(let [f (io/file @pods-repo-dir (str qsym) (str version) "manifest.edn")] (let [f (pod-manifest-file qsym version)]
(if (and (not force?) (if (and (not force?)
(.exists f)) (.exists f))
(edn/read-string (slurp f)) (edn/read-string (slurp f))
@ -185,41 +186,51 @@
(.digest digest)) (.digest digest))
(String. "UTF-8")))) (String. "UTF-8"))))
(defn install-pod-artifacts
[artifacts {cdir :cache-dir, ddir :data-dir, :keys [:force? :pod/options]}]
(let [execs (mapv (fn [artifact]
(let [url (:artifact/url artifact)
file-name (last (str/split url #"/"))
cache-file (io/file cdir file-name)
exe-file-name (:artifact/executable artifact)
executable (io/file ddir exe-file-name)]
(when (or force? (not (.exists executable)))
(warn (format "Downloading pod %s" url))
(download url cache-file false)
(when-let [expected-sha (:artifact/hash artifact)]
(let [sha (sha256 cache-file)]
(when-not (= (str/replace expected-sha #"^sha256:" "")
sha)
(throw (ex-info (str "Wrong SHA-256 for file" (str cache-file))
{:sha sha
:expected-sha expected-sha})))))
(let [filename (.getName cache-file)]
(cond (str/ends-with? filename ".zip")
(unzip {:zip-file cache-file
:destination-dir ddir
:verbose false})
(or (str/ends-with? filename ".tgz")
(str/ends-with? filename ".tar.gz"))
(un-tgz cache-file ddir
false))
(.delete cache-file))
(make-executable ddir [exe-file-name] false)
(warn (format "Successfully installed pod %s" exe-file-name))
(io/file ddir exe-file-name))
(io/file ddir exe-file-name)))
artifacts)]
{:executable (.getAbsolutePath ^java.io.File (first execs))
:options options}))
(defn resolve [qsym version force?] (defn resolve [qsym version force?]
(when-not (string? version) (when-not (string? version)
(throw (IllegalArgumentException. "Version must be provided for resolving from pod registry!"))) (throw (IllegalArgumentException. "Version must be provided for resolving from pod registry!")))
(when-let [manifest (pod-manifest qsym version force?)] (when-let [manifest (pod-manifest qsym version force?)]
(let [artifacts (match-artifacts manifest) (let [cdir (cache-dir manifest)
cdir (cache-dir manifest)
ddir (data-dir manifest) ddir (data-dir manifest)
execs (mapv (fn [artifact] artifacts (match-artifacts manifest)
(let [url (:artifact/url artifact) pod-options (:pod/options manifest)]
file-name (last (str/split url #"/")) (install-pod-artifacts artifacts {:cache-dir cdir
cache-file (io/file cdir file-name) :data-dir ddir
executable (io/file ddir (:artifact/executable artifact))] :force? force?
(when (or force? (not (.exists executable))) :pod/options pod-options}))))
(warn (format "Downloading pod %s (%s)" qsym version))
(download url cache-file false)
(when-let [expected-sha (:artifact/hash artifact)]
(let [sha (sha256 cache-file)]
(when-not (= (str/replace expected-sha #"^sha256:" "")
sha)
(throw (ex-info (str "Wrong SHA-256 for file" (str cache-file))
{:sha sha
:expected-sha expected-sha})))))
(let [filename (.getName cache-file)]
(cond (str/ends-with? filename ".zip")
(unzip {:zip-file cache-file
:destination-dir ddir
:verbose false})
(or (str/ends-with? filename ".tgz")
(str/ends-with? filename ".tar.gz"))
(un-tgz cache-file ddir
false))
(.delete cache-file))
(make-executable ddir [(:artifact/executable artifact)] false)
(warn (format "Successfully installed pod %s (%s)" qsym version))
(io/file ddir (:artifact/executable artifact)))
(io/file ddir (:artifact/executable artifact)))) artifacts)]
{:executable (.getAbsolutePath ^java.io.File (first execs))
:options (:pod/options manifest)})))

View file

@ -12,12 +12,11 @@
(defn- process-namespace [{:keys [:name :vars]}] (defn- process-namespace [{:keys [:name :vars]}]
(binding [*ns* (load-string (format "(ns %s) *ns*" name))] (binding [*ns* (load-string (format "(ns %s) *ns*" name))]
(doseq [[var-sym v] vars] (doseq [[var-sym v] vars]
(when-let [maybe-core (some-> (ns-resolve *ns* var-sym) meta :ns str symbol)]
(when (= 'clojure.core maybe-core)
(ns-unmap *ns* var-sym)))
(cond (cond
(ifn? v) (ifn? v)
(intern name var-sym v) (do
(ns-unmap *ns* var-sym)
(intern name var-sym v))
(string? v) (string? v)
(load-string v))))) (load-string v)))))

View file

@ -41,17 +41,32 @@
(with-open [r (PushbackInputStream. (io/input-stream cache-file))] (with-open [r (PushbackInputStream. (io/input-stream cache-file))]
(impl/read r))))) (impl/read r)))))
(defn load-pod-metadata* [bb-edn-file pod-spec {:keys [:version :cache] :as opts}] (defn cache-pod-metadata! [metadata {:keys [:bb-edn-file :pod-name :pod-version]}]
(let [metadata (impl/load-pod-metadata pod-spec opts) (let [cache-file (metadata-cache-file bb-edn-file pod-name
cache-file (when (and metadata cache) {:version pod-version})]
(metadata-cache-file bb-edn-file pod-spec opts))]
(when cache-file (when cache-file
(io/make-parents cache-file) (io/make-parents cache-file)
(when (fs/writable? (fs/parent cache-file)) (when (fs/writable? (fs/parent cache-file))
(with-open [w (io/output-stream cache-file)] (with-open [w (io/output-stream cache-file)]
(impl/write w metadata)))) (impl/write w metadata))))))
(defn load-pod-metadata* [bb-edn-file pod-spec {:keys [:version :cache] :as opts}]
(let [metadata (impl/load-pod-metadata pod-spec opts)]
(when (and metadata cache)
(cache-pod-metadata! metadata {:bb-edn-file bb-edn-file
:pod-name pod-spec
:pod-version version}))
metadata)) metadata))
(defn pod-namespaces
[pod-name metadata opts]
(reduce
(fn [pod-namespaces ns]
(let [ns-sym (-> ns (get "name") impl/bytes->string symbol)]
(assoc pod-namespaces ns-sym {:pod-spec pod-name
:opts (assoc opts :metadata metadata)})))
{} (get metadata "namespaces")))
(defn load-pod-metadata (defn load-pod-metadata
([pod-spec opts] (load-pod-metadata nil pod-spec opts)) ([pod-spec opts] (load-pod-metadata nil pod-spec opts))
([bb-edn-file pod-spec {:keys [:cache] :as opts}] ([bb-edn-file pod-spec {:keys [:cache] :as opts}]
@ -62,12 +77,7 @@
opts))] opts))]
cached-metadata cached-metadata
(load-pod-metadata* bb-edn-file pod-spec opts))] (load-pod-metadata* bb-edn-file pod-spec opts))]
(reduce (pod-namespaces pod-spec metadata opts))))
(fn [pod-namespaces ns]
(let [ns-sym (-> ns (get "name") impl/bytes->string symbol)]
(assoc pod-namespaces ns-sym {:pod-spec pod-spec
:opts (assoc opts :metadata metadata)})))
{} (get metadata "namespaces")))))
(defn load-pod (defn load-pod
([ctx pod-spec] (load-pod ctx pod-spec nil)) ([ctx pod-spec] (load-pod ctx pod-spec nil))
@ -122,6 +132,29 @@
(sci/future (impl/processor pod)) (sci/future (impl/processor pod))
{:pod/id (:pod-id pod)}))) {:pod/id (:pod-id pod)})))
(defn load-pod-metadata-from-manifest
[manifest {:keys [:bb-edn-file]}]
(let [artifacts (resolver/match-artifacts manifest)
pod-name (:pod/name manifest)]
(when artifacts
(let [cdir (resolver/cache-dir manifest)
ddir (resolver/data-dir manifest)
pod (resolver/install-pod-artifacts
artifacts {:cache-dir cdir
:data-dir ddir
:pod/options (:pod/options manifest)})
metadata (impl/run-pod-for-metadata [(:executable pod)] nil)]
(let [pod-version (:pod/version manifest)
opts {:bb-edn-file bb-edn-file
:pod-name pod-name
:pod-version pod-version}]
(cache-pod-metadata! metadata opts)
(pod-namespaces pod-name metadata {:version pod-version}))))))
(defn pod-manifest-file
[manifest]
(resolver/pod-manifest-file (-> manifest :pod/name symbol) (:pod/version manifest)))
(defn unload-pod (defn unload-pod
([pod-id] (unload-pod pod-id {})) ([pod-id] (unload-pod pod-id {}))
([pod-id _opts] ([pod-id _opts]

View file

@ -61,12 +61,6 @@
(transit/write w s) (transit/write w s)
(str baos)))) (str baos))))
(defn transit-json-write-meta [s]
(with-open [baos (java.io.ByteArrayOutputStream. 4096)]
(let [w (transit/writer baos :json {:transform transit/write-meta})]
(transit/write w s)
(str baos))))
(defn run-pod [cli-args] (defn run-pod [cli-args]
(let [format (cond (contains? cli-args "--json") :json (let [format (cond (contains? cli-args "--json") :json
(contains? cli-args "--transit+json") :transit+json (contains? cli-args "--transit+json") :transit+json
@ -136,10 +130,6 @@
{"name" "read-other-tag" {"name" "read-other-tag"
"code" "(defn read-other-tag [x] [x x])" "code" "(defn read-other-tag [x] [x x])"
"meta" "{:doc \"unread\"}"} "meta" "{:doc \"unread\"}"}
{"name" "round-trip-meta"
"arg-meta" "true"}
{"name" "dont-round-trip-meta"
"arg-meta" "false"}
{"name" "-local-date-time"} {"name" "-local-date-time"}
{"name" "transit-stuff" {"name" "transit-stuff"
"code" " "code" "
@ -161,8 +151,7 @@
(babashka.pods/add-transit-read-handler! \"java.array\" (babashka.pods/add-transit-read-handler! \"java.array\"
into-array) into-array)
"} "}]
{"name" "incorrect-edn"}]
dependents)} dependents)}
{"name" "pod.test-pod.loaded" {"name" "pod.test-pod.loaded"
"defer" "true"} "defer" "true"}
@ -186,91 +175,70 @@
pod.test-pod/add-sync pod.test-pod/add-sync
(try (let [ret (apply + args)] (try (let [ret (apply + args)]
(write out (write out
{"value" (write-fn ret) {"value" (write-fn ret)
"id" id "id" id
"status" ["done"]})) "status" ["done"]}))
(catch Exception e (catch Exception e
(write out (write out
{"ex-data" (write-fn {:args args}) {"ex-data" (write-fn {:args args})
"ex-message" (.getMessage e) "ex-message" (.getMessage e)
"status" ["done" "error"] "status" ["done" "error"]
"id" id}))) "id" id})))
pod.test-pod/range-stream pod.test-pod/range-stream
(let [rng (apply range args)] (let [rng (apply range args)]
(doseq [v rng] (doseq [v rng]
(write out (write out
{"value" (write-fn v) {"value" (write-fn v)
"id" id}) "id" id})
(Thread/sleep 100)) (Thread/sleep 100))
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id})) "id" id}))
pod.test-pod/assoc pod.test-pod/assoc
(write out (write out
{"value" (write-fn (apply assoc args)) {"value" (write-fn (apply assoc args))
"status" ["done"] "status" ["done"]
"id" id}) "id" id})
pod.test-pod/error pod.test-pod/error
(write out (write out
{"ex-data" (write-fn {:args args}) {"ex-data" (write-fn {:args args})
"ex-message" (str "Illegal arguments") "ex-message" (str "Illegal arguments")
"status" ["done" "error"] "status" ["done" "error"]
"id" id}) "id" id})
pod.test-pod/print pod.test-pod/print
(do (write out (do (write out
{"out" (with-out-str (prn args)) {"out" (with-out-str (prn args))
"id" id}) "id" id})
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id})) "id" id}))
pod.test-pod/print-err pod.test-pod/print-err
(do (write out (do (write out
{"err" (with-out-str (prn args)) {"err" (with-out-str (prn args))
"id" id}) "id" id})
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id})) "id" id}))
pod.test-pod/return-nil pod.test-pod/return-nil
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"value" (write-fn nil)}) "value" (write-fn nil)})
pod.test-pod/reader-tag pod.test-pod/reader-tag
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"value" "#my/tag[1 2 3]"}) "value" "#my/tag[1 2 3]"})
pod.test-pod/other-tag pod.test-pod/other-tag
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"value" "#my/other-tag[1]"}) "value" "#my/other-tag[1]"})
pod.test-pod/round-trip-meta
(write out
{"status" ["done"]
"id" id
"value"
(case format
:transit+json (transit-json-write-meta (first args))
(write-fn (first args)))})
pod.test-pod/dont-round-trip-meta
(write out
{"status" ["done"]
"id" id
"value"
(case format
:transit+json (transit-json-write-meta (first args))
(write-fn (first args)))})
pod.test-pod/-local-date-time pod.test-pod/-local-date-time
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"value" (write-fn (first args))}) "value" (write-fn (first args))}))
pod.test-pod/incorrect-edn
(write out
{"status" ["done"]
"id" id
"value" (write-fn {(keyword "foo bar") 1})}))
(recur)) (recur))
:shutdown (System/exit 0) :shutdown (System/exit 0)
:load-ns (let [ns (-> (get message "ns") :load-ns (let [ns (-> (get message "ns")
@ -281,20 +249,20 @@
(case ns (case ns
pod.test-pod.loaded pod.test-pod.loaded
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"name" "pod.test-pod.loaded" "name" "pod.test-pod.loaded"
"vars" [{"name" "loaded" "vars" [{"name" "loaded"
"code" "(defn loaded [x] (inc x))"}]}) "code" "(defn loaded [x] (inc x))"}]})
pod.test-pod.loaded2 pod.test-pod.loaded2
(write out (write out
{"status" ["done"] {"status" ["done"]
"id" id "id" id
"name" "pod.test-pod.loaded2" "name" "pod.test-pod.loaded2"
"vars" [{"name" "x" "vars" [{"name" "x"
"code" "(require '[pod.test-pod.loaded :as loaded])"} "code" "(require '[pod.test-pod.loaded :as loaded])"}
{"name" "loaded" {"name" "loaded"
"code" "(defn loaded [x] (loaded/loaded x))"}]})) "code" "(defn loaded [x] (loaded/loaded x))"}]}))
(recur))))))) (recur)))))))
(catch Exception e (catch Exception e
(binding [*out* *err*] (binding [*out* *err*]

View file

@ -84,31 +84,12 @@
(.isArray (class v))) (.isArray (class v)))
true)) true))
(def round-trip-meta
(if (= "transit+json" fmt)
(= {:my-meta 2} (meta (pod.test-pod/round-trip-meta (with-meta [2] {:my-meta 2}))))
true))
(def round-trip-meta-nested
(if (= "transit+json" fmt)
(= {:my-meta 3} (meta (first (pod.test-pod/round-trip-meta [(with-meta [3] {:my-meta 3})]))))
true))
(def dont-round-trip-meta
(if (= "transit+json" fmt)
(= nil (meta (pod.test-pod/dont-round-trip-meta (with-meta [2] {:my-meta 2}))))
true))
(require '[pod.test-pod.only-code :as only-code]) (require '[pod.test-pod.only-code :as only-code])
(def should-be-1 (only-code/foo)) (def should-be-1 (only-code/foo))
(require '[pod.test-pod.loaded2 :as loaded2]) (require '[pod.test-pod.loaded2 :as loaded2])
(def loaded (loaded2/loaded 1)) (def loaded (loaded2/loaded 1))
(def incorrect-edn-response
(try (pod.test-pod/incorrect-edn)
(catch Exception e (ex-message e))))
(pods/unload-pod pod-id) (pods/unload-pod pod-id)
(def successfully-removed (nil? (find-ns 'pod.test-pod))) (def successfully-removed (nil? (find-ns 'pod.test-pod)))
@ -131,11 +112,7 @@
fn-called fn-called
local-date-time local-date-time
assoc-string-array assoc-string-array
round-trip-meta
round-trip-meta-nested
dont-round-trip-meta
should-be-1 should-be-1
add-sync-meta add-sync-meta
error-meta error-meta
read-other-tag-meta read-other-tag-meta]
incorrect-edn-response]

View file

@ -19,7 +19,7 @@
(let [out (java.io.StringWriter.) (let [out (java.io.StringWriter.)
err (java.io.StringWriter.) err (java.io.StringWriter.)
ex (binding [*out* out ex (binding [*out* out
*err* err] *err* err]
(try (load-string (try (load-string
pod-registry) pod-registry)
(catch Exception e (catch Exception e

View file

@ -23,9 +23,7 @@
_ (vreset! ctx-ref ctx) _ (vreset! ctx-ref ctx)
ret (sci/binding [sci/out out ret (sci/binding [sci/out out
sci/err err] sci/err err]
(binding [*out* out (sci/eval-string* ctx test-program))]
*err* err]
(sci/eval-string* ctx test-program)))]
(assertions out err ret))) (assertions out err ret)))
(deftest pod-registry-test (deftest pod-registry-test

View file

@ -1,6 +1,5 @@
(ns babashka.pods.test-common (ns babashka.pods.test-common
(:require [clojure.java.io :as io] (:require [clojure.java.io :as io]
[clojure.string :as str]
[clojure.test :refer [is]])) [clojure.test :refer [is]]))
(def test-program (slurp (io/file "test-resources" "test_program.clj"))) (def test-program (slurp (io/file "test-resources" "test_program.clj")))
@ -10,45 +9,34 @@
;; (.println System/err out) ;; (.println System/err out)
;; (.println System/err err) ;; (.println System/err err)
(doseq [[expected actual] (doseq [[expected actual]
(map vector (replace (map vector '["pod.test-pod"
{::edn-error (if (= "edn" pod.test-pod
(System/getenv "BABASHKA_POD_TEST_FORMAT")) {:a 1, :b 2}
"Map literal must contain an even number of forms" 6
::dont-care)} 3
'["pod.test-pod" [1 2 3 4 5 6 7 8 9]
pod.test-pod #"Illegal arguments / \{:args [\(\[]1 2 3[\)\]]\}"
{:a 1, :b 2} nil
6 3
3 #"cast"
[1 2 3 4 5 6 7 8 9] {:args ["1" 2]}
#"Illegal arguments / \{:args [\(\[]1 2 3[\)\]]\}" true
nil 9
3 [1 2 3]
#"cast" [[1] [1]]
{:args ["1" 2]} 2
true 3
9 true ;; local-date
[1 2 3] true ;; roundtrip string array
[[1] [1]] 1
2 "add the arguments"
3 nil
true ;; local-date nil]
true ;; roundtrip string array
true ;; roundtrip metadata
true ;; roundtrip metadata nested
true ;; dont roundtrip metadata (when arg-meta "false"/ absent)
1
"add the arguments"
nil
nil
::edn-error])
(concat ret (repeat ::nil)))] (concat ret (repeat ::nil)))]
(cond (instance? java.util.regex.Pattern expected) (if (instance? java.util.regex.Pattern expected)
(is (re-find expected actual)) (is (re-find expected actual))
(= ::dont-care expected) nil (is (= expected actual))))
:else
(is (= expected actual))))
(is (= "(\"hello\" \"print\" \"this\" \"debugging\" \"message\")\n:foo\n:foo\n" (str out))) (is (= "(\"hello\" \"print\" \"this\" \"debugging\" \"message\")\n:foo\n:foo\n" (str out)))
(is (str/starts-with? (str err) "(\"hello\" \"print\" \"this\" \"error\")" ))) (is (= "(\"hello\" \"print\" \"this\" \"error\")\n" (str err))))
(def pod-registry (slurp (io/file "test-resources" "pod_registry.clj"))) (def pod-registry (slurp (io/file "test-resources" "pod_registry.clj")))