lazy namespaces experiment

.circleci/config.yml

@@ -1,28 +0,0 @@
-version: 2.1
-jobs:
-  test:
-    docker:
-      - image: clojure:openjdk-11-tools-deps-1.10.3.1087-slim-bullseye
-    working_directory: ~/repo
-    environment:
-      LEIN_ROOT: "true"
-      BABASHKA_PLATFORM: linux
-    resource_class: large
-    steps:
-      - checkout
-      - restore_cache:
-          keys:
-            - v1-dependencies-{{ checksum "deps.edn" }}
-            # fallback to using latest cache if no exact match is found
-            - v1-dependencies-
-      - run: |
-          script/test
-      - save_cache:
-          paths:
-            - ~/.m2
-          key: v1-dependencies-{{ checksum "deps.edn" }}
-workflows:
-  version: 2
-  ci:
-    jobs:
-      - test

.gitignore

@@ -3,7 +3,3 @@
 .lein-failures
 /pom.xml
 .lein-repl-history
-.cache
-.clj-kondo/babashka
-.clj-kondo/rewrite-clj
-src/scratch.clj

CHANGELOG.md

@@ -1,21 +0,0 @@
-# Changelog
-
-## Unreleased
-
-- [#63](https://github.com/babashka/pods/issues/63): create directory before un-tarring
-- [#59](https://github.com/babashka/pods/issues/59): delete port file on exit
-- [#65](https://github.com/babashka/pods/issues/65): fix warnings when defining var with core name in JVM
-- [#66](https://github.com/babashka/pods/issues/66): Allow metadata on fn arguments for transit+json
-
-## v0.2.0
-
-- [#61](https://github.com/babashka/pods/issues/61): add transit as explicit JVM dependency
-- [#60](https://github.com/babashka/pods/issues/60): transform pod reader error into exception of caller
-- Switch "out" and "err" messages to print and flush instead of `println` ([@justone](https://github.com/justone))
-- Set TCP_NODELAY on transport socket ([@retrogradeorbit](https://github.com/retrogradeorbit))
--  Allow env vars OS_NAME & OS_ARCH to override os props ([@cap10morgan](https://github.com/cap10morgan))
-- [#49](https://github.com/babashka/pods/issues/49): don't log socket closed exception
-
-## v0.1.0
-
-Initial version

README.md

@@ -1,11 +1,7 @@
-# Babashka pods
+# babashka.pods
-
-[![Clojars Project](https://img.shields.io/clojars/v/babashka/babashka.pods.svg)](https://clojars.org/babashka/babashka.pods)
-
-Babashka pods are programs that can be used as Clojure libraries by babashka.
 
 This is the library to load babashka pods. It is used by
-[babashka](https://github.com/babashka/babashka/) but also usable from the JVM
+[babashka](https://github.com/borkdude/babashka/) but also usable from the JVM
 and [sci](https://github.com/borkdude/sci)-based projects other than babashka.
 
 <a title="Wa17gs / CC BY-SA (https://creativecommons.org/licenses/by-sa/4.0)" href="https://commons.wikimedia.org/wiki/File:Below_Golden_Gate_Bridge.jpeg"><img width="512" alt="Below Golden Gate Bridge" src="https://upload.wikimedia.org/wikipedia/commons/thumb/9/9a/Below_Golden_Gate_Bridge.jpeg/512px-Below_Golden_Gate_Bridge.jpeg"></a>
@@ -26,7 +22,7 @@ babashka is the pod client. When a JVM invokes a pod, the JVM is the pod client.
 - _message_: a message sent from the pod client to the pod or vice versa,
   encoded in [bencode](https://en.wikipedia.org/wiki/Bencode) format.
 - _payload_: a particular field of a _message_ encoded in a _payload format_
-  (currently JSON, EDN or Transit JSON). Examples are `args`, `value` and `ex-data`.  _
+  (currently only JSON or EDN). Examples are `args`, `value` and `ex-data`.  _
 -  _pod protocol_: the documented way of exchanging messages between a _pod
   client_ and _pod_.
 
@@ -47,8 +43,24 @@ _below_ in Polish and Russian. In Romanian it means _bridge_
 
 ## Available pods
 
-For a list of available pods, take a look
+Currently the following pods are available:
-[here](https://github.com/babashka/babashka/blob/master/doc/projects.md#pods).
+
+- [bootleg](https://github.com/retrogradeorbit/bootleg): static HTML website
+  generation.
+- [clj-kondo](https://github.com/borkdude/clj-kondo/#babashka-pod): a Clojure
+  linter
+- [pod-babashka-filewatcher](https://github.com/babashka/pod-babashka-filewatcher): a
+  filewatcher pod based on Rust notify.
+- [pod-babashka-hsqldb](https://github.com/babashka/pod-babashka-hsqldb): a pod
+  that allows you to create and fire queries at a
+  [HSQLDB](http://www.hsqldb.org/) database.
+- [pod-janet-peg](https://github.com/sogaiu/pod-janet-peg): a pod for
+  calling [Janet](https://github.com/janet-lang/janet)'s PEG
+  functionality
+- [pod-jaydeesimon-jsoup](https://github.com/jaydeesimon/pod-jaydeesimon-jsoup):
+    a pod for parsing HTML using CSS queries backed by Jsoup.
+- [pod-lispyclouds-docker](https://github.com/lispyclouds/pod-lispyclouds-docker):
+  A pod for interacting with docker
 
 ## Status
 
@@ -72,47 +84,6 @@ On the JVM:
 ;;=> [#:next.jdbc{:update-count 0}]
 ```
 
-### Where does the pod come from?
-
-When calling `load-pod` with a string or vector of strings (or declaring it in your `bb.edn`),
-the pod is looked up on the local file system (either using the PATH, or using an absolute path).
-When it is called with a qualified symbol and a version - like `(load-pod 'org.babashka/aws "0.0.5")`
-then it will be looked up in and downloaded from the [pod-registry](https://github.com/babashka/pod-registry). You can customize the file system location that `load-pod` will use by setting the `BABASHKA_PODS_DIR` environment variable.
-
-By default babashka will search for a pod binary matching your system's OS and arch. If you want to download
-pods for a different OS / arch (e.g. for deployment to servers), you can set one or both of the following
-environment variables:
-
-- `BABASHKA_PODS_OS_NAME=Linux` (or `Mac OS X` or any other value returned by Java's `os.name` property)
-- `BABASHKA_PODS_OS_ARCH=aarch64` (or `amd64` or any other value returned by Java's `os.arch` property)
-
-### In a babashka project
-
-As of babashka 0.8.0 you can declare the pods your babashka project uses in your `bb.edn` file like so:
-
-```clojure
-:pods {org.babashka/hsqldb {:version "0.1.0"} ; will be downloaded from the babashka pod registry
-       my.local/pod {:path "../pod-my-local/my-pod-binary"
-                     :cache false}} ; optionally disable namespace caching if you're actively working on this pod
-```
-
-Then you can just require the pods in your code like any other clojure lib:
-
-```clojure
-(ns my.project
-  (:require [pod.babashka.hsqldb :as sql]
-            [my.local.pod :as my-pod]))
-
-(def db "jdbc:hsqldb:mem:testdb;sql.syntax_mys=true")
-(sql/execute! db ["create table foo ( foo int );"])
-;;=> [#:next.jdbc{:update-count 0}]
-
-(my-pod/do-a-thing "foo")
-;;=> "something"
-```
-
-The pods will then be loaded on demand when you require them. No need to call `load-pod` explicitly.
-
 ## Sci
 
 To use pods in a [sci](https://github.com/borkdude/sci) based project, see
@@ -130,10 +101,10 @@ light weight replacement for native interop (JNI, JNA, etc.).
 
 ### Examples
 
-Beyond the already available pods mentioned above, educational examples of pods
+Beyond the already available pods mentioned above, eductional examples of pods
-can be found [here](examples):
+can be found [here](../examples/pods):
 
-- [pod-lispyclouds-sqlite](examples/pod-lispyclouds-sqlite): a pod that
+- [pod-lispyclouds-sqlite](../examples/pods/pod-lispyclouds-sqlite): a pod that
   allows you to create and fire queries at a [sqlite](https://www.sqlite.org/)
   database. Implemented in Python.
 
@@ -179,10 +150,10 @@ format that only has four types:
 - byte strings
 
 Additionally, _payloads_ like `args` (arguments) or `value` (a function return
-value) are encoded in either EDN, JSON or Transit JSON.
+value) are encoded in either JSON or EDN.
 
 So remember: messages are in bencode, payloads (particular fields in the
-message) are in either EDN, JSON or Transit JSON.
+message) are in either JSON or EDN.
 
 Bencode is chosen as the message format because it is a light-weight format
 which can be implemented in 200-300 lines of code in most languages. If pods are
@@ -190,9 +161,12 @@ implemented in Clojure, they only need to depend on the
 [bencode](https://github.com/nrepl/bencode) library and use `pr-str` and
 `edn/read-string` for encoding and decoding payloads.
 
-So we use bencode as the first encoding and choose one of multiple richer encodings on top of this, similar to how the nREPL protocol is implemented. More
+Why isn't EDN or JSON chosen as the message format instead of bencode, you may
-payload formats might be added in the future.
+ask.  Assuming EDN or JSON as the message and payload format for all pods is too
-Other languages typically use a bencode library + a JSON library to encode payloads.
+constraining: other languages might already have built-in JSON support and there
+might not be a good EDN library available. So we use bencode as the first
+encoding and choose one of multiple richer encodings on top of this. More
+payload formats might be added in the future (e.g. transit).
 
 When calling the `babashka.pods/load-pod` function, the pod client will start
 the pod and leave the pod running throughout the duration of a babashka script.
@@ -212,7 +186,7 @@ Encoded in bencode this looks like:
 ;;=> d2:op8:describee
 ```
 
-The pod should reply to this request with a message similar to:
+The pod should reply to this request with a message in the vein of:
 
 ``` clojure
 {"format" "json"
@@ -226,9 +200,7 @@ In this reply, the pod declares that payloads will be encoded and decoded using
 JSON. It also declares that the pod exposes one namespace,
 `pod.lispyclouds.sqlite` with one var `execute!`.
 
-To encode payloads in EDN use `"edn"` and for Transit JSON use `"transit+json"`.
+The pod encodes the above map to bencode and writes it to stdoud. The pod client
-
-The pod encodes the above map to bencode and writes it to stdout. The pod client
 reads this message from the pod's stdout.
 
 Upon receiving this message, the pod client creates these namespaces and vars.
@@ -254,8 +226,8 @@ As a pod user, you can load the pod with:
 #### invoke
 
 When invoking a var that is related to the pod, let's call it a _proxy var_, the
-pod client reaches out to the pod with the arguments encoded in EDN, JSON or Transit JSON. The
+pod client reaches out to the pod with the arguments encoded in JSON or EDN. The
-pod will then respond with a return value encoded in EDN, JSON or Transit JSON. The pod client
+pod will then respond with a return value encoded in JSON or EDN. The pod client
 will then decode the return value and present the user with that.
 
 Example: the user invokes `(sql/execute! "select * from foo")`. The pod client
@@ -374,98 +346,6 @@ In the pod client:
 nil
 ```
 
-#### Metadata
-
-**From pod to pod client**
-
-*Fixed Metadata on vars*
-
-Pods may attach metadata to functions and macros by sending data to the pod client
-in a `"meta"` field as part of a `"var"` section. The metadata must be an appropriate
-map, encoded as an EDN string. This is only applicable to vars in the pod and will be
-ignored if the var refers to Client-side code, since metadata can already be defined
-in those code blocks (see 'Dynamic Metadata' below to enable the encoding of metadata).
-
-For example, a pod can define a function called `add`:
-
-``` clojure
-{"format" "json"
- "namespaces"
- [{"name" "pod.babashka.demo"
-   "vars" [{"name" "add"
-            "meta" "{:doc \"arithmetic addition of 2 arguments\" :arglists ([a b])}"}]}]}
-```
-
-*Dynamic Metadata*
-
-Pods may send metadata on values returned to the client if metadata encoding is enabled
-for the particular transport format used by the pod.
-
-For example, if your pod uses `:transit+json` as its format, you can enable metadata
-encoding by adding `:transform transit/write-meta` (or whatever transit is aliased to)
-to the optional map passed to `transit/writer`. e.g.:
-
-````clojure
-(transit/writer baos :json {:transform transit/write-meta})
-````
-
-##### From pod client to pod
-
-Currently sending metadata on arguments passed to a pod function is available only for the
-`transit+json` format and can be enabled on a per var basis.
-
-A pod can enable metadata to be read on arguments by sending the "arg-meta" field to "true"
-for the var representing that function. For example:
-
-````clojure
-{:format :transit+json
-    :namespaces [{:name "pod.babashka.demo"
-                  :vars [{"name" "round-trip" "arg-meta" "true"}]}]}
-````
-
-#### Deferred namespace loading
-
-When your pod exposes multiple namespaces that can be used independently from
-each other, consider implementing the `load-ns` op which allows the pod client
-to load the namespace and process the client side code when it is loaded using
-`require`. This will speed up the initial setup of the pod in `load-pod`.
-
-In `describe` the pod will mark the namespaces as deferred:
-
-``` clojure
-{"name" "pod.lispyclouds.deferred-ns"
- "defer" "true"}
-```
-
-When the user requires the namespace with `(require
-'[pod.lispyclouds.deferred-ns])` the pod client will then send a message:
-
-``` clojure
-{"op" "load-ns"
- "ns" "pod.lispyclouds.deferred-ns"
- "id  "..."}
-```
-
-upon which the pod will reply with the namespace data:
-
-``` clojure
-{"name" "pod.lispyclouds.deferred-ns"
- "vars" [{"name" "myfunc" "code" "(defn my-func [])"}]
- "id" "..."}
-```
-
-If a deferred namespace depends on another deferred namespace, provide explicit
-`require`s in `code` segments:
-
-``` clojure
-{"name" "pod.lispyclouds.another-deferred-ns"
- "vars"
- [{"name" "myfunc"
-   "code" "(require '[pod.lispyclouds.deferred-ns :as dns])
-           (defn my-func [] (dns/x))"}]
- "id" "..."}
-```
-
 #### Async
 
 Asynchronous functions can be implemented using callbacks.
@@ -497,7 +377,7 @@ The arguments to `babashka.pods/invoke` are:
 - a pod identifier string derived from the first described namespace.
 - the symbol of the var to invoke
 - the arguments to the var
-- an opts map containing `:handlers` containing callback functions: `:success`, `:error` and `:done`
+- an opts map containing `:handler` containing callback functions: `:success`, `:error` and `:done`
 
 The return value of `babashka.pods/invoke` is a map containing `:result`. When
 not using callbacks, this is the return value from the pod var invocation. When
@@ -523,9 +403,9 @@ callback is only called if no errors were sent by the pod.
 
 In the above example the wrapper function calls the pod identified by
 `"pod.babashka.filewatcher"`. It calls the var
-`pod.babashka.filewatcher/watch*`. In `:success` it pulls out received
+`pod.babashka.filewatcher/watch*`. In `:on-success` it pulls out received
 values, passing them to the user-provided callback. Additionally, it prints any
-errors received from the pod library in `:error` to `*err*`.
+errors received from the pod library in `:on-error` to `*err*`.
 
 A user will then use `pod.babashka.filewatcher/watch` like this:
 
@@ -542,7 +422,7 @@ user=> (fw/watch "/tmp" (fn [result] (prn "result" result)))
 nil
 user=> (spit "/tmp/foobar123.txt" "foo")
 nil
-user=> "result" {:path "/private/tmp/foobar123.txt", :type :create}
+user=> "result" {:path "/private/tmp/foobar123.txt", :type "create"}
 ```
 
 ## Run tests

bb.edn

@@ -1,2 +0,0 @@
-{:tasks {test {:doc "Run tests"
-               :task (shell "script/test")}}}

deps.edn

@@ -1,15 +1,12 @@
 {:deps {nrepl/bencode {:mvn/version "1.1.0"}
-        cheshire/cheshire {:mvn/version "5.10.0"}
+        cheshire {:mvn/version "5.10.0"}}
-        com.cognitect/transit-clj {:mvn/version "1.0.324"}
-        babashka/fs {:mvn/version "0.1.6"}}
  :aliases
  {:sci
   {:extra-deps
-   {borkdude/sci {:git/url "https://github.com/borkdude/sci"
+   {borkdude/sci {:mvn/version "0.0.13-alpha.26"}}}
-                  :sha "5aa9031eb3692a2207106076088fcab7347c2299"}}}
   :test
   {:extra-deps
-   {cognitect/test-runner
+   {test-runner
     {:git/url "https://github.com/cognitect-labs/test-runner"
      :sha "cb96e80f6f3d3b307c59cbeb49bb0dcb3a2a780b"}}
    :extra-paths ["test"]
@@ -19,5 +16,5 @@
    {lambdaisland/kaocha {:mvn/version "1.0.632"}}
    :main-opts ["-m" "kaocha.runner"]}
   :test-pod
-  {:extra-paths ["src" "test-pod"]
+  {:paths ["src" "test-pod"]
    :main-opts ["-m" "pod.test-pod"]}}}

project.clj

@@ -1,17 +1,15 @@
-(defproject babashka/babashka.pods "0.2.0"
+(defproject babashka/babashka.pods "0.0.1-SNAPSHOT"
   :description "babashka pods"
   :url "https://github.com/babashka/babashka.pods"
   :scm {:name "git"
         :url "https://github.com/babashka/babashka.pods"}
   :license {:name "EPL-1.0"
             :url "https://www.eclipse.org/legal/epl-1.0/"}
-  :dependencies [[org.clojure/clojure "1.10.3"]
+  :dependencies [[org.clojure/clojure "1.10.2-alpha1"]
                  [nrepl/bencode "1.1.0"]
-                 [cheshire "5.10.0"]
+                 [cheshire "5.10.0"]]
-                 [babashka/fs "0.1.6"]
-                 [com.cognitect/transit-clj "1.0.329"]]
   :deploy-repositories [["clojars" {:url "https://clojars.org/repo"
-                                    :username :env/clojars_user
+                                    :username :env/babashka_nrepl_clojars_user
-                                    :password :env/clojars_pass
+                                    :password :env/babashka_nrepl_clojars_pass
                                     :sign-releases false}]]
-  :profiles {:test {:dependencies [[borkdude/sci "0.2.4"]]}})
+  :profiles {:test {:dependencies [[borkdude/sci "0.0.13-alpha.26"]]}})

script/changelog.clj

@@ -1,17 +0,0 @@
-#!/usr/bin/env bb
-
-(ns changelog
-  (:require [clojure.string :as str]))
-
-(let [changelog (slurp "CHANGELOG.md")
-      replaced (str/replace changelog
-                            #" #(\d+)"
-                            (fn [[_ issue after]]
-                              (format " [#%s](https://github.com/babashka/pods/issues/%s)%s"
-                                      issue issue (str after))))
-      replaced (str/replace replaced
-                            #"@([a-zA-Z0-9-_]+)([, \.)])"
-                            (fn [[_ name after]]
-                              (format "[@%s](https://github.com/%s)%s"
-                                      name name after)))]
-  (spit "CHANGELOG.md" replaced))

script/test

@@ -1,32 +1,7 @@
 #!/usr/bin/env bash
 
-set -eou pipefail
+clojure -A:test -n babashka.pods.jvm-test
+#clojure -A:test:kaocha --focus babashka.pods.jvm-test
 
-export BABASHKA_POD_TEST_FORMAT
+clojure -A:sci:test -n babashka.pods.sci-test
-export BABASHKA_POD_TEST_SOCKET
+#clojure -A:sci:test:kaocha --focus babashka.pods.sci-test
-
-# format = edn
-BABASHKA_POD_TEST_FORMAT=edn
-echo "Testing edn"
-clojure -M:test -n babashka.pods.jvm-test
-clojure -M:sci:test -n babashka.pods.sci-test
-clojure -M:test -n babashka.pods.impl-test
-
-# format = json
-BABASHKA_POD_TEST_FORMAT=json
-echo "Testing json"
-clojure -M:test -n babashka.pods.jvm-test
-clojure -M:sci:test -n babashka.pods.sci-test
-
-# format = json
-BABASHKA_POD_TEST_FORMAT="transit+json"
-echo "Testing transit"
-clojure -M:test -n babashka.pods.jvm-test
-clojure -M:sci:test -n babashka.pods.sci-test
-
-# socket = true
-unset BABASHKA_POD_TEST_FORMAT
-BABASHKA_POD_TEST_SOCKET=true
-echo "Testing socket"
-clojure -M:test -n babashka.pods.jvm-test
-clojure -M:sci:test -n babashka.pods.sci-test

src/babashka/pods.clj

@@ -3,29 +3,12 @@
 
 (defn load-pod
   ([pod-spec] (load-pod pod-spec nil))
-  ([pod-spec version opts]
-   (load-pod pod-spec (assoc opts :version version)))
   ([pod-spec opts] (jvm/load-pod pod-spec opts)))
 
 (defn unload-pod
-  ([pod-id-or-pod] (unload-pod pod-id-or-pod {}))
+  ([pod-id] (unload-pod pod-id {}))
-  ([pod-id-or-pod opts] (jvm/unload-pod pod-id-or-pod opts)))
+  ([pod-id opts] (jvm/unload-pod pod-id opts)))
 
 (defn invoke
-  ([pod-id-or-pod sym args] (invoke pod-id-or-pod sym args {}))
+  ([pod-id sym args] (invoke pod-id sym args {}))
-  ([pod-id-or-pod sym args opts] (jvm/invoke pod-id-or-pod sym args opts)))
+  ([pod-id sym args opts] (jvm/invoke pod-id sym args opts)))
-
-(defmacro copy-var [name var]
-  `(do (def ~name ~var)
-       (let [m# (meta (var ~var))
-             doc# (:doc m#)
-             arglists# (:arglists m#)]
-         (alter-meta! (var ~name) assoc
-                      :arglists arglists#
-                      :doc doc#))))
-
-#_:clj-kondo/ignore
-(do
-  (copy-var add-transit-read-handler! jvm/add-transit-read-handler!)
-  (copy-var add-transit-write-handler! jvm/add-transit-write-handler!)
-  (copy-var set-default-transit-write-handler! jvm/set-default-transit-write-handler!))

src/babashka/pods/impl.clj

@@ -1,15 +1,9 @@
 (ns babashka.pods.impl
   {:no-doc true}
   (:refer-clojure :exclude [read])
-  (:require [babashka.pods.impl.resolver :as resolver]
+  (:require [bencode.core :as bencode]
-            [bencode.core :as bencode]
             [cheshire.core :as cheshire]
-            [clojure.edn :as edn]
+            [clojure.edn :as edn]))
-            [clojure.java.io :as io]
-            [clojure.string :as str]
-            [cognitect.transit :as transit])
-  (:import [java.io PushbackInputStream]
-           [java.net Socket]))
 
 (set! *warn-on-reflection* true)
 
@@ -28,9 +22,6 @@
 (defn bytes->string [^"[B" bytes]
   (String. bytes))
 
-(defn bytes->boolean [^"[B" bytes]
-  (= "true" (String. bytes)))
-
 (defn get-string [m k]
   (-> (get m k)
       bytes->string))
@@ -39,66 +30,83 @@
   (some-> (get m k)
           bytes->string))
 
-(defn get-maybe-boolean [m k]
+(defn processor [pod]
-  (some-> (get m k)
+  (let [stdout (:stdout pod)
-          bytes->boolean))
+        format (:format pod)
+        chans (:chans pod)
+        out-stream (:out pod)
+        err-stream (:err pod)
+        readers (:readers pod)
+        read-fn (case format
+                  :edn #(edn/read-string {:readers readers} %)
+                  :json #(cheshire/parse-string-strict % true))]
+    (try
+      (loop []
+        (let [reply (try (read stdout)
+                         (catch java.io.EOFException _
+                           ::EOF))]
+          (when-not (identical? ::EOF reply)
+            (let [id    (get reply "id")
+                  id    (bytes->string id)
+                  value* (find reply "value")
+                  value (some-> value*
+                                second
+                                bytes->string
+                                read-fn)
+                  status (get reply "status")
+                  status (set (map (comp keyword bytes->string) status))
+                  error? (contains? status :error)
+                  done? (or error? (contains? status :done))
+                  [ex-message ex-data]
+                  (when error?
+                    [(or (some-> (get reply "ex-message")
+                                 bytes->string)
+                         "")
+                     (or (some-> (get reply "ex-data")
+                                 bytes->string
+                                 read-fn)
+                         {})])
+                  chan (get @chans id)
+                  promise? (instance? clojure.lang.IPending chan)
+                  exception (when (and promise? error?)
+                              (ex-info ex-message ex-data))
+                  ;; NOTE: if we need more fine-grained handlers, we will add
+                  ;; a :raw handler that will just get the bencode message's raw
+                  ;; data
+                  {error-handler :error
+                   done-handler :done
+                   success-handler :success} (when (map? chan)
+                                               chan)
+                  out (some-> (get reply "out")
+                              bytes->string)
+                  err (some-> (get reply "err")
+                              bytes->string)]
+              (when (or value* error?)
+                (cond promise?
+                      (deliver chan (if error? exception value))
+                      (and (not error?) success-handler)
+                      (success-handler value)
+                      (and error? error-handler)
+                      (error-handler {:ex-message ex-message
+                                      :ex-data ex-data})))
+              (when (and done? (not error?))
+                (when promise?
+                  (deliver chan nil))
+                (when done-handler
+                  (done-handler)))
+              (when out
+                (binding [*out* out-stream]
+                  (println out)))
+              (when err (binding [*out* err-stream]
+                          (println err))))
+            (recur))))
+      (catch Exception e
+        (binding [*out* *err* #_err-stream]
+          (prn e))))))
 
 (defn next-id []
   (str (java.util.UUID/randomUUID)))
 
-(def ^:dynamic *pod-id* nil)
-
-(defonce transit-read-handlers (atom {}))
-(defonce transit-read-handler-maps (atom {}))
-
-(defn update-transit-read-handler-map []
-  (swap! transit-read-handler-maps assoc *pod-id*
-         (transit/read-handler-map (get @transit-read-handlers *pod-id*))))
-
-(defn transit-json-read [pod-id ^String s]
-  (with-open [bais (java.io.ByteArrayInputStream. (.getBytes s "UTF-8"))]
-    (let [r (transit/reader bais :json {:handlers (get @transit-read-handler-maps pod-id)})]
-      (transit/read r))))
-
-;; https://www.cognitect.com/blog/2015/9/10/extending-transit
-(defn add-transit-read-handler!
-  ([tag fn]
-   (let [rh (transit/read-handler fn)]
-     (swap! transit-read-handlers assoc-in [*pod-id* tag] rh)
-     (update-transit-read-handler-map)
-     nil)))
-
-(defonce transit-write-handlers (atom {}))
-(defonce transit-write-handler-maps (atom {}))
-
-(defn update-transit-write-handler-map []
-  (swap! transit-write-handler-maps assoc *pod-id*
-         (transit/write-handler-map (get @transit-write-handlers *pod-id*))))
-
-;; https://www.cognitect.com/blog/2015/9/10/extending-transit
-(defn add-transit-write-handler!
-  [classes tag fn]
-  (let [rh (transit/write-handler tag fn)]
-    (doseq [class classes]
-      (swap! transit-write-handlers assoc-in [*pod-id* class] rh)))
-  (update-transit-write-handler-map)
-  nil)
-
-(defonce transit-default-write-handlers (atom {}))
-
-(defn set-default-transit-write-handler! [tag-fn val-fn]
-  (let [wh (transit/write-handler tag-fn val-fn)]
-    (swap! transit-default-write-handlers assoc *pod-id* wh)))
-
-(defn transit-json-write
-  [pod-id ^String s metadata?]
-  (with-open [baos (java.io.ByteArrayOutputStream. 4096)]
-    (let [w (transit/writer baos :json (cond-> {:handlers (get @transit-write-handler-maps pod-id)
-                                                :default-handler (get @transit-default-write-handlers pod-id)}
-                                         metadata? (assoc :transform transit/write-meta)))]
-      (transit/write w s)
-      (str baos))))
-
 (defn invoke [pod pod-var args opts]
   (let [handlers (:handlers opts)
         stream (:stdin pod)
@@ -106,8 +114,7 @@
         chans (:chans pod)
         write-fn (case format
                    :edn pr-str
-                   :json cheshire/generate-string
+                   :json cheshire/generate-string)
-                   :transit+json #(transit-json-write (:pod-id pod) % (:arg-meta opts)))
         id (next-id)
         chan (if handlers handlers
                  (promise))
@@ -122,170 +129,28 @@
                     (throw v)
                     v)))))
 
-(defn bencode->vars [pod ns-name-str vars]
-  (mapv
-   (fn [var]
-     (let [name (get-string var "name")
-           async? (some-> (get var "async")
-                          bytes->string
-                          #(Boolean/parseBoolean %))
-           name-sym (symbol name)
-           sym (symbol ns-name-str name)
-           code (get-maybe-string var "code")
-           vmeta (some-> (get var "meta")
-                         bytes->string
-                         edn/read-string)
-           name-sym (if vmeta
-                      (with-meta name-sym vmeta)
-                      name-sym)
-           metadata? (get-maybe-boolean var "arg-meta")]
-       [name-sym
-        (or code
-            (fn [& args]
-              (let [res (invoke pod sym args {:async async? :arg-meta metadata?})]
-                res)))]))
-   vars))
-
-(defn processor [pod]
-  (let [stdout (:stdout pod)
-        format (:format pod)
-        chans (:chans pod)
-        out-stream (:out pod)
-        err-stream (:err pod)
-        readers (:readers pod)
-        read-fn (case format
-                  :edn (fn [s]
-                         (try (edn/read-string {:readers readers} s)
-                              (catch Exception e
-                                (binding [*out* *err*]
-                                  (println "Cannot read EDN: " (pr-str s))
-                                  (throw e)))))
-                  :json (fn [s]
-                          (try (cheshire/parse-string-strict s true)
-                               (catch Exception e
-                                 (binding [*out* *err*]
-                                   (println "Cannot read JSON: " (pr-str s))
-                                   (throw e)))))
-                  :transit+json
-                  (fn [s]
-                    (try (transit-json-read (:pod-id pod) s)
-                         (catch Exception e
-                           (binding [*out* *err*]
-                             (println "Cannot read Transit JSON: " (pr-str s))
-                             (throw e))))))]
-    (binding [*pod-id* (:pod-id pod)]
-      (try
-        (loop []
-          (let [reply (try (read stdout)
-                           (catch java.io.EOFException _
-                             ::EOF)
-                           (catch java.net.SocketException e
-                             (if (= "Socket closed" (ex-message e))
-                               ::EOF
-                               (throw e))))]
-            (when-not (identical? ::EOF reply)
-              (let [id (get reply "id")
-                    id    (bytes->string id)
-                    value* (find reply "value")
-                    [exception value] (try (some->> value*
-                                                    second
-                                                    bytes->string
-                                                    read-fn
-                                                    (vector nil))
-                                           (catch Exception e
-                                             [e nil]))
-                    status (get reply "status")
-                    status (set (map (comp keyword bytes->string) status))
-                    error? (or exception (contains? status :error))
-                    done? (or error? exception (contains? status :done))
-                    [ex-message ex-data]
-                    (when error?
-                      [(or (some-> (get reply "ex-message")
-                                   bytes->string)
-                           "")
-                       (or (some-> (get reply "ex-data")
-                                   bytes->string
-                                   read-fn)
-                           {})])
-                    namespace (when-let [v (get reply "vars")]
-                                (let [name-str (-> (get reply "name")
-                                                   bytes->string)
-                                      name (symbol name-str)]
-                                  {:name name
-                                   :vars (bencode->vars pod name-str v)}))
-                    chan (get @chans id)
-                    promise? (instance? clojure.lang.IPending chan)
-                    exception (or exception
-                                  (when (and promise? error?)
-                                    (ex-info ex-message ex-data)))
-                    ;; NOTE: if we need more fine-grained handlers, we will add
-                    ;; a :raw handler that will just get the bencode message's raw
-                    ;; data
-                    {error-handler :error
-                     done-handler :done
-                     success-handler :success} (when (map? chan)
-                                                 chan)
-                    out (some-> (get reply "out")
-                                bytes->string)
-                    err (some-> (get reply "err")
-                                bytes->string)]
-                ;; NOTE: write to out and err before delivering promise for making
-                ;; listening to output synchronous.
-                (when out
-                  (binding [*out* out-stream]
-                    (print out)
-                    (.flush ^java.io.Writer out-stream)))
-                (when err
-                  (binding [*out* err-stream]
-                    (print err)
-                    (.flush ^java.io.Writer err-stream)))
-                (when (or value* error? namespace)
-                  (cond promise?
-                        (deliver chan (cond error? exception
-                                            value value
-                                            namespace namespace))
-                        (and (not error?) success-handler)
-                        (success-handler value)
-                        (and error? error-handler)
-                        (error-handler {:ex-message ex-message
-                                        :ex-data ex-data})))
-                (when (and done? (not error?))
-                  (when promise?
-                    (deliver chan nil))
-                  (when done-handler
-                    (done-handler))))
-              (recur))))
-        (catch Exception e
-          (binding [*out* *err* #_err-stream]
-            (prn e)))))))
-
 (def pods (atom {}))
 
-(defn get-pod-id [x]
-  (if (map? x)
-    (:pod/id x)
-    x))
-
 (defn lookup-pod [pod-id]
   (get @pods pod-id))
 
-(defn destroy* [{:keys [:stdin :process :ops]}]
+(defn destroy [pod-id]
-  (if (contains? ops :shutdown)
+  (when-let [pod (lookup-pod pod-id)]
-    (do (write stdin
+    (if (contains? (:ops pod) :shutdown)
-               {"op" "shutdown"
+      (do (write (:stdin pod)
-                "id" (next-id)})
+                 {"op" "shutdown"
-        (.waitFor ^Process process))
+                  "id" (next-id)})
-    (.destroy ^Process process)))
+          (.waitFor ^Process (:process pod)))
+      (.destroy ^Process (:process pod)))
+    (when-let [rns (:remove-ns pod)]
+      (doseq [[ns-name _] (:namespaces pod)]
+        (rns ns-name)))))
 
-(defn destroy [pod-id-or-pod]
+(def next-pod-id
-  (let [pod-id (get-pod-id pod-id-or-pod)]
+  (let [counter (atom 0)]
-    (when-let [pod (lookup-pod pod-id)]
+    (fn []
-      (destroy* pod)
+      (let [[o _] (swap-vals! counter inc)]
-      (when-let [rns (:remove-ns pod)]
+        o))))
-        (doseq [[ns-name _] (:namespaces pod)]
-          (rns ns-name))))
-    (swap! pods dissoc pod-id)
-    nil))
 
 (def bytes->symbol
   (comp symbol bytes->string))
@@ -296,138 +161,25 @@
           dict-vals (map (comp resolve-fn bytes->symbol) (vals dict))]
       (zipmap dict-keys dict-vals))))
 
-(defn bencode->namespace [pod namespace]
-  (let [name-str (-> namespace (get "name") bytes->string)
-        name-sym (symbol name-str)
-        vars (get namespace "vars")
-        vars (bencode->vars pod name-str vars)
-        defer? (some-> namespace (get-maybe-string "defer") (= "true"))]
-    [name-sym vars defer?]))
-
-(defn create-socket
-  "Connect a socket to a remote host. The call blocks until
-   the socket is connected."
-  ^Socket
-  [^String hostname ^Integer port]
-  (doto (Socket. hostname port)
-    (.setTcpNoDelay true)))
-
-(defn close-socket
-  "Close the socket, and also closes its input and output streams."
-  [^Socket socket]
-  (try (.close socket)
-       nil
-       (catch java.net.SocketException _ nil)))
-
-(defn port-file [pid]
-  (doto (io/file (str ".babashka-pod-" pid ".port"))
-    (.deleteOnExit)))
-
-(defn read-port [^java.io.File port-file]
-  (loop []
-    (let [f port-file]
-      (if-let [s (when (.exists f)
-                   (let [s (slurp f)]
-                     (when (str/ends-with? s "\n")
-                       (str/trim s))))]
-        (Integer/parseInt s)
-        (recur)))))
-
-(defn debug [& strs]
-  (binding [*out* *err*]
-    (println (str/join " " (map pr-str strs)))))
-
-(defn resolve-pod [pod-spec {:keys [:version :path :force] :as opts}]
-  (when (qualified-symbol? pod-spec)
-    (when (and (not version) (not path))
-      (throw (IllegalArgumentException. "Version or path must be provided")))
-    (when (and version path)
-      (throw (IllegalArgumentException. "You must provide either version or path, not both"))))
-  (let [resolved (when (and (qualified-symbol? pod-spec) version)
-                   (resolver/resolve pod-spec version force))
-        opts (if resolved
-               (if-let [extra-opts (:options resolved)]
-                 (merge opts extra-opts)
-                 opts)
-               opts)
-        pod-spec (cond
-                   resolved [(:executable resolved)]
-                   path [path]
-                   (string? pod-spec) [pod-spec]
-                   :else pod-spec)]
-    {:pod-spec pod-spec, :opts opts}))
-
-(defn run-pod [pod-spec {:keys [:transport] :as _opts}]
-  (let [pb (ProcessBuilder. ^java.util.List pod-spec)
-        socket? (identical? :socket transport)
-        _ (if socket?
-            (.inheritIO pb)
-            (.redirectError pb java.lang.ProcessBuilder$Redirect/INHERIT))
-        _ (cond-> (doto (.environment pb)
-                    (.put "BABASHKA_POD" "true"))
-            socket? (.put "BABASHKA_POD_TRANSPORT" "socket"))
-        p (.start pb)
-        port-file (when socket? (port-file (.pid p)))
-        socket-port (when socket? (read-port port-file))
-        [socket stdin stdout]
-        (if socket?
-          (let [^Socket socket
-                (loop []
-                  (if-let [sock (try (create-socket "localhost" socket-port)
-                                     (catch java.net.ConnectException _
-                                       nil))]
-                    sock
-                    (recur)))]
-            [socket
-             (.getOutputStream socket)
-             (PushbackInputStream. (.getInputStream socket))])
-          [nil (.getOutputStream p) (java.io.PushbackInputStream. (.getInputStream p))])]
-    {:process p
-     :socket socket
-     :stdin stdin
-     :stdout stdout}))
-
-(defn describe-pod [{:keys [:stdin :stdout]}]
-  (write stdin {"op" "describe"
-                "id" (next-id)})
-  (read stdout))
-
-(defn describe->ops [describe-reply]
-  (some->> (get describe-reply "ops") keys (map keyword) set))
-
-(defn describe->metadata [describe-reply resolve-fn]
-  (let [format (-> (get describe-reply "format") bytes->string keyword)
-        ops (describe->ops describe-reply)
-        readers (when (identical? :edn format)
-                  (read-readers describe-reply resolve-fn))]
-    {:format format, :ops ops, :readers readers}))
-
-(defn run-pod-for-metadata [pod-spec opts]
-  (let [running-pod    (run-pod pod-spec opts)
-        describe-reply (describe-pod running-pod)
-        ops (describe->ops describe-reply)]
-    (destroy* (assoc running-pod :ops ops))
-    describe-reply))
-
-(defn load-pod-metadata [unresolved-pod-spec {:keys [:download-only] :as opts}]
-  (let [{:keys [:pod-spec :opts]} (resolve-pod unresolved-pod-spec opts)]
-    (if download-only
-      (resolver/warn "Not running pod" unresolved-pod-spec "to pre-cache metadata because OS and/or arch are different than system")
-      (run-pod-for-metadata pod-spec opts))))
-
 (defn load-pod
   ([pod-spec] (load-pod pod-spec nil))
-  ([pod-spec opts]
+  ([pod-spec {:keys [:remove-ns :resolve]}]
-   (let [{:keys [:pod-spec :opts]} (resolve-pod pod-spec opts)
+   (let [pod-spec (if (string? pod-spec) [pod-spec] pod-spec)
-         {:keys [:remove-ns :resolve]} opts
+         pb (ProcessBuilder. ^java.util.List pod-spec)
-
+         _ (.redirectError pb java.lang.ProcessBuilder$Redirect/INHERIT)
-         {p :process, stdin :stdin, stdout :stdout, socket :socket
+         _ (doto (.environment pb)
-          :as running-pod}
+             (.put "BABASHKA_POD" "true"))
-         (run-pod pod-spec opts)
+         p (.start pb)
-
+         stdin (.getOutputStream p)
-         reply (or (:metadata opts)
+         stdout (.getInputStream p)
-                   (describe-pod running-pod))
+         stdout (java.io.PushbackInputStream. stdout)
-         {:keys [:format :ops :readers]} (describe->metadata reply resolve)
+         _ (write stdin {"op" "describe"
+                         "id" (next-id)})
+         reply (read stdout)
+         format (-> (get reply "format") bytes->string keyword)
+         ops (some->> (get reply "ops") keys (map keyword) set)
+         readers (when (identical? :edn format)
+                   (read-readers reply resolve))
          pod {:process p
               :pod-spec pod-spec
               :stdin stdin
@@ -439,38 +191,42 @@
               :err *err*
               :remove-ns remove-ns
               :readers readers}
+         _ (add-shutdown-hook! #(destroy pod))
          pod-namespaces (get reply "namespaces")
          pod-id (or (when-let [ns (first pod-namespaces)]
                       (get-string ns "name"))
                     (next-id))
-         _ (add-shutdown-hook! #(do
-                                  (destroy pod-id)
-                                  (when socket
-                                    ;; this probably isn't necessary because we
-                                    ;; killed the process, but anyway
-                                    (close-socket socket))))
          pod (assoc pod :pod-id pod-id)
-         pod-namespaces (mapv #(bencode->namespace pod %)
+         vars-fn (fn [ns-name-str vars]
+                   (mapv
+                    (fn [var]
+                      (let [name (get-string var "name")
+                            async? (some-> (get var "async")
+                                           bytes->string
+                                           #(Boolean/parseBoolean %))
+                            name-sym (symbol name)
+                            sym (symbol ns-name-str name)
+                            code (get-maybe-string var "code")]
+                        [name-sym
+                         (or code
+                             (fn [& args]
+                               (let [res (invoke pod sym args {:async async?})]
+                                 res)))]))
+                    vars))
+         pod-namespaces (mapv (fn [namespace]
+                                (let [name-str (-> namespace (get "name") bytes->string)
+                                      name-sym (symbol name-str)
+                                      vars (get namespace "vars")
+                                      vars (vars-fn name-str vars)]
+                                  [name-sym vars]))
                               pod-namespaces)
          pod (assoc pod :namespaces pod-namespaces)]
      (swap! pods assoc pod-id pod)
      pod)))
 
-(defn load-ns [pod namespace]
-  (let [prom (promise)
-        chans (:chans pod)
-        id (next-id)
-        _ (swap! chans assoc id prom)]
-    (write (:stdin pod)
-           {"op" "load-ns"
-            "ns" (str namespace)
-            "id" id})
-    @prom))
-
 (defn invoke-public [pod-id fn-sym args opts]
-  (let [pod-id (get-pod-id pod-id)
+  (let [pod (lookup-pod pod-id)]
-        pod (lookup-pod pod-id)]
     (invoke pod fn-sym args opts)))
 
-(defn unload-pod [pod-id-or-pod]
+(defn unload-pod [pod-id]
-  (destroy pod-id-or-pod))
+  (destroy pod-id))

src/babashka/pods/impl/resolver.clj

@@ -1,225 +0,0 @@
-(ns babashka.pods.impl.resolver
-  {:no-doc true}
-  (:refer-clojure :exclude [resolve])
-  (:require [clojure.edn :as edn]
-            [clojure.java.io :as io]
-            [clojure.java.shell :refer [sh]]
-            [clojure.string :as str])
-  (:import [java.net URL HttpURLConnection]
-           [java.nio.file Files]))
-
-(set! *warn-on-reflection* true)
-
-(defn normalize-arch [arch]
-  (if (= "amd64" arch)
-    "x86_64"
-    arch))
-
-(defn normalize-os [os]
-  (-> os str/lower-case (str/replace #"\s+" "_")))
-
-(def os
-  (delay
-    {:os/name (or (System/getenv "BABASHKA_PODS_OS_NAME")
-                  (System/getProperty "os.name"))
-     :os/arch (let [arch (or (System/getenv "BABASHKA_PODS_OS_ARCH")
-                             (System/getProperty "os.arch"))]
-                (normalize-arch arch))}))
-
-(defn warn [& strs]
-  (binding [*out* *err*]
-    (apply println strs)))
-
-(defn match-artifacts
-  ([package] (match-artifacts package (:os/arch @os)))
-  ([package arch]
-   (let [artifacts (:pod/artifacts package)
-         res (filter (fn [{os-name :os/name
-                           os-arch :os/arch}]
-                       (let [os-arch (normalize-arch os-arch)]
-                         (and (re-matches (re-pattern os-name) (:os/name @os))
-                              (re-matches (re-pattern os-arch)
-                                          arch))))
-                     artifacts)]
-     (if (empty? res)
-       (if (and (= "Mac OS X" (:os/name @os))
-                (= "aarch64" (:os/arch @os)))
-         ;; Rosetta2 fallback on Apple M1 machines
-         (match-artifacts package "x86_64")
-         (throw (IllegalArgumentException. (format "No executable found for pod %s (%s) and OS %s/%s"
-                                                   (:pod/name package)
-                                                   (:pod/version package)
-                                                   (:os/name @os)
-                                                   (:os/arch @os)))))
-       res))))
-
-(defn unzip [{:keys [^java.io.File zip-file
-                     ^java.io.File destination-dir
-                     verbose]}]
-  (when verbose (warn "Unzipping" (.getPath zip-file) "to" (.getPath destination-dir)))
-  (let [output-path (.toPath destination-dir)
-        zip-file (io/file zip-file)
-        _ (.mkdirs (io/file destination-dir))]
-    (with-open
-      [fis (Files/newInputStream (.toPath zip-file) (into-array java.nio.file.OpenOption []))
-       zis (java.util.zip.ZipInputStream. fis)]
-      (loop []
-        (let [entry (.getNextEntry zis)]
-          (when entry
-            (let [entry-name (.getName entry)
-                  new-path (.resolve output-path entry-name)]
-              (if (.isDirectory entry)
-                (Files/createDirectories new-path ^"[Ljava.nio.file.attribute.FileAttribute;"
-                                         (into-array java.nio.file.attribute.FileAttribute []))
-                (Files/copy ^java.io.InputStream zis
-                            new-path
-                            ^"[Ljava.nio.file.CopyOption;"
-                            (into-array
-                             java.nio.file.CopyOption
-                             [java.nio.file.StandardCopyOption/REPLACE_EXISTING]))))
-            (recur)))))))
-
-(defn un-tgz [^java.io.File zip-file ^java.io.File destination-dir verbose?]
-  (when verbose? (warn "Unzipping" (.getPath zip-file) "to" (.getPath destination-dir)))
-  (let [tmp-file (java.io.File/createTempFile "glam" ".tar")
-        output-path (.toPath tmp-file)]
-    (with-open
-      [fis (Files/newInputStream (.toPath zip-file) (into-array java.nio.file.OpenOption []))
-       zis (java.util.zip.GZIPInputStream. fis)]
-      (Files/copy ^java.io.InputStream zis
-                  output-path
-                  ^"[Ljava.nio.file.CopyOption;"
-                  (into-array
-                   [java.nio.file.StandardCopyOption/REPLACE_EXISTING])))
-    (.mkdirs destination-dir)
-    (let [res (sh "tar" "xf" (.getPath tmp-file) "--directory" (.getPath destination-dir))]
-      (when-not (zero? (:exit res))
-        (throw (ex-info (:err res) res))))
-    (.delete tmp-file)))
-
-(defn make-executable [dest-dir executables verbose?]
-  (doseq [e executables]
-    (let [f (io/file dest-dir e)]
-      (when verbose? (warn "Making" (.getPath f) "executable."))
-      (.setExecutable f true))))
-
-(defn download [source ^java.io.File dest verbose?]
-  (when verbose? (warn "Downloading" source "to" (.getPath dest)))
-  (let [source (URL. source)
-        dest (io/file dest)
-        conn ^HttpURLConnection (.openConnection ^URL source)]
-    (.setInstanceFollowRedirects conn true)
-    (.connect conn)
-    (io/make-parents dest)
-    (with-open [is (.getInputStream conn)]
-      (io/copy is dest))))
-
-(defn repo-dir []
-  (io/file (if-let [pods-dir (System/getenv "BABASHKA_PODS_DIR")]
-             (io/file pods-dir)
-             (io/file (or
-                       (System/getenv "XDG_DATA_HOME")
-                       (System/getProperty "user.home"))
-                      ".babashka"
-                      "pods"))
-           "repository"))
-
-(def pods-repo-dir
-  ;; wrapped in delay for GraalVM native-image
-  (delay
-   (repo-dir)))
-
-(defn github-url [qsym version]
-  (format
-   "https://raw.githubusercontent.com/babashka/pod-registry/master/manifests/%s/%s/manifest.edn"
-   qsym version))
-
-(defn pod-manifest
-  [qsym version force?]
-  (let [f (io/file @pods-repo-dir (str qsym) (str version) "manifest.edn")]
-    (if (and (not force?)
-             (.exists f))
-      (edn/read-string (slurp f))
-      (do (download (github-url qsym version) f false)
-          (edn/read-string (slurp f))))))
-
-(defn cache-dir
-  ^java.io.File
-  [{pod-name :pod/name
-    pod-version :pod/version}]
-  (let [base-file
-        (if-let [pods-dir (System/getenv "BABASHKA_PODS_DIR")]
-          (io/file pods-dir)
-          (io/file (or
-                    (System/getenv "XDG_CACHE_HOME")
-                    (System/getProperty "user.home"))
-                   ".babashka"
-                   "pods"))]
-    (io/file base-file
-             "repository"
-             (str pod-name)
-             pod-version
-             (normalize-os (:os/name @os))
-             (:os/arch @os))))
-
-(defn data-dir
-  ^java.io.File
-  [{pod-name :pod/name
-    pod-version :pod/version}]
-  (io/file @pods-repo-dir
-           (str pod-name)
-           pod-version
-           (normalize-os (:os/name @os))
-           (:os/arch @os)))
-
-(defn sha256 [file]
-  (let [buf (byte-array 8192)
-        digest (java.security.MessageDigest/getInstance "SHA-256")]
-    (with-open [bis (io/input-stream (io/file file))]
-      (loop []
-        (let [count (.read bis buf)]
-          (when (pos? count)
-            (.update digest buf 0 count)
-            (recur)))))
-    (-> (.encode (java.util.Base64/getEncoder)
-                 (.digest digest))
-        (String. "UTF-8"))))
-
-(defn resolve [qsym version force?]
-  (when-not (string? version)
-    (throw (IllegalArgumentException. "Version must be provided for resolving from pod registry!")))
-  (when-let [manifest (pod-manifest qsym version force?)]
-    (let [artifacts (match-artifacts manifest)
-          cdir (cache-dir manifest)
-          ddir (data-dir manifest)
-          execs (mapv (fn [artifact]
-                        (let [url (:artifact/url artifact)
-                              file-name (last (str/split url #"/"))
-                              cache-file (io/file cdir file-name)
-                              executable (io/file ddir (:artifact/executable artifact))]
-                          (when (or force? (not (.exists executable)))
-                            (warn (format "Downloading pod %s (%s)" qsym version))
-                            (download url cache-file false)
-                            (when-let [expected-sha (:artifact/hash artifact)]
-                              (let [sha (sha256 cache-file)]
-                                (when-not (= (str/replace expected-sha #"^sha256:" "")
-                                             sha)
-                                  (throw (ex-info (str "Wrong SHA-256 for file" (str cache-file))
-                                                  {:sha sha
-                                                   :expected-sha expected-sha})))))
-                            (let [filename (.getName cache-file)]
-                              (cond (str/ends-with? filename ".zip")
-                                    (unzip {:zip-file cache-file
-                                            :destination-dir ddir
-                                            :verbose false})
-                                    (or (str/ends-with? filename ".tgz")
-                                        (str/ends-with? filename ".tar.gz"))
-                                    (un-tgz cache-file ddir
-                                            false))
-                              (.delete cache-file))
-                            (make-executable ddir [(:artifact/executable artifact)] false)
-                            (warn (format "Successfully installed pod %s (%s)" qsym version))
-                            (io/file ddir (:artifact/executable artifact)))
-                          (io/file ddir (:artifact/executable artifact)))) artifacts)]
-      {:executable (.getAbsolutePath ^java.io.File (first execs))
-       :options (:pod/options manifest)})))

src/babashka/pods/jvm.clj

@@ -1,65 +1,28 @@
 (ns babashka.pods.jvm
   (:require [babashka.pods.impl :as impl]))
 
-(def ^:private namespaces-to-load (atom {}))
-
-(defn- unroot-resource [^String path]
-  (symbol (.. path
-              (substring 1)
-              (replace \/ \.)
-              (replace \_ \-))))
-
-(defn- process-namespace [{:keys [:name :vars]}]
-  (binding [*ns* (load-string (format "(ns %s) *ns*" name))]
-    (doseq [[var-sym v] vars]
-      (when-let [maybe-core (some-> (ns-resolve *ns* var-sym) meta :ns str symbol)]
-        (when (= 'clojure.core maybe-core)
-          (ns-unmap *ns* var-sym)))
-      (cond
-        (ifn? v)
-        (intern name var-sym v)
-        (string? v)
-        (load-string v)))))
-
-(let [core-load clojure.core/load]
-  (intern 'clojure.core 'load
-          (fn [& paths]
-            (let [nss @namespaces-to-load]
-              (doseq [path paths]
-                (let [lib (unroot-resource path)]
-                  (if-let [pod (get nss lib)]
-                    (let [ns (impl/load-ns pod lib)]
-                      (process-namespace ns))
-                    (core-load path))))))))
-
 (defn load-pod
   ([pod-spec] (load-pod pod-spec nil))
-  ([pod-spec version opts] (load-pod pod-spec (assoc opts :version version)))
+  ([pod-spec _opts]
-  ([pod-spec opts]
+   (let [pod (impl/load-pod
-   (let [opts (if (string? opts)
-                {:version opts}
-                opts)
-         pod (impl/load-pod
               pod-spec
-              (merge {:remove-ns remove-ns
+              {:remove-ns remove-ns
-                      :resolve (fn [sym]
+               :resolve (fn [sym]
-                                 (or (resolve sym)
+                          (or (resolve sym)
-                                     (intern
+                              (intern
-                                      (create-ns (symbol (namespace sym)))
+                               (create-ns (symbol (namespace sym)))
-                                      (symbol (name sym)))))}
+                               (symbol (name sym)))))})
-                     opts))
          namespaces (:namespaces pod)]
-     (swap! namespaces-to-load
+     (doseq [[ns-sym v] namespaces]
-            merge
+       (binding [*ns* (load-string (format "(ns %s) *ns*" ns-sym))]
-            (into {}
+         (doseq [[var-sym v] v]
-                  (keep (fn [[ns-name _ defer?]]
+           (cond
-                          (when defer?
+             (ifn? v)
-                            [ns-name pod]))
+             (do
-                        namespaces)))
+               (ns-unmap *ns* var-sym)
-     (binding [impl/*pod-id* (:pod-id pod)]
+               (intern ns-sym var-sym v))
-       (doseq [[ns-sym vars lazy?] namespaces
+             (string? v)
-               :when (not lazy?)]
+             (load-string v)))))
-         (process-namespace {:name ns-sym :vars vars})))
      (future (impl/processor pod))
      {:pod/id (:pod-id pod)})))
 
@@ -71,18 +34,3 @@
 (defn invoke
   ([pod-id sym args] (invoke pod-id sym args {}))
   ([pod-id sym args opts] (impl/invoke-public pod-id sym args opts)))
-
-(defmacro copy-var [name var]
-  `(do (def ~name ~var)
-       (let [m# (meta (var ~var))
-             doc# (:doc m#)
-             arglists# (:arglists m#)]
-         (alter-meta! (var ~name) assoc
-                      :arglists arglists#
-                      :doc doc#))))
-
-#_:clj-kondo/ignore
-(do
-  (copy-var add-transit-read-handler! impl/add-transit-read-handler!)
-  (copy-var add-transit-write-handler! impl/add-transit-write-handler!)
-  (copy-var set-default-transit-write-handler! impl/set-default-transit-write-handler!))

src/babashka/pods/sci.clj

@@ -1,126 +1,57 @@
 (ns babashka.pods.sci
   (:require [babashka.pods.impl :as impl]
-            [sci.core :as sci]
+            [sci.core :as sci]))
-            [clojure.java.io :as io]
-            [babashka.pods.impl.resolver :as resolver]
-            [babashka.fs :as fs])
-  (:import (java.io PushbackInputStream File)))
 
-(set! *warn-on-reflection* true)
+(def load-pod
-
+  (with-meta
-(defn- process-namespace [ctx {:keys [:name :vars]}]
+    (fn
-  (let [env (:env ctx)
+      ([ctx pod-spec] (load-pod ctx pod-spec nil))
-        ns-name name
+      ([ctx pod-spec _opts]
-        sci-ns (sci/create-ns (symbol ns-name))]
+       (let [ns-load-fns (atom {})
-    (sci/binding [sci/ns sci-ns]
+             load-fn (fn load-fn [{:keys [:namespace]}]
-      ;; ensure ns map in ctx, see #20
+                       (when-let [f (get @ns-load-fns namespace)]
-      (swap! env update-in [:namespaces ns-name]
+                         (f)
-             (fn [ns-map]
+                         ;; return empty source, for sci to evaluate
-               (if ns-map ns-map {:obj sci-ns})))
+                         ""))
-      (doseq [[var-name var-value :as var] vars]
+             env (:env ctx)
-        (cond (ifn? var-value)
-              (swap! env assoc-in [:namespaces ns-name var-name]
-                     (sci/new-var
-                      (symbol (str ns-name) (str var-name)) var-value (meta var-name)))
-              (string? var-value)
-              (sci/eval-string* ctx var-value))))))
-
-(defn metadata-cache-file ^File [^File bb-edn-file pod-spec {:keys [:version :path]}]
-  (if version
-    (io/file (resolver/cache-dir {:pod/name pod-spec :pod/version version})
-             "metadata.cache")
-    (let [config-dir (.getParentFile bb-edn-file)
-          cache-dir (io/file config-dir ".babashka")
-          pod-file (-> path io/file .getName)
-          cache-file (io/file cache-dir (str pod-file ".metadata.cache"))]
-      cache-file)))
-
-(defn load-metadata-from-cache [bb-edn-file pod-spec opts]
-  (let [cache-file (metadata-cache-file bb-edn-file pod-spec opts)]
-    (when (.exists cache-file)
-      (with-open [r (PushbackInputStream. (io/input-stream cache-file))]
-        (impl/read r)))))
-
-(defn load-pod-metadata* [bb-edn-file pod-spec {:keys [:version :cache] :as opts}]
-  (let [metadata (impl/load-pod-metadata pod-spec opts)
-        cache-file (when (and metadata cache)
-                     (metadata-cache-file bb-edn-file pod-spec opts))]
-    (when cache-file
-      (io/make-parents cache-file)
-      (when (fs/writable? (fs/parent cache-file))
-        (with-open [w (io/output-stream cache-file)]
-          (impl/write w metadata))))
-    metadata))
-
-(defn load-pod-metadata
-  ([pod-spec opts] (load-pod-metadata nil pod-spec opts))
-  ([bb-edn-file pod-spec {:keys [:cache] :as opts}]
-   (let [metadata
-         (if-let [cached-metadata (when cache
-                                    (load-metadata-from-cache bb-edn-file
-                                                              pod-spec
-                                                              opts))]
-           cached-metadata
-           (load-pod-metadata* bb-edn-file pod-spec opts))]
-     (reduce
-       (fn [pod-namespaces ns]
-         (let [ns-sym (-> ns (get "name") impl/bytes->string symbol)]
-           (assoc pod-namespaces ns-sym {:pod-spec pod-spec
-                                         :opts (assoc opts :metadata metadata)})))
-       {} (get metadata "namespaces")))))
-
-(defn load-pod
-  ([ctx pod-spec] (load-pod ctx pod-spec nil))
-  ([ctx pod-spec version opts] (load-pod ctx pod-spec (assoc opts :version version)))
-  ([ctx pod-spec opts]
-   (let [opts (if (string? opts)
-                {:version opts}
-                opts)
-         env (:env ctx)
-         pod (binding [*out* @sci/out
-                       *err* @sci/err]
-               (impl/load-pod
-                pod-spec
-                (merge
-                 {:remove-ns
-                  (fn [sym]
-                    (swap! env update :namespaces dissoc sym))
-                  :resolve
-                  (fn [sym]
-                    (let [sym-ns (or (some-> (namespace sym)
-                                             symbol)
-                                     'clojure.core)
-                          sym-name (symbol (name sym))]
-                      (or (get-in @env [:namespaces sym-ns sym-name])
-                          (let [v (sci/new-var sym {:predefined true})]
-                            (swap! env assoc-in [:namespaces sym-ns sym-name]
-                                   v)
-                            v))))}
-                 opts)))
-         namespaces (:namespaces pod)
-         namespaces-to-load (set (keep (fn [[ns-name _ defer?]]
-                                         (when defer?
-                                           ns-name))
-                                       namespaces))]
-     (when (seq namespaces-to-load)
-       (let [load-fn (fn load-fn [{:keys [:namespace]}]
-                       (when (contains? namespaces-to-load namespace)
-                         (let [ns (impl/load-ns pod namespace)]
-                           (process-namespace ctx ns))
-                         {:file nil
-                          :source ""}))
              prev-load-fn (:load-fn @env)
              new-load-fn (fn [m]
                            (or (load-fn m)
                                (when prev-load-fn
-                                 (prev-load-fn m))))]
+                                 (prev-load-fn m))))
-         (swap! env assoc :load-fn new-load-fn)))
+             _ (swap! env assoc :load-fn new-load-fn)
-     (binding [impl/*pod-id* (:pod-id pod)]
+             pod (binding [*out* @sci/out
-       (doseq [[ns-name vars lazy?] namespaces
+                           *err* @sci/err]
-               :when (not lazy?)]
+                   (impl/load-pod
-         (process-namespace ctx {:name ns-name :vars vars})))
+                    pod-spec
-     (sci/future (impl/processor pod))
+                    {:remove-ns
-     {:pod/id (:pod-id pod)})))
+                     (fn [sym]
+                       (swap! env update :namespaces dissoc sym))
+                     :resolve
+                     (fn [sym]
+                       (let [sym-ns (or (some-> (namespace sym)
+                                                symbol)
+                                        'clojure.core)
+                             sym-name (symbol (name sym))]
+                         (or (get-in @env [:namespaces sym-ns sym-name])
+                             (let [v (sci/new-var sym {:predefined true})]
+                               (swap! env assoc-in [:namespaces sym-ns sym-name]
+                                      v)
+                               v))))}))
+             namespaces (:namespaces pod)]
+         (doseq [[ns-name vars] namespaces]
+           (swap! ns-load-fns assoc ns-name
+                  #(sci/binding [sci/ns (sci/create-ns ns-name)]
+                     (doseq [[var-name var-value] vars]
+                       (cond (ifn? var-value)
+                             (swap! env assoc-in [:namespaces ns-name var-name]
+                                    (sci/new-var
+                                     (symbol (str ns-name) (str var-name)) var-value))
+                             (string? var-value)
+                             (sci/eval-string* ctx var-value))))))
+         (sci/future (impl/processor pod))
+         {:pod/id (:pod-id pod)})))
+    {:sci.impl/op :needs-ctx}))
 
 (defn unload-pod
   ([pod-id] (unload-pod pod-id {}))
@@ -130,18 +61,3 @@
 (defn invoke
   ([pod-id sym args] (invoke pod-id sym args {}))
   ([pod-id sym args opts] (impl/invoke-public pod-id sym args opts)))
-
-(defmacro copy-var [name var]
-  `(do (def ~name ~var)
-       (let [m# (meta (var ~var))
-             doc# (:doc m#)
-             arglists# (:arglists m#)]
-         (alter-meta! (var ~name) assoc
-                      :arglists arglists#
-                      :doc doc#))))
-
-#_:clj-kondo/ignore
-(do
-  (copy-var add-transit-read-handler! impl/add-transit-read-handler!)
-  (copy-var add-transit-write-handler! impl/add-transit-write-handler!)
-  (copy-var set-default-transit-write-handler! impl/set-default-transit-write-handler!))

test-pod/pod/test_pod.clj

@@ -3,10 +3,8 @@
   (:require [bencode.core :as bencode]
             [cheshire.core :as cheshire]
             [clojure.edn :as edn]
-            [clojure.java.io :as io]
+            [clojure.java.io :as io])
-            [cognitect.transit :as transit])
+  (:import [java.io PushbackInputStream])
-  (:import [java.io PushbackInputStream]
-           [java.net ServerSocket])
   (:gen-class))
 
 (def debug? false)
@@ -16,15 +14,17 @@
     (binding [*out* (io/writer "/tmp/log.txt" :append true)]
       (apply println args))))
 
-(defn write [stream v]
+(def stdin (PushbackInputStream. System/in))
-  (bencode/write-bencode stream v)
+
-  (flush))
+(defn write [v]
+  (bencode/write-bencode System/out v)
+  (.flush System/out))
 
 (defn read-string [^"[B" v]
   (String. v))
 
-(defn read [stream]
+(defn read []
-  (bencode/read-bencode stream))
+  (bencode/read-bencode stdin))
 
 (def dependents
   (for [i (range 10)]
@@ -34,70 +34,19 @@
        (format "(def x%s (inc x%s))" i (dec i))
        "(def x0 0)")}))
 
-(defn transit-json-read [^String s]
-  (with-open [bais (java.io.ByteArrayInputStream. (.getBytes s "UTF-8"))]
-    (let [r (transit/reader bais :json {:handlers
-                                        {"local-date-time"
-                                         (transit/read-handler
-                                          (fn [s]
-                                            (java.time.LocalDateTime/parse s)))
-                                         "java.array"
-                                         (transit/read-handler
-                                          (fn [v]
-                                            (into-array v)))}})]
-      (transit/read r))))
-
-(defn transit-json-write [s]
-  (with-open [baos (java.io.ByteArrayOutputStream. 4096)]
-    (let [w (transit/writer baos :json {:handlers
-                                        {java.time.LocalDateTime
-                                         (transit/write-handler
-                                          "local-date-time"
-                                          str)}
-                                        :default-handler
-                                        (transit/write-handler
-                                         (fn [x] (when (.isArray (class x)) "java.array"))
-                                         vec)})]
-      (transit/write w s)
-      (str baos))))
-
-(defn transit-json-write-meta [s]
-  (with-open [baos (java.io.ByteArrayOutputStream. 4096)]
-    (let [w (transit/writer baos :json {:transform transit/write-meta})]
-      (transit/write w s)
-      (str baos))))
-
 (defn run-pod [cli-args]
-  (let [format (cond (contains? cli-args "--json") :json
+  (let [format (if (contains? cli-args "--json")
-                     (contains? cli-args "--transit+json") :transit+json
+                 :json
-                     :else :edn)
+                 :edn)
-        write-fn (case format
+        write-fn (if (identical? :json format)
-                   :edn pr-str
+                   cheshire/generate-string
-                   :json cheshire/generate-string
+                   pr-str)
-                   :transit+json transit-json-write)
+        read-fn (if (identical? :json format)
-        read-fn (case format
+                  #(cheshire/parse-string % true)
-                  :edn edn/read-string
+                  edn/read-string)]
-                  :json #(cheshire/parse-string % true)
-                  :transit+json transit-json-read)
-        socket (= "socket" (System/getenv "BABASHKA_POD_TRANSPORT"))
-        [in out] (if socket
-                   (let [server (ServerSocket. 0)
-                         port (.getLocalPort server)
-                         pid (.pid (java.lang.ProcessHandle/current))
-                         port-file (io/file (str ".babashka-pod-" pid ".port"))
-                         _ (.addShutdownHook (Runtime/getRuntime)
-                                             (Thread. (fn [] (.delete port-file))))
-                         _ (spit port-file
-                                 (str port "\n"))
-                         socket (.accept server)
-                         in (PushbackInputStream. (.getInputStream socket))
-                         out (.getOutputStream socket)]
-                     [in out])
-                   [(PushbackInputStream. System/in)
-                    System/out])]
     (try
       (loop []
-        (let [message (try (read in)
+        (let [message (try (read)
                            (catch java.io.EOFException _
                              ::EOF))]
           (when-not (identical? ::EOF message)
@@ -106,72 +55,33 @@
                   op (keyword op)]
               (case op
                 :describe
-                (do (write out {"format" (case format
+                (do (write {"format" (if (= format :json)
-                                           :edn "edn"
+                                       "json"
-                                           :json "json"
+                                       "edn")
-                                           :transit+json "transit+json")
+                            "readers" {"my/tag" "identity"
-                                "readers" {"my/tag" "identity"
+                                       ;; NOTE: this function is defined later,
-                                           ;; NOTE: this function is defined later,
+                                       ;; which should be supported
-                                           ;; which should be supported
+                                       "my/other-tag" "pod.test-pod/read-other-tag"}
-                                           "my/other-tag" "pod.test-pod/read-other-tag"}
+                            "namespaces"
-                                "namespaces"
+                            [{"name" "pod.test-pod"
-                                [{"name" "pod.test-pod"
+                              "vars" (into [{"name" "add-sync"}
-                                  "vars" (into [{"name" "add-sync"
+                                            {"name" "range-stream"
-                                                 "meta" "{:doc \"add the arguments\"}"}
+                                             "async" "true"}
-                                                {"name" "range-stream"
+                                            {"name" "assoc"}
-                                                 "async" "true"}
+                                            {"name" "error"}
-                                                {"name" "assoc"}
+                                            {"name" "print"}
-                                                {"name" "error"}
+                                            {"name" "print-err"}
-                                                {"name" "print"}
+                                            {"name" "return-nil"}
-                                                {"name" "print-err"}
+                                            {"name" "do-twice"
-                                                {"name" "return-nil"}
+                                             "code" "(defmacro do-twice [x] `(do ~x ~x))"}
-                                                {"name" "do-twice"
+                                            {"name" "reader-tag"}
-                                                 "code" "(defmacro do-twice [x] `(do ~x ~x))"}
+                                            ;; returns thing with other tag
-                                                {"name" "fn-call"
+                                            {"name" "other-tag"}
-                                                 "code" "(defn fn-call [f x] (f x))"}
+                                            ;; reads thing with other tag
-                                                {"name" "reader-tag"}
+                                            {"name" "read-other-tag"
-                                                ;; returns thing with other tag
+                                             "code" "(defn read-other-tag [x] [x x])"}]
-                                                {"name" "other-tag"}
+                                           dependents)}]
-                                                ;; reads thing with other tag
+                            "ops" {"shutdown" {}}})
-                                                {"name" "read-other-tag"
-                                                 "code" "(defn read-other-tag [x] [x x])"
-                                                 "meta" "{:doc \"unread\"}"}
-                                                {"name" "round-trip-meta"
-                                                 "arg-meta" "true"}
-                                                {"name" "dont-round-trip-meta"
-                                                 "arg-meta" "false"}
-                                                {"name" "-local-date-time"}
-                                                {"name" "transit-stuff"
-                                                 "code" "
-(babashka.pods/add-transit-read-handler! \"local-date-time\"
-  (fn [s] (java.time.LocalDateTime/parse s)))
-
-(babashka.pods/add-transit-write-handler! #{java.time.LocalDateTime}
-  \"local-date-time\"
-  str )
-
-(defn local-date-time [x]
-  (-local-date-time x))
-
-;; serialize Java arrays as vectors with tag java.array
-(babashka.pods/set-default-transit-write-handler!
-  (fn [x] (when (.isArray (class x)) \"java.array\"))
-  vec)
-
-(babashka.pods/add-transit-read-handler! \"java.array\"
-  into-array)
-
-"}
-                                                {"name" "incorrect-edn"}]
-                                               dependents)}
-                                 {"name" "pod.test-pod.loaded"
-                                  "defer" "true"}
-                                 {"name" "pod.test-pod.loaded2"
-                                  "defer" "true"}
-                                 {"name" "pod.test-pod.only-code"
-                                  "vars" [{"name" "foo"
-                                           "code" "(defn foo [] 1)"}]}]
-                                "ops" {"shutdown" {}}})
                     (recur))
                 :invoke (let [var (-> (get message "var")
                                       read-string
@@ -185,123 +95,72 @@
                           (case var
                             pod.test-pod/add-sync
                             (try (let [ret (apply + args)]
-                                   (write out
+                                   (write
-                                          {"value" (write-fn ret)
+                                    {"value" (write-fn ret)
-                                           "id" id
+                                     "id" id
-                                           "status" ["done"]}))
+                                     "status" ["done"]}))
                                  (catch Exception e
-                                   (write out
+                                   (write
-                                          {"ex-data" (write-fn {:args args})
+                                    {"ex-data" (write-fn {:args args})
-                                           "ex-message" (.getMessage e)
+                                     "ex-message" (.getMessage e)
-                                           "status" ["done" "error"]
+                                     "status" ["done" "error"]
-                                           "id" id})))
+                                     "id" id})))
                             pod.test-pod/range-stream
                             (let [rng (apply range args)]
                               (doseq [v rng]
-                                (write out
+                                (write
-                                       {"value" (write-fn v)
+                                 {"value" (write-fn v)
-                                        "id" id})
+                                  "id" id})
                                 (Thread/sleep 100))
-                              (write out
+                              (write
-                                     {"status" ["done"]
+                               {"status" ["done"]
-                                      "id" id}))
+                                "id" id}))
                             pod.test-pod/assoc
-                            (write out
+                            (write
-                                   {"value" (write-fn (apply assoc args))
+                             {"value" (write-fn (apply assoc args))
-                                    "status" ["done"]
+                              "status" ["done"]
-                                    "id" id})
+                              "id" id})
                             pod.test-pod/error
-                            (write out
+                            (write
-                                   {"ex-data" (write-fn {:args args})
+                             {"ex-data" (write-fn {:args args})
-                                    "ex-message" (str "Illegal arguments")
+                              "ex-message" (str "Illegal arguments")
-                                    "status" ["done" "error"]
+                              "status" ["done" "error"]
-                                    "id" id})
+                              "id" id})
                             pod.test-pod/print
-                            (do (write out
+                            (do (write
-                                       {"out" (with-out-str (prn args))
+                                 {"out" (pr-str args)
-                                        "id" id})
+                                  "id" id})
-                                (write out
+                                (write
-                                       {"status" ["done"]
+                                 {"status" ["done"]
-                                        "id" id}))
+                                  "id" id}))
                             pod.test-pod/print-err
-                            (do (write out
+                            (do (write
-                                       {"err" (with-out-str (prn args))
+                                 {"err" (pr-str args)
-                                        "id" id})
+                                  "id" id})
-                                (write out
+                                (write
-                                       {"status" ["done"]
+                                 {"status" ["done"]
-                                        "id" id}))
+                                  "id" id}))
                             pod.test-pod/return-nil
-                            (write out
+                            (write
-                                   {"status" ["done"]
+                             {"status" ["done"]
-                                    "id" id
+                              "id" id
-                                    "value" (write-fn nil)})
+                              "value" "nil"})
                             pod.test-pod/reader-tag
-                            (write out
+                            (write
-                                   {"status" ["done"]
+                             {"status" ["done"]
-                                    "id" id
+                              "id" id
-                                    "value" "#my/tag[1 2 3]"})
+                              "value" "#my/tag[1 2 3]"})
                             pod.test-pod/other-tag
-                            (write out
+                            (write
-                                   {"status" ["done"]
+                             {"status" ["done"]
-                                    "id" id
+                              "id" id
-                                    "value" "#my/other-tag[1]"})
+                              "value" "#my/other-tag[1]"}))
-                            pod.test-pod/round-trip-meta
-                            (write out
-                                   {"status" ["done"]
-                                    "id" id
-                                    "value"
-                                    (case format
-                                      :transit+json (transit-json-write-meta (first args))
-                                      (write-fn (first args)))})
-                            pod.test-pod/dont-round-trip-meta
-                            (write out
-                                   {"status" ["done"]
-                                    "id" id
-                                    "value"
-                                    (case format
-                                      :transit+json (transit-json-write-meta (first args))
-                                      (write-fn (first args)))})
-                            pod.test-pod/-local-date-time
-                            (write out
-                                   {"status" ["done"]
-                                    "id" id
-                                    "value" (write-fn (first args))})
-                            pod.test-pod/incorrect-edn
-                            (write out
-                                   {"status" ["done"]
-                                    "id" id
-                                    "value" (write-fn {(keyword "foo bar") 1})}))
                           (recur))
-                :shutdown (System/exit 0)
+                :shutdown (System/exit 0))))))
-                :load-ns (let [ns (-> (get message "ns")
-                                      read-string
-                                      symbol)
-                               id (-> (get message "id")
-                                      read-string)]
-                           (case ns
-                             pod.test-pod.loaded
-                             (write out
-                                    {"status" ["done"]
-                                     "id" id
-                                     "name" "pod.test-pod.loaded"
-                                     "vars" [{"name" "loaded"
-                                              "code" "(defn loaded [x] (inc x))"}]})
-                             pod.test-pod.loaded2
-                             (write out
-                                    {"status" ["done"]
-                                     "id" id
-                                     "name" "pod.test-pod.loaded2"
-                                     "vars" [{"name" "x"
-                                              "code" "(require '[pod.test-pod.loaded :as loaded])"}
-                                             {"name" "loaded"
-                                              "code" "(defn loaded [x] (loaded/loaded x))"}]}))
-                           (recur)))))))
       (catch Exception e
         (binding [*out* *err*]
           (prn e))))))
 
 (defn -main [& args]
-  #_(binding [*out* *err*]
-      (prn :args args))
   (when (= "true" (System/getenv "BABASHKA_POD"))
     (run-pod (set args))))

test-resources/pod_registry.clj

@@ -1,11 +0,0 @@
-(require '[babashka.pods :as pods])
-
-(pods/load-pod 'org.babashka/buddy "0.1.0")
-
-(require '[pod.babashka.buddy.codecs :as codecs]
-         '[pod.babashka.buddy.hash :as hash])
-
-(println (-> (hash/sha256 "foobar")
-             (codecs/bytes->hex)))
-
-(pods/load-pod 'org.babashka/etaoin) ;; should cause error when version & path are missing

test-resources/test_program.clj

@@ -1,19 +1,5 @@
 (require '[babashka.pods :as pods])
-
+(def pod-id (:pod/id (pods/load-pod ["clojure" "-A:test-pod"])))
-(def fmt (or (System/getenv "BABASHKA_POD_TEST_FORMAT")
-             "edn"))
-
-(def socket (System/getenv "BABASHKA_POD_TEST_SOCKET"))
-
-(def cmd (cond-> ["clojure" "-M:test-pod"]
-           (= "json" fmt) (conj "--json")
-           (= "transit+json" fmt) (conj "--transit+json")))
-
-;; (.println System/err cmd)
-
-(def pod-id (:pod/id (pods/load-pod cmd
-                                    {:socket (boolean socket)})))
-
 (require '[pod.test-pod :as pod])
 (def pod-ns-name (ns-name (find-ns 'pod.test-pod)))
 
@@ -55,59 +41,10 @@
 (def add-result (pod.test-pod/add-sync 1 2 3))
 (def nil-result (pod.test-pod/return-nil))
 
-(def add-sync-meta (:doc (meta #'pod.test-pod/add-sync)))
-(def error-meta (:doc (meta #'pod.test-pod/error)))
-(def read-other-tag-meta (:doc (meta #'pod.test-pod/read-other-tag)))
-
 (def x9 pod.test-pod/x9)
 
-(def tagged (if (= "edn" fmt)
+(def tagged (pod/reader-tag))
-              (pod/reader-tag)
+(def other-tagged (pod/other-tag))
-              [1 2 3]))
-
-(def other-tagged
-  (if (= "edn" fmt)
-    (pod/other-tag)
-    [[1] [1]]))
-
-(def fn-called (pod.test-pod/fn-call inc 2))
-
-;; (.println System/err (str :fmt " " fmt))
-(def local-date-time
-  (if (= "transit+json" fmt)
-    (instance? java.time.LocalDateTime (pod.test-pod/local-date-time (java.time.LocalDateTime/now)))
-    true))
-
-(def assoc-string-array
-  (if (= "transit+json" fmt)
-    (let [v (:a (pod.test-pod/assoc {} :a (into-array String ["foo"])))]
-      (.isArray (class v)))
-    true))
-
-(def round-trip-meta
-  (if (= "transit+json" fmt)
-    (= {:my-meta 2} (meta (pod.test-pod/round-trip-meta (with-meta [2] {:my-meta 2}))))
-    true))
-
-(def round-trip-meta-nested
-  (if (= "transit+json" fmt)
-    (= {:my-meta 3} (meta (first (pod.test-pod/round-trip-meta [(with-meta [3] {:my-meta 3})]))))
-    true))
-
-(def dont-round-trip-meta
-  (if (= "transit+json" fmt)
-    (= nil (meta (pod.test-pod/dont-round-trip-meta (with-meta [2] {:my-meta 2}))))
-    true))
-
-(require '[pod.test-pod.only-code :as only-code])
-(def should-be-1 (only-code/foo))
-
-(require '[pod.test-pod.loaded2 :as loaded2])
-(def loaded (loaded2/loaded 1))
-
-(def incorrect-edn-response
-  (try (pod.test-pod/incorrect-edn)
-       (catch Exception e (ex-message e))))
 
 (pods/unload-pod pod-id)
 (def successfully-removed (nil? (find-ns 'pod.test-pod)))
@@ -126,16 +63,4 @@
  successfully-removed
  x9
  tagged
- other-tagged
+ other-tagged]
- loaded
- fn-called
- local-date-time
- assoc-string-array
- round-trip-meta
- round-trip-meta-nested
- dont-round-trip-meta
- should-be-1
- add-sync-meta
- error-meta
- read-other-tag-meta
- incorrect-edn-response]

test/babashka/pods/impl_test.clj

@@ -1,11 +0,0 @@
-(ns babashka.pods.impl-test
-  (:require [clojure.test :refer :all]
-            [babashka.pods.impl :refer :all]))
-
-(deftest load-pod-test
-  (testing "resolve fn gets called when pod has EDN data readers"
-    (let [resolved? (atom false)
-          test-resolve (fn [_sym]
-                         (reset! resolved? true))]
-      (load-pod ["clojure" "-M:test-pod"] {:resolve test-resolve})
-      (is @resolved?))))

test/babashka/pods/jvm_test.clj

@@ -1,8 +1,6 @@
 (ns babashka.pods.jvm-test
-  (:require [babashka.pods.test-common :refer [test-program assertions
+  (:require [babashka.pods.test-common :refer [test-program assertions]]
-                                               pod-registry]]
+            [clojure.test :refer [deftest]]))
-            [clojure.string :as str]
-            [clojure.test :refer [deftest is]]))
 
 (deftest jvm-test
   (let [out (java.io.StringWriter.)
@@ -14,15 +12,3 @@
                    (catch Exception e (prn e))))]
 
     (assertions out err ret)))
-
-(deftest pod-registry-test
-  (let [out (java.io.StringWriter.)
-        err (java.io.StringWriter.)
-        ex (binding [*out* out
-                     *err* err]
-              (try (load-string
-                    pod-registry)
-                   (catch Exception e
-                     e)))]
-    (is (str/includes? (str out) "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2"))
-    (is (str/includes? (pr-str ex) "Version or path must be provided"))))

test/babashka/pods/sci_test.clj

@@ -1,41 +1,18 @@
 (ns babashka.pods.sci-test
   (:require [babashka.pods.sci :as pods]
-            [babashka.pods.test-common :refer [test-program assertions pod-registry]]
+            [babashka.pods.test-common :refer [test-program assertions]]
-            [clojure.string :as str]
+            [clojure.test :refer [deftest]]
-            [clojure.test :refer [deftest is]]
             [sci.core :as sci]))
 
 (deftest sci-test
   (let [out (java.io.StringWriter.)
         err (java.io.StringWriter.)
-        ctx-ref (volatile! nil)
-        ctx (sci/init {:namespaces {'babashka.pods
-                                    {'load-pod (fn [& args]
-                                                 (apply pods/load-pod @ctx-ref args))
-                                     'invoke pods/invoke
-                                     'unload-pod pods/unload-pod
-                                     'add-transit-read-handler!  pods/add-transit-read-handler!
-                                     'add-transit-write-handler! pods/add-transit-write-handler!
-                                     'set-default-transit-write-handler! pods/set-default-transit-write-handler!}}
-                       :classes {'System System
-                                 'java.time.LocalDateTime java.time.LocalDateTime
-                                 'java.lang.Class Class}})
-        _ (vreset! ctx-ref ctx)
         ret (sci/binding [sci/out out
                           sci/err err]
-              (binding [*out* out
+              (sci/eval-string
-                        *err* err]
+               test-program
-                (sci/eval-string* ctx test-program)))]
+               {:namespaces {'babashka.pods
+                             {'load-pod pods/load-pod
+                              'invoke pods/invoke
+                              'unload-pod pods/unload-pod}}}))]
     (assertions out err ret)))
-
-(deftest pod-registry-test
-  (let [out (java.io.StringWriter.)
-        err (java.io.StringWriter.)
-        ex (binding [*out* out
-                     *err* err]
-             (try (load-string
-                   pod-registry)
-                  (catch Exception e
-                    e)))]
-    (is (str/includes? (str out) "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2"))
-    (is (str/includes? (pr-str ex) "Version or path must be provided"))))

test/babashka/pods/test_common.clj

@@ -1,54 +1,29 @@
 (ns babashka.pods.test-common
   (:require [clojure.java.io :as io]
-            [clojure.string :as str]
             [clojure.test :refer [is]]))
 
 (def test-program (slurp (io/file "test-resources" "test_program.clj")))
 
 (defn assertions [out err ret]
-  ;; (.println System/err ret)
-  ;; (.println System/err out)
-  ;; (.println System/err err)
   (doseq [[expected actual]
-          (map vector (replace
+          (map vector '["pod.test-pod"
-                       {::edn-error (if (= "edn"
+                        pod.test-pod
-                                           (System/getenv "BABASHKA_POD_TEST_FORMAT"))
+                        {:a 1, :b 2}
-                                      "Map literal must contain an even number of forms"
+                        6
-                                      ::dont-care)}
+                        3
-                       '["pod.test-pod"
+                        [1 2 3 4 5 6 7 8 9]
-                         pod.test-pod
+                        "Illegal arguments / {:args (1 2 3)}"
-                         {:a 1, :b 2}
+                        nil
-                         6
+                        3
-                         3
+                        #"cast"
-                         [1 2 3 4 5 6 7 8 9]
+                        {:args ["1" 2]}
-                         #"Illegal arguments / \{:args [\(\[]1 2 3[\)\]]\}"
+                        true
-                         nil
+                        9
-                         3
+                        [1 2 3]
-                         #"cast"
+                        [[1] [1]]]
-                         {:args ["1" 2]}
-                         true
-                         9
-                         [1 2 3]
-                         [[1] [1]]
-                         2
-                         3
-                         true ;; local-date
-                         true ;; roundtrip string array
-                         true ;; roundtrip metadata
-                         true ;; roundtrip metadata nested
-                         true ;; dont roundtrip metadata (when arg-meta "false"/ absent)
-                         1
-                         "add the arguments"
-                         nil
-                         nil
-                         ::edn-error])
                (concat ret (repeat ::nil)))]
-    (cond (instance? java.util.regex.Pattern expected)
+    (if (instance? java.util.regex.Pattern expected)
-          (is (re-find expected actual))
+      (is (re-find expected actual))
-          (= ::dont-care expected) nil
+      (is (= expected actual))))
-          :else
-          (is (= expected actual))))
   (is (= "(\"hello\" \"print\" \"this\" \"debugging\" \"message\")\n:foo\n:foo\n" (str out)))
-  (is (str/starts-with? (str err) "(\"hello\" \"print\" \"this\" \"error\")" )))
+  (is (= "(\"hello\" \"print\" \"this\" \"error\")\n" (str err))))
-
-(def pod-registry (slurp (io/file "test-resources" "pod_registry.clj")))