luciano/babashka
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

README [skip ci]

Browse source
This commit is contained in:
Michiel Borkent 2020-11-08 23:06:06 +01:00
parent 6c9006036a
commit 526a5daf5c
1 changed files with 14 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
README.md
View file

@ -870,16 +870,21 @@ For making HTTP requests you can use:
If memory usage is a concern and you are downloading big files, choose If memory usage is a concern and you are downloading big files, choose
`babashka.curl` with `:as :stream` over `org.httpkit.client` since httpkit holds `babashka.curl` with `:as :stream` over `org.httpkit.client` since httpkit holds
the entire response in memory at once: the entire response in memory at once. Let's download a 200mb file with 10mb
heap size:
``` shell ``` shell
$ time bb -e '(io/copy (:body (curl/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))' $ time bb -Xmx10m -e '(io/copy (:body (curl/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'
... ```
max memory: 20064 MB
$ time bb -e '(io/copy (:body @(org.httpkit.client/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))' With `babashka.curl` this forks fine. However with `org.httpkit.client` that
bb -e 0.87s user 1.26s system 23% cpu 9.150 total won't work. Not even 190mb of heap will do:
max memory: 686584 MB
``` shell
$ time bb -Xmx190m -e '(io/copy (:body @(org.httpkit.client/get "http://ipv4.download.thinkbroadband.com/200MB.zip" {:as :stream})) (io/file "/tmp/200mb.zip"))'
Sun Nov 08 23:01:46 CET 2020 [client-loop] ERROR - select exception, should not happen
java.lang.OutOfMemoryError: Array allocation too large.
``` ```
If your script creates many requests with relatively small payloads, choose If your script creates many requests with relatively small payloads, choose
@ -889,7 +894,8 @@ process to shell out to curl for each request.
In the future babashka (1.0.0?) may come with an HTTP client based on the JVM 11 In the future babashka (1.0.0?) may come with an HTTP client based on the JVM 11
`java.net.http` package that ticks all the boxes (async, HTTP/2, websockets, `java.net.http` package that ticks all the boxes (async, HTTP/2, websockets,
multi-part file uploads, sane memory usage) and is a suitable replacement for multi-part file uploads, sane memory usage) and is a suitable replacement for
all of the above options. all of the above options. If you know about a GraalVM-friendly feature-complete
well-maintained library, please reach out!
### HTTP over Unix sockets ### HTTP over Unix sockets