Compare commits

..

No commits in common. "master" and "bandalore-0.0.2" have entirely different histories.

5 changed files with 30 additions and 91 deletions

View file

@ -14,17 +14,17 @@ Bandalore is available in Maven central. Add it to your Maven project's `pom.xm
<dependency> <dependency>
<groupId>com.cemerick</groupId> <groupId>com.cemerick</groupId>
<artifactId>bandalore</artifactId> <artifactId>bandalore</artifactId>
<version>0.0.6</version> <version>0.0.2</version>
</dependency> </dependency>
``` ```
or your leiningen project.clj: or your leiningen project.clj:
```clojure ```clojure
[com.cemerick/bandalore "0.0.6"] [com.cemerick/bandalore "0.0.2"]
``` ```
Bandalore is compatible with Clojure 1.2.0+. Bandalore is compatible with Clojure 1.2.0 - 1.4.0.
## Logging ## Logging
@ -42,7 +42,7 @@ Translate as necessary if you're using log4j, etc.
## Usage ## Usage
You should be familiar with [SQS itself](http://aws.amazon.com/sqs/) You should be familiar with http://aws.amazon.com/sqs/[SQS itself]
before sensibly using this library. That said, Bandalore's API before sensibly using this library. That said, Bandalore's API
is well-documented. is well-documented.
@ -54,19 +54,6 @@ to do anything:
(def client (sqs/create-client "your aws id" "your aws secret-key")) (def client (sqs/create-client "your aws id" "your aws secret-key"))
``` ```
**Security Note** If your application using Bandalore is deployed to EC2, _you
should not put your AWS credentials on those EC2 nodes_. Rather,
[give your EC2 instances IAM roles](http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html),
and use the nullary arity of `create-client`:
```clojure
(require '[cemerick.bandalore :as sqs])
(def client (sqs/create-client))
```
This will use credentials assigned to your EC2 node based on its
role that are automatically rotated.
You can create, delete, and list queues: You can create, delete, and list queues:
```clojure ```clojure
@ -108,28 +95,6 @@ That's cleaner than having to interop directly with the Java SDK, but it's all
pretty pedestrian stuff. You can do more interesting things with some pretty pedestrian stuff. You can do more interesting things with some
simple higher-order functions and other nifty Clojure facilities. simple higher-order functions and other nifty Clojure facilities.
### Enabling SQS Long Polling
[Long polling](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html) reduces the number of empty responses by allowing Amazon SQS service to wait until a message is available in the queue before sending a response. You can enable long polling on an individual receive request by supplying the optional kwarg `:wait-time-seconds`:
:wait-time-seconds - time in seconds (bewteen 0 and 20) for SQS to wait if there are no messages in the queue. A value of 0 indicates no long polling.
```clojure
; ensure our queue is empty to start
#> (get (sqs/queue-attrs client q) "ApproximateNumberOfMessages")
"0"
#> (let [no-polling (future (sqs/receive client q))
long-polling (future (sqs/receive client q :wait-time-seconds 20))]
(Thread/sleep 10000) ;; Sleep 10s before sending message
(sqs/send client q "my message body")
(println (count @no-polling))
(println (count @long-polling)))
0
1
nil
```
### Sending and receiving Clojure values ### Sending and receiving Clojure values
SQS' message bodies are strings, so you can stuff anything in them that you can SQS' message bodies are strings, so you can stuff anything in them that you can
@ -257,7 +222,7 @@ and another consumes those messages using a lazy seq provided by `polling-receiv
#> (defn consume-dummy-messages #> (defn consume-dummy-messages
[client q] [client q]
(future (dorun (map (sqs/deleting-consumer client (comp println :body)) (future (dorun (map (sqs/deleting-consumer client (comp println :body))
(sqs/polling-receive client q :max-wait Long/MAX_VALUE :limit 10))))) (sqs/polling-receive client q :max-wait Integer/MAX_VALUE :limit 10)))))
#'cemerick.bandalore-test/consume-dummy-messages #'cemerick.bandalore-test/consume-dummy-messages
#> (consume-dummy-messages client q) ;; start the consumer #> (consume-dummy-messages client q) ;; start the consumer
#<core$future_call$reify__5500@a6f00bc: :pending> #<core$future_call$reify__5500@a6f00bc: :pending>
@ -299,13 +264,13 @@ Since the tests are live, you either need to add your AWS credentials to your
using `-D` switches: using `-D` switches:
``` ```
$ mvn -Daws.id=XXXXXXX -Daws.secret-key=YYYYYYY clean install $ mvn -Daws.id#XXXXXXX -Daws.secret-key#YYYYYYY clean install
``` ```
Or, you can skip the tests entirely: Or, you can skip the tests entirely:
``` ```
$ mvn -Dmaven.test.skip=true clean install $ mvn -Dmaven.test.skip#true clean install
``` ```
In any case, you'll find a built `.jar` file in the `target` directory, and in In any case, you'll find a built `.jar` file in the `target` directory, and in
@ -319,6 +284,6 @@ or would like to contribute patches.
## License ## License
Copyright © 2011-2013 Chas Emerick and contributors. Copyright © 2011-2012 Chas Emerick
Licensed under the EPL. (See the file epl-v10.html.) Licensed under the EPL. (See the file epl-v10.html.)

View file

@ -2,7 +2,7 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<groupId>com.cemerick</groupId> <groupId>com.cemerick</groupId>
<artifactId>bandalore</artifactId> <artifactId>bandalore</artifactId>
<version>0.0.7-SNAPSHOT</version> <version>0.0.2</version>
<name>bandalore</name> <name>bandalore</name>
<description>A Clojure library for Amazon's Simple Queue Service (SQS).</description> <description>A Clojure library for Amazon's Simple Queue Service (SQS).</description>
<url>http://github.com/cemerick/bandalore</url> <url>http://github.com/cemerick/bandalore</url>
@ -29,14 +29,14 @@
</scm> </scm>
<properties> <properties>
<clojure.version>1.4.0</clojure.version> <clojure.version>1.3.0</clojure.version>
</properties> </properties>
<dependencies> <dependencies>
<dependency> <dependency>
<groupId>com.amazonaws</groupId> <groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId> <artifactId>aws-java-sdk</artifactId>
<version>1.8.0</version> <version>1.1.5</version>
</dependency> </dependency>
</dependencies> </dependencies>

View file

@ -3,14 +3,14 @@
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<groupId>clojure.tools</groupId> <groupId>clojure.tools</groupId>
<version>0.0.0</version> <version>0.0.0</version>
<artifactId>test-clojure-1.3.0</artifactId> <artifactId>test-clojure-1.4.0</artifactId>
<name>(Clojure 1.3.0 tests)</name> <name>(Clojure 1.4.0 tests)</name>
<dependencies> <dependencies>
<dependency> <dependency>
<groupId>org.clojure</groupId> <groupId>org.clojure</groupId>
<artifactId>clojure</artifactId> <artifactId>clojure</artifactId>
<version>1.3.0</version> <version>1.4.0-master-SNAPSHOT</version>
</dependency> </dependency>
<dependency> <dependency>
<groupId>@project.groupId@</groupId> <groupId>@project.groupId@</groupId>

View file

@ -19,28 +19,22 @@
(defn create-client (defn create-client
"Creates a synchronous AmazonSQSClient using the provided account id, secret key, "Creates a synchronous AmazonSQSClient using the provided account id, secret key,
and optional com.amazonaws.ClientConfiguration." and optional com.amazonaws.ClientConfiguration."
([]
(create-client (com.amazonaws.ClientConfiguration.)))
([client-config]
(AmazonSQSClient.
(.withUserAgent client-config "Bandalore - SQS for Clojure")))
([id secret-key] ([id secret-key]
(create-client id secret-key (com.amazonaws.ClientConfiguration.))) (create-client id secret-key (com.amazonaws.ClientConfiguration.)))
([id secret-key client-config] ([id secret-key client-config]
(AmazonSQSClient. (com.amazonaws.auth.BasicAWSCredentials. id secret-key) (AmazonSQSClient. (com.amazonaws.auth.BasicAWSCredentials. id secret-key)
(.withUserAgent client-config "Bandalore - SQS for Clojure")))) (.withUserAgent client-config "Bandalore - SQS for Clojure"))))
(def ^{:private true} visibility-warned? (atom false))
(defn create-queue (defn create-queue
"Creates a queue with the given name, returning the corresponding URL string. "Creates a queue with the given name, returning the corresponding URL string.
Returns successfully if the queue already exists." Returns successfully if the queue already exists.
[^AmazonSQSClient client queue-name & {:as options}]
(when (and (:visibility options) (not @visibility-warned?)) Specify an optional :visibility keyword arg to set the new queue's default
(println "[WARNING] :visibility option to cemerick.bandalore/create-queue no longer supported;") visibility timeout in seconds."
(println "[WARNING] See https://github.com/cemerick/bandalore/issues/3") [^AmazonSQSClient client queue-name & {:keys [visibility]}]
(reset! visibility-warned? true)) (->> (if visibility
(->> (CreateQueueRequest. queue-name) (CreateQueueRequest. queue-name visibility)
(CreateQueueRequest. queue-name))
(.createQueue client) (.createQueue client)
.getQueueUrl)) .getQueueUrl))
@ -50,10 +44,9 @@
(.deleteQueue client (DeleteQueueRequest. queue-url))) (.deleteQueue client (DeleteQueueRequest. queue-url)))
(defn list-queues (defn list-queues
"Returns a seq of all queues' URL strings. Takes an optional string prefix "Returns a seq of all queues' URL strings."
argument to only list queues with names that start with the prefix." [^AmazonSQSClient client]
[^AmazonSQSClient client & {:keys [prefix]}] (->> (ListQueuesRequest.)
(->> (ListQueuesRequest. prefix)
(.listQueues client) (.listQueues client)
.getQueueUrls .getQueueUrls
seq)) seq))
@ -106,11 +99,6 @@
Defaults to the empty set (i.e. no attributes will be included in Defaults to the empty set (i.e. no attributes will be included in
received messages). received messages).
See the SQS documentation for all support message attributes. See the SQS documentation for all support message attributes.
:wait-time-seconds - enables long poll support. time is in seconds, bewteen
0 (default - no long polling) and 20.
Allows Amazon SQS service to wait until a message is available
in the queue before sending a response.
See the SQS documentation at (http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html)
Returns a seq of maps with these slots: Returns a seq of maps with these slots:
@ -121,17 +109,13 @@
:receipt-handle - the ID used to delete the message from the queue after :receipt-handle - the ID used to delete the message from the queue after
it has been fully processed. it has been fully processed.
:source-queue - the URL of the queue from which the message was received" :source-queue - the URL of the queue from which the message was received"
[^AmazonSQSClient client queue-url & {:keys [limit [^AmazonSQSClient client queue-url & {:keys [limit visibility ^java.util.Collection attributes]
visibility
wait-time-seconds
^java.util.Collection attributes]
:or {limit 1 :or {limit 1
attributes #{}}}] attributes #{}}}]
(let [req (-> (ReceiveMessageRequest. queue-url) (let [req (-> (ReceiveMessageRequest. queue-url)
(.withMaxNumberOfMessages (-> limit (min 10) (max 1) int Integer/valueOf)) (.withMaxNumberOfMessages (-> limit (min 10) (max 1) Integer.))
(.withAttributeNames attributes)) (.withAttributeNames attributes))
req (if wait-time-seconds (.withWaitTimeSeconds req (Integer/valueOf (int wait-time-seconds))) req) req (if visibility (.withVisibilityTimeout req (Integer. visibility)) req)]
req (if visibility (.withVisibilityTimeout req (Integer/valueOf (int visibility))) req)]
(->> (.receiveMessage client req) (->> (.receiveMessage client req)
.getMessages .getMessages
(map (partial message-map queue-url))))) (map (partial message-map queue-url)))))

View file

@ -59,9 +59,7 @@
; sending a msg seems to "force" the queue's existence in listings ; sending a msg seems to "force" the queue's existence in listings
(send client *test-queue-url* msg) (send client *test-queue-url* msg)
(wait-for-condition #((set (list-queues client)) *test-queue-url*) (wait-for-condition #((set (list-queues client)) *test-queue-url*)
"Created queue not visible in result of list-queues") "Created queue not visible in result of list-queues")))
(wait-for-condition #((set (list-queues client :prefix test-queue-name-prefix)) *test-queue-url*)
"Created queue not visible in result of list-queues with prefix")))
(defsqstest test-queue-attrs (defsqstest test-queue-attrs
(let [{:strs [MaximumMessageSize] :as base-attrs} (queue-attrs client *test-queue-url*) (let [{:strs [MaximumMessageSize] :as base-attrs} (queue-attrs client *test-queue-url*)
@ -121,11 +119,3 @@
(let [v (-> (receive client *test-queue-url* :visibility 5) first :body read-string)] (let [v (-> (receive client *test-queue-url* :visibility 5) first :body read-string)]
(is (some #(= v (-> % :body read-string)) (polling-receive client *test-queue-url* :max-wait 10000))))) (is (some #(= v (-> % :body read-string)) (polling-receive client *test-queue-url* :max-wait 10000)))))
(defsqstest test-receive-long-polling
(let [q *test-queue-url*
no-poll (future (receive client q))
long-poll (future (receive client q :wait-time-seconds 20))]
(Thread/sleep 10000)
(send client q "1")
(is (== 0 (count @no-poll)) "Should not return messages")
(is (== 1 (count @long-poll)) "Should return 1 message")))