Merge branch 'master' into core-cache-integration

Conflicts:
	project.clj
This commit is contained in:
Michael S. Klishin 2012-04-11 06:28:20 +04:00
commit fff83160be
19 changed files with 425 additions and 204 deletions

View file

@ -1,5 +1,70 @@
## Changes between 1.0.0-beta4 and 1.0.0-beta5
No changes yet.
## Changes between 1.0.0-beta3 and 1.0.0-beta4
### Support for URI connections (and thus PaaS provides like Heroku)
`monger.core/connect-via-uri!` is a new function that combines `monger.core/connect!`, `monger.core/set-db!` and `monger.core/authenticate`
and works with string URIs like `mongodb://userb71148a:0da0a696f23a4ce1ecf6d11382633eb2049d728e@cluster1.mongohost.com:27034/app81766662`.
It can be used to connect with or without authentication, for example:
``` clojure
;; connect without authentication
(monger.core/connect-via-uri! "mongodb://127.0.0.1/monger-test4")
;; connect with authentication
(monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:monger!@127.0.0.1/monger-test4")
;; connect using connection URI stored in an env variable, in this case, MONGOHQ_URL
(monger.core/connect-via-uri! (System/genenv "MONGOHQ_URL"))
```
It is also possible to pass connection options and query parameters:
``` clojure
(monger.core/connect-via-uri! "mongodb://localhost/test?maxPoolSize=128&waitQueueMultiple=5;waitQueueTimeoutMS=150;socketTimeoutMS=5500&autoConnectRetry=true;safe=false&w=1;wtimeout=2500;fsync=true")
```
## Changes between 1.0.0-beta2 and 1.0.0-beta3
### Support for field negation in queries
Previously to load only a subset of document fields with Monger, one had to specify them all. Starting
with 1.0.0-beta3, Monger supports [field negation](http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields#RetrievingaSubsetofFields-FieldNegation) feature of MongoDB: it is possible to exclude
certain fields instead.
To do so, pass a map as field selector, with fields that should be omitted set to 0:
``` clojure
;; will retrieve all fields except body
(monger.collection/find-one-map "documents" {:author "John"} {:body 0})
```
### Validateur 1.1.0-beta1
[Validateur](https://github.com/michaelklishin/validateur) dependency has been upgraded to 1.1.0-beta1.
### Index Options support for monger.collection/ensure-index and /create-index
`monger.collection/ensure-index` and `/create-index` now accept index options as additional argument.
**Breaking change**: 3-arity versions of those functions now become 4-arity versions.
### Support serialization of Clojure ratios
Documents that contain Clojure ratios (for example, `26/5`) now can be converted to DBObject instances
and thus stored. On load, ratios will be presented as doubles: this way we ensure interoperability with
other languages and clients.
### Factories/fixtures DSL
When working with even moderately complex data sets, fixture data quickly becomes difficult to

View file

@ -15,11 +15,12 @@ wanted a client that will
* Be well documented.
* Be well tested.
* Be maintained, do not carry technical debt from 2009 forever.
* Target Clojure 1.3.0 and later from the ground up.
* Integrate with libraries like clojure.data.json and Joda Time.
* Provide support for unit testing: factories/fixtures DSL, collection cleaner functions, clojure.test integration and so on.
* Integrate usage of JavaScript files and ClojureScript (as soon as the compiler gets artifact it is possible to depend on for easy embedding).
* Support URI connections to be friendly to Heroku and other PaaS providers.
* Learn from other clients like the Java and Ruby ones.
* Target Clojure 1.3.0 and later from the ground up.
* Integrate usage of JavaScript files and ClojureScript (as soon as the compiler gets artifact it is possible to depend on for easy embedding).
## Documentation & Examples
@ -45,7 +46,7 @@ together with documentation guides and dedicated website.
With Leiningen:
[com.novemberain/monger "1.0.0-beta2"]
[com.novemberain/monger "1.0.0-beta4"]
With Maven:
@ -53,7 +54,7 @@ With Maven:
<dependency>
<groupId>com.novemberain</groupId>
<artifactId>monger</artifactId>
<version>1.0.0-beta2</version>
<version>1.0.0-beta4</version>
</dependency>
@ -223,7 +224,7 @@ Here is what monger.query DSL feels like:
(with-collection "movies"
(find { :year { $lt 2010 $gte 2000 }, :revenue { $gt 20000000 } })
(fields [ :year :title :producer :cast :budget :revenue ])
(sort-by { :revenue -1 })
(sort { :revenue -1 })
(skip 10)
(limit 20)
(hint "year-by-year-revenue-idx")
@ -362,7 +363,7 @@ Neocons is part of the group of libraries known as ClojureWerkz, together with
## Development
Monger uses [Leiningen 2](https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md). Make sure you have it installed and then run tests against
Clojure 1.3.0 and 1.4.0[-beta4] using
supported Clojure versions using
lein2 all test

View file

@ -4,27 +4,27 @@
:license {:name "Eclipse Public License"}
:dependencies [[org.clojure/clojure "1.3.0"]
[org.mongodb/mongo-java-driver "2.7.3"]
[com.novemberain/validateur "1.0.0"]]
:test-selectors {:focus (fn [v] (:focus v))}
[com.novemberain/validateur "1.1.0-beta1"]]
:test-selectors {:default (complement :performance)
:focus :focus
:indexing :indexing
:external :external
:performance :performance
:all (constantly true)}
:codox {:exclude [monger.internal.pagination]}
:mailing-list {:name "clojure-monger",
:archive "https://groups.google.com/group/clojure-monger",
:mailing-list {:name "clojure-monger"
:archive "https://groups.google.com/group/clojure-monger"
:post "clojure-monger@googlegroups.com"}
:profiles {:1.4 {:resource-paths ["test/resources"],
:dependencies [[org.clojure/clojure "1.4.0-beta5"]]},
:dev {:resource-paths ["test/resources"],
:dependencies [[org.mongodb/mongo-java-driver "2.7.3"]
[com.novemberain/validateur "1.0.0"]
[clj-time "0.3.6" :exclusions [org.clojure/clojure]]
[codox "0.3.4" :exclusions [org.clojure/clojure]]
[org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]]
[org.clojure/data.json "0.1.2" :exclusions [org.clojure/clojure]]
[clj-time "0.3.6" :exclusions [org.clojure/clojure]]
[org.clojure/core.cache "0.5.0" :exclusions [org.clojure/clojure]]
[codox "0.3.4" :exclusions [org.clojure/clojure]]
[org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]]]}}
:profiles {:1.4 {:resource-paths ["test/resources"]
:dependencies [[org.clojure/clojure "1.4.0-beta7"]]}
:dev {:resource-paths ["test/resources"]
:dependencies [[clj-time "0.3.6" :exclusions [org.clojure/clojure]]
[codox "0.3.4" :exclusions [org.clojure/clojure]]
[org.clojure/data.json "0.1.2" :exclusions [org.clojure/clojure]]
[org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]]
[org.clojure/core.cache "0.5.0" :exclusions [org.clojure/clojure]]]}}
:aliases { "all" ["with-profile" "dev:dev,1.4"] }
:repositories {"sonatype" {:url "http://oss.sonatype.org/content/repositories/releases",
:snapshots false,
:releases {:checksum :fail, :update :always}}}
:repositories {"sonatype" {:url "http://oss.sonatype.org/content/repositories/releases"
:snapshots false
:releases {:checksum :fail :update :always}}}
:warn-on-reflection true)

View file

@ -21,10 +21,6 @@
;; Implementation
;;
(defn- fields-to-db-object
[^List fields]
(zipmap fields (repeat 1)))
(definline check-not-nil!
[ref ^String message]
`(when (nil? ~ref)
@ -104,13 +100,13 @@
([^String collection ^Map ref]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)]
(.find ^DBCollection coll ^DBObject (to-db-object ref))))
([^String collection ^Map ref ^List fields]
([^String collection ^Map ref fields]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)
map-of-fields (fields-to-db-object fields)]
map-of-fields (as-field-selector fields)]
(.find ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields))))
([^DB db ^String collection ^Map ref ^List fields]
([^DB db ^String collection ^Map ref fields]
(let [^DBCollection coll (.getCollection db collection)
map-of-fields (fields-to-db-object fields)]
map-of-fields (as-field-selector fields)]
(.find ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields)))))
(defn ^ISeq find-maps
@ -122,9 +118,9 @@
(map (fn [x] (from-db-object x true)) (find collection)))
([^String collection ^Map ref]
(map (fn [x] (from-db-object x true)) (find collection ref)))
([^String collection ^Map ref ^List fields]
([^String collection ^Map ref fields]
(map (fn [x] (from-db-object x true)) (find collection ref fields)))
([^DB db ^String collection ^Map ref ^List fields]
([^DB db ^String collection ^Map ref fields]
(map (fn [x] (from-db-object x true)) (find db collection ref fields))))
(defn ^ISeq find-seq
@ -133,9 +129,9 @@
(seq (find collection)))
([^String collection ^Map ref]
(seq (find collection ref)))
([^String collection ^Map ref ^List fields]
([^String collection ^Map ref fields]
(seq (find collection ref fields)))
([^DB db ^String collection ^Map ref ^List fields]
([^DB db ^String collection ^Map ref fields]
(seq (find db collection ref fields))))
;;
@ -157,22 +153,22 @@
([^String collection ^Map ref]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)]
(.findOne ^DBCollection coll ^DBObject (to-db-object ref))))
([^String collection ^Map ref ^List fields]
([^String collection ^Map ref fields]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)
map-of-fields (fields-to-db-object fields)]
map-of-fields (as-field-selector fields)]
(.findOne ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields))))
([^DB db ^String collection ^Map ref ^List fields]
([^DB db ^String collection ^Map ref fields]
(let [^DBCollection coll (.getCollection db collection)
map-of-fields (fields-to-db-object fields)]
map-of-fields (as-field-selector fields)]
(.findOne ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields)))))
(defn ^IPersistentMap find-one-as-map
"Returns a single object converted to Map from this collection matching the query."
([^String collection ^Map ref]
(from-db-object ^DBObject (find-one collection ref) true))
([^String collection ^Map ref ^List fields]
([^String collection ^Map ref fields]
(from-db-object ^DBObject (find-one collection ref fields) true))
([^String collection ^Map ref ^List fields keywordize]
([^String collection ^Map ref fields keywordize]
(from-db-object ^DBObject (find-one collection ref fields) keywordize)))
@ -195,10 +191,10 @@
([^String collection id]
(check-not-nil! id "id must not be nil")
(find-one collection { :_id id }))
([^String collection id ^List fields]
([^String collection id fields]
(check-not-nil! id "id must not be nil")
(find-one collection { :_id id } fields))
([^DB db ^String collection id ^List fields]
([^DB db ^String collection id fields]
(check-not-nil! id "id must not be nil")
(find-one db collection { :_id id } fields)))
@ -207,10 +203,10 @@
([^String collection id]
(check-not-nil! id "id must not be nil")
(from-db-object ^DBObject (find-one-as-map collection { :_id id }) true))
([^String collection id ^List fields]
([^String collection id fields]
(check-not-nil! id "id must not be nil")
(from-db-object ^DBObject (find-one-as-map collection { :_id id } fields) true))
([^String collection id ^List fields keywordize]
([^String collection id fields keywordize]
(check-not-nil! id "id must not be nil")
(from-db-object ^DBObject (find-one-as-map collection { :_id id } fields) keywordize)))
@ -388,16 +384,20 @@
EXAMPLES
;; Will create an index on \"language\" field
;; Will create an index on the \"language\" field
(monger.collection/create-index collection { \"language\" 1 })
(monger.collection/create-index collection { \"language\" 1 } { :unique true :name \"unique_language\" })
"
([^String collection ^Map keys]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)]
(.createIndex coll (to-db-object keys))))
([^DB db ^String collection ^Map keys]
([^String collection ^Map keys options]
(let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)]
(.createIndex coll (to-db-object keys) (to-db-object options))))
([^DB db ^String collection ^Map keys ^Map options]
(let [^DBCollection coll (.getCollection db collection)]
(.createIndex coll (to-db-object keys)))))
(.createIndex coll (to-db-object keys) (to-db-object options)))))
;;
@ -416,9 +416,12 @@
([^String collection, ^Map keys]
(let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)]
(.ensureIndex ^DBCollection coll ^DBObject (to-db-object keys))))
([^String collection, ^Map keys, ^String name]
([^String collection, ^Map keys ^Map options]
(let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)]
(.ensureIndex coll ^DBObject (to-db-object keys) ^String name))))
(.ensureIndex ^DBCollection coll ^DBObject (to-db-object keys) (to-db-object options))))
([^String collection ^Map keys ^String name ^Boolean unique?]
(let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)]
(.ensureIndex coll ^DBObject (to-db-object keys) ^String name unique?))))
;;

View file

@ -23,12 +23,12 @@
(ns monger.conversion
(:import [com.mongodb DBObject BasicDBObject BasicDBList DBCursor]
[clojure.lang IPersistentMap Keyword]
[clojure.lang IPersistentMap Keyword Ratio]
[java.util List Map Date]
[org.bson.types ObjectId]))
org.bson.types.ObjectId))
(defprotocol ConvertToDBObject
(to-db-object [input] "Converts given piece of Clojure data to BasicDBObject MongoDB Java driver uses"))
(^com.mongodb.DBObject to-db-object [input] "Converts given piece of Clojure data to BasicDBObject MongoDB Java driver uses"))
(extend-protocol ConvertToDBObject
nil
@ -39,6 +39,10 @@
(to-db-object [input]
input)
Ratio
(to-db-object [^Ratio input]
(double input))
Keyword
(to-db-object [^Keyword input] (.getName input))
@ -105,7 +109,7 @@
(defprotocol ConvertToObjectId
(to-object-id [input] "Instantiates ObjectId from input unless the input itself is an ObjectId instance. In that case, returns input as is."))
(^org.bson.types.ObjectId to-object-id [input] "Instantiates ObjectId from input unless the input itself is an ObjectId instance. In that case, returns input as is."))
(extend-protocol ConvertToObjectId
String
@ -121,3 +125,19 @@
input))
(defprotocol FieldSelector
(^com.mongodb.DBObject as-field-selector [input] "Converts values to DBObject that can be used to specify a list of document fields (including negation support)"))
(extend-protocol FieldSelector
DBObject
(as-field-selector [^DBObject input]
input)
List
(as-field-selector [^List input]
(to-db-object (zipmap input (repeat 1))))
Object
(as-field-selector [input]
(to-db-object input)))

View file

@ -14,7 +14,7 @@
monger.core
(:refer-clojure :exclude [count])
(:use [monger.conversion])
(:import [com.mongodb Mongo DB WriteConcern DBObject DBCursor CommandResult Bytes MongoOptions ServerAddress]
(:import [com.mongodb Mongo MongoURI DB WriteConcern DBObject DBCursor CommandResult Bytes MongoOptions ServerAddress MapReduceOutput]
[com.mongodb.gridfs GridFS]
[java.util Map]))
@ -22,7 +22,7 @@
;; Defaults
;;
(def ^:dynamic ^String *mongodb-host* "localhost")
(def ^:dynamic ^String *mongodb-host* "127.0.0.1")
(def ^:dynamic ^long *mongodb-port* 27017)
(declare ^:dynamic ^Mongo *mongodb-connection*)
@ -31,6 +31,7 @@
(declare ^:dynamic ^GridFS *mongodb-gridfs*)
;;
;; API
;;
@ -51,11 +52,10 @@
(Mongo.))
([^ServerAddress server-address ^MongoOptions options]
(Mongo. server-address options))
([{ :keys [host port] :or { host *mongodb-host*, port *mongodb-port* }}]
([{ :keys [host port uri] :or { host *mongodb-host* port *mongodb-port* }}]
(Mongo. ^String host ^Long port)))
(defn ^DB get-db-names
"Gets a list of all database names present on the server"
([]
@ -105,7 +105,7 @@
(defn server-address
([^String hostname]
(ServerAddress. hostname))
([^String hostname ^long port]
([^String hostname ^Long port]
(ServerAddress. hostname port)))
@ -172,6 +172,32 @@
and WebScale fast second."
(def ^:dynamic *mongodb-write-concern* wc))
(defn connect-via-uri!
"Connects to MongoDB using a URI, sets up default connection and database. Commonly used for PaaS-based applications,
for example, running on Heroku. If username and password are provided, performs authentication."
[uri]
(let [uri (MongoURI. uri)
;; yes, you are not hallucinating. A class named MongoURI has a method called connectDB.
;; I call it "college OOP". Or maybe "don't give a shit" OOP.
db (.connectDB uri)
conn (.getMongo db)
user (.getUsername uri)
pwd (.getPassword uri)]
;; I hope that whoever wrote the MongoDB Java driver connection/authentication parts
;; wasn't sober while at it. MK.
;;
;; First we set connection, then DB, then authentcate
(set-connection! conn)
(when (and user pwd)
(when-not (authenticate (.getName db) user pwd)
(throw (IllegalArgumentException. "Could not authenticate. Either database name or credentials are invalid."))))
;; only do this *after* we authenticated because set-db! will try to set up a default GridFS instance. MK.
(when db
(set-db! db))
conn))
(defn ^CommandResult command
"Runs a database command (please check MongoDB documentation for the complete list of commands). Some common commands
are:
@ -237,7 +263,11 @@
(extend-protocol Countable
DBCursor
(count [^DBCursor this]
(.count this)))
(.count this))
MapReduceOutput
(count [^MapReduceOutput this]
(.count (.results this))))
(defn ^DBObject get-last-error
"Returns the the error (if there is one) from the previous operation on this connection.

View file

@ -31,7 +31,7 @@
;;
;; Existing query fields:
;;
;; :fields - selects which fields are returned. The default is all fields. _id is always returned.
;; :fields - selects which fields are returned. The default is all fields. _id is included by default.
;; :sort - adds a sort to the query.
;; :fields - set of fields to retrieve during query execution
;; :skip - Skips the first N results.
@ -58,13 +58,9 @@
([^DBCollection coll]
(merge (empty-query) { :collection coll })))
(defn- fields-to-db-object
[^List fields]
(to-db-object (zipmap fields (repeat 1))))
(defn exec
[{ :keys [collection query fields skip limit sort batch-size hint snapshot read-preference keywordize-fields] :or { limit 0 batch-size 256 skip 0 } }]
(let [cursor (doto ^DBCursor (.find ^DBCollection collection (to-db-object query) (fields-to-db-object fields))
(let [cursor (doto ^DBCursor (.find ^DBCollection collection (to-db-object query) (as-field-selector fields))
(.limit limit)
(.skip skip)
(.sort (to-db-object sort))

View file

@ -7,7 +7,7 @@
;; the terms of this license.
;; You must not remove this notice, or any other, from this software.
(ns monger.testing
(ns monger.testkit
(:require [monger.collection :as mc]
[monger.result :as mr])
(:use [monger.internal.fn :only (expand-all expand-all-with) :as fntools])

View file

@ -2,18 +2,15 @@
(ns monger.test.collection
(:import [com.mongodb WriteResult WriteConcern DBCursor DBObject CommandResult$CommandFailure MapReduceOutput MapReduceCommand MapReduceCommand$OutputType]
[org.bson.types ObjectId]
[java.util Date])
org.bson.types.ObjectId
java.util.Date)
(:require [monger core util]
[clojure stacktrace]
[monger.collection :as mgcol]
[monger.result :as mgres]
[monger.conversion :as mgcnv]
[monger.js :as js]
[monger.collection :as mgcol]
[monger.result :as mgres]
[monger.test.helper :as helper])
(:use [clojure.test]
[monger.operators]
[monger.test.fixtures]))
(:use clojure.test
monger.operators
monger.test.fixtures))
(helper/connect!)
@ -71,27 +68,6 @@
(is (nil? (mgcol/find-by-id collection oid)))))
;;
;; indexes
;;
(deftest index-operations
(let [collection "libraries"]
(mgcol/drop-indexes collection)
(is (= "_id_"
(:name (first (mgcol/indexes-on collection)))))
(is (nil? (second (mgcol/indexes-on collection))))
(mgcol/create-index collection { "language" 1 })
(is (= "language_1"
(:name (second (mgcol/indexes-on collection)))))
(mgcol/drop-index collection "language_1")
(is (nil? (second (mgcol/indexes-on collection))))
(mgcol/ensure-index collection { "language" 1 })
(is (= "language_1"
(:name (second (mgcol/indexes-on collection)))))
(mgcol/ensure-index collection { "language" 1 })))
;;
;; exists?, drop, create
;;
@ -116,65 +92,36 @@
(is (mgcol/exists? "gadgets"))
(mgcol/drop "gadgets")))
;;
;; Map/Reduce
;; any?, empty?
;;
(let [collection "widgets"
mapper (js/load-resource "resources/mongo/js/mapfun1.js")
reducer "function(key, values) {
var result = 0;
values.forEach(function(v) { result += v });
(deftest test-any-on-empty-collection
(let [collection "things"]
(is (not (mgcol/any? collection)))))
return result;
}"
batch [{ :state "CA" :quantity 1 :price 199.00 }
{ :state "NY" :quantity 2 :price 199.00 }
{ :state "NY" :quantity 1 :price 299.00 }
{ :state "IL" :quantity 2 :price 11.50 }
{ :state "CA" :quantity 2 :price 2.95 }
{ :state "IL" :quantity 3 :price 5.50 }]
expected [{:_id "CA", :value 204.9} {:_id "IL", :value 39.5} {:_id "NY", :value 697.0}]]
(deftest basic-inline-map-reduce-example
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(let [output (mgcol/map-reduce collection mapper reducer nil MapReduceCommand$OutputType/INLINE {})
results (mgcnv/from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
(mgres/ok? output)
(is (= expected results))))
(deftest test-any-on-non-empty-collection
(let [collection "things"
_ (mgcol/insert collection { :language "Clojure", :name "langohr" })]
(is (mgcol/any? "things"))
(is (mgcol/any? monger.core/*mongodb-database* "things" {:language "Clojure"}))))
(deftest basic-map-reduce-example-that-replaces-named-collection
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(let [output (mgcol/map-reduce collection mapper reducer "mr_outputs" {})
results (mgcnv/from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
(mgres/ok? output)
(is (= 3 (monger.core/count results)))
(is (= expected
(map #(mgcnv/from-db-object % true) (seq results))))
(is (= expected
(map #(mgcnv/from-db-object % true) (mgcol/find "mr_outputs"))))
(.drop ^MapReduceOutput output)))
(deftest test-empty-on-empty-collection
(let [collection "things"]
(is (mgcol/empty? collection))
(is (mgcol/empty? monger.core/*mongodb-database* collection))))
(deftest basic-map-reduce-example-that-merged-results-into-named-collection
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})
(is (mgres/ok? (mgcol/insert collection { :state "OR" :price 17.95 :quantity 4 })))
(let [output (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})]
(mgres/ok? output)
(is (= 4 (monger.core/count (.results ^MapReduceOutput output))))
(is (= ["CA" "IL" "NY" "OR"]
(map :_id (mgcol/find-maps "merged_mr_outputs"))))
(.drop ^MapReduceOutput output))))
(deftest test-empty-on-non-empty-collection
(let [collection "things"
_ (mgcol/insert collection { :language "Clojure", :name "langohr" })]
(is (not (mgcol/empty? "things")))))
;;
;; distinct
;;
(deftest distinct-values
(deftest test-distinct-values
(let [collection "widgets"
batch [{ :state "CA" :quantity 1 :price 199.00 }
{ :state "NY" :quantity 2 :price 199.00 }
@ -185,28 +132,3 @@
(mgcol/insert-batch collection batch)
(is (= ["CA" "IL" "NY"] (sort (mgcol/distinct monger.core/*mongodb-database* collection :state {}))))
(is (= ["CA" "NY"] (sort (mgcol/distinct collection :state { :price { $gt 100.00 } }))))))
;;
;; any?, empty?
;;
(deftest any-on-empty-collection
(let [collection "things"]
(is (not (mgcol/any? collection)))))
(deftest any-on-non-empty-collection
(let [collection "things"
_ (mgcol/insert collection { :language "Clojure", :name "langohr" })]
(is (mgcol/any? "things"))
(is (mgcol/any? monger.core/*mongodb-database* "things" {:language "Clojure"}))))
(deftest empty-on-empty-collection
(let [collection "things"]
(is (mgcol/empty? collection))
(is (mgcol/empty? monger.core/*mongodb-database* collection))))
(deftest empty-on-non-empty-collection
(let [collection "things"
_ (mgcol/insert collection { :language "Clojure", :name "langohr" })]
(is (not (mgcol/empty? "things")))))

View file

@ -1,10 +1,9 @@
(ns monger.test.conversion
(:require [monger core collection]
[monger.conversion :as cnv])
(:require [monger core collection])
(:import [com.mongodb DBObject BasicDBObject BasicDBList]
[java.util Date Calendar List ArrayList]
[org.bson.types ObjectId])
(:use [clojure.test]))
(:use clojure.test monger.conversion))
;;
@ -13,28 +12,33 @@
(deftest convert-nil-to-dbobject
(let [input nil
output (cnv/to-db-object input)]
output (to-db-object input)]
(is (nil? output))))
(deftest convert-integer-to-dbobject
(let [input 1
output (cnv/to-db-object input)]
output (to-db-object input)]
(is (= input output))))
(deftest convert-float-to-dbobject
(let [input 11.12
output (cnv/to-db-object input)]
output (to-db-object input)]
(is (= input output))))
(deftest convert-rationale-to-dbobject
(let [input 11/2
output (to-db-object input)]
(is (= 5.5 output))))
(deftest convert-string-to-dbobject
(let [input "MongoDB"
output (cnv/to-db-object input)]
output (to-db-object input)]
(is (= input output))))
(deftest convert-map-to-dbobject
(let [input { :int 1, :string "Mongo", :float 22.23 }
output ^DBObject (cnv/to-db-object input)]
output ^DBObject (to-db-object input)]
(is (= 1 (.get output "int")))
(is (= "Mongo" (.get output "string")))
(is (= 22.23 (.get output "float")))))
@ -42,7 +46,7 @@
(deftest convert-nested-map-to-dbobject
(let [input { :int 1, :string "Mongo", :float 22.23, :map { :int 10, :string "Clojure", :float 11.9, :list '(1 "a" :b), :map { :key "value" } } }
output ^DBObject (cnv/to-db-object input)
output ^DBObject (to-db-object input)
inner ^DBObject (.get output "map")]
(is (= 10 (.get inner "int")))
(is (= "Clojure" (.get inner "string")))
@ -54,19 +58,19 @@
;; to obtain _id that was generated. MK.
(deftest convert-dbobject-to-dbobject
(let [input (BasicDBObject.)
output (cnv/to-db-object input)]
output (to-db-object input)]
(is (= input output))))
(deftest convert-java-date-to-dbobject
(let [date (Date.)
input { :int 1, :string "Mongo", :date date }
output ^DBObject (cnv/to-db-object input)]
output ^DBObject (to-db-object input)]
(is (= date (.get output "date")))))
(deftest convert-java-calendar-instance-to-dbobject
(let [date (Calendar/getInstance)
input { :int 1, :string "Mongo", :date date }
output ^DBObject (cnv/to-db-object input)]
output ^DBObject (to-db-object input)]
(is (= date (.get output "date")))))
@ -77,16 +81,16 @@
;;
(deftest convert-nil-from-db-object
(is (nil? (cnv/from-db-object nil false)))
(is (nil? (cnv/from-db-object nil true))))
(is (nil? (from-db-object nil false)))
(is (nil? (from-db-object nil true))))
(deftest convert-integer-from-dbobject
(is (= 2 (cnv/from-db-object 2 false)))
(is (= 2 (cnv/from-db-object 2 true))))
(is (= 2 (from-db-object 2 false)))
(is (= 2 (from-db-object 2 true))))
(deftest convert-float-from-dbobject
(is (= 3.3 (cnv/from-db-object 3.3 false)))
(is (= 3.3 (cnv/from-db-object 3.3 true))))
(is (= 3.3 (from-db-object 3.3 false)))
(is (= 3.3 (from-db-object 3.3 true))))
(deftest convert-flat-db-object-to-map-without-keywordizing
(let [name "Michael"
@ -94,7 +98,7 @@
input (doto (BasicDBObject.)
(.put "name" name)
(.put "age" age))
output (cnv/from-db-object input false)]
output (from-db-object input false)]
(is (= (output { "name" name, "age" age })))
(is (= (output "name") name))
(is (nil? (output :name)))
@ -107,7 +111,7 @@
input (doto (BasicDBObject.)
(.put "name" name)
(.put "age" age))
output (cnv/from-db-object input true)]
output (from-db-object input true)]
(is (= (output { :name name, :age age })))
(is (= (output :name) name))
(is (nil? (output "name")))
@ -126,7 +130,7 @@
input (doto (BasicDBObject.)
(.put "_id" did)
(.put "nested" nested))
output (cnv/from-db-object input false)]
output (from-db-object input false)]
(is (= (output "_id") did))
(is (= (-> output (get "nested") (get "int")) 101))
(is (= (-> output (get "nested") (get "list")) ["red" "green" "blue"]))
@ -140,5 +144,18 @@
(deftest test-conversion-to-object-id
(let [output (ObjectId. "4efb39370364238a81020502")]
(is (= output (cnv/to-object-id "4efb39370364238a81020502")))
(is (= output (cnv/to-object-id output)))))
(is (= output (to-object-id "4efb39370364238a81020502")))
(is (= output (to-object-id output)))))
;;
;; Field selector coercion
;;
(deftest test-field-selector-coercion
(are [i o] (is (= (from-db-object (as-field-selector i) true) o))
[:a :b :c] {:a 1 :b 1 :c 1}
'(:a :b :c) {:a 1 :b 1 :c 1}
{:a 1 :b 1 :c 1} {:a 1 :b 1 :c 1}
{"a" 1 "b" 1 "c" 1} {:a 1 :b 1 :c 1}
{:comments 0} {:comments 0}))

View file

@ -23,6 +23,30 @@
(let [connection (monger.core/connect { :host "127.0.0.1" })]
(is (instance? com.mongodb.Mongo connection))))
(deftest connect-to-mongo-via-uri-without-credentials
(let [connection (monger.core/connect-via-uri! "mongodb://127.0.0.1/monger-test4")]
(is (= (-> connection .getAddress (.sameHost "127.0.0.1")))))
;; reconnect using regular host
(helper/connect!))
(deftest connect-to-mongo-via-uri-with-valid-credentials
(let [connection (monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:monger!@127.0.0.1/monger-test4")]
(is (= (-> connection .getAddress (.sameHost "127.0.0.1")))))
;; reconnect using regular host
(helper/connect!))
(if-let [uri (System/getenv "MONGOHQ_URL")]
(deftest ^{:external true} connect-to-mongo-via-uri-with-valid-credentials
(let [connection (monger.core/connect-via-uri! uri)]
(is (= (-> connection .getAddress (.sameHost "127.0.0.1")))))
;; reconnect using regular host
(helper/connect!)))
(deftest connect-to-mongo-via-uri-with-invalid-credentials
(is (thrown? IllegalArgumentException
(monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:ahsidaysd78jahsdi8@127.0.0.1/monger-test4"))))
(deftest test-mongo-options-builder
(let [max-wait-time (* 1000 60 2)

View file

@ -1,6 +1,6 @@
(ns monger.test.factory-dsl
(:use [clojure.test]
[monger testing joda-time]
[monger testkit joda-time]
[monger.test.fixtures]
[clj-time.core :only [days ago weeks now]])
(:require [monger.collection :as mc]

View file

@ -1,6 +1,6 @@
(ns monger.test.fixtures
(:require [monger.collection :as mgcol])
(:use [monger.testing]))
(:use monger.testkit))
;;
;; fixture functions

View file

@ -0,0 +1,35 @@
(ns monger.test.map-reduce
(:import org.bson.types.ObjectId
java.util.Date)
(:require [monger core util]
[monger.collection :as mgcol]
[monger.result :as mgres]
[monger.test.helper :as helper])
(:use clojure.test
[monger operators conversion]
monger.test.fixtures))
(helper/connect!)
;;
;; indexes
;;
(deftest ^{:indexing true} test-creating-and-dropping-indexes
(let [collection "libraries"]
(mgcol/drop-indexes collection)
(is (= "_id_"
(:name (first (mgcol/indexes-on collection)))))
(is (nil? (second (mgcol/indexes-on collection))))
(mgcol/create-index collection { "language" 1 })
(is (= "language_1"
(:name (second (mgcol/indexes-on collection)))))
(mgcol/drop-index collection "language_1")
(is (nil? (second (mgcol/indexes-on collection))))
(mgcol/ensure-index collection { "language" 1 } {:unique true})
(is (= "language_1"
(:name (second (mgcol/indexes-on collection)))))
(mgcol/ensure-index collection { "language" 1 })
(mgcol/ensure-index collection { "language" 1 } { :unique true })
(mgcol/drop-indexes collection)))

View file

@ -55,6 +55,24 @@
result (mgcol/insert "people" doc)]
(is (= id (monger.util/get-id doc)))))
(deftest insert-a-document-with-clojure-ratio-in-it
(let [collection "widgets"
id (ObjectId.)
doc { :ratio 11/2 "_id" id }
result (mgcol/insert "widgets" doc)]
(is (= 5.5 (:ratio (mgcol/find-map-by-id collection id))))))
(defrecord Metrics
[rps eps])
(deftest ^:focus insert-a-document-with-clojure-record-in-it
(let [collection "widgets"
id (ObjectId.)
doc { :record (Metrics. 10 20) "_id" id }
result (mgcol/insert "widgets" doc)]
(is (= {:rps 10 :eps 20} (:record (mgcol/find-map-by-id collection id))))))
;;

View file

@ -0,0 +1,69 @@
(ns monger.test.map-reduce
(:import [com.mongodb WriteResult WriteConcern DBCursor DBObject CommandResult$CommandFailure MapReduceOutput MapReduceCommand MapReduceCommand$OutputType]
org.bson.types.ObjectId
java.util.Date)
(:require [monger core util]
[monger.collection :as mgcol]
[monger.result :as mgres]
[monger.js :as js]
[monger.test.helper :as helper])
(:use clojure.test
[monger operators conversion]
[monger.test.fixtures]))
(helper/connect!)
(use-fixtures :each purge-people purge-docs purge-things purge-libraries)
;;
;; Map/Reduce
;;
(let [collection "widgets"
mapper (js/load-resource "resources/mongo/js/mapfun1.js")
reducer "function(key, values) {
var result = 0;
values.forEach(function(v) { result += v });
return result;
}"
batch [{ :state "CA" :quantity 1 :price 199.00 }
{ :state "NY" :quantity 2 :price 199.00 }
{ :state "NY" :quantity 1 :price 299.00 }
{ :state "IL" :quantity 2 :price 11.50 }
{ :state "CA" :quantity 2 :price 2.95 }
{ :state "IL" :quantity 3 :price 5.50 }]
expected [{:_id "CA", :value 204.9} {:_id "IL", :value 39.5} {:_id "NY", :value 697.0}]]
(deftest test-basic-inline-map-reduce-example
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(let [output (mgcol/map-reduce collection mapper reducer nil MapReduceCommand$OutputType/INLINE {})
results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
(mgres/ok? output)
(is (= expected results))))
(deftest test-basic-map-reduce-example-that-replaces-named-collection
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(let [output (mgcol/map-reduce collection mapper reducer "mr_outputs" {})
results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
(mgres/ok? output)
(is (= 3 (monger.core/count results)))
(is (= expected
(map #(from-db-object % true) (seq results))))
(is (= expected
(map #(from-db-object % true) (mgcol/find "mr_outputs"))))
(.drop ^MapReduceOutput output)))
(deftest test-basic-map-reduce-example-that-merged-results-into-named-collection
(mgcol/remove monger.core/*mongodb-database* collection {})
(is (mgres/ok? (mgcol/insert-batch collection batch)))
(mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})
(is (mgres/ok? (mgcol/insert collection { :state "OR" :price 17.95 :quantity 4 })))
(let [^MapReduceOutput output (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})]
(mgres/ok? output)
(is (= 4 (monger.core/count output)))
(is (= ["CA" "IL" "NY" "OR"]
(map :_id (mgcol/find-maps "merged_mr_outputs"))))
(.drop ^MapReduceOutput output))))

View file

@ -54,21 +54,31 @@
(let [collection "docs"
doc-id (monger.util/random-uuid)
doc { :data-store "MongoDB", :language "Clojure", :_id doc-id }
fields [:language]
_ (mgcol/insert collection doc)
loaded (mgcol/find-one collection { :language "Clojure" } fields)]
loaded (mgcol/find-one collection { :language "Clojure" } [:language])]
(is (nil? (.get ^DBObject loaded "data-store")))
(is (= doc-id (monger.util/get-id loaded)))
(is (= "Clojure" (.get ^DBObject loaded "language")))))
(deftest find-one-partial-document-using-field-negation-when-collection-has-matches
(let [collection "docs"
doc-id (monger.util/random-uuid)
doc { :data-store "MongoDB", :language "Clojure", :_id doc-id }
_ (mgcol/insert collection doc)
^DBObject loaded (mgcol/find-one collection { :language "Clojure" } {:data-store 0 :_id 0})]
(is (nil? (.get loaded "data-store")))
(is (nil? (.get loaded "_id")))
(is (nil? (monger.util/get-id loaded)))
(is (= "Clojure" (.get loaded "language")))))
(deftest find-one-partial-document-as-map-when-collection-has-matches
(let [collection "docs"
doc-id (monger.util/random-uuid)
doc { :data-store "MongoDB", :language "Clojure", :_id doc-id }
fields [:data-store]]
doc { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
(mgcol/insert collection doc)
(is (= { :data-store "MongoDB", :_id doc-id } (mgcol/find-one-as-map collection { :language "Clojure" } fields)))))
(is (= { :data-store "MongoDB", :_id doc-id } (mgcol/find-one-as-map collection { :language "Clojure" } [:data-store])))))
(deftest find-one-partial-document-as-map-when-collection-has-matches-with-keywordize

View file

@ -30,7 +30,7 @@
(monger.core/set-default-write-concern! WriteConcern/NORMAL)
(deftest insert-large-batches-of-documents-without-object-ids
(deftest ^{:performance true} insert-large-batches-of-documents-without-object-ids
(doseq [n [1000 10000 100000]]
(let [collection "things"
docs (map (fn [i]

View file

@ -44,6 +44,17 @@
(mgcol/update-by-id collection doc-id { :language "Erlang" })
(is (= (modified-doc (mgcol/find-by-id collection doc-id))))))
(deftest update-nested-document-fields-without-upsert-using-update-by-id
(let [collection "libraries"
doc-id (ObjectId.)
date (Date.)
doc { :created-at date :data-store "MongoDB" :language { :primary "Clojure" } :_id doc-id }
modified-doc { :created-at date :data-store "MongoDB" :language { :primary "Erlang" } :_id doc-id }]
(mgcol/insert collection doc)
(is (= (doc (mgcol/find-by-id collection doc-id))))
(mgcol/update-by-id collection doc-id { $set { "language.primary" "Erlang" }})
(is (= (modified-doc (mgcol/find-by-id collection doc-id))))))
(deftest update-multiple-documents
(let [collection "libraries"]