diff --git a/ChangeLog.md b/ChangeLog.md index ce145b4..b01885c 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,5 +1,70 @@ +## Changes between 1.0.0-beta4 and 1.0.0-beta5 + +No changes yet. + + + +## Changes between 1.0.0-beta3 and 1.0.0-beta4 + +### Support for URI connections (and thus PaaS provides like Heroku) + +`monger.core/connect-via-uri!` is a new function that combines `monger.core/connect!`, `monger.core/set-db!` and `monger.core/authenticate` +and works with string URIs like `mongodb://userb71148a:0da0a696f23a4ce1ecf6d11382633eb2049d728e@cluster1.mongohost.com:27034/app81766662`. + +It can be used to connect with or without authentication, for example: + +``` clojure +;; connect without authentication +(monger.core/connect-via-uri! "mongodb://127.0.0.1/monger-test4") + +;; connect with authentication +(monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:monger!@127.0.0.1/monger-test4") + +;; connect using connection URI stored in an env variable, in this case, MONGOHQ_URL +(monger.core/connect-via-uri! (System/genenv "MONGOHQ_URL")) +``` + +It is also possible to pass connection options and query parameters: + +``` clojure +(monger.core/connect-via-uri! "mongodb://localhost/test?maxPoolSize=128&waitQueueMultiple=5;waitQueueTimeoutMS=150;socketTimeoutMS=5500&autoConnectRetry=true;safe=false&w=1;wtimeout=2500;fsync=true") +``` + + ## Changes between 1.0.0-beta2 and 1.0.0-beta3 +### Support for field negation in queries + +Previously to load only a subset of document fields with Monger, one had to specify them all. Starting +with 1.0.0-beta3, Monger supports [field negation](http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields#RetrievingaSubsetofFields-FieldNegation) feature of MongoDB: it is possible to exclude +certain fields instead. + +To do so, pass a map as field selector, with fields that should be omitted set to 0: + +``` clojure +;; will retrieve all fields except body +(monger.collection/find-one-map "documents" {:author "John"} {:body 0}) +``` + + +### Validateur 1.1.0-beta1 + +[Validateur](https://github.com/michaelklishin/validateur) dependency has been upgraded to 1.1.0-beta1. + + +### Index Options support for monger.collection/ensure-index and /create-index + +`monger.collection/ensure-index` and `/create-index` now accept index options as additional argument. +**Breaking change**: 3-arity versions of those functions now become 4-arity versions. + + +### Support serialization of Clojure ratios + +Documents that contain Clojure ratios (for example, `26/5`) now can be converted to DBObject instances +and thus stored. On load, ratios will be presented as doubles: this way we ensure interoperability with +other languages and clients. + + ### Factories/fixtures DSL When working with even moderately complex data sets, fixture data quickly becomes difficult to diff --git a/README.md b/README.md index 0ca1f13..fd59e41 100644 --- a/README.md +++ b/README.md @@ -15,11 +15,12 @@ wanted a client that will * Be well documented. * Be well tested. * Be maintained, do not carry technical debt from 2009 forever. + * Target Clojure 1.3.0 and later from the ground up. * Integrate with libraries like clojure.data.json and Joda Time. * Provide support for unit testing: factories/fixtures DSL, collection cleaner functions, clojure.test integration and so on. - * Integrate usage of JavaScript files and ClojureScript (as soon as the compiler gets artifact it is possible to depend on for easy embedding). + * Support URI connections to be friendly to Heroku and other PaaS providers. * Learn from other clients like the Java and Ruby ones. - * Target Clojure 1.3.0 and later from the ground up. + * Integrate usage of JavaScript files and ClojureScript (as soon as the compiler gets artifact it is possible to depend on for easy embedding). ## Documentation & Examples @@ -45,7 +46,7 @@ together with documentation guides and dedicated website. With Leiningen: - [com.novemberain/monger "1.0.0-beta2"] + [com.novemberain/monger "1.0.0-beta4"] With Maven: @@ -53,7 +54,7 @@ With Maven: com.novemberain monger - 1.0.0-beta2 + 1.0.0-beta4 @@ -223,7 +224,7 @@ Here is what monger.query DSL feels like: (with-collection "movies" (find { :year { $lt 2010 $gte 2000 }, :revenue { $gt 20000000 } }) (fields [ :year :title :producer :cast :budget :revenue ]) - (sort-by { :revenue -1 }) + (sort { :revenue -1 }) (skip 10) (limit 20) (hint "year-by-year-revenue-idx") @@ -362,7 +363,7 @@ Neocons is part of the group of libraries known as ClojureWerkz, together with ## Development Monger uses [Leiningen 2](https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md). Make sure you have it installed and then run tests against -Clojure 1.3.0 and 1.4.0[-beta4] using +supported Clojure versions using lein2 all test diff --git a/project.clj b/project.clj index b146cd0..7674642 100644 --- a/project.clj +++ b/project.clj @@ -4,27 +4,27 @@ :license {:name "Eclipse Public License"} :dependencies [[org.clojure/clojure "1.3.0"] [org.mongodb/mongo-java-driver "2.7.3"] - [com.novemberain/validateur "1.0.0"]] - :test-selectors {:focus (fn [v] (:focus v))} + [com.novemberain/validateur "1.1.0-beta1"]] + :test-selectors {:default (complement :performance) + :focus :focus + :indexing :indexing + :external :external + :performance :performance + :all (constantly true)} :codox {:exclude [monger.internal.pagination]} - :mailing-list {:name "clojure-monger", - :archive "https://groups.google.com/group/clojure-monger", + :mailing-list {:name "clojure-monger" + :archive "https://groups.google.com/group/clojure-monger" :post "clojure-monger@googlegroups.com"} - :profiles {:1.4 {:resource-paths ["test/resources"], - :dependencies [[org.clojure/clojure "1.4.0-beta5"]]}, - :dev {:resource-paths ["test/resources"], - :dependencies [[org.mongodb/mongo-java-driver "2.7.3"] - [com.novemberain/validateur "1.0.0"] - [clj-time "0.3.6" :exclusions [org.clojure/clojure]] - [codox "0.3.4" :exclusions [org.clojure/clojure]] - [org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]] - [org.clojure/data.json "0.1.2" :exclusions [org.clojure/clojure]] - [clj-time "0.3.6" :exclusions [org.clojure/clojure]] - [org.clojure/core.cache "0.5.0" :exclusions [org.clojure/clojure]] - [codox "0.3.4" :exclusions [org.clojure/clojure]] - [org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]]]}} + :profiles {:1.4 {:resource-paths ["test/resources"] + :dependencies [[org.clojure/clojure "1.4.0-beta7"]]} + :dev {:resource-paths ["test/resources"] + :dependencies [[clj-time "0.3.6" :exclusions [org.clojure/clojure]] + [codox "0.3.4" :exclusions [org.clojure/clojure]] + [org.clojure/data.json "0.1.2" :exclusions [org.clojure/clojure]] + [org.clojure/tools.cli "0.2.1" :exclusions [org.clojure/clojure]] + [org.clojure/core.cache "0.5.0" :exclusions [org.clojure/clojure]]]}} :aliases { "all" ["with-profile" "dev:dev,1.4"] } - :repositories {"sonatype" {:url "http://oss.sonatype.org/content/repositories/releases", - :snapshots false, - :releases {:checksum :fail, :update :always}}} + :repositories {"sonatype" {:url "http://oss.sonatype.org/content/repositories/releases" + :snapshots false + :releases {:checksum :fail :update :always}}} :warn-on-reflection true) \ No newline at end of file diff --git a/src/monger/collection.clj b/src/monger/collection.clj index be130ff..ce6afca 100644 --- a/src/monger/collection.clj +++ b/src/monger/collection.clj @@ -21,10 +21,6 @@ ;; Implementation ;; -(defn- fields-to-db-object - [^List fields] - (zipmap fields (repeat 1))) - (definline check-not-nil! [ref ^String message] `(when (nil? ~ref) @@ -104,13 +100,13 @@ ([^String collection ^Map ref] (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)] (.find ^DBCollection coll ^DBObject (to-db-object ref)))) - ([^String collection ^Map ref ^List fields] + ([^String collection ^Map ref fields] (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection) - map-of-fields (fields-to-db-object fields)] + map-of-fields (as-field-selector fields)] (.find ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields)))) - ([^DB db ^String collection ^Map ref ^List fields] + ([^DB db ^String collection ^Map ref fields] (let [^DBCollection coll (.getCollection db collection) - map-of-fields (fields-to-db-object fields)] + map-of-fields (as-field-selector fields)] (.find ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields))))) (defn ^ISeq find-maps @@ -122,9 +118,9 @@ (map (fn [x] (from-db-object x true)) (find collection))) ([^String collection ^Map ref] (map (fn [x] (from-db-object x true)) (find collection ref))) - ([^String collection ^Map ref ^List fields] + ([^String collection ^Map ref fields] (map (fn [x] (from-db-object x true)) (find collection ref fields))) - ([^DB db ^String collection ^Map ref ^List fields] + ([^DB db ^String collection ^Map ref fields] (map (fn [x] (from-db-object x true)) (find db collection ref fields)))) (defn ^ISeq find-seq @@ -133,9 +129,9 @@ (seq (find collection))) ([^String collection ^Map ref] (seq (find collection ref))) - ([^String collection ^Map ref ^List fields] + ([^String collection ^Map ref fields] (seq (find collection ref fields))) - ([^DB db ^String collection ^Map ref ^List fields] + ([^DB db ^String collection ^Map ref fields] (seq (find db collection ref fields)))) ;; @@ -157,22 +153,22 @@ ([^String collection ^Map ref] (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)] (.findOne ^DBCollection coll ^DBObject (to-db-object ref)))) - ([^String collection ^Map ref ^List fields] + ([^String collection ^Map ref fields] (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection) - map-of-fields (fields-to-db-object fields)] + map-of-fields (as-field-selector fields)] (.findOne ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields)))) - ([^DB db ^String collection ^Map ref ^List fields] + ([^DB db ^String collection ^Map ref fields] (let [^DBCollection coll (.getCollection db collection) - map-of-fields (fields-to-db-object fields)] + map-of-fields (as-field-selector fields)] (.findOne ^DBCollection coll ^DBObject (to-db-object ref) ^DBObject (to-db-object map-of-fields))))) (defn ^IPersistentMap find-one-as-map "Returns a single object converted to Map from this collection matching the query." ([^String collection ^Map ref] (from-db-object ^DBObject (find-one collection ref) true)) - ([^String collection ^Map ref ^List fields] + ([^String collection ^Map ref fields] (from-db-object ^DBObject (find-one collection ref fields) true)) - ([^String collection ^Map ref ^List fields keywordize] + ([^String collection ^Map ref fields keywordize] (from-db-object ^DBObject (find-one collection ref fields) keywordize))) @@ -195,10 +191,10 @@ ([^String collection id] (check-not-nil! id "id must not be nil") (find-one collection { :_id id })) - ([^String collection id ^List fields] + ([^String collection id fields] (check-not-nil! id "id must not be nil") (find-one collection { :_id id } fields)) - ([^DB db ^String collection id ^List fields] + ([^DB db ^String collection id fields] (check-not-nil! id "id must not be nil") (find-one db collection { :_id id } fields))) @@ -207,10 +203,10 @@ ([^String collection id] (check-not-nil! id "id must not be nil") (from-db-object ^DBObject (find-one-as-map collection { :_id id }) true)) - ([^String collection id ^List fields] + ([^String collection id fields] (check-not-nil! id "id must not be nil") (from-db-object ^DBObject (find-one-as-map collection { :_id id } fields) true)) - ([^String collection id ^List fields keywordize] + ([^String collection id fields keywordize] (check-not-nil! id "id must not be nil") (from-db-object ^DBObject (find-one-as-map collection { :_id id } fields) keywordize))) @@ -388,16 +384,20 @@ EXAMPLES - ;; Will create an index on \"language\" field + ;; Will create an index on the \"language\" field (monger.collection/create-index collection { \"language\" 1 }) + (monger.collection/create-index collection { \"language\" 1 } { :unique true :name \"unique_language\" }) " ([^String collection ^Map keys] (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)] (.createIndex coll (to-db-object keys)))) - ([^DB db ^String collection ^Map keys] + ([^String collection ^Map keys options] + (let [^DBCollection coll (.getCollection monger.core/*mongodb-database* collection)] + (.createIndex coll (to-db-object keys) (to-db-object options)))) + ([^DB db ^String collection ^Map keys ^Map options] (let [^DBCollection coll (.getCollection db collection)] - (.createIndex coll (to-db-object keys))))) + (.createIndex coll (to-db-object keys) (to-db-object options))))) ;; @@ -416,9 +416,12 @@ ([^String collection, ^Map keys] (let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)] (.ensureIndex ^DBCollection coll ^DBObject (to-db-object keys)))) - ([^String collection, ^Map keys, ^String name] + ([^String collection, ^Map keys ^Map options] (let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)] - (.ensureIndex coll ^DBObject (to-db-object keys) ^String name)))) + (.ensureIndex ^DBCollection coll ^DBObject (to-db-object keys) (to-db-object options)))) + ([^String collection ^Map keys ^String name ^Boolean unique?] + (let [coll ^DBCollection (.getCollection monger.core/*mongodb-database* collection)] + (.ensureIndex coll ^DBObject (to-db-object keys) ^String name unique?)))) ;; diff --git a/src/monger/conversion.clj b/src/monger/conversion.clj index 23ca31b..d5176d0 100644 --- a/src/monger/conversion.clj +++ b/src/monger/conversion.clj @@ -23,12 +23,12 @@ (ns monger.conversion (:import [com.mongodb DBObject BasicDBObject BasicDBList DBCursor] - [clojure.lang IPersistentMap Keyword] + [clojure.lang IPersistentMap Keyword Ratio] [java.util List Map Date] - [org.bson.types ObjectId])) + org.bson.types.ObjectId)) (defprotocol ConvertToDBObject - (to-db-object [input] "Converts given piece of Clojure data to BasicDBObject MongoDB Java driver uses")) + (^com.mongodb.DBObject to-db-object [input] "Converts given piece of Clojure data to BasicDBObject MongoDB Java driver uses")) (extend-protocol ConvertToDBObject nil @@ -39,6 +39,10 @@ (to-db-object [input] input) + Ratio + (to-db-object [^Ratio input] + (double input)) + Keyword (to-db-object [^Keyword input] (.getName input)) @@ -105,7 +109,7 @@ (defprotocol ConvertToObjectId - (to-object-id [input] "Instantiates ObjectId from input unless the input itself is an ObjectId instance. In that case, returns input as is.")) + (^org.bson.types.ObjectId to-object-id [input] "Instantiates ObjectId from input unless the input itself is an ObjectId instance. In that case, returns input as is.")) (extend-protocol ConvertToObjectId String @@ -121,3 +125,19 @@ input)) + +(defprotocol FieldSelector + (^com.mongodb.DBObject as-field-selector [input] "Converts values to DBObject that can be used to specify a list of document fields (including negation support)")) + +(extend-protocol FieldSelector + DBObject + (as-field-selector [^DBObject input] + input) + + List + (as-field-selector [^List input] + (to-db-object (zipmap input (repeat 1)))) + + Object + (as-field-selector [input] + (to-db-object input))) diff --git a/src/monger/core.clj b/src/monger/core.clj index 8584312..be62984 100644 --- a/src/monger/core.clj +++ b/src/monger/core.clj @@ -14,7 +14,7 @@ monger.core (:refer-clojure :exclude [count]) (:use [monger.conversion]) - (:import [com.mongodb Mongo DB WriteConcern DBObject DBCursor CommandResult Bytes MongoOptions ServerAddress] + (:import [com.mongodb Mongo MongoURI DB WriteConcern DBObject DBCursor CommandResult Bytes MongoOptions ServerAddress MapReduceOutput] [com.mongodb.gridfs GridFS] [java.util Map])) @@ -22,7 +22,7 @@ ;; Defaults ;; -(def ^:dynamic ^String *mongodb-host* "localhost") +(def ^:dynamic ^String *mongodb-host* "127.0.0.1") (def ^:dynamic ^long *mongodb-port* 27017) (declare ^:dynamic ^Mongo *mongodb-connection*) @@ -31,6 +31,7 @@ (declare ^:dynamic ^GridFS *mongodb-gridfs*) + ;; ;; API ;; @@ -51,11 +52,10 @@ (Mongo.)) ([^ServerAddress server-address ^MongoOptions options] (Mongo. server-address options)) - ([{ :keys [host port] :or { host *mongodb-host*, port *mongodb-port* }}] + ([{ :keys [host port uri] :or { host *mongodb-host* port *mongodb-port* }}] (Mongo. ^String host ^Long port))) - (defn ^DB get-db-names "Gets a list of all database names present on the server" ([] @@ -105,7 +105,7 @@ (defn server-address ([^String hostname] (ServerAddress. hostname)) - ([^String hostname ^long port] + ([^String hostname ^Long port] (ServerAddress. hostname port))) @@ -172,6 +172,32 @@ and WebScale fast second." (def ^:dynamic *mongodb-write-concern* wc)) + +(defn connect-via-uri! + "Connects to MongoDB using a URI, sets up default connection and database. Commonly used for PaaS-based applications, + for example, running on Heroku. If username and password are provided, performs authentication." + [uri] + (let [uri (MongoURI. uri) + ;; yes, you are not hallucinating. A class named MongoURI has a method called connectDB. + ;; I call it "college OOP". Or maybe "don't give a shit" OOP. + db (.connectDB uri) + conn (.getMongo db) + user (.getUsername uri) + pwd (.getPassword uri)] + ;; I hope that whoever wrote the MongoDB Java driver connection/authentication parts + ;; wasn't sober while at it. MK. + ;; + ;; First we set connection, then DB, then authentcate + (set-connection! conn) + (when (and user pwd) + (when-not (authenticate (.getName db) user pwd) + (throw (IllegalArgumentException. "Could not authenticate. Either database name or credentials are invalid.")))) + ;; only do this *after* we authenticated because set-db! will try to set up a default GridFS instance. MK. + (when db + (set-db! db)) + conn)) + + (defn ^CommandResult command "Runs a database command (please check MongoDB documentation for the complete list of commands). Some common commands are: @@ -237,7 +263,11 @@ (extend-protocol Countable DBCursor (count [^DBCursor this] - (.count this))) + (.count this)) + + MapReduceOutput + (count [^MapReduceOutput this] + (.count (.results this)))) (defn ^DBObject get-last-error "Returns the the error (if there is one) from the previous operation on this connection. diff --git a/src/monger/query.clj b/src/monger/query.clj index 0b7a64c..c9224e3 100644 --- a/src/monger/query.clj +++ b/src/monger/query.clj @@ -31,7 +31,7 @@ ;; ;; Existing query fields: ;; -;; :fields - selects which fields are returned. The default is all fields. _id is always returned. +;; :fields - selects which fields are returned. The default is all fields. _id is included by default. ;; :sort - adds a sort to the query. ;; :fields - set of fields to retrieve during query execution ;; :skip - Skips the first N results. @@ -58,13 +58,9 @@ ([^DBCollection coll] (merge (empty-query) { :collection coll }))) -(defn- fields-to-db-object - [^List fields] - (to-db-object (zipmap fields (repeat 1)))) - (defn exec [{ :keys [collection query fields skip limit sort batch-size hint snapshot read-preference keywordize-fields] :or { limit 0 batch-size 256 skip 0 } }] - (let [cursor (doto ^DBCursor (.find ^DBCollection collection (to-db-object query) (fields-to-db-object fields)) + (let [cursor (doto ^DBCursor (.find ^DBCollection collection (to-db-object query) (as-field-selector fields)) (.limit limit) (.skip skip) (.sort (to-db-object sort)) diff --git a/src/monger/testing.clj b/src/monger/testkit.clj similarity index 99% rename from src/monger/testing.clj rename to src/monger/testkit.clj index 89c9aa0..64cebad 100644 --- a/src/monger/testing.clj +++ b/src/monger/testkit.clj @@ -7,7 +7,7 @@ ;; the terms of this license. ;; You must not remove this notice, or any other, from this software. -(ns monger.testing +(ns monger.testkit (:require [monger.collection :as mc] [monger.result :as mr]) (:use [monger.internal.fn :only (expand-all expand-all-with) :as fntools]) diff --git a/test/monger/test/collection.clj b/test/monger/test/collection.clj index 1ac1650..65e6935 100644 --- a/test/monger/test/collection.clj +++ b/test/monger/test/collection.clj @@ -2,18 +2,15 @@ (ns monger.test.collection (:import [com.mongodb WriteResult WriteConcern DBCursor DBObject CommandResult$CommandFailure MapReduceOutput MapReduceCommand MapReduceCommand$OutputType] - [org.bson.types ObjectId] - [java.util Date]) + org.bson.types.ObjectId + java.util.Date) (:require [monger core util] - [clojure stacktrace] - [monger.collection :as mgcol] - [monger.result :as mgres] - [monger.conversion :as mgcnv] - [monger.js :as js] + [monger.collection :as mgcol] + [monger.result :as mgres] [monger.test.helper :as helper]) - (:use [clojure.test] - [monger.operators] - [monger.test.fixtures])) + (:use clojure.test + monger.operators + monger.test.fixtures)) (helper/connect!) @@ -71,27 +68,6 @@ (is (nil? (mgcol/find-by-id collection oid))))) -;; -;; indexes -;; - -(deftest index-operations - (let [collection "libraries"] - (mgcol/drop-indexes collection) - (is (= "_id_" - (:name (first (mgcol/indexes-on collection))))) - (is (nil? (second (mgcol/indexes-on collection)))) - (mgcol/create-index collection { "language" 1 }) - (is (= "language_1" - (:name (second (mgcol/indexes-on collection))))) - (mgcol/drop-index collection "language_1") - (is (nil? (second (mgcol/indexes-on collection)))) - (mgcol/ensure-index collection { "language" 1 }) - (is (= "language_1" - (:name (second (mgcol/indexes-on collection))))) - (mgcol/ensure-index collection { "language" 1 }))) - - ;; ;; exists?, drop, create ;; @@ -116,65 +92,36 @@ (is (mgcol/exists? "gadgets")) (mgcol/drop "gadgets"))) - ;; -;; Map/Reduce +;; any?, empty? ;; -(let [collection "widgets" - mapper (js/load-resource "resources/mongo/js/mapfun1.js") - reducer "function(key, values) { - var result = 0; - values.forEach(function(v) { result += v }); +(deftest test-any-on-empty-collection + (let [collection "things"] + (is (not (mgcol/any? collection))))) - return result; - }" - batch [{ :state "CA" :quantity 1 :price 199.00 } - { :state "NY" :quantity 2 :price 199.00 } - { :state "NY" :quantity 1 :price 299.00 } - { :state "IL" :quantity 2 :price 11.50 } - { :state "CA" :quantity 2 :price 2.95 } - { :state "IL" :quantity 3 :price 5.50 }] - expected [{:_id "CA", :value 204.9} {:_id "IL", :value 39.5} {:_id "NY", :value 697.0}]] - (deftest basic-inline-map-reduce-example - (mgcol/remove monger.core/*mongodb-database* collection {}) - (is (mgres/ok? (mgcol/insert-batch collection batch))) - (let [output (mgcol/map-reduce collection mapper reducer nil MapReduceCommand$OutputType/INLINE {}) - results (mgcnv/from-db-object ^DBObject (.results ^MapReduceOutput output) true)] - (mgres/ok? output) - (is (= expected results)))) +(deftest test-any-on-non-empty-collection + (let [collection "things" + _ (mgcol/insert collection { :language "Clojure", :name "langohr" })] + (is (mgcol/any? "things")) + (is (mgcol/any? monger.core/*mongodb-database* "things" {:language "Clojure"})))) - (deftest basic-map-reduce-example-that-replaces-named-collection - (mgcol/remove monger.core/*mongodb-database* collection {}) - (is (mgres/ok? (mgcol/insert-batch collection batch))) - (let [output (mgcol/map-reduce collection mapper reducer "mr_outputs" {}) - results (mgcnv/from-db-object ^DBObject (.results ^MapReduceOutput output) true)] - (mgres/ok? output) - (is (= 3 (monger.core/count results))) - (is (= expected - (map #(mgcnv/from-db-object % true) (seq results)))) - (is (= expected - (map #(mgcnv/from-db-object % true) (mgcol/find "mr_outputs")))) - (.drop ^MapReduceOutput output))) +(deftest test-empty-on-empty-collection + (let [collection "things"] + (is (mgcol/empty? collection)) + (is (mgcol/empty? monger.core/*mongodb-database* collection)))) - (deftest basic-map-reduce-example-that-merged-results-into-named-collection - (mgcol/remove monger.core/*mongodb-database* collection {}) - (is (mgres/ok? (mgcol/insert-batch collection batch))) - (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {}) - (is (mgres/ok? (mgcol/insert collection { :state "OR" :price 17.95 :quantity 4 }))) - (let [output (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})] - (mgres/ok? output) - (is (= 4 (monger.core/count (.results ^MapReduceOutput output)))) - (is (= ["CA" "IL" "NY" "OR"] - (map :_id (mgcol/find-maps "merged_mr_outputs")))) - (.drop ^MapReduceOutput output)))) +(deftest test-empty-on-non-empty-collection + (let [collection "things" + _ (mgcol/insert collection { :language "Clojure", :name "langohr" })] + (is (not (mgcol/empty? "things"))))) ;; ;; distinct ;; -(deftest distinct-values +(deftest test-distinct-values (let [collection "widgets" batch [{ :state "CA" :quantity 1 :price 199.00 } { :state "NY" :quantity 2 :price 199.00 } @@ -185,28 +132,3 @@ (mgcol/insert-batch collection batch) (is (= ["CA" "IL" "NY"] (sort (mgcol/distinct monger.core/*mongodb-database* collection :state {})))) (is (= ["CA" "NY"] (sort (mgcol/distinct collection :state { :price { $gt 100.00 } })))))) - - -;; -;; any?, empty? -;; - -(deftest any-on-empty-collection - (let [collection "things"] - (is (not (mgcol/any? collection))))) - -(deftest any-on-non-empty-collection - (let [collection "things" - _ (mgcol/insert collection { :language "Clojure", :name "langohr" })] - (is (mgcol/any? "things")) - (is (mgcol/any? monger.core/*mongodb-database* "things" {:language "Clojure"})))) - -(deftest empty-on-empty-collection - (let [collection "things"] - (is (mgcol/empty? collection)) - (is (mgcol/empty? monger.core/*mongodb-database* collection)))) - -(deftest empty-on-non-empty-collection - (let [collection "things" - _ (mgcol/insert collection { :language "Clojure", :name "langohr" })] - (is (not (mgcol/empty? "things"))))) \ No newline at end of file diff --git a/test/monger/test/conversion.clj b/test/monger/test/conversion.clj index 2d15643..945f4f5 100644 --- a/test/monger/test/conversion.clj +++ b/test/monger/test/conversion.clj @@ -1,10 +1,9 @@ (ns monger.test.conversion - (:require [monger core collection] - [monger.conversion :as cnv]) + (:require [monger core collection]) (:import [com.mongodb DBObject BasicDBObject BasicDBList] [java.util Date Calendar List ArrayList] [org.bson.types ObjectId]) - (:use [clojure.test])) + (:use clojure.test monger.conversion)) ;; @@ -13,28 +12,33 @@ (deftest convert-nil-to-dbobject (let [input nil - output (cnv/to-db-object input)] + output (to-db-object input)] (is (nil? output)))) (deftest convert-integer-to-dbobject (let [input 1 - output (cnv/to-db-object input)] + output (to-db-object input)] (is (= input output)))) (deftest convert-float-to-dbobject (let [input 11.12 - output (cnv/to-db-object input)] + output (to-db-object input)] (is (= input output)))) +(deftest convert-rationale-to-dbobject + (let [input 11/2 + output (to-db-object input)] + (is (= 5.5 output)))) + (deftest convert-string-to-dbobject (let [input "MongoDB" - output (cnv/to-db-object input)] + output (to-db-object input)] (is (= input output)))) (deftest convert-map-to-dbobject (let [input { :int 1, :string "Mongo", :float 22.23 } - output ^DBObject (cnv/to-db-object input)] + output ^DBObject (to-db-object input)] (is (= 1 (.get output "int"))) (is (= "Mongo" (.get output "string"))) (is (= 22.23 (.get output "float"))))) @@ -42,7 +46,7 @@ (deftest convert-nested-map-to-dbobject (let [input { :int 1, :string "Mongo", :float 22.23, :map { :int 10, :string "Clojure", :float 11.9, :list '(1 "a" :b), :map { :key "value" } } } - output ^DBObject (cnv/to-db-object input) + output ^DBObject (to-db-object input) inner ^DBObject (.get output "map")] (is (= 10 (.get inner "int"))) (is (= "Clojure" (.get inner "string"))) @@ -54,19 +58,19 @@ ;; to obtain _id that was generated. MK. (deftest convert-dbobject-to-dbobject (let [input (BasicDBObject.) - output (cnv/to-db-object input)] + output (to-db-object input)] (is (= input output)))) (deftest convert-java-date-to-dbobject (let [date (Date.) input { :int 1, :string "Mongo", :date date } - output ^DBObject (cnv/to-db-object input)] + output ^DBObject (to-db-object input)] (is (= date (.get output "date"))))) (deftest convert-java-calendar-instance-to-dbobject (let [date (Calendar/getInstance) input { :int 1, :string "Mongo", :date date } - output ^DBObject (cnv/to-db-object input)] + output ^DBObject (to-db-object input)] (is (= date (.get output "date"))))) @@ -77,16 +81,16 @@ ;; (deftest convert-nil-from-db-object - (is (nil? (cnv/from-db-object nil false))) - (is (nil? (cnv/from-db-object nil true)))) + (is (nil? (from-db-object nil false))) + (is (nil? (from-db-object nil true)))) (deftest convert-integer-from-dbobject - (is (= 2 (cnv/from-db-object 2 false))) - (is (= 2 (cnv/from-db-object 2 true)))) + (is (= 2 (from-db-object 2 false))) + (is (= 2 (from-db-object 2 true)))) (deftest convert-float-from-dbobject - (is (= 3.3 (cnv/from-db-object 3.3 false))) - (is (= 3.3 (cnv/from-db-object 3.3 true)))) + (is (= 3.3 (from-db-object 3.3 false))) + (is (= 3.3 (from-db-object 3.3 true)))) (deftest convert-flat-db-object-to-map-without-keywordizing (let [name "Michael" @@ -94,7 +98,7 @@ input (doto (BasicDBObject.) (.put "name" name) (.put "age" age)) - output (cnv/from-db-object input false)] + output (from-db-object input false)] (is (= (output { "name" name, "age" age }))) (is (= (output "name") name)) (is (nil? (output :name))) @@ -107,7 +111,7 @@ input (doto (BasicDBObject.) (.put "name" name) (.put "age" age)) - output (cnv/from-db-object input true)] + output (from-db-object input true)] (is (= (output { :name name, :age age }))) (is (= (output :name) name)) (is (nil? (output "name"))) @@ -126,7 +130,7 @@ input (doto (BasicDBObject.) (.put "_id" did) (.put "nested" nested)) - output (cnv/from-db-object input false)] + output (from-db-object input false)] (is (= (output "_id") did)) (is (= (-> output (get "nested") (get "int")) 101)) (is (= (-> output (get "nested") (get "list")) ["red" "green" "blue"])) @@ -140,5 +144,18 @@ (deftest test-conversion-to-object-id (let [output (ObjectId. "4efb39370364238a81020502")] - (is (= output (cnv/to-object-id "4efb39370364238a81020502"))) - (is (= output (cnv/to-object-id output))))) + (is (= output (to-object-id "4efb39370364238a81020502"))) + (is (= output (to-object-id output))))) + + +;; +;; Field selector coercion +;; + +(deftest test-field-selector-coercion + (are [i o] (is (= (from-db-object (as-field-selector i) true) o)) + [:a :b :c] {:a 1 :b 1 :c 1} + '(:a :b :c) {:a 1 :b 1 :c 1} + {:a 1 :b 1 :c 1} {:a 1 :b 1 :c 1} + {"a" 1 "b" 1 "c" 1} {:a 1 :b 1 :c 1} + {:comments 0} {:comments 0})) diff --git a/test/monger/test/core.clj b/test/monger/test/core.clj index 29a5803..d0a8b95 100644 --- a/test/monger/test/core.clj +++ b/test/monger/test/core.clj @@ -23,6 +23,30 @@ (let [connection (monger.core/connect { :host "127.0.0.1" })] (is (instance? com.mongodb.Mongo connection)))) +(deftest connect-to-mongo-via-uri-without-credentials + (let [connection (monger.core/connect-via-uri! "mongodb://127.0.0.1/monger-test4")] + (is (= (-> connection .getAddress (.sameHost "127.0.0.1"))))) + ;; reconnect using regular host + (helper/connect!)) + +(deftest connect-to-mongo-via-uri-with-valid-credentials + (let [connection (monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:monger!@127.0.0.1/monger-test4")] + (is (= (-> connection .getAddress (.sameHost "127.0.0.1"))))) + ;; reconnect using regular host + (helper/connect!)) + +(if-let [uri (System/getenv "MONGOHQ_URL")] + (deftest ^{:external true} connect-to-mongo-via-uri-with-valid-credentials + (let [connection (monger.core/connect-via-uri! uri)] + (is (= (-> connection .getAddress (.sameHost "127.0.0.1"))))) + ;; reconnect using regular host + (helper/connect!))) + + +(deftest connect-to-mongo-via-uri-with-invalid-credentials + (is (thrown? IllegalArgumentException + (monger.core/connect-via-uri! "mongodb://clojurewerkz/monger!:ahsidaysd78jahsdi8@127.0.0.1/monger-test4")))) + (deftest test-mongo-options-builder (let [max-wait-time (* 1000 60 2) diff --git a/test/monger/test/factory_dsl.clj b/test/monger/test/factory_dsl.clj index 08f6364..cf6c577 100644 --- a/test/monger/test/factory_dsl.clj +++ b/test/monger/test/factory_dsl.clj @@ -1,6 +1,6 @@ (ns monger.test.factory-dsl (:use [clojure.test] - [monger testing joda-time] + [monger testkit joda-time] [monger.test.fixtures] [clj-time.core :only [days ago weeks now]]) (:require [monger.collection :as mc] diff --git a/test/monger/test/fixtures.clj b/test/monger/test/fixtures.clj index 3d3a79a..ec1ee49 100644 --- a/test/monger/test/fixtures.clj +++ b/test/monger/test/fixtures.clj @@ -1,6 +1,6 @@ (ns monger.test.fixtures (:require [monger.collection :as mgcol]) - (:use [monger.testing])) + (:use monger.testkit)) ;; ;; fixture functions diff --git a/test/monger/test/indexing.clj b/test/monger/test/indexing.clj new file mode 100644 index 0000000..250352c --- /dev/null +++ b/test/monger/test/indexing.clj @@ -0,0 +1,35 @@ +(ns monger.test.map-reduce + (:import org.bson.types.ObjectId + java.util.Date) + (:require [monger core util] + [monger.collection :as mgcol] + [monger.result :as mgres] + [monger.test.helper :as helper]) + (:use clojure.test + [monger operators conversion] + monger.test.fixtures)) + +(helper/connect!) + + +;; +;; indexes +;; + +(deftest ^{:indexing true} test-creating-and-dropping-indexes + (let [collection "libraries"] + (mgcol/drop-indexes collection) + (is (= "_id_" + (:name (first (mgcol/indexes-on collection))))) + (is (nil? (second (mgcol/indexes-on collection)))) + (mgcol/create-index collection { "language" 1 }) + (is (= "language_1" + (:name (second (mgcol/indexes-on collection))))) + (mgcol/drop-index collection "language_1") + (is (nil? (second (mgcol/indexes-on collection)))) + (mgcol/ensure-index collection { "language" 1 } {:unique true}) + (is (= "language_1" + (:name (second (mgcol/indexes-on collection))))) + (mgcol/ensure-index collection { "language" 1 }) + (mgcol/ensure-index collection { "language" 1 } { :unique true }) + (mgcol/drop-indexes collection))) diff --git a/test/monger/test/inserting.clj b/test/monger/test/inserting.clj index 366e97e..6538096 100644 --- a/test/monger/test/inserting.clj +++ b/test/monger/test/inserting.clj @@ -55,6 +55,24 @@ result (mgcol/insert "people" doc)] (is (= id (monger.util/get-id doc))))) +(deftest insert-a-document-with-clojure-ratio-in-it + (let [collection "widgets" + id (ObjectId.) + doc { :ratio 11/2 "_id" id } + result (mgcol/insert "widgets" doc)] + (is (= 5.5 (:ratio (mgcol/find-map-by-id collection id)))))) + + +(defrecord Metrics + [rps eps]) + +(deftest ^:focus insert-a-document-with-clojure-record-in-it + (let [collection "widgets" + id (ObjectId.) + doc { :record (Metrics. 10 20) "_id" id } + result (mgcol/insert "widgets" doc)] + (is (= {:rps 10 :eps 20} (:record (mgcol/find-map-by-id collection id)))))) + ;; diff --git a/test/monger/test/map_reduce.clj b/test/monger/test/map_reduce.clj new file mode 100644 index 0000000..7204fa8 --- /dev/null +++ b/test/monger/test/map_reduce.clj @@ -0,0 +1,69 @@ +(ns monger.test.map-reduce + (:import [com.mongodb WriteResult WriteConcern DBCursor DBObject CommandResult$CommandFailure MapReduceOutput MapReduceCommand MapReduceCommand$OutputType] + org.bson.types.ObjectId + java.util.Date) + (:require [monger core util] + [monger.collection :as mgcol] + [monger.result :as mgres] + [monger.js :as js] + [monger.test.helper :as helper]) + (:use clojure.test + [monger operators conversion] + [monger.test.fixtures])) + +(helper/connect!) + +(use-fixtures :each purge-people purge-docs purge-things purge-libraries) + + +;; +;; Map/Reduce +;; + +(let [collection "widgets" + mapper (js/load-resource "resources/mongo/js/mapfun1.js") + reducer "function(key, values) { + var result = 0; + values.forEach(function(v) { result += v }); + + return result; + }" + batch [{ :state "CA" :quantity 1 :price 199.00 } + { :state "NY" :quantity 2 :price 199.00 } + { :state "NY" :quantity 1 :price 299.00 } + { :state "IL" :quantity 2 :price 11.50 } + { :state "CA" :quantity 2 :price 2.95 } + { :state "IL" :quantity 3 :price 5.50 }] + expected [{:_id "CA", :value 204.9} {:_id "IL", :value 39.5} {:_id "NY", :value 697.0}]] + (deftest test-basic-inline-map-reduce-example + (mgcol/remove monger.core/*mongodb-database* collection {}) + (is (mgres/ok? (mgcol/insert-batch collection batch))) + (let [output (mgcol/map-reduce collection mapper reducer nil MapReduceCommand$OutputType/INLINE {}) + results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)] + (mgres/ok? output) + (is (= expected results)))) + + (deftest test-basic-map-reduce-example-that-replaces-named-collection + (mgcol/remove monger.core/*mongodb-database* collection {}) + (is (mgres/ok? (mgcol/insert-batch collection batch))) + (let [output (mgcol/map-reduce collection mapper reducer "mr_outputs" {}) + results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)] + (mgres/ok? output) + (is (= 3 (monger.core/count results))) + (is (= expected + (map #(from-db-object % true) (seq results)))) + (is (= expected + (map #(from-db-object % true) (mgcol/find "mr_outputs")))) + (.drop ^MapReduceOutput output))) + + (deftest test-basic-map-reduce-example-that-merged-results-into-named-collection + (mgcol/remove monger.core/*mongodb-database* collection {}) + (is (mgres/ok? (mgcol/insert-batch collection batch))) + (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {}) + (is (mgres/ok? (mgcol/insert collection { :state "OR" :price 17.95 :quantity 4 }))) + (let [^MapReduceOutput output (mgcol/map-reduce collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})] + (mgres/ok? output) + (is (= 4 (monger.core/count output))) + (is (= ["CA" "IL" "NY" "OR"] + (map :_id (mgcol/find-maps "merged_mr_outputs")))) + (.drop ^MapReduceOutput output)))) diff --git a/test/monger/test/regular_finders.clj b/test/monger/test/regular_finders.clj index 79f5753..f5af0eb 100644 --- a/test/monger/test/regular_finders.clj +++ b/test/monger/test/regular_finders.clj @@ -54,21 +54,31 @@ (let [collection "docs" doc-id (monger.util/random-uuid) doc { :data-store "MongoDB", :language "Clojure", :_id doc-id } - fields [:language] _ (mgcol/insert collection doc) - loaded (mgcol/find-one collection { :language "Clojure" } fields)] + loaded (mgcol/find-one collection { :language "Clojure" } [:language])] (is (nil? (.get ^DBObject loaded "data-store"))) (is (= doc-id (monger.util/get-id loaded))) (is (= "Clojure" (.get ^DBObject loaded "language"))))) +(deftest find-one-partial-document-using-field-negation-when-collection-has-matches + (let [collection "docs" + doc-id (monger.util/random-uuid) + doc { :data-store "MongoDB", :language "Clojure", :_id doc-id } + _ (mgcol/insert collection doc) + ^DBObject loaded (mgcol/find-one collection { :language "Clojure" } {:data-store 0 :_id 0})] + (is (nil? (.get loaded "data-store"))) + (is (nil? (.get loaded "_id"))) + (is (nil? (monger.util/get-id loaded))) + (is (= "Clojure" (.get loaded "language"))))) + + (deftest find-one-partial-document-as-map-when-collection-has-matches (let [collection "docs" doc-id (monger.util/random-uuid) - doc { :data-store "MongoDB", :language "Clojure", :_id doc-id } - fields [:data-store]] + doc { :data-store "MongoDB", :language "Clojure", :_id doc-id }] (mgcol/insert collection doc) - (is (= { :data-store "MongoDB", :_id doc-id } (mgcol/find-one-as-map collection { :language "Clojure" } fields))))) + (is (= { :data-store "MongoDB", :_id doc-id } (mgcol/find-one-as-map collection { :language "Clojure" } [:data-store]))))) (deftest find-one-partial-document-as-map-when-collection-has-matches-with-keywordize diff --git a/test/monger/test/stress.clj b/test/monger/test/stress.clj index e0d981b..9efd301 100644 --- a/test/monger/test/stress.clj +++ b/test/monger/test/stress.clj @@ -30,7 +30,7 @@ (monger.core/set-default-write-concern! WriteConcern/NORMAL) -(deftest insert-large-batches-of-documents-without-object-ids +(deftest ^{:performance true} insert-large-batches-of-documents-without-object-ids (doseq [n [1000 10000 100000]] (let [collection "things" docs (map (fn [i] diff --git a/test/monger/test/updating.clj b/test/monger/test/updating.clj index df86048..475e498 100644 --- a/test/monger/test/updating.clj +++ b/test/monger/test/updating.clj @@ -44,6 +44,17 @@ (mgcol/update-by-id collection doc-id { :language "Erlang" }) (is (= (modified-doc (mgcol/find-by-id collection doc-id)))))) +(deftest update-nested-document-fields-without-upsert-using-update-by-id + (let [collection "libraries" + doc-id (ObjectId.) + date (Date.) + doc { :created-at date :data-store "MongoDB" :language { :primary "Clojure" } :_id doc-id } + modified-doc { :created-at date :data-store "MongoDB" :language { :primary "Erlang" } :_id doc-id }] + (mgcol/insert collection doc) + (is (= (doc (mgcol/find-by-id collection doc-id)))) + (mgcol/update-by-id collection doc-id { $set { "language.primary" "Erlang" }}) + (is (= (modified-doc (mgcol/find-by-id collection doc-id)))))) + (deftest update-multiple-documents (let [collection "libraries"]