Merge pull request #235 from quelist/fix/update-before-scripts

Add MongoDB shell version detection to before scripts
2024-11-28 15:12:00 -05:00 · 2024-11-28 12:54:54 +05:30 · 2024-04-07 21:30:25 -04:00 · 2024-04-04 10:09:20 -03:00 · 2023-01-19 08:37:42 -06:00 · 2023-01-19 08:37:13 -06:00
47 changed files with 950 additions and 472 deletions
--- a/.gitignore
+++ b/.gitignore
@ -10,3 +10,7 @@ deploy.docs.sh
 target/*
 todo.org
 .nrepl-*
+.idea/
+*.iml
+/.clj-kondo/.cache
+/.lsp/.cache
--- a/.travis.yml
+++ b/.travis.yml
@ -1,15 +1,20 @@
 language: clojure
-lein: lein2
+sudo: required
+lein: lein
+dist: xenial
 before_script:
+  # Give MongoDB server some time to boot
+  - sleep 15
+  - mongod --version
  - ./bin/ci/before_script.sh
-script: lein2 do clean, javac, test
+script: lein do clean, javac, test
 jdk:
-  - openjdk7
-  - oraclejdk7
-  - oraclejdk8
+  - openjdk10
+  - oraclejdk11
+  - openjdk12
 services:
  - mongodb
 branches:
  only:
    - master
-    - 2.1.x-stable
+    - 3.5.x-stable
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,10 +1,11 @@
 ## Pre-requisites

-The project uses [Leiningen 2](https://leiningen.org) and requires MongoDB `2.4+` to be running
-locally. Make
-sure you have those two installed and then run tests against all supported Clojure versions using
+The project uses [Leiningen 2](http://leiningen.org) and requires a recent MongoDB to be running
+locally. Make sure you have those two installed and then run tests against all supported Clojure versions using
+
+    ./bin/ci/before_script.sh
+    lein all do clean, javac, test

-    lein2 all test

 ## Pull Requests

--- a/ChangeLog.md
+++ b/ChangeLog.md
@ -1,6 +1,86 @@
+## Changes between 3.5.x and 3.6.0 (unreleased)
+
+### UUID Representation Option
+
+Added a new connection option, `:uuid-representation`.
+
+Contributed by @okorz001.
+
+GitHub issue: [#212](https://github.com/michaelklishin/monger/issues/212)
+
+### Operator List Update
+
+For MongoDB 4.x.
+
+Contributed by @mjrb.
+
+GitHub issue: [#196](https://github.com/michaelklishin/monger/pull/196)
+
+### Dependency Update
+
+Contributed by @robhanlon22.
+
+GitHub issue: [#206](https://github.com/michaelklishin/monger/pull/206)
+
+
+## Changes between 3.1.x and 3.5.0 (Dec 10th, 2018)
+
+### MongoDB Java Driver Update
+
+MongoDB Java driver dependency has been updated to `3.9.x`.
+
+This means that Monger now **requires JDK 8**.
+
+Contributed by @Linicks.
+
+### 3rd Party Library Compatibility
+
+ * Cheshire `5.8.x`
+ * clj-time `0.15.1`
+ * ring-core `0.15.1`
+ * Ragtime `0.7.x`.
+
+### URI Connection Usability Improvement
+
+URIs that don't specify a database will now be rejected as invalid.
+
+Contributed by Chris Broome.
+
+
+## Changes between 3.0.x and 3.1.0 (September 17th, 2016)
+
+### MongoDB Java Driver Update
+
+MongoDB Java driver dependency has been updated to `3.3.0`.
+
+### Cursor Hinting Option Fix
+
+Contributed by Stijn Opheide.
+
+### Improved DBObject to Clojure Map conversion performance
+
+New `from-db-object` implementation for `DBObject` avoids creation of an unnecessary
+sequence and instead directly accesses `DBObject` instance in reduce. This should
+offer performance improvement of about 20%. A performance test can be found
+at [monger.test.stress-test](https://github.com/michaelklishin/monger/blob/master/test/monger/test/stress_test.clj).
+
+Contributed by Juho Teperi.
+
+### Authencation Function No Longer Ignores Credentials
+
+In some cases Monger ignored provided credentials.
+
+Contributed by Artem Chistyakov.
+
+### Macro Type Hint Fixes
+
+Contributed by Andre Ambrosio Boechat.
+
+
+
 ## Changes between 2.1.0 and 3.0.0

-Monger 3.0 is based on the [MongoDB Java driver 3.0]()
+Monger 3.0 is based on the [MongoDB Java driver 3.0](https://www.mongodb.com/blog/post/introducing-30-java-driver)
 and has some (relatively minor) **breaking API changes**.

 ### Error Handling Built Around Write Concerns
@ -54,6 +134,12 @@ An example of a search query in 3.0:
  (mc/find-maps db "collection" {$text {$search "hello"}}))
 ```

+### Add allow-disk-use and Cursor Options to Aggregates
+
+`monger.collection/aggregate` now supports `:cursor` and `:allow-disk-use` options.
+
+Contributed by Bartek Marcinowski.
+

 ### JSON Serialization of BSON Timestamps

--- a/README.md
+++ b/README.md
@ -1,16 +1,21 @@
 # Monger, a modern Clojure MongoDB Driver
-
+[![Build Status](https://travis-ci.org/xingzhefeng/monger.svg?branch=master)](https://travis-ci.org/xingzhefeng/monger)
 Monger is an idiomatic [Clojure MongoDB driver](http://clojuremongodb.info) for a more civilized age.

-It has batteries included, offers powerful expressive query DSL, strives to support every MongoDB 2.0+ feature and has sane defaults. Monger is built from for modern Clojure versions and sits on top of the official MongoDB Java driver.
+It has batteries included, offers powerful expressive query DSL,
+strives to support modern MongoDB features and have the "look and feel" and
+flexibility of the MongoDB shell.
+
+Monger is built from for modern Clojure versions and sits on top of
+the official MongoDB Java driver.


 ## Project Goals

 There is one MongoDB client for Clojure that has been around since 2009. So, why create another one? Monger authors
-wanted a client that will
+wanted a client that would

- * Support most of MongoDB 2.0+ features, focus on those that really matter.
+ * Support most of modern MongoDB features, focus on those that really matter.
 * Be [well documented](http://clojuremongodb.info).
 * Be [well tested](https://github.com/michaelklishin/monger/tree/master/test/monger/test).
 * Target modern Clojure versions.
@ -24,7 +29,7 @@ wanted a client that will

 ## Project Maturity

-Monger is not a young project: started in July 2011, it is over 3
+Monger is not a young project: started in July 2011, it is over 7
 years old with active production use from week 1.


@ -45,14 +50,14 @@ Maven, add the following repository definition to your `pom.xml`:

 With Leiningen:

-    [com.novemberain/monger "2.1.0"]
+    [com.novemberain/monger "3.5.0"]

 With Maven:

    <dependency>
      <groupId>com.novemberain</groupId>
      <artifactId>monger</artifactId>
-      <version>2.1.0</version>
+      <version>3.5.0</version>
    </dependency>


@ -70,15 +75,13 @@ questions, too!

 Please see our [documentation guides site](http://clojuremongodb.info/) and [API reference](http://reference.clojuremongodb.info).

-Our [test
-suite](https://github.com/michaelklishin/monger/tree/master/test/monger/test)
+Our [test suite](https://github.com/michaelklishin/monger/tree/master/test/monger/test)
 also has many code examples.


 ## Community

-[Monger has a mailing
-list](https://groups.google.com/forum/#!forum/clojure-mongodb). Feel
+[Monger has a mailing list](https://groups.google.com/forum/#!forum/clojure-mongodb). Feel
 free to join it and ask any questions you may have.

 To subscribe for announcements of releases, important changes and so
@ -88,20 +91,19 @@ on Twitter.

 ## Supported Clojure versions

-Monger requires Clojure 1.6+. The most recent
+Monger requires Clojure 1.8+. The most recent
 stable release is highly recommended.


 ## Continuous Integration Status

 [![Continuous Integration status](https://secure.travis-ci.org/michaelklishin/monger.svg)](http://travis-ci.org/michaelklishin/monger)
-[![Dependencies Status](http://jarkeeper.com/michaelklishin/monger/status.svg)](http://jarkeeper.com/michaelklishin/monger)


 ## Monger Is a ClojureWerkz Project

 Monger is part of the [group of Clojure libraries known as ClojureWerkz](http://clojurewerkz.org), together with
-[Cassaforte](http://clojurecassandra.info), [Langohr](http://clojurerabbitmq.info), [Elastisch](http://clojureelasticsearch.info), [Titanium](http://titanium.clojurewerkz.org), [Quartzite](http://clojurequartz.info) and several others.
+[Cassaforte](http://clojurecassandra.info), [Langohr](http://clojurerabbitmq.info), [Elastisch](http://clojureelasticsearch.info), [Quartzite](http://clojurequartz.info) and several others.



@ -110,6 +112,13 @@ Monger is part of the [group of Clojure libraries known as ClojureWerkz](http://
 Monger uses [Leiningen 2](https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md). Make sure you have it installed and then run tests against
 supported Clojure versions using

+    ./bin/ci/before_script.sh
+    lein all do clean, javac, test
+
+Or, if you don't have mongodb installed, you can use docker
+
+    docker-compose up
+    ./bin/ci/before_script_docker.sh
    lein all do clean, javac, test

 Then create a branch and make your changes on it. Once you are done with your changes and all tests pass, submit a pull request
@ -119,7 +128,7 @@ on Github.

 ## License

-Copyright (C) 2011-2015 [Michael S. Klishin](http://twitter.com/michaelklishin), Alex Petrov, and the ClojureWerkz team.
+Copyright (C) 2011-2018 [Michael S. Klishin](http://twitter.com/michaelklishin), Alex Petrov, and the ClojureWerkz team.

 Double licensed under the [Eclipse Public License](http://www.eclipse.org/legal/epl-v10.html) (the same as Clojure) or
 the [Apache Public License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html).
--- a/bin/ci/before_script.sh
+++ b/bin/ci/before_script.sh
@ -1,11 +1,18 @@
 #!/bin/sh

-# MongoDB seems to need some time to boot first. MK.
-sleep 5
+# Check which MongoDB shell is available
+if command -v mongosh >/dev/null 2>&1; then
+    MONGO_SHELL="mongosh"
+elif command -v mongo >/dev/null 2>&1; then
+    MONGO_SHELL="mongo"
+else
+    echo "Error: Neither mongo nor mongosh shell found. Please install MongoDB shell."
+    exit 1
+fi

 # MongoDB Java driver won't run authentication twice on the same DB instance,
 # so we need to use multiple DBs.
-mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"]})' monger-test
-mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"]})' monger-test2
-mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"]})' monger-test3
-mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"]})' monger-test4
+$MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test
+$MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test2
+$MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test3
+$MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test4
--- a/bin/ci/before_script_docker.sh
+++ b/bin/ci/before_script_docker.sh
@ -0,0 +1,18 @@
+#!/bin/sh
+
+# Check which MongoDB shell is available in the container
+if docker exec mongo_test which mongosh >/dev/null 2>&1; then
+    MONGO_SHELL="mongosh"
+elif docker exec mongo_test which mongo >/dev/null 2>&1; then
+    MONGO_SHELL="mongo"
+else
+    echo "Error: Neither mongo nor mongosh shell found in the container."
+    exit 1
+fi
+
+# MongoDB Java driver won't run authentication twice on the same DB instance,
+# so we need to use multiple DBs.
+docker exec mongo_test $MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test
+docker exec mongo_test $MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test2
+docker exec mongo_test $MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test3
+docker exec mongo_test $MONGO_SHELL --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], mechanisms: ["SCRAM-SHA-1"], passwordDigestor: "client"})' monger-test4
--- a/bin/ci/before_script_server_3.6.x.sh
+++ b/bin/ci/before_script_server_3.6.x.sh
@ -0,0 +1,11 @@
+#!/bin/sh
+
+# MongoDB seems to need some time to boot first. MK.
+sleep 5
+
+# MongoDB Java driver won't run authentication twice on the same DB instance,
+# so we need to use multiple DBs.
+mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], passwordDigestor: "client"})' monger-test
+mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], passwordDigestor: "client"})' monger-test2
+mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], passwordDigestor: "client"})' monger-test3
+mongo --eval 'db.createUser({"user": "clojurewerkz/monger", "pwd": "monger", roles: ["dbAdmin"], passwordDigestor: "client"})' monger-test4
--- a/bin/ci/install_mongodb.sh
+++ b/bin/ci/install_mongodb.sh
@ -0,0 +1,8 @@
+#!/bin/sh
+
+sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
+
+echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list
+
+sudo apt-get update
+sudo apt-get install -y mongodb-org
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -0,0 +1,11 @@
+# Use root/example as user/password credentials
+version: '3.1'
+
+services:
+
+  mongo:
+    image: mongo
+    container_name: mongo_test
+    restart: always
+    ports:
+      - "27017:27017"
--- a/project.clj
+++ b/project.clj
@ -1,12 +1,12 @@
-(defproject com.novemberain/monger "3.0.0-rc1"
+(defproject com.novemberain/monger "4.0.0-SNAPSHOT"
  :description "Monger is a Clojure MongoDB client for a more civilized age: friendly, flexible and with batteries included"
  :url "http://clojuremongodb.info"
  :min-lein-version "2.5.1"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
-  :dependencies [[org.clojure/clojure           "1.6.0"]
-                 [org.mongodb/mongo-java-driver "3.0.1"]
-                 [clojurewerkz/support          "1.1.0"]]
+  :dependencies [[org.clojure/clojure        "1.11.1"]
+                 [org.mongodb/mongodb-driver "3.12.11"]
+                 [clojurewerkz/support       "1.5.0"]]
  :test-selectors {:default     (fn [m]
                                  (and (not (:performance m))
                                       (not (:edge-features m))
@ -27,41 +27,32 @@
                   :all           (constantly true)}
  :source-paths      ["src/clojure"]
  :java-source-paths ["src/java"]
-  :javac-options     ["-target" "1.6" "-source" "1.6"]
+  :javac-options     ["-target" "1.8" "-source" "1.8"]
  :mailing-list {:name "clojure-mongodb"
                 :archive "https://groups.google.com/group/clojure-mongodb"
                 :post "clojure-mongodb@googlegroups.com"}
-  :profiles {:dj01x {:dependencies [[org.clojure/data.json  "0.1.2" :exclusions [org.clojure/clojure]]]}
-             :dj02x {:dependencies [[org.clojure/data.json  "0.2.4" :exclusions [org.clojure/clojure]]]}
-             :1.5    {:dependencies [[org.clojure/clojure "1.5.1"]]}
-             :1.7    {:dependencies [[org.clojure/clojure "1.7.0-alpha5"]]}
-             :master {:dependencies [[org.clojure/clojure "1.7.0-master-SNAPSHOT"]]}
+  :profiles {:1.10 {:dependencies [[org.clojure/clojure "1.10.2"]]}
+             :1.9 {:dependencies [[org.clojure/clojure "1.9.0"]]}
             :dev {:resource-paths ["test/resources"]
-                   :dependencies  [[clj-time               "0.8.0" :exclusions [org.clojure/clojure]]
-                                   [cheshire               "5.3.1" :exclusions [org.clojure/clojure]]
-                                   [org.clojure/data.json  "0.2.5" :exclusions [org.clojure/clojure]]
-                                   [org.clojure/tools.cli  "0.3.1" :exclusions [org.clojure/clojure]]
-                                   [org.clojure/core.cache "0.6.3" :exclusions [org.clojure/clojure]]
-                                   [ring/ring-core         "1.3.0" :exclusions [org.clojure/clojure]]
-                                   [com.novemberain/validateur "2.4.2" :exclusions [org.clojure/clojure]]
-                                   [ch.qos.logback/logback-classic "1.1.3" :exclusions [org.slf4j/slf4j-api]]
-                                   [ragtime/ragtime.core   "0.3.7" :exclusions [org.clojure/clojure]]]
-                   :plugins [[codox "0.8.10"]]
-                   :codox {:sources ["src/clojure"]
-                           :output-dir "doc/api"
-                           :exclude [monger.internal.pagination
-                                     monger.internal.fn
-                                     ;; these are not fully baked yet or have changes
-                                     ;; that are not entirely backwards compatible with 1.0. MK.
-                                     monger.testkit
-                                     monger.ring.session-store]}}
+                   :dependencies  [[clj-time               "0.15.1" :exclusions [org.clojure/clojure]]
+                                   [cheshire               "5.8.1" :exclusions [org.clojure/clojure]]
+                                   [org.clojure/data.json  "2.5.0" :exclusions [org.clojure/clojure]]
+                                   [org.clojure/tools.cli  "0.4.1" :exclusions [org.clojure/clojure]]
+                                   [org.clojure/core.cache "0.7.1" :exclusions [org.clojure/clojure]]
+                                   [ring/ring-core         "1.7.1" :exclusions [org.clojure/clojure]]
+                                   [com.novemberain/validateur "2.6.0" :exclusions [org.clojure/clojure]]
+                                   [ch.qos.logback/logback-classic "1.2.3" :exclusions [org.slf4j/slf4j-api]]
+                                   [ragtime/core "0.7.2" :exclusions [org.clojure/clojure]]]
+                   :plugins [[lein-codox "0.10.5"]]
+                   :codox {:source-paths ["src/clojure"]
+                           :namespaces [#"^monger\.(?!internal)"]}}
             ;; only clj-time/JodaTime available, used to test monger.joda-time w/o clojure.data.json
             :dev2 {:resource-paths ["test/resources"]
-                    :dependencies  [[clj-time "0.8.0"               :exclusions [org.clojure/clojure]]]}}
-  :aliases {"all" ["with-profile" "dev:dev,1.5:dev,dj01x:dev,dj02x:dev,1.7"]}
-  :repositories {"sonatype" {:url "http://oss.sonatype.org/content/repositories/releases"
+                    :dependencies  [[clj-time "0.15.2"               :exclusions [org.clojure/clojure]]]}}
+  :aliases {"all" ["with-profile" "dev:dev,1.10:dev,1.9:dev"]}
+  :repositories {"sonatype" {:url "https://oss.sonatype.org/content/repositories/releases"
                             :snapshots false
                             :releases {:checksum :fail :update :always}}
-                 "sonatype-snapshots" {:url "http://oss.sonatype.org/content/repositories/snapshots"
+                 "sonatype-snapshots" {:url "https://oss.sonatype.org/content/repositories/snapshots"
                                       :snapshots true
                                       :releases {:checksum :fail :update :always}}})
--- a/src/clojure/monger/cache.clj
+++ b/src/clojure/monger/cache.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/collection.clj
+++ b/src/clojure/monger/collection.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;; Copyright (c) 2012 Toby Hede
 ;; Copyright (c) 2012 Baishampayan Ghose
 ;;
@ -24,7 +24,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; Copyright (c) 2012 Toby Hede
 ;; Copyright (c) 2012 Baishampayan Ghose
 ;;
@ -50,16 +50,19 @@
   * http://clojuremongodb.info/articles/updating.html
   * http://clojuremongodb.info/articles/deleting.html
   * http://clojuremongodb.info/articles/aggregation.html"
-  (:refer-clojure :exclude [find remove count drop distinct empty? update])
-  (:import [com.mongodb Mongo DB DBCollection WriteResult DBObject WriteConcern 
-            DBCursor MapReduceCommand MapReduceCommand$OutputType]
+  (:refer-clojure :exclude [find remove count drop distinct empty? any? update])
+  (:import [com.mongodb Mongo DB DBCollection WriteResult DBObject WriteConcern
+            DBCursor MapReduceCommand MapReduceCommand$OutputType AggregationOutput
+            AggregationOptions AggregationOptions$OutputMode]
           [java.util List Map]
+           [java.util.concurrent TimeUnit]
           [clojure.lang IPersistentMap ISeq]
           org.bson.types.ObjectId)
  (:require [monger.core :as mc]
            [monger.result :as mres]
            [monger.conversion :refer :all]
-            [monger.constraints :refer :all]))
+            [monger.constraints :refer :all]
+            [monger.util :refer [into-array-list]]))


 ;;
@ -106,7 +109,7 @@


 (defn ^WriteResult insert-batch
-  "Saves documents do collection. You can optionally specify WriteConcern as a third argument."
+  "Saves documents to collection. You can optionally specify WriteConcern as a third argument."
  ([^DB db ^String coll ^List documents]
     (.insert (.getCollection db (name coll))
              ^List (to-db-object documents)
@ -145,8 +148,10 @@
     (with-open [dbc (find db coll ref)]
       (map (fn [x] (from-db-object x true)) dbc)))
  ([^DB db ^String coll ^Map ref fields]
+     (find-maps db coll ref fields true))
+  ([^DB db ^String coll ^Map ref fields keywordize]
     (with-open [dbc (find db coll ref fields)]
-       (map (fn [x] (from-db-object x true)) dbc))))
+       (map (fn [x] (from-db-object x keywordize)) dbc))))

 (defn find-seq
  "Queries for objects in this collection, returns ISeq of DBObjects."
@ -221,13 +226,13 @@
  "Returns a single object, converted to map with matching _id field."
  ([^DB db ^String coll id]
     (check-not-nil! id "id must not be nil")
-     (from-db-object ^DBObject (find-one-as-map db coll {:_id id}) true))
+     (find-one-as-map db coll {:_id id}))
  ([^DB db ^String coll id fields]
     (check-not-nil! id "id must not be nil")
-     (from-db-object ^DBObject (find-one-as-map db coll {:_id id} fields) true))
+     (find-one-as-map db coll {:_id id} fields))
  ([^DB db ^String coll id fields keywordize]
     (check-not-nil! id "id must not be nil")
-     (from-db-object ^DBObject (find-one-as-map db coll {:_id id} fields) keywordize)))
+     (find-one-as-map db coll {:_id id} fields keywordize)))

 ;;
 ;; monger.collection/count
@ -422,10 +427,10 @@
     (.createIndex (.getCollection db (name coll))
                   (as-field-selector keys)
                   (to-db-object options)))
-  ([^DB db ^String coll ^Map keys ^String name unique?]
+  ([^DB db ^String coll ^Map keys ^String index-name unique?]
     (.createIndex (.getCollection db (name coll))
                   (as-field-selector keys)
-                   name
+                   index-name
                   unique?)))


@ -446,9 +451,9 @@
 (defn drop-index
  "Drops an index from this collection."
  [^DB db ^String coll idx]
-  (.dropIndex (.getCollection db (name coll)) (if (string? idx)
-                                                idx
-                                                (to-db-object idx))))
+  (if (string? idx)
+    (.dropIndex (.getCollection db (name coll)) ^String idx)
+    (.dropIndex (.getCollection db (name coll)) (to-db-object idx))))

 (defn drop-indexes
  "Drops all indixes from this collection."
@ -462,7 +467,7 @@


 (defn exists?
-  "Checks weather collection with certain name exists."
+  "Checks whether collection with certain name exists."
  ([^DB db ^String coll]
     (.collectionExists db coll)))

@ -485,9 +490,9 @@
 (defn rename
  "Renames collection."
  ([^DB db ^String from, ^String to]
-     (.rename (.getCollection db from) to))
+     (.rename (.getCollection db (name from)) (name to)))
  ([^DB db ^String from ^String to drop-target?]
-     (.rename (.getCollection db from) to drop-target?)))
+     (.rename (.getCollection db (name from)) (name to) drop-target?)))

 ;;
 ;; Map/Reduce
@ -519,17 +524,45 @@
 ;; Aggregation
 ;;

+(defn- build-aggregation-options
+  ^AggregationOptions
+  [{:keys [^Boolean allow-disk-use cursor ^Long max-time]}]
+  (cond-> (AggregationOptions/builder)
+     allow-disk-use       (.allowDiskUse allow-disk-use)
+     cursor               (.outputMode AggregationOptions$OutputMode/CURSOR)
+     max-time             (.maxTime max-time TimeUnit/MILLISECONDS)
+     (:batch-size cursor) (.batchSize (int (:batch-size cursor)))
+     true                 .build))
+
 (defn aggregate
  "Executes an aggregation query. MongoDB 2.2+ only.
+   Accepts the options :allow-disk-use and :cursor (a map with the :batch-size
+   key), as described in the MongoDB manual. Additionally, the :max-time option
+  is supported, for specifying a limit on the execution time of the query in
+  milliseconds.
+
+  :keywordize option that control if resulting map keys will be turned into keywords, default is true.

  See http://docs.mongodb.org/manual/applications/aggregation/ to learn more."
-  [^DB db ^String coll stages]
-  (let [res (mc/command db {:aggregate coll :pipeline stages})]
-    ;; this is what DBCollection#distinct does. Turning a blind's eye!
-    (.throwOnError res)
-    (map #(from-db-object % true) (.get res "result"))))
+  [^DB db ^String coll stages & opts]
+  (let [coll (.getCollection db (name coll))
+        agg-opts (build-aggregation-options opts)
+        pipe (into-array-list (to-db-object stages))
+        res (.aggregate coll pipe agg-opts)
+        {:keys [^Boolean keywordize]
+         :or            {keywordize true}} opts]
+    (map #(from-db-object % keywordize) (iterator-seq res))))

+(defn explain-aggregate
+  "Returns the explain plan for an aggregation query. MongoDB 2.2+ only.

+  See http://docs.mongodb.org/manual/applications/aggregation/ to learn more."
+  [^DB db ^String coll stages & opts]
+  (let [coll (.getCollection db (name coll))
+        agg-opts (build-aggregation-options opts)
+        pipe (into-array-list (to-db-object stages))
+        res (.explainAggregate coll pipe agg-opts)]
+    (from-db-object res true)))
 ;;
 ;; Misc
 ;;
--- a/src/clojure/monger/command.clj
+++ b/src/clojure/monger/command.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;; Copyright (c) 2012 Toby Hede
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
@ -23,7 +23,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; Copyright (c) 2012 Toby Hede
 ;; All rights reserved.
 ;;
--- a/src/clojure/monger/constraints.clj
+++ b/src/clojure/monger/constraints.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/conversion.clj
+++ b/src/clojure/monger/conversion.clj
@ -5,7 +5,7 @@
 ;;
 ;; ----------------------------------------------------------------------------------
 ;; Portions of the code are Copyright (c) 2009 Andrew Boekhoff
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -24,7 +24,7 @@
 ;;
 ;; ----------------------------------------------------------------------------------
 ;; Portions of the code are Copyright (c) 2009 Andrew Boekhoff
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -46,7 +46,8 @@
  (:import [com.mongodb DBObject BasicDBObject BasicDBList DBCursor]
           [clojure.lang IPersistentMap Named Keyword Ratio]
           [java.util List Map Date Set]
-           org.bson.types.ObjectId))
+           org.bson.types.ObjectId
+           (org.bson.types Decimal128)))

 (defprotocol ConvertToDBObject
  (^com.mongodb.DBObject to-db-object [input] "Converts given piece of Clojure data to BasicDBObject MongoDB Java driver uses"))
@ -104,55 +105,44 @@



-
-(declare associate-pairs)
 (defprotocol ConvertFromDBObject
  (from-db-object [input keywordize] "Converts given DBObject instance to a piece of Clojure data"))

 (extend-protocol ConvertFromDBObject
  nil
-  (from-db-object [input keywordize] input)
+  (from-db-object [_ _] nil)

  Object
-  (from-db-object [input keywordize] input)
+  (from-db-object [input _] input)

-  Map
-  (from-db-object [^Map input keywordize]
-    (associate-pairs (.entrySet input) keywordize))
+  Decimal128
+  (from-db-object [^Decimal128 input _]
+    (.bigDecimalValue input))

  List
  (from-db-object [^List input keywordize]
-    (vec (map #(from-db-object % keywordize) input)))
+    (mapv #(from-db-object % keywordize) input))

  BasicDBList
  (from-db-object [^BasicDBList input keywordize]
-    (vec (map #(from-db-object % keywordize) input)))
+    (mapv #(from-db-object % keywordize) input))

  com.mongodb.DBRef
-  (from-db-object [^com.mongodb.DBRef input keywordize]
+  (from-db-object [^com.mongodb.DBRef input _]
    input)

  DBObject
  (from-db-object [^DBObject input keywordize]
    ;; DBObject provides .toMap, but the implementation in
    ;; subclass GridFSFile unhelpfully throws
-    ;; UnsupportedOperationException. This part is taken from congomongo and
-    ;; may need revisiting at a later point. MK.
-    (associate-pairs (for [key-set (.keySet input)] [key-set (.get input key-set)])
-                     keywordize)))
-
-
-(defn- associate-pairs [pairs keywordize]
-  ;; Taking the keywordize test out of the fn reduces derefs
-  ;; dramatically, which was the main barrier to matching pure-Java
-  ;; performance for this marshalling. Taken from congomongo. MK.
-  (reduce (if keywordize
-            (fn [m [^String k v]]
-              (assoc m (keyword k) (from-db-object v true)))
-            (fn [m [^String k v]]
-              (assoc m k (from-db-object v false))))
-          {} (reverse pairs)))
-
+    ;; UnsupportedOperationException.
+    (persistent!
+     (reduce (if keywordize
+               (fn [m ^String k]
+                 (assoc! m (keyword k) (from-db-object (.get input k) true)))
+               (fn [m ^String k]
+                 (assoc! m k (from-db-object (.get input k) false))))
+             (transient {}) (.keySet input)))))


 (defprotocol ConvertToObjectId
--- a/src/clojure/monger/core.clj
+++ b/src/clojure/monger/core.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -42,11 +42,12 @@
   * http://clojuremongodb.info/articles/commands.html
   * http://clojuremongodb.info/articles/gridfs.html"
  (:refer-clojure :exclude [count])
-  (:require [monger.conversion :refer :all])
+  (:require [monger.conversion :refer :all]
+            [monger.util :refer [into-array-list]])
  (:import [com.mongodb MongoClient MongoClientURI MongoCredential DB WriteConcern DBObject DBCursor Bytes
            MongoClientOptions MongoClientOptions$Builder ServerAddress MapReduceOutput MongoException]
           [com.mongodb.gridfs GridFS]
-           [java.util Map ArrayList]))
+           [java.util Map]))

 ;;
 ;; Defaults
@ -69,40 +70,43 @@
     :host (\"127.0.0.1\" by default)
     :port (27017 by default)"
  {:arglists '([]
-                 [server-address options]
-                   [[server-address & more] options]
-                     [{:keys [host port uri] :or { host *mongodb-host* port *mongodb-port*}}])}
+               [server-address options]
+               [server-address options credentials]
+               [[server-address & more] options]
+               [{:keys [host port uri] :or { host *mongodb-host* port *mongodb-port*}}])}
  ([]
     (MongoClient.))
  ([server-address ^MongoClientOptions options]
     (if (coll? server-address)
       ;; connect to a replica set
-       (let [server-list ^ArrayList (ArrayList. ^java.util.Collection server-address)]
+       (let [server-list (into-array-list server-address)]
         (MongoClient. server-list options))
       ;; connect to a single instance
       (MongoClient. ^ServerAddress server-address options)))
  ([server-address ^MongoClientOptions options credentials]
-     (let [creds (if (coll? credentials)
-                   credentials
-                   [credentials])]
+   (let [creds (into-array-list (if (coll? credentials)
+                                  credentials
+                                  [credentials]))]
       (if (coll? server-address)
-         (let [server-list ^ArrayList (ArrayList. ^java.util.Collection server-address)]
-           (MongoClient. server-list creds options))
-       (MongoClient. ^ServerAddress server-address options))))
+         (let [server-list (into-array-list server-address)]
+           (MongoClient. server-list ^java.util.List creds options))
+         (MongoClient. ^ServerAddress server-address ^java.util.List creds options))))
  ([{ :keys [host port uri] :or { host *mongodb-host* port *mongodb-port* }}]
-     (MongoClient. ^String host ^Long port)))
+    (if uri
+       (MongoClient. (MongoClientURI. uri))
+       (MongoClient. ^String host ^Long port))))

 (defn ^MongoClient connect-with-credentials
  "Connect with provided credentials and default options"
  ([credentials]
-     (connect-with-credentials *mongodb-host* *mongodb-port* credentials))  
+     (connect-with-credentials *mongodb-host* *mongodb-port* credentials))
  ([^String hostname credentials]
     (connect-with-credentials hostname *mongodb-port* credentials))
-  ([^String hostname port credentials]
-     (MongoClient. [(ServerAddress. hostname port)]
-                   (if (coll? credentials)
-                     credentials
-                     [credentials]))))
+  ([^String hostname ^long port credentials]
+     (MongoClient. (into-array-list [(ServerAddress. hostname port)])
+                   (into-array-list (if (coll? credentials)
+                                      credentials
+                                      [credentials])))))

 (defn get-db-names
  "Gets a list of all database names present on the server"
@ -132,38 +136,112 @@
     (ServerAddress. hostname port)))

 (defn ^MongoClientOptions$Builder mongo-options-builder
-  [{:keys [connections-per-host threads-allowed-to-block-for-connection-multiplier
-           max-wait-time connect-timeout socket-timeout socket-keep-alive auto-connect-retry max-auto-connect-retry-time
-           description write-concern cursor-finalizer-enabled read-preference
-           required-replica-set-name] :or [auto-connect-retry true]}]
+  [{:keys [add-cluster-listener add-cluster-listeners add-command-listener add-command-listeners
+           add-connection-pool-listener add-connection-pool-listeners add-server-listener add-server-listeners
+           add-server-monitor-listener add-server-monitor-listeners always-use-mbeans application-name
+           codec-registry compressor-list connect-timeout connections-per-host cursor-finalizer-enabled
+           db-decoder-factory db-encoder-factory description heartbeat-connect-timeout heartbeat-frequency
+           heartbeat-socket-timeout local-threshold max-connection-idle-time max-connection-life-time
+           max-wait-time min-connections-per-host min-heartbeat-frequency read-concern read-preference
+           required-replica-set-name retry-writes server-selection-timeout server-selector socket-keep-alive 
+           socket-factory socket-timeout ssl-context ssl-enabled ssl-invalid-host-name-allowed
+           threads-allowed-to-block-for-connection-multiplier uuid-representation write-concern]}]
  (let [mob (MongoClientOptions$Builder.)]
+    (when add-cluster-listener
+      (.addClusterListener mob add-cluster-listener))
+    (when add-cluster-listeners
+      (doseq [cluster-listener add-cluster-listeners]
+        (.addClusterListener mob cluster-listener)))
+    (when add-command-listener
+      (.addCommandListener mob add-command-listener))
+    (when add-command-listeners
+      (doseq [command-listener add-command-listeners]
+        (.addCommandListener mob command-listener)))
+    (when add-connection-pool-listener
+      (.addConnectionPoolListener mob add-connection-pool-listener))
+    (when add-connection-pool-listeners
+      (doseq [connection-pool-listener add-connection-pool-listeners]
+        (.addConnectionPoolListener mob connection-pool-listener)))
+    (when add-server-listener
+      (.addServerListener mob add-server-listener))
+    (when add-server-listeners
+      (doseq [server-listener add-server-listeners]
+        (.addServerListener mob server-listener)))
+    (when add-server-monitor-listener
+      (.addServerMonitorListener mob add-server-monitor-listener))
+    (when add-server-monitor-listeners
+      (doseq [server-monitor-listener add-server-monitor-listeners]
+        (.addServerMonitorListener mob server-monitor-listener)))
+    (when always-use-mbeans
+      (.alwaysUseMBeans mob always-use-mbeans))
+    (when application-name
+      (.applicationName mob application-name))
+    (when always-use-mbeans
+      (.alwaysUseMBeans mob always-use-mbeans))
+    (when codec-registry
+      (.codecRegistry mob codec-registry))
+    (when compressor-list
+      (.compressorList mob compressor-list))
    (when connections-per-host
      (.connectionsPerHost mob connections-per-host))
-    (when threads-allowed-to-block-for-connection-multiplier
-      (.threadsAllowedToBlockForConnectionMultiplier mob threads-allowed-to-block-for-connection-multiplier))
-    (when max-wait-time
-      (.maxWaitTime mob max-wait-time))
    (when connect-timeout
      (.connectTimeout mob connect-timeout))
-    (when socket-timeout
-      (.socketTimeout mob socket-timeout))
-    (when socket-keep-alive
-      (.socketKeepAlive mob socket-keep-alive))
-    (when auto-connect-retry
-      (.autoConnectRetry mob auto-connect-retry))
-    ;; deprecated
-    (when max-auto-connect-retry-time
-      (.maxAutoConnectRetryTime mob max-auto-connect-retry-time))
-    (when read-preference
-      (.readPreference mob read-preference))
-    (when description
-      (.description mob description))
-    (when write-concern
-      (.writeConcern mob write-concern))
    (when cursor-finalizer-enabled
      (.cursorFinalizerEnabled mob cursor-finalizer-enabled))
+    (when db-decoder-factory
+      (.dbDecoderFactory mob db-decoder-factory))
+    (when db-encoder-factory
+      (.dbEncoderFactory mob db-encoder-factory))
+    (when description
+      (.description mob description))
+    (when heartbeat-connect-timeout
+      (.heartbeatConnectTimeout mob heartbeat-connect-timeout))
+    (when heartbeat-frequency
+      (.heartbeatFrequency mob heartbeat-frequency))
+    (when heartbeat-socket-timeout
+      (.heartbeatSocketTimeout mob heartbeat-socket-timeout))
+    (when ssl-context
+      (.sslContext mob ssl-context))
+    (when local-threshold
+      (.localThreshold mob local-threshold))
+    (when max-connection-idle-time
+      (.maxConnectionIdleTime mob max-connection-idle-time))
+    (when max-wait-time
+      (.maxWaitTime mob max-wait-time))
+    (when max-connection-life-time
+      (.maxConnectionLifeTime mob max-connection-life-time))
+    (when min-connections-per-host
+      (.minConnectionsPerHost mob min-connections-per-host))
+    (when min-heartbeat-frequency
+      (.minHeartbeatFrequency mob min-heartbeat-frequency))
+    (when read-concern
+      (.readConcern mob read-concern))
+    (when read-preference
+      (.readPreference mob read-preference))
    (when required-replica-set-name
      (.requiredReplicaSetName mob required-replica-set-name))
+    (when retry-writes
+      (.retryWrites mob retry-writes))
+    (when server-selection-timeout
+      (.serverSelectionTimeout mob server-selection-timeout))
+    (when server-selector
+      (.serverSelector mob server-selector))
+    (when socket-keep-alive
+      (.socketKeepAlive mob socket-keep-alive))
+    (when socket-factory
+      (.socketFactory mob socket-factory))
+    (when socket-timeout
+      (.socketTimeout mob socket-timeout))
+    (when ssl-enabled
+      (.sslEnabled mob ssl-enabled))
+    (when ssl-invalid-host-name-allowed
+      (.sslInvalidHostNameAllowed mob ssl-invalid-host-name-allowed))
+    (when threads-allowed-to-block-for-connection-multiplier
+      (.threadsAllowedToBlockForConnectionMultiplier mob threads-allowed-to-block-for-connection-multiplier))
+    (when uuid-representation
+      (.uuidRepresentation mob uuid-representation))
+    (when write-concern
+      (.writeConcern mob write-concern))
    mob))

 (defn ^MongoClientOptions mongo-options
@ -195,10 +273,11 @@
   Commonly used for PaaS-based applications, for example, running on Heroku.
   If username and password are provided, performs authentication."
  [^String uri-string]
-  (let [uri  (MongoClientURI. uri-string)
-        conn (MongoClient. uri)
-        db   (.getDB conn (.getDatabase uri))]
-    {:conn conn :db db}))
+  (let [uri    (MongoClientURI. uri-string)
+        conn   (MongoClient. uri)]
+    (if-let [dbName (.getDatabase uri)]
+      {:conn conn :db (.getDB conn dbName)}
+      (throw (IllegalArgumentException. "No database name specified in URI. Monger requires a database to be explicitly configured.")))))

 (defn ^com.mongodb.CommandResult command
  "Runs a database command (please check MongoDB documentation for the complete list of commands).
--- a/src/clojure/monger/credentials.clj
+++ b/src/clojure/monger/credentials.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -41,7 +41,7 @@
 ;; API
 ;;

-(defn ^MongoCredential for
+(defn ^MongoCredential create
  "Creates a MongoCredential instance with an unspecified mechanism.
   The client will negotiate the best mechanism based on the
   version of the server that the client is authenticating to."
--- a/src/clojure/monger/cursor.clj
+++ b/src/clojure/monger/cursor.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/db.clj
+++ b/src/clojure/monger/db.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;; Copyright (c) 2012 Toby Hede
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
@ -23,7 +23,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; Copyright (c) 2012 Toby Hede
 ;; All rights reserved.
 ;;
--- a/src/clojure/monger/gridfs.clj
+++ b/src/clojure/monger/gridfs.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -114,9 +114,30 @@
  (to-input-stream [^InputStream input]
    input))

+(defprotocol GridFSInputFileFactory
+  (^GridFSInputFile create-gridfs-file [input ^GridFS fs] "Creates a file entry"))
+
+(extend byte-array-type
+  GridFSInputFileFactory
+  {:create-gridfs-file (fn [^bytes input ^GridFS fs]
+                         (.createFile fs input))})
+
+(extend-protocol GridFSInputFileFactory
+  String
+  (create-gridfs-file [^String input ^GridFS fs]
+    (.createFile fs (io/file input)))
+
+  File
+  (create-gridfs-file [^File input ^GridFS fs]
+    (.createFile fs input))
+
+  InputStream
+  (create-gridfs-file [^InputStream input ^GridFS fs]
+    (.createFile fs input)))
+
 (defn ^GridFSInputFile make-input-file
  [^GridFS fs input]
-  (.createFile fs (to-input-stream input)))
+  (create-gridfs-file input fs))

 (defmacro store
  [^GridFSInputFile input & body]
@ -188,4 +209,4 @@

 (defn find-map-by-id
  [^GridFS fs ^ObjectId id]
-  (converter (find-one fs id)))
+  (converter (find-by-id fs id)))
--- a/src/clojure/monger/internal/pagination.clj
+++ b/src/clojure/monger/internal/pagination.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/joda_time.clj
+++ b/src/clojure/monger/joda_time.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/js.clj
+++ b/src/clojure/monger/js.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/json.clj
+++ b/src/clojure/monger/json.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -70,13 +70,19 @@
            (try
              (extend-protocol clojure.data.json/JSONWriter
                ObjectId
-                (-write [^ObjectId object out]
-                  (clojure.data.json/write (.toString object) out)))
+                (-write
+                  ([^ObjectId object out]
+                   (clojure.data.json/write (.toString object) out))
+                  ([^ObjectId object out options]
+                   (clojure.data.json/write (.toString object) out options))))

              (extend-protocol clojure.data.json/JSONWriter
                BSONTimestamp
-                  (-write [^BSONTimestamp object out]
-                    (clojure.data.json/write {:time (.getTime object) :inc (.getInc object)} out)))
+                (-write
+                  ([^BSONTimestamp object out]
+                   (clojure.data.json/write {:time (.getTime object) :inc (.getInc object)} out))
+                  ([^BSONTimestamp object out options]
+                   (clojure.data.json/write {:time (.getTime object) :inc (.getInc object)} out options))))

              (catch Throwable _
                false))
--- a/src/clojure/monger/operators.clj
+++ b/src/clojure/monger/operators.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -46,6 +46,8 @@
 ;; QUERY OPERATORS
 ;;

+(declare $gt $gte $lt $lte $all $in $nin $eq $ne $elemMatch $regex $options)
+
 ;; $gt is "greater than" comparator
 ;; $gte is "greater than or equals" comparator
 ;; $gt is "less than" comparator
@ -80,10 +82,16 @@
 ;;   (mgcol/find-maps "languages" { :tags { $nin [ "functional" ] } } )
 (defoperator $nin)

+;; $eq is "equals" comparator
+;;
+;; EXAMPLES:
+;;   (monger.collection/find "libraries" { :language { $eq "Clojure" }})
+(defoperator $eq)
+
 ;; $ne is "non-equals" comparator
 ;;
 ;; EXAMPLES:
-;;   (monger.collection/find "libraries" {$ne { :language "Clojure" }})
+;;   (monger.collection/find "libraries" { :language { $ne "Clojure" }})
 (defoperator $ne)

 ;; $elemMatch checks if an element in an array matches the specified expression
@ -96,6 +104,37 @@
 (defoperator $regex)
 (defoperator $options)

+;; comment on a query predicate
+
+(declare $comment $explain $hint $maxTimeMS $orderBy $query $returnKey $showDiskLoc $natural)
+
+(defoperator $comment)
+(defoperator $explain)
+(defoperator $hint)
+(defoperator $maxTimeMS)
+(defoperator $orderBy)
+(defoperator $query)
+(defoperator $returnKey)
+(defoperator $showDiskLoc)
+(defoperator $natural)
+
+
+;;
+;; EVALUATION (QUERY)
+;;
+
+(declare $expr $jsonSchema $where $and $or $nor)
+
+(defoperator $expr)
+(defoperator $jsonSchema)
+
+;; Matches documents that satisfy a JavaScript expression.
+;;
+;; EXAMPLES:
+;;
+;;   (monger.collection/find "people" { $where "this.placeOfBirth === this.address.city" })
+(defoperator $where)
+
 ;;
 ;; LOGIC OPERATORS
 ;;
@ -128,6 +167,8 @@
 ;; ATOMIC MODIFIERS
 ;;

+(declare $inc $mul $set $unset $setOnInsert $rename $push $position $each $addToSet $pop $pull $pullAll $bit $bitsAllClear $bitsAllSet $bitsAnyClear $bitsAnySet $exists $mod $size $type $not)
+
 ;; $inc increments one or many fields for the given value, otherwise sets the field to value
 ;;
 ;; EXAMPLES:
@ -171,6 +212,9 @@
 ;;   (mgcol/update "docs" { :_id oid } { $push { :tags "modifiers" } })
 (defoperator $push)

+;; $position modifies the behavior of $push per https://docs.mongodb.com/manual/reference/operator/update/position/
+(defoperator $position)
+
 ;; $each is a modifier for the $push and $addToSet operators for appending multiple values to an array field.
 ;; Without the $each modifier $push and $addToSet will append an array as a single value.
 ;; MongoDB 2.4 adds support for the $each modifier to the $push operator.
@ -180,14 +224,6 @@
 ;;   (mgcol/update coll { :_id oid } { $push { :tags { $each ["mongodb" "docs"] } } })
 (defoperator $each)

-;; $pushAll appends each value in value_array to field, if field is an existing array, otherwise sets field to the array value_array
-;; if field is not present. If field is present but is not an array, an error condition is raised.
-;; Deprecated since MongoDB 2.4, $push with $each modifier should be used instead.
-;;
-;; EXAMPLES:
-;;   (mgcol/update coll { :_id oid } { $pushAll { :tags ["mongodb" "docs"] } })
-(defoperator $pushAll)
-
 ;; $addToSet Adds value to the array only if its not in the array already, if field is an existing array, otherwise sets field to the
 ;; array value if field is not present. If field is present but is not an array, an error condition is raised.
 ;;
@ -214,11 +250,15 @@
 ;; an error condition is raised.
 ;;
 ;; EXAMPLES:
-;;   (mgcol/update coll { :_id oid } { $pull { :measurements 1.2 } })
-;;   (mgcol/update coll { :_id oid } { $pull { :measurements { $gte 1.2 } } })
+;;   (mgcol/update coll { :_id oid } { $pullAll { :measurements 1.2 } })
+;;   (mgcol/update coll { :_id oid } { $pullAll { :measurements { $gte 1.2 } } })
 (defoperator $pullAll)

 (defoperator $bit)
+(defoperator $bitsAllClear)
+(defoperator $bitsAllSet)
+(defoperator $bitsAnyClear)
+(defoperator $bitsAnySet)

 (defoperator $exists)
 (defoperator $mod)
@ -228,36 +268,132 @@


 ;;
-;; Aggregation in 2.2
+;; Aggregation in 4.2
 ;;

+(declare $addFields $bucket $bucketAuto $collStats $facet $geoNear $graphLookup $indexStats $listSessions $lookup $match $merge $out $planCacheStats $project $redact $replaceRoot $replaceWith $sample $limit $skip $unwind $group $sort $sortByCount $currentOp $listLocalSessions $cmp $min $max $avg $stdDevPop $stdDevSamp $sum $let $first $last $abs $add $ceil $divide $exp $floor $ln $log $log10 $multiply $pow $round $sqrt $subtract $trunc $literal $arrayElemAt $arrayToObject $concatArrays $filter $indexOfArray $isArray $map $objectToArray $range $reduce $reverseArray $zip $mergeObjects $allElementsTrue $anyElementsTrue $setDifference $setEquals $setIntersection $setIsSubset $setUnion $strcasecmp $substr $substrBytes $substrCP $toLower $toString $toUpper $concat $indexOfBytes $indexOfCP $ltrim $regexFind $regexFindAll $regexMatch $rtrim $split $strLenBytes $subLenCP $trim $sin $cos $tan $asin $acos $atan $atan2 $asinh $acosh $atanh $radiansToDegrees $degreesToRadians $convert $toBool $toDecimal $toDouble $toInt $toLong $toObjectId $dayOfMonth $dayOfWeek $dayOfYear $hour $minute $month $second $millisecond $week $year $isoDate $dateFromParts $dateFromString $dateToParts $dateToString $isoDayOfWeek $isoWeek $isoWeekYear $toDate $ifNull $cond $switch)
+
+(defoperator $addFields)
+(defoperator $bucket)
+(defoperator $bucketAuto)
+(defoperator $collStats)
+(defoperator $facet)
+(defoperator $geoNear)
+(defoperator $graphLookup)
+(defoperator $indexStats)
+(defoperator $listSessions)
+(defoperator $lookup)
 (defoperator $match)
+(defoperator $merge)
+(defoperator $out)
+(defoperator $planCacheStats)
 (defoperator $project)
+(defoperator $redact)
+(defoperator $replaceRoot)
+(defoperator $replaceWith)
+(defoperator $sample)
 (defoperator $limit)
 (defoperator $skip)
 (defoperator $unwind)
 (defoperator $group)
 (defoperator $sort)
+(defoperator $sortByCount)
+
+(defoperator $currentOp)
+(defoperator $listLocalSessions)

 (defoperator $cmp)

 (defoperator $min)
 (defoperator $max)
 (defoperator $avg)
+(defoperator $stdDevPop)
+(defoperator $stdDevSamp)
 (defoperator $sum)
+(defoperator $let)

 (defoperator $first)
 (defoperator $last)

+(defoperator $abs)
 (defoperator $add)
+(defoperator $ceil)
 (defoperator $divide)
+(defoperator $exp)
+(defoperator $floor)
+(defoperator $ln)
+(defoperator $log)
+(defoperator $log10)
 (defoperator $multiply)
-(defoperator $substract)
+(defoperator $pow)
+(defoperator $round)
+(defoperator $sqrt)
+(defoperator $subtract)
+(defoperator $trunc)
+(defoperator $literal)
+
+(defoperator $arrayElemAt)
+(defoperator $arrayToObject)
+(defoperator $concatArrays)
+(defoperator $filter)
+(defoperator $indexOfArray)
+(defoperator $isArray)
+(defoperator $map)
+(defoperator $objectToArray)
+(defoperator $range)
+(defoperator $reduce)
+(defoperator $reverseArray)
+(defoperator $zip)
+(defoperator $mergeObjects)
+
+(defoperator $allElementsTrue)
+(defoperator $anyElementsTrue)
+(defoperator $setDifference)
+(defoperator $setEquals)
+(defoperator $setIntersection)
+(defoperator $setIsSubset)
+(defoperator $setUnion)

 (defoperator $strcasecmp)
 (defoperator $substr)
+(defoperator $substrBytes)
+(defoperator $substrCP)
 (defoperator $toLower)
+(defoperator $toString)
 (defoperator $toUpper)
+(defoperator $concat)
+(defoperator $indexOfBytes)
+(defoperator $indexOfCP)
+(defoperator $ltrim)
+(defoperator $regexFind)
+(defoperator $regexFindAll)
+(defoperator $regexMatch)
+(defoperator $rtrim)
+(defoperator $split)
+(defoperator $strLenBytes)
+(defoperator $subLenCP)
+(defoperator $trim)
+
+(defoperator $sin)
+(defoperator $cos)
+(defoperator $tan)
+(defoperator $asin)
+(defoperator $acos)
+(defoperator $atan)
+(defoperator $atan2)
+(defoperator $asinh)
+(defoperator $acosh)
+(defoperator $atanh)
+(defoperator $radiansToDegrees)
+(defoperator $degreesToRadians)
+
+(defoperator $convert)
+(defoperator $toBool)
+(defoperator $toDecimal)
+(defoperator $toDouble)
+(defoperator $toInt)
+(defoperator $toLong)
+(defoperator $toObjectId)

 (defoperator $dayOfMonth)
 (defoperator $dayOfWeek)
@ -270,19 +406,54 @@
 (defoperator $week)
 (defoperator $year)
 (defoperator $isoDate)
+(defoperator $dateFromParts)
+(defoperator $dateFromString)
+(defoperator $dateToParts)
+(defoperator $dateToString)
+(defoperator $isoDayOfWeek)
+(defoperator $isoWeek)
+(defoperator $isoWeekYear)
+(defoperator $toDate)


 (defoperator $ifNull)
 (defoperator $cond)
+(defoperator $switch)

+;; Geospatial
+(declare $geoWithin $geoIntersects $near $nearSphere $geometry $maxDistance $minDistance $center $centerSphere $box $polygon $slice)
 (defoperator $geoWithin)
 (defoperator $geoIntersects)
 (defoperator $near)
+(defoperator $nearSphere)
+(defoperator $geometry)
+(defoperator $maxDistance)
+(defoperator $minDistance)
+(defoperator $center)
+(defoperator $centerSphere)
+(defoperator $box)
+(defoperator $polygon)

 (defoperator $slice)

 ;; full text search
+(declare $text $meta $search $language $natural $currentDate $isolated $count)
 (defoperator $text)
+(defoperator $meta)
 (defoperator $search)
 (defoperator $language)
 (defoperator $natural)
+
+;; $currentDate operator sets the value of a field to the current date, either as a Date or a timestamp. The default type is Date.
+;;
+;; EXAMPLES:
+;;   (mgcol/update coll { :_id oid } { $currentDate { :lastModified true } })
+(defoperator $currentDate)
+
+;; Isolates intermediate multi-document updates from other clients.
+;;
+;; EXAMPLES:
+;;   (mgcol/update "libraries" { :language "Clojure", $isolated 1 } { $inc { :popularity 1 } } {:multi true})
+(defoperator $isolated)
+
+(defoperator $count)
--- a/src/clojure/monger/query.clj
+++ b/src/clojure/monger/query.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -43,6 +43,7 @@
            [monger.conversion :refer :all]
            [monger.operators :refer :all])
  (:import [com.mongodb DB DBCollection DBObject DBCursor ReadPreference]
+           [java.util.concurrent TimeUnit]
           java.util.List))


@ -96,6 +97,7 @@
           snapshot
           read-preference
           keywordize-fields
+           max-time
           options]
    :or { limit 0 batch-size 256 skip 0 } }]
  (with-open [cursor (doto (.find collection (to-db-object query) (as-field-selector fields))
@ -106,9 +108,11 @@
    (when snapshot
      (.snapshot cursor))
    (when hint
-      (.hint (to-db-object hint)))
+      (.hint cursor (to-db-object hint)))
    (when read-preference
      (.setReadPreference cursor read-preference))
+    (when max-time
+      (.maxTime cursor max-time TimeUnit/MILLISECONDS))
    (when options
      (add-options cursor options))
    (map (fn [x] (from-db-object x keywordize-fields))
@ -154,6 +158,10 @@
  [m ^ReadPreference rp]
  (merge m { :read-preference rp }))

+(defn max-time
+  [m ^long max-time]
+  (merge m { :max-time max-time }))
+
 (defn options
  [m opts]
  (merge m { :options opts }))
@ -167,10 +175,11 @@
  (merge m { :limit per-page :skip (monger.internal.pagination/offset-for page per-page) }))

 (defmacro with-collection
-  [^DB db ^String coll & body]
+  [db coll & body]
  `(let [coll# ~coll
+         ^DB db# ~db
         db-coll# (if (string? coll#)
-                    (.getCollection ~db ^String coll#)
+                    (.getCollection db# coll#)
                    coll#)
         query# (-> (empty-query db-coll#) ~@body)]
     (exec query#)))
--- a/src/clojure/monger/ragtime.clj
+++ b/src/clojure/monger/ragtime.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -34,7 +34,7 @@
 (ns monger.ragtime
  "Ragtime integration"
  (:refer-clojure :exclude [find sort])
-  (:require [ragtime.core      :as ragtime]
+  (:require [ragtime.protocols :as proto]
            [monger.core       :as mg]
            [monger.collection :as mc]
            [monger.query :refer [with-collection find sort]])
@ -47,7 +47,7 @@


 (extend-type com.mongodb.DB
-  ragtime/Migratable
+  proto/DataStore
  (add-migration-id [db id]
    (mc/insert db migrations-collection {:_id id :created_at (Date.)} WriteConcern/FSYNC_SAFE))
  (remove-migration-id [db id]
--- a/src/clojure/monger/result.clj
+++ b/src/clojure/monger/result.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -65,3 +65,8 @@
  (acknowledged?
    [^CommandResult result]
    (.ok result)))
+
+(defn affected-count
+  "Get the number of documents affected"
+  [^WriteResult result]
+  (.getN result))
--- a/src/clojure/monger/ring/session_store.clj
+++ b/src/clojure/monger/ring/session_store.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
--- a/src/clojure/monger/util.clj
+++ b/src/clojure/monger/util.clj
@ -4,7 +4,7 @@
 ;; The APL v2.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team
 ;;
 ;; Licensed under the Apache License, Version 2.0 (the "License");
 ;; you may not use this file except in compliance with the License.
@ -22,7 +22,7 @@
 ;; The EPL v1.0:
 ;;
 ;; ----------------------------------------------------------------------------------
-;; Copyright (c) 2011-2015 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
+;; Copyright (c) 2011-2018 Michael S. Klishin, Alex Petrov, and the ClojureWerkz Team.
 ;; All rights reserved.
 ;;
 ;; This program and the accompanying materials are made available under the terms of
@ -32,12 +32,14 @@
 ;; ----------------------------------------------------------------------------------

 (ns ^{:doc "Provides various utility functions, primarily for working with document ids."} monger.util
+  (:refer-clojure :exclude [random-uuid])
  (:import java.security.SecureRandom
           java.math.BigInteger
           org.bson.types.ObjectId
           com.mongodb.DBObject
           clojure.lang.IPersistentMap
-           java.util.Map))
+           java.util.Map)
+  (:refer-clojure :exclude [random-uuid]))

 ;;
 ;; API
@ -57,7 +59,7 @@
  "Returns a new BSON object id, or converts str to BSON object id"
  ([]
     (ObjectId.))
-  ([s]
+  ([^String s]
     (ObjectId. s)))

 (defprotocol GetDocumentId
@ -73,3 +75,8 @@
  (get-id
    [^IPersistentMap object]
    (or (:_id object) (object "_id"))))
+
+(defn into-array-list
+  "Coerce a j.u.Collection into a j.u.ArrayList."
+  ^java.util.ArrayList [^java.util.Collection coll]
+  (java.util.ArrayList. coll))
--- a/test/monger/test/aggregation_framework_test.clj
+++ b/test/monger/test/aggregation_framework_test.clj
@ -15,6 +15,27 @@

  (use-fixtures :each purge-collections)

+  (deftest test-basic-single-stage-$project-aggregation-no-keywordize
+    (let [batch      [{"state" "CA" "quantity" 1 "price" 199.00}
+                      {"state" "NY" "quantity" 2 "price" 199.00}
+                      {"state" "NY" "quantity" 1 "price" 299.00}
+                      {"state" "IL" "quantity" 2 "price" 11.50 }
+                      {"state" "CA" "quantity" 2 "price" 2.95  }
+                      {"state" "IL" "quantity" 3 "price" 5.50  }]
+          expected    #{{"quantity" 1 "state" "CA"}
+                        {"quantity" 2 "state" "NY"}
+                        {"quantity" 1 "state" "NY"}
+                        {"quantity" 2 "state" "IL"}
+                        {"quantity" 2 "state" "CA"}
+                        {"quantity" 3 "state" "IL"}}]
+      (mc/insert-batch db coll batch)
+      (is (= 6 (mc/count db coll)))
+      (let [result (->>
+                     (mc/aggregate db coll [{$project {"state" 1 "quantity" 1}}] :keywordize false)
+                     (map #(select-keys % ["state" "quantity"]))
+                     (set))]
+        (is (= expected result)))))
+
  (deftest test-basic-single-stage-$project-aggregation
    (let [batch      [{:state "CA" :quantity 1 :price 199.00}
                      {:state "NY" :quantity 2 :price 199.00}
@ -85,4 +106,31 @@
          expected   "IL"]
      (mc/insert-batch db coll batch)
      (let [result (:state (first (mc/aggregate db coll [{$group {:_id 1 :state {$last "$state"}}}])))]
-        (is (= expected result))))))
+        (is (= expected result)))))
+
+  (deftest test-cursor-aggregation
+    (let [batch      [{:state "CA" :quantity 1 :price 199.00}
+                      {:state "NY" :quantity 2 :price 199.00}
+                      {:state "NY" :quantity 1 :price 299.00}
+                      {:state "IL" :quantity 2 :price 11.50 }
+                      {:state "CA" :quantity 2 :price 2.95  }
+                      {:state "IL" :quantity 3 :price 5.50  }]
+          expected    #{{:quantity 1 :state "CA"}
+                        {:quantity 2 :state "NY"}
+                        {:quantity 1 :state "NY"}
+                        {:quantity 2 :state "IL"}
+                        {:quantity 2 :state "CA"}
+                        {:quantity 3 :state "IL"}}]
+      (mc/insert-batch db coll batch)
+      (is (= 6 (mc/count db coll)))
+      (let [result (set (map #(select-keys % [:state :quantity])
+                             (mc/aggregate db coll [{$project {:state 1 :quantity 1}}] :cursor {:batch-size 10})))]
+        (is (= expected result)))))
+
+  (deftest test-explain-aggregate
+    (let [batch [{:state "CA" :price 100}
+                 {:state "CA" :price 10}
+                 {:state "IL" :price 50}]]
+      (mc/insert-batch db coll batch)
+      (let [result (mc/explain-aggregate db coll [{$match {:state "CA"}}])]
+        (is (:ok result))))))
--- a/test/monger/test/atomic_modifiers_test.clj
+++ b/test/monger/test/atomic_modifiers_test.clj
@ -174,7 +174,7 @@


  ;; this is a common mistake, I leave it here to demonstrate it. You almost never
-  ;; actually want to do this! What you really want is to use $pushAll instead of $push. MK.
+  ;; actually want to do this! What you really want is to use $push with $each instead of $push. MK.
  (deftest add-array-value-to-an-existing-array-using-$push-modifier
    (let [coll  "docs"
          oid   (ObjectId.)
@ -228,34 +228,34 @@
             (mc/find-map-by-id db coll oid)))))

  ;;
-  ;; $pushAll
+  ;; $push + $each (formerly $pushAll)
  ;;

-  (deftest initialize-an-array-using-$pushAll-modifier
+  (deftest initialize-an-array-using-$push-and-$each-modifiers
    (let [coll  "docs"
          oid   (ObjectId.)
          title "$pushAll modifier appends multiple values to field"]
      (mc/insert db coll {:_id oid :title title})
-      (mc/update db coll {:_id oid} {$pushAll {:tags ["mongodb" "docs"]}})
+      (mc/update db coll {:_id oid} {$push {:tags {$each ["mongodb" "docs"]}}})
      (is (= {:_id oid :title title :tags ["mongodb" "docs"]}
             (mc/find-map-by-id db coll oid)))))

-  (deftest add-value-to-an-existing-array-using-$pushAll-modifier
+  (deftest add-value-to-an-existing-array-using-$push-and-$each-modifier
    (let [coll  "docs"
          oid   (ObjectId.)
          title "$pushAll modifier appends multiple values to field"]
      (mc/insert db coll {:_id oid :title title :tags ["mongodb"]})
-      (mc/update db coll {:_id oid} {$pushAll {:tags ["modifiers" "docs"]}})
+      (mc/update db coll {:_id oid} {$push {:tags {$each ["modifiers" "docs"]}}})
      (is (= {:_id oid :title title :tags ["mongodb" "modifiers" "docs"]}
             (mc/find-map-by-id db coll oid)))))


-  (deftest double-add-value-to-an-existing-array-using-$pushAll-modifier
+  (deftest double-add-value-to-an-existing-array-using-$push-and-$each-modifier
    (let [coll  "docs"
          oid   (ObjectId.)
          title "$pushAll modifier appends multiple values to field"]
      (mc/insert db coll {:_id oid :title title :tags ["mongodb" "docs"]})
-      (mc/update db coll {:_id oid} {$pushAll {:tags ["modifiers" "docs"]}})
+      (mc/update db coll {:_id oid} {$push {:tags {$each ["modifiers" "docs"]}}})
      (is (= {:_id oid :title title :tags ["mongodb" "docs" "modifiers" "docs"]}
             (mc/find-map-by-id db coll oid)))))

--- a/test/monger/test/authentication_test.clj
+++ b/test/monger/test/authentication_test.clj
@ -12,12 +12,12 @@
 (when-not (System/getenv "CI")
  (deftest ^{:authentication true} connect-to-mongo-via-uri-without-credentials
    (let [{:keys [conn db]} (mg/connect-via-uri "mongodb://127.0.0.1/monger-test4")]
-      (is (= (-> conn .getAddress ^InetAddress (.sameHost "127.0.0.1"))))))
+      (is (-> conn .getAddress (.sameHost "127.0.0.1")))))

  (deftest ^{:authentication true} connect-to-mongo-via-uri-with-valid-credentials
-    (let [{:keys [conn db]} (mg/connect-via-uri "mongodb://clojurewerkz/monger:monger@127.0.0.1/monger-test4")]
+    (let [{:keys [conn db]} (mg/connect-via-uri "mongodb://clojurewerkz%2Fmonger:monger@127.0.0.1/monger-test4")]
      (is (= "monger-test4" (.getName db)))
-      (is (= (-> conn .getAddress ^InetAddress (.sameHost "127.0.0.1"))))
+      (is (-> conn .getAddress (.sameHost "127.0.0.1")))
      (mc/remove db "documents")
      ;; make sure that the database is selected
      ;; and operations get through.
@ -27,7 +27,7 @@
 (if-let [uri (System/getenv "MONGOHQ_URL")]
  (deftest ^{:external true :authentication true} connect-to-mongo-via-uri-with-valid-credentials
    (let [{:keys [conn db]} (mg/connect-via-uri uri)]
-      (is (= (-> conn .getAddress ^InetAddress (.sameHost "127.0.0.1")))))))
+      (is (-> conn .getAddress (.sameHost "127.0.0.1"))))))


 ;;
@ -37,6 +37,6 @@
 (deftest ^{:authentication true} test-authentication-with-valid-credentials
    ;; see ./bin/ci/before_script.sh. MK.
  (doseq [s ["monger-test" "monger-test2" "monger-test3" "monger-test4"]]
-    (let [creds (mcr/for "clojurewerkz/monger" "monger-test" "monger")
+    (let [creds (mcr/create "clojurewerkz/monger" "monger-test" "monger")
          conn  (mg/connect-with-credentials "127.0.0.1" creds)]
      (mc/remove (mg/get-db conn "monger-test") "documents"))))
--- a/test/monger/test/cache_test.clj
+++ b/test/monger/test/cache_test.clj
@ -1,126 +0,0 @@
-(ns monger.test.cache-test
-  (:require [monger.core        :as mg]
-            [monger.collection  :as mc]
-            [clojure.core.cache :refer :all]
-            [clojure.test :refer :all]
-            [monger.cache :refer :all])
-  (:import [clojure.core.cache BasicCache FIFOCache LRUCache TTLCache]
-           java.util.UUID))
-
-;;
-;; Playground/Tests. These were necessary because clojure.core.cache has
-;; little documentation, incomplete test suite and
-;; slightly non-standard (although necessary to support all those cache variations)
-;; cache operations protocol.
-;;
-;; This is by no means clear or complete either but it did the job of helping me
-;; explore the API.
-
-(deftest ^{:cache true}
-  test-has?-with-basic-cache
-  (testing "that has? returns false for misses"
-    (let [c (BasicCache. {})]
-      (are [v] (is (false? (has? c v)))
-           :missing-key
-           "missing-key"
-           (gensym "missing-key"))))
-  (testing "that has? returns true for hits"
-    (let [c (BasicCache. {:skey "Value" :lkey (Long/valueOf 10000) "kkey" :keyword})]
-      (are [v] (is (has? c v))
-           :skey
-           :lkey
-           "kkey"))))
-
-
-(deftest ^{:cache true}
-  test-lookup-with-basic-cache
-  (testing "that lookup returns nil for misses"
-    (let [c (BasicCache. {})]
-      (are [v] (is (nil? (lookup c v)))
-           :missing-key
-           "missing-key"
-           (gensym "missing-key"))))
-  (testing "that lookup returns cached values for hits"
-    (let [l (Long/valueOf 10000)
-          c (BasicCache. {:skey "Value" :lkey l "kkey" :keyword})]
-      (are [v k] (is (= v (lookup c k)))
-           "Value"   :skey
-           l         :lkey
-           :keyword  "kkey"))))
-
-(deftest ^{:cache true}
-  test-evict-with-basic-cache
-  (testing "that evict has no effect for keys that do not exist"
-    (let [c (atom (BasicCache. {:a 1 :b 2}))]
-      (swap! c evict :missing-key)
-      (is (has? @c :a))
-      (is (has? @c :b))))
-  (testing "that evict removes keys that did exist"
-    (let [c (atom (BasicCache. {:skey "Value" "kkey" :keyword}))]
-      (is (has? @c :skey))
-      (is (= "Value"  (lookup @c :skey)))
-      (swap! c evict :skey)
-      (is (not (has? @c :skey)))
-      (is (= nil  (lookup @c :skey)))
-      (is (has? @c "kkey"))
-      (is (= :keyword (lookup @c "kkey"))))))
-
-(deftest ^{:cache true}
-  test-seed-with-basic-cache
-  (testing "that seed returns a new value"
-    (let [c (atom (BasicCache. {}))]
-      (swap! c seed {:a 1 :b "b" "c" :d})
-      (are [k v] (do
-                   (is (has? @c k))
-                   (is (= v (lookup @c k))))
-           :a 1
-           :b "b"
-           "c" :d))))
-
-
-;;
-;; Tests
-;;
-
-(let [conn (mg/connect)
-      db   (mg/get-db conn "monger-test")]
-  (use-fixtures :each (fn [f]
-                        (mc/remove db "basic_monger_cache_entries")
-                        (f)
-                        (mc/remove db "basic_monger_cache_entries")))
-
-
-  (deftest ^{:cache true}
-    test-has?-with-basic-monger-cache
-    (testing "that has? returns false for misses"
-      (let [coll "basic_monger_cache_entries"
-            c    (basic-monger-cache-factory db coll)]
-        (is (not (has? c (str (UUID/randomUUID)))))
-        (is (not (has? c (str (UUID/randomUUID)))))))
-    (testing "that has? returns true for hits"
-      (let [coll "basic_monger_cache_entries"
-            c    (basic-monger-cache-factory db coll {"a" 1 "b" "cache" "c" 3/4})]
-        (is (has? c "a"))
-        (is (has? c "b"))
-        (is (has? c "c"))
-        (is (not (has? c "d"))))))
-
-
-  (deftest ^{:cache true}
-    test-lookup-with-basic-monger-cache
-    (testing "that lookup returns nil for misses"
-      (let [coll "basic_monger_cache_entries"
-            c    (basic-monger-cache-factory db coll)]
-        (are [v] (is (nil? (lookup c v)))
-             :missing-key
-             "missing-key"
-             (gensym "missing-key"))))
-    (testing "that lookup returns cached values for hits"
-      (let [l (Long/valueOf 10000)
-            coll "basic_monger_cache_entries"
-            c    (basic-monger-cache-factory db coll {:skey "Value" :lkey l "kkey" :keyword})]
-        (are [v k] (is (= v (lookup c k)))
-             "Value"   :skey
-             l         :lkey
-             "keyword" "kkey")))))
-
--- a/test/monger/test/collection_test.clj
+++ b/test/monger/test/collection_test.clj
@ -134,6 +134,7 @@
                      {:state "CA" :quantity 2 :price 2.95  }
                      {:state "IL" :quantity 3 :price 5.50  }]]
      (mc/insert-batch db collection batch)
+      (is (= ["CA" "IL" "NY"] (sort (mc/distinct db collection :state))))
      (is (= ["CA" "IL" "NY"] (sort (mc/distinct db collection :state {}))))
      (is (= ["CA" "NY"] (sort (mc/distinct db collection :state {:price {$gt 100.00}}))))))

--- a/test/monger/test/conversion_test.clj
+++ b/test/monger/test/conversion_test.clj
@ -4,7 +4,8 @@
            [monger.conversion :refer :all])
  (:import [com.mongodb DBObject BasicDBObject BasicDBList]
           [java.util Date Calendar List ArrayList]
-           org.bson.types.ObjectId))
+           org.bson.types.ObjectId
+           (org.bson.types Decimal128)))


 ;;
@ -101,6 +102,13 @@
  (is (= 2 (from-db-object 2 false)))
  (is (= 2 (from-db-object 2 true))))

+(deftest convert-decimal-from-dbobject
+  (is (= 2.3M (from-db-object (Decimal128. 2.3M) false)))
+  (is (= 2.3M (from-db-object (Decimal128. 2.3M) true)))
+  (is (= 2.3M (from-db-object (Decimal128/parse "2.3") true)))
+  (is (not= 2.32M (from-db-object (Decimal128/parse "2.3") true)))
+  )
+
 (deftest convert-float-from-dbobject
  (is (= 3.3 (from-db-object 3.3 false)))
  (is (= 3.3 (from-db-object 3.3 true))))
@ -112,20 +120,20 @@
                (.put "name" name)
                (.put "age"  age))
        output (from-db-object input false)]
-    (is (= (output { "name" name, "age" age })))
+    (is (= output { "name" name, "age" age }))
    (is (= (output "name") name))
    (is (nil? (output :name)))
    (is (= (output "age") age))
    (is (nil? (output "points")))))

-(deftest convert-flat-db-object-to-map-without-keywordizing
+(deftest convert-flat-db-object-to-map-with-keywordizing
  (let [name   "Michael"
        age    26
        input (doto (BasicDBObject.)
                (.put "name" name)
                (.put "age"  age))
        output (from-db-object input true)]
-    (is (= (output { :name name, :age age })))
+    (is (= output { :name name, :age age }))
    (is (= (output :name) name))
    (is (nil? (output "name")))
    (is (= (output :age) age))
--- a/test/monger/test/core_test.clj
+++ b/test/monger/test/core_test.clj
@ -55,3 +55,35 @@
        dbs  (mg/get-db-names conn)]  
    (is (not (empty? dbs)))
    (is (dbs "monger-test"))))
+
+(deftest monger-options-test
+  (let [opts {:always-use-mbeans true
+              :application-name "app"
+              :connect-timeout 1
+              :connections-per-host 1
+              :cursor-finalizer-enabled true
+              :description "Description"
+              :heartbeat-connect-timeout 1
+              :heartbeat-frequency 1
+              :heartbeat-socket-timeout 1
+              :local-threshold 1
+              :max-connection-idle-time 1
+              :max-connection-life-time 1
+              :max-wait-time 1
+              :min-connections-per-host 1
+              :min-heartbeat-frequency 1
+              :required-replica-set-name "rs"
+              :retry-writes true
+              :server-selection-timeout 1
+              :socket-keep-alive true
+              :socket-timeout 1
+              :ssl-enabled true
+              :ssl-invalid-host-name-allowed true
+              :threads-allowed-to-block-for-connection-multiplier 1
+              :uuid-representation org.bson.UuidRepresentation/STANDARD
+              :write-concern com.mongodb.WriteConcern/JOURNAL_SAFE}]
+    (is (instance? com.mongodb.MongoClientOptions$Builder (mg/mongo-options-builder opts)))))
+
+(deftest connect-to-uri-without-db-name
+  (let [uri "mongodb://localhost:27017"]
+    (is (thrown? IllegalArgumentException (mg/connect-via-uri uri)))))
--- a/test/monger/test/gridfs_test.clj
+++ b/test/monger/test/gridfs_test.clj
@ -78,11 +78,21 @@
                  (content-type "application/octet-stream"))
      (is (= 1 (count (gridfs/all-files fs))))))

+  (deftest ^{:gridfs true} test-deleting-file-instance-on-disk-after-storing
+    (let [tmp-file (File/createTempFile "monger.test.gridfs" "test-deleting-file-instance-on-disk-after-storing")
+          _        (spit tmp-file "to be deleted")]
+      (is (= 0 (count (gridfs/all-files fs))))
+      (store-file (make-input-file fs tmp-file)
+                  (filename "test-deleting-file-instance-on-disk-after-storing")
+                  (content-type "application/octet-stream"))
+      (is (= 1 (count (gridfs/all-files fs))))
+      (is (.delete tmp-file))))
+


  (deftest ^{:gridfs true} test-finding-individual-files-on-gridfs
    (testing "gridfs/find-one"
-      (purge-gridfs*)    
+      (purge-gridfs*)
      (let [input   "./test/resources/mongo/js/mapfun1.js"
            ct     "binary/octet-stream"
            fname  "monger.test.gridfs.file5"
@ -121,7 +131,46 @@
          (is (= {:meta "data"} (:metadata m))))
        (are [a query] (is (= a (:md5 (gridfs/find-one-as-map fs query))))
             md5 {:_id (:_id stored)}
-             md5 {:md5 md5}))))
+             md5 {:md5 md5})))
+    (testing "gridfs/find-by-id"
+      (purge-gridfs*)
+      (let [input   "./test/resources/mongo/js/mapfun1.js"
+            ct     "binary/octet-stream"
+            fname  "monger.test.gridfs.file5"
+            md5    "14a09deabb50925a3381315149017bbd"
+            stored (store-file (make-input-file fs input)
+                               (filename fname)
+                               (content-type ct))]
+        (is (= 1 (count (gridfs/all-files fs))))
+        (is (:_id stored))
+        (is (:uploadDate stored))
+        (is (= 62 (:length stored)))
+        (is (= md5 (:md5 stored)))
+        (is (= fname (:filename stored)))
+        (is (= ct (:contentType stored)))
+        (are [a id] (is (= a (:md5 (from-db-object (gridfs/find-by-id fs id) true))))
+             md5 (:_id stored))))
+    (testing "gridfs/find-map-by-id"
+      (purge-gridfs*)
+      (let [input   "./test/resources/mongo/js/mapfun1.js"
+            ct      "binary/octet-stream"
+            fname "monger.test.gridfs.file6"
+            md5      "14a09deabb50925a3381315149017bbd"
+            stored  (store-file (make-input-file fs input)
+                                (filename fname)
+                                (metadata (to-db-object {:meta "data"}))
+                                (content-type ct))]
+        (is (= 1 (count (gridfs/all-files fs))))
+        (is (:_id stored))
+        (is (:uploadDate stored))
+        (is (= 62 (:length stored)))
+        (is (= md5 (:md5 stored)))
+        (is (= fname (:filename stored)))
+        (is (= ct (:contentType stored)))
+        (let [m (gridfs/find-map-by-id fs (:_id stored))]
+          (is (= {:meta "data"} (:metadata m))))
+        (are [a id] (is (= a (:md5 (gridfs/find-map-by-id fs id))))
+          md5 (:_id stored)))))

  (deftest ^{:gridfs true} test-finding-multiple-files-on-gridfs
    (let [input   "./test/resources/mongo/js/mapfun1.js"
--- a/test/monger/test/indexing_test.clj
+++ b/test/monger/test/indexing_test.clj
@ -13,34 +13,35 @@
    (let [collection "libraries"]
      (mc/drop-indexes db collection)
      (mc/create-index db collection {"language" 1})
-      (is (= "language_"
+      (is (= "language_1"
             (:name (second (mc/indexes-on db collection)))))
      (mc/drop-indexes db collection)
-      (mc/create-index db collection ["language"])
-      (mc/drop-index db collection {"language" 1})
      (is (nil? (second (mc/indexes-on db collection))))
      (mc/ensure-index db collection (array-map "language" 1) {:unique true})
-      (is (= "language_"
+      (is (= "language_1"
             (:name (second (mc/indexes-on db collection)))))
      (mc/drop-indexes db collection)
      (mc/ensure-index db collection (array-map "language" 1))
      (mc/drop-indexes db collection)
      (mc/ensure-index db collection (array-map "language" 1) {:unique true})
+      (mc/drop-indexes db collection)
+      (mc/ensure-index db collection (array-map "language" 1) "index-name" true)
      (mc/drop-indexes db collection)))

  (deftest ^{:indexing true :time-consuming true} test-ttl-collections
    (let [coll  "recent_events"
-          ttl   30
-          sleep 120]
+          ttl   15
+          sleep 65]
      (mc/remove db coll)
+      (mc/drop-indexes db coll)
      (mc/ensure-index db coll (array-map :created-at 1) {:expireAfterSeconds ttl})
      (dotimes [i 100]
        (mc/insert db coll {:type "signup" :created-at (-> i seconds ago) :i i}))
      (dotimes [i 100]
        (mc/insert db coll {:type "signup" :created-at (-> i seconds from-now) :i i}))
      (is (= 200 (mc/count db coll {:type "signup"})))
-      ;; sleep for 65 seconds. MongoDB 2.1.2 seems to run TTLMonitor once per minute, according to
-      ;; the log. MK.
+      ;; sleep for > 60 seconds. MongoDB seems to run TTLMonitor once per minute, according to
+      ;; the log.
      (println (format "Now sleeping for %d seconds to test TTL collections!" sleep))
      (Thread/sleep (* sleep 1000))
      (println (format "Documents in the TTL collection: %d" (mc/count db coll {:type "signup"})))
--- a/test/monger/test/map_reduce_test.clj
+++ b/test/monger/test/map_reduce_test.clj
@ -1,62 +0,0 @@
-(ns monger.test.map-reduce-test
-  (:import  [com.mongodb WriteResult WriteConcern DBCursor DBObject MapReduceOutput MapReduceCommand MapReduceCommand$OutputType]
-            org.bson.types.ObjectId
-            java.util.Date)
-  (:require [monger.collection       :as mc]
-            [monger.core             :as mg]
-            [clojurewerkz.support.js :as js]
-            [clojure.test :refer :all]
-            [monger.operators :refer :all]
-            [monger.conversion :refer :all]))
-
-(let [conn (mg/connect)
-      db   (mg/get-db conn "monger-test")]
-  (use-fixtures :each (fn [f]
-                        (mc/remove db "widgets")
-                        (f)
-                        (mc/remove db "widgets")))
-
-  (let [collection "widgets"
-        mapper     (js/load-resource "resources/mongo/js/mapfun1.js")
-        reducer    "function(key, values) {
-                    var result = 0;
-                    values.forEach(function(v) { result += v });
-
-                    return result;
-                   }"
-        batch      [{ :state "CA" :quantity 1 :price 199.00 }
-                    { :state "NY" :quantity 2 :price 199.00 }
-                    { :state "NY" :quantity 1 :price 299.00 }
-                    { :state "IL" :quantity 2 :price 11.50  }
-                    { :state "CA" :quantity 2 :price 2.95   }
-                    { :state "IL" :quantity 3 :price 5.50   }]
-        expected    [{:_id "CA", :value 204.9} {:_id "IL", :value 39.5} {:_id "NY", :value 697.0}]]
-    (deftest test-basic-inline-map-reduce-example
-      (mc/remove db collection)
-      (mc/insert-batch db collection batch)
-      (let [output  (mc/map-reduce db collection mapper reducer nil MapReduceCommand$OutputType/INLINE {})
-            results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
-        (is (= expected results))))
-
-    (deftest test-basic-map-reduce-example-that-replaces-named-collection
-      (mc/remove db collection)
-      (mc/insert-batch db collection batch)
-      (let [output  (mc/map-reduce db collection mapper reducer "mr_outputs" {})
-            results (from-db-object ^DBObject (.results ^MapReduceOutput output) true)]
-        (is (= 3 (mg/count results)))
-        (is (= expected
-               (map #(from-db-object % true) (seq results))))
-        (is (= expected
-               (map #(from-db-object % true) (mc/find db "mr_outputs"))))
-        (.drop ^MapReduceOutput output)))
-
-    (deftest test-basic-map-reduce-example-that-merged-results-into-named-collection
-      (mc/remove db collection)
-      (mc/insert-batch db collection batch)
-      (mc/map-reduce db collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})
-      (mc/insert db collection { :state "OR" :price 17.95 :quantity 4 })
-      (let [^MapReduceOutput output (mc/map-reduce db collection mapper reducer "merged_mr_outputs" MapReduceCommand$OutputType/MERGE {})]
-        (is (= 4 (mg/count output)))
-        (is (= ["CA" "IL" "NY" "OR"]
-               (map :_id (mc/find-maps db "merged_mr_outputs"))))
-        (.drop ^MapReduceOutput output)))))
--- a/test/monger/test/query_operators_test.clj
+++ b/test/monger/test/query_operators_test.clj
@ -3,10 +3,28 @@
            [monger.collection :as mc]
            [monger.js         :as js]
            [clojure.test :refer :all]
-            [monger.operators :refer :all]))
+            [clojure.set :refer [difference]]
+            [monger.operators :refer :all])
+  (:import [com.mongodb QueryOperators]))

 ;; (use-fixtures :each purge-people purge-docs purge-things purge-libraries)

+(deftest every-query-operator-is-defined
+  (let [driver-query-operators (->> (.getDeclaredFields QueryOperators) (map #(.get % nil)) set)
+        monger-query-operators (->> (ns-publics 'monger.operators) (map (comp name first)) set)
+        ; $within is deprecated and replaced by $geoWithin since v2.4.
+        ; $uniqueDocs is deprecated since v2.6.
+        deprecated-query-operators #{"$within" "$uniqueDocs"}
+        ; Query modifier operators that are deprecated in the mongo shell since v3.2
+        deprecated-meta-operators #{"$comment" "$explain" "$hint" "$maxScan"
+                                    "$maxTimeMS" "$max" "$min" "$orderby"
+                                    "$returnKey" "$showDiskLoc" "$snapshot" "$query"}
+        undefined-non-deprecated-operators (difference driver-query-operators
+                                                       deprecated-query-operators
+                                                       deprecated-meta-operators
+                                                       monger-query-operators)]
+    (is (= #{} undefined-non-deprecated-operators))))
+
 (let [conn (mg/connect)
      db   (mg/get-db conn "monger-test")]
  (defn purge-collections
@ -36,17 +54,29 @@
           2 {:users {$lte 5}}
           1 {:users {$gt 10 $lt 150}})))

+  ;;
+  ;; $eq
+  ;;
+
+  (deftest find-with-eq-operator
+    (let [collection "libraries"]
+      (mc/insert-batch db collection [{:language "Ruby"    :name "mongoid"  :users 1 :displayName nil}
+                                      {:language "Clojure" :name "langohr"  :users 5}
+                                      {:language "Clojure" :name "incanter" :users 15}
+                                      {:language "Scala"   :name "akka"     :users 150}])
+      (is (= 2 (.count (mc/find db collection {:language {$eq "Clojure"}}))))))
+
  ;;
  ;; $ne
  ;;

-  (deftest find-with-and-or-operators
+  (deftest find-with-ne-operator
    (let [collection "libraries"]
      (mc/insert-batch db collection [{:language "Ruby"    :name "mongoid"  :users 1}
                                      {:language "Clojure" :name "langohr"  :users 5}
                                      {:language "Clojure" :name "incanter" :users 15}
                                      {:language "Scala"   :name "akka"     :users 150}])
-      (is (= 2 (.count (mc/find db collection {$ne {:language "Clojure"}}))))))
+      (is (= 2 (.count (mc/find db collection {:language {$ne "Clojure"}}))))))


  ;;
@ -107,4 +137,10 @@
           {:language {$regex "clo.*" $options "i"}} 2
           {:name     {$regex "aK.*" $options "i"}} 1
           {:language {$regex ".*by"}} 1
-           {:language {$regex ".*ala.*"}} 1))))
+           {:language {$regex ".*ala.*"}} 1)))
+
+  (deftest find-with-js-expression
+    (let [collection "people"]
+      (mc/insert-batch db collection [{:name "Bob" :placeOfBirth "New York" :address {:city "New York"}}
+                                      {:name "Alice" :placeOfBirth "New York" :address {:city "Los Angeles"}}])
+      (is (= 1 (.count (mc/find db collection {$where "this.placeOfBirth === this.address.city"})))))))
--- a/test/monger/test/ragtime_test.clj
+++ b/test/monger/test/ragtime_test.clj
@ -2,7 +2,7 @@
  (:require [monger.core :as mg]
            [monger.collection  :as mc]
            monger.ragtime
-            [ragtime.core :refer :all]
+            [ragtime.protocols :refer :all]
            [clojure.test :refer :all]))


--- a/test/monger/test/regular_finders_test.clj
+++ b/test/monger/test/regular_finders_test.clj
@ -8,7 +8,8 @@
            [monger.result     :as mgres]
            [monger.conversion :as mgcnv]
            [clojure.test :refer :all]
-            [monger.operators :refer :all]))
+            [monger.operators :refer :all]
+            [monger.conversion :refer [to-db-object]]))

 (let [conn (mg/connect)
      db   (mg/get-db conn "monger-test")]
@ -147,36 +148,36 @@
          doc-id     (mu/random-uuid)
          doc        { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-by-id db collection doc-id))))))
+      (is (= (to-db-object doc) (mc/find-by-id db collection doc-id)))))

  (deftest find-full-document-by-object-id-when-document-does-exist
    (let [collection "libraries"
          doc-id     (ObjectId.)
          doc        { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-by-id db collection doc-id))))))
+      (is (= (to-db-object doc) (mc/find-by-id db collection doc-id)))))

  (deftest find-full-document-map-by-string-id-when-document-does-exist
    (let [collection "libraries"
          doc-id     (mu/random-uuid)
          doc        { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-map-by-id db collection doc-id))))))
+      (is (= doc (mc/find-map-by-id db collection doc-id)))))

  (deftest find-full-document-map-by-object-id-when-document-does-exist
    (let [collection "libraries"
          doc-id     (ObjectId.)
          doc        { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-map-by-id db collection doc-id))))))
+      (is (= doc (mc/find-map-by-id db collection doc-id)))))

  (deftest find-partial-document-by-id-when-document-does-exist
    (let [collection "libraries"
          doc-id     (mu/random-uuid)
          doc        { :data-store "MongoDB", :language "Clojure", :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= ({ :language "Clojure" }
-              (mc/find-by-id db collection doc-id [ :language ]))))))
+      (is (= (to-db-object { :_id doc-id :language "Clojure" })
+             (mc/find-by-id db collection doc-id [ :language ])))))


  (deftest find-partial-document-as-map-by-id-when-document-does-exist
@ -232,7 +233,7 @@
    (let [collection "libraries"
          _          (mc/insert db collection { :language "Clojure", :name "monger" })
          result     (mc/find db collection { :language "Clojure"} [:language])]
-      (is (= (seq [:_id :language]) (keys (mgcnv/from-db-object (.next result) true))))))
+      (is (= (set [:_id :language]) (-> (mgcnv/from-db-object (.next result) true) keys set)))))

  (deftest find-and-iterate-over-multiple-documents-the-hard-way
    (let [collection "libraries"]
@ -281,4 +282,12 @@
        (doseq [i clojure-libs]
          (let [doc (mgcnv/from-db-object i true)]
            (is (= (:language doc) "Clojure"))))
-        (is (empty? (mc/find db collection { :language "Erlang" } [:name])))))))
+        (is (empty? (mc/find db collection { :language "Erlang" } [:name]))))))
+
+  (deftest find-maps-with-keywordize-false
+    (let [collection "libraries"]
+      (mc/insert-batch db collection [{ :language "Clojure", :name "monger" }
+                                      { :language "Clojure", :name "langohr" }])
+      (let [results (mc/find-maps db collection {:name "langohr"} [] false)]
+        (is (= 1 (.count results)))
+        (is (= (get (first results) "language") "Clojure"))))))
--- a/test/monger/test/result_test.clj
+++ b/test/monger/test/result_test.clj
@ -15,8 +15,11 @@
          date         (Date.)
          doc          { :created-at date :data-store "MongoDB" :language "Clojure" :_id doc-id }
          modified-doc { :created-at date :data-store "MongoDB" :language "Erlang"  :_id doc-id }]
-      (is (not (mgres/updated-existing? (mc/update db collection { :language "Clojure" } doc {:upsert true}))))
-      (is (mgres/updated-existing? (mc/update db collection { :language "Clojure" }      doc {:upsert true})))
-      (mgres/updated-existing? (mc/update db collection { :language "Clojure" } modified-doc {:multi false :upsert true}))
+      (let [result (mc/update db collection { :language "Clojure" } doc {:upsert true})]
+        (is (not (mgres/updated-existing? result)))
+        (is (= 1 (mgres/affected-count result))))
+      (is (mgres/updated-existing? (mc/update db collection { :language "Clojure" } doc {:upsert true})))
+      (is (mgres/updated-existing? (mc/update db collection { :language "Clojure" } modified-doc {:multi false :upsert true})))
+      (is (= 1 (mgres/affected-count (mc/remove db collection { :_id doc-id }))))
      (mc/remove db collection)
      (mg/disconnect conn))))
--- a/test/monger/test/stress_test.clj
+++ b/test/monger/test/stress_test.clj
@ -1,7 +1,7 @@
 (ns monger.test.stress-test
  (:require [monger.core :as mg]
            [monger.collection :as mc]
-            [monger.conversion :refer [to-db-object]]
+            [monger.conversion :refer [to-db-object from-db-object]]
            [clojure.test :refer :all])
  (:import [com.mongodb WriteConcern]
           java.util.Date))
@ -30,4 +30,11 @@
        (mc/remove db collection)
        (println "Inserting " n " documents...")
        (time (mc/insert-batch db collection docs))
-        (is (= n (mc/count db collection)))))))
+        (is (= n (mc/count db collection))))))
+
+  (deftest ^{:performance true} convert-large-number-of-dbojects-to-maps
+    (doseq [n [10 100 1000 20000 40000]]
+      (let [docs (map (fn [i]
+                        (to-db-object {:title "Untitled" :created-at (Date.) :number i}))
+                      (take n (iterate inc 1)))]
+        (time (doall (map (fn [x] (from-db-object x true)) docs)))))))
--- a/test/monger/test/updating_test.clj
+++ b/test/monger/test/updating_test.clj
@ -33,9 +33,9 @@
          doc          { :created-at date, :data-store "MongoDB", :language "Clojure", :_id doc-id }
          modified-doc { :created-at date, :data-store "MongoDB", :language "Erlang",  :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-by-id db collection doc-id))))
-      (mc/update db collection { :_id doc-id } { :language "Erlang" })
-      (is (= (modified-doc (mc/find-by-id db collection doc-id))))))
+      (is (= (to-db-object doc) (mc/find-by-id db collection doc-id)))
+      (mc/update db collection { :_id doc-id } { $set { :language "Erlang" } })
+      (is (= (to-db-object modified-doc) (mc/find-by-id db collection doc-id)))))

  (deftest ^{:updating true} update-document-by-id-without-upsert-using-update-by-id
    (let [collection "libraries"
@ -44,9 +44,9 @@
          doc          { :created-at date, :data-store "MongoDB", :language "Clojure", :_id doc-id }
          modified-doc { :created-at date, :data-store "MongoDB", :language "Erlang",  :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-by-id db collection doc-id))))
-      (mc/update-by-id db collection doc-id { :language "Erlang" })
-      (is (= (modified-doc (mc/find-by-id db collection doc-id))))))
+      (is (= (to-db-object doc) (mc/find-by-id db collection doc-id)))
+      (mc/update-by-id db collection doc-id { $set { :language "Erlang" } })
+      (is (= (to-db-object modified-doc) (mc/find-by-id db collection doc-id)))))

  (deftest ^{:updating true} update-nested-document-fields-without-upsert-using-update-by-id
    (let [collection "libraries"
@ -55,9 +55,9 @@
          doc          { :created-at date :data-store "MongoDB" :language { :primary "Clojure" } :_id doc-id }
          modified-doc { :created-at date :data-store "MongoDB" :language { :primary "Erlang"  } :_id doc-id }]
      (mc/insert db collection doc)
-      (is (= (doc (mc/find-by-id db collection doc-id))))
+      (is (= (to-db-object doc) (mc/find-by-id db collection doc-id)))
      (mc/update-by-id db collection doc-id { $set { "language.primary" "Erlang" }})
-      (is (= (modified-doc (mc/find-by-id db collection doc-id))))))
+      (is (= (to-db-object modified-doc) (mc/find-by-id db collection doc-id)))))


  (deftest ^{:updating true} update-multiple-documents
@ -151,7 +151,7 @@
      (is (= 1 (mc/count db collection)))
      (is (mr/updated-existing? (mc/update db collection { :language "Clojure" } modified-doc {:multi false :upsert true})))
      (is (= 1 (mc/count db collection)))
-      (is (= (modified-doc (mc/find-by-id db collection doc-id))))
+      (is (= (to-db-object modified-doc) (mc/find-by-id db collection doc-id)))
      (mc/remove db collection)))

  (deftest ^{:updating true} upsert-a-document-using-upsert
@ -165,5 +165,5 @@
      (is (= 1 (mc/count db collection)))
      (is (mr/updated-existing? (mc/upsert db collection {:language "Clojure"} modified-doc {:multi false})))
      (is (= 1 (mc/count db collection)))
-      (is (= (modified-doc (mc/find-by-id db collection doc-id))))
+      (is (= (to-db-object modified-doc) (mc/find-by-id db collection doc-id)))
      (mc/remove db collection))))