Configuring MongoDB Java driver logging from Clojure using timbre

I’ve mentioned in the past how you can configure the MongoDB Java driver output from Java. Most Clojure applications that use MongoDB use a database driver that wraps the official MongoDB Java driver. I personally use monger for a lot of my projects, but also occasionally created my own wrapper. The methods described in this post should be applicable to other Clojure MongoDB drivers as long as they wrap the official MongoDB Java driver. They should also work if your application directly wraps the MongoDB Java driver itself rather than using a more idiomatic driver.

The MongoDB Java driver helpfully logs a lot of information via slf4j under the assumption that your application configures its output to an appropriate place. That’s all very nice if you already use slf4j and have it configured it accordingly. However, if you use Clojure (or you’re me) that’s probably not the case. At least I didn’t set up slf4j in any of my Clojure code.

Enter timbre. Timbre is a logging library for Clojure that optionally interoperates with Java logging libraries like slf4j. You can use the interop for slf4j via the slf4j-timbre library and most importantly for me, control the slf4j logging output from Clojure directly.

The timbre configuration for timbre 5.1 that I use in some of my Clojure projects  that interact with MongoDB is as follows:

(timbre/merge-config! {:min-level `[[#{"org.mongodb.*"} :error]]})

This configuration sets a minimum log level of error for any log output from the org.mongodb namespace. This is the namespace the Java driver uses for logging. Normally you wouldn’t just configure the minimum log level for the MongoDB Java driver, but for several different tools. As a result the timbre configuration for my projets tends to look more like this:

(timbre/merge-config! {:min-level `[[#{"org.mongodb.*"} :error] [#{"clj-ssh.*"} :warn] [#{"*"} :debug]]})

The configuration above sets a default minimum log output level of :debug. It also sets two additional log levels specific to the org.mongodb and clj-ssh namespaces. Obviously if you use other tools or have other namespaces that require special handling, you would add those to the configuration settings as well.

Note that using timbre and slj4f-timbre requires the following dependencies in project.clj:

:dependencies [[org.clojure/clojure        "1.10.0"]
               [com.taoensso/timbre        "5.1.0"] 
               [com.fzakaria/slf4j-timbre  "0.3.14"] ;; Attempt to send all log output through timbre
               [org.slf4j/jul-to-slf4j     "1.7.14"]]

MongoDB tips – How to find documents containing a non-empty array in MongoDB

Why do we need another post cluttering up the Interpipes on how to find a set of documents in a MongoDB collection that contain a non-empty array field? It’s not like we suddenly have a shortage of posts and articles on this topic after all. Well, it maybe a shocking revelation but not all of these posts and Stack Overflow answers are, well, as correct as I’d like them to be.

Go on, what’s wrong with the typical approach then?

When you search for ‘how to find MongoDB documents with non-empty arrays’, you usually end up finding suggestions like db.test.find({ my_huge_array: { $exists: true, $ne: [] } }) or variations thereof. And this approach works most of the time, especially if you have a fairly rigid schema and a little helping of luck. The only problem with this approach is that, well, it’s almost correct but not correct enough to be robust. But, I hear you say you’ve used this find clause in production for years and it just works, what I am talking about?

Read More

How to rename a database in MongoDB

MongoDB has a handy command to rename a collection, db.collectionName.renameCollection(). There is currently no equivalent to rename a database. Now if we accept that from time to time, one positively, absolutely just has to rename a database in MongoDB, well, there are a couple of options. Unfortunately they aren’t quite as straight forward as single MongoDB command. All methods for renaming a database in MongoDB also take a fair amount of time and/or disk space to complete. Keep this in mind when you try to use any of them.

Read More

Talk – Getting started with geospatial data in MongoDB (MDBW 2017)

I’ve been meaning to post this link for quite a while now but keep forgetting to do so. If you are planning to store geospatial data in MongoDB, the database offers you a variety of ways to deal with geospatial-specific data storage and queries.

I gave an introductory talk on this subject at MongoDB World 2017 and you can find a recording of the talk here.

Disclaimer: I work for MongoDB as a Consulting Engineer and this is my personal blog. Any opinions expressed herein are entirely my own and do not reflect opinions of my employers, past present or future.

Using tuned.conf to disable mongod startup warnings on RHEL/CentOS 7

RHEL 7 – and CentOS 7, which I used for this test – use tuned.conf to set a lot of system settings. Several of the tuned settings affect MongoDB’s performance; some are important enough that mongod actually triggers startup warnings. The main setting is transparent huge pages, which is a setting that does not work very well with databases in general.

The MongoDB documentation already describes how to disable Transparent Huge Pages (aka THP) using tuned.conf, but there are several other settings that mongod tends to warn users about if you run it on an out-of-the-box CentOS 7.

Read More

How to enable logging in the MongoDB Java driver

I will show you how to enable logging in the MongoDB Java driver and also how to set and change the log level. The official mongoDB Java driver uses java.util.logging as its default logging framework or sl4j if the latter is present. It can be very useful to enable logging in the MongoDB drivers to trace how the driver is interacting with the database.

Read More