HACK: Creating triggers for MongoDB

14 05 2012

I visited the MongoDB conference in Berlin. At one talk about tips, tricks and hacks for MongoDB the speaker mentioned that there is a little hack which you can use to create a trigger for MongoDB. I wanted to try this out because he only mentioned how to do this theoretically very shortly.

When you have configured MongoDB to work as a replicaset you maybe have noticed that on the local database a new collection called “oplog.rs” is created. Inside this collection MongoDB stores all insert / update and delete operations which are executed against this replicaset (it’s comparable to the transaction log on a SQL Server). The oplog collection is used to distribute all the operations from the primary node to all secondary’s. With the help of this collection and a little javascript file we are able to create something which behaves like a trigger.

Let’s start with the oplog collection. If you look at an entry from this collection you can see something which can look similar to the following extract.

{
    "ts" : {
        "$timestamp" : NumberLong("5724119038133534721")
    },
    "h" : NumberLong("-7041921609633449468"),
    "op" : "i",
    "ns" : "TestApplication.BlogPost",
    "o" : {
        "_id" : ObjectId("4f7027f0df6e252390d2332a"),
        "Author" : "Test Author",
        "CreationDate" : new Date("Mon, 12 Mar 2012 00:00:00 GMT +01:00"),
        "Comment" : "My Comment"
    }
}

ts: is the timestamp. We need the timestamp to avoid that an element can be triggered twice.

op: is the operation. The interesting operations are “i” for insert / “u” for update and “d” for delete.

ns: is the namespace (database and collection) were the operation was executed.

o: is the object which is created or updated.

If you need more information about the oplog have a look at the following page about the oplog on the website from MongoDB:

http://www.mongodb.org/display/DOCS/Replica+Sets+-+Oplog

Now we create the javascript file. This script has a while loop without any option to exit this loop. We want to watch for all changes on the oplog and want to react on these changes. As long the script is running we have a behavior similar to a trigger.

Two features of MongoDB are used to allow the execution of this script (have a look at the links if you want to have further information):

Now have a look at the script and modify and reuse it if you like.

var coll = db.oplog.rs;
var lastTimeStamp = coll.find().sort({ '$natural' : -1 })[0].ts;

while(1){
    cursor = coll.find({ ts: { $gt: lastTimeStamp } });
    // tailable
    cursor.addOption( 2 );
    // await data
    cursor.addOption( 32 );

    while( cursor.hasNext() ){
        var doc = cursor.next();
        lastTimeStamp = doc.ts;
        printjson( doc );
    }
}

What the current script does is checking for operations inside the oplog and print out the oplog entry. Just change the line with the printjson command to the operation you want to perform as the result of the trigger. On the line where you initialize the cursor you can enhance the query if you maybe only want to react on update operations.

I developed on a project with MongoDB nearly 1.5 years now and I didn’t came across a problem were I really need a trigger. I saw a couple of people asking for triggers at different pages and hope I can help some of them with this little hack. This is not tested on a high traffic environments.

About these ads

Actions

Information

4 responses

4 09 2012
zakwanhaj

useful post, thanks

9 01 2013
Herman Junge (@hermanjunge)

Thank you.

8 05 2013
MongoDB Client-Trigger mit Reactive Extensions (Rx) | Nove Rosso

[…] Link: Daniel Weber: HACK: Creating triggers for MongoDB […]

6 06 2013
pb

Hi,

Though its a old post but it seems the issue still exist. I have a question – Capped Collections cannot be sharded – I have a situation where i want to do some pre-aggregation whenever a inset happens on this collection ( which is a candidate to be sharded as we expect around 10 mil row insert per day ) . The pre-aggregation tables are different from the collection where the insert is being performed.

Am using mongo for the first time – will there be an impact.

Thanks

P

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Follow

Get every new post delivered to your Inbox.

%d bloggers like this: