Testing MsgPack and BSON

Wed, Aug 28, 2013

In one of the projects I am working on we are trying to select a message encoding scheme. The use case is pretty simple. We have a server that accepts data records from different clients, and our goal is support 100K messages per second per node. Our selection criteria are pretty simple as well:

Out of the various encoding schemes out there, including BSON, MessagePack, ProtoBuf, Thrift, etc etc, we decided to test BSON and MessagePack due to their closeness to JSON (we use JSON format quite a bit internally).

Here’s a very short and simple Go program we wrote to test the performance. The libraries I am using are Go codec and Go BSON library. (I am sure one can argue that the library used affects the result, which I would agree. However, these are the best libraries for Go AFAICT.)

The results are as follows:

So looks like msgpack is the way to go!

Special thanks(!) to realrocker in #go-nuts and Ugorji for their help in cleaning up my code and helping me figure out my problems.

Some notes about the codec library:

discussion

{% gist 6371433 %}

comments powered by Disqus