BT

New Early adopter or innovator? InfoQ has been working on some new features for you. Learn more

Google Snappy–A Fast Compressing Library

| by Abel Avram on Apr 06, 2011. Estimated reading time: 1 minute |

Google has open sourced Snappy, a compressing/decompressing library that processes data streams at minimum 250MB/s-500MB/s on a single core Intel Core i7 processor.

Snappy, previously known as Zippy, is a compression library used by Google in production internally by many projects including BigTable, MapReduce and RPC. Google says the library and its algorithm has been tuned for speed, and not for compression output size nor compatibility with other similar tools. Snappy has been optimized for 64-bit x86 processors, on a single core Intel Core i7 processor achieving a compression rate of at least 250MB/s and a decompression one of 500MB/s.

The high compression speed is achieved by losing on the compression ratio, the resulting output being 20-100% larger than that of other libraries, Snappy having a compression ratio of “1.5-1.7x for plain text, about 2-4x for HTML, and of course 1.0x for JPEGs, PNGs and other already-compressed data”.

Google touts Snappy as robust being “designed not to crash in the face of corrupted or malicious input”, and stable after being used to compress petabytes of data in Google’s production environment.

Snappy can be used to benchmark itself against a number of other compression libraries - zlib, LZO, LZF, FastLZ and QuickLZ –, if they are installed on the same machine. The tool comes as a C++ library that is linked into the product that is supposed to use it, but there are several other bindings for it: Haskell, Java, Perl, Python, and Ruby.

Snappy has been open sourced under the New BSD License.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and dont miss out on content that matters to you

BT