BT

Google Snappy–A Fast Compressing Library

by Abel Avram on Apr 06, 2011 |

Google has open sourced Snappy, a compressing/decompressing library that processes data streams at minimum 250MB/s-500MB/s on a single core Intel Core i7 processor.

Snappy, previously known as Zippy, is a compression library used by Google in production internally by many projects including BigTable, MapReduce and RPC. Google says the library and its algorithm has been tuned for speed, and not for compression output size nor compatibility with other similar tools. Snappy has been optimized for 64-bit x86 processors, on a single core Intel Core i7 processor achieving a compression rate of at least 250MB/s and a decompression one of 500MB/s.

The high compression speed is achieved by losing on the compression ratio, the resulting output being 20-100% larger than that of other libraries, Snappy having a compression ratio of “1.5-1.7x for plain text, about 2-4x for HTML, and of course 1.0x for JPEGs, PNGs and other already-compressed data”.

Google touts Snappy as robust being “designed not to crash in the face of corrupted or malicious input”, and stable after being used to compress petabytes of data in Google’s production environment.

Snappy can be used to benchmark itself against a number of other compression libraries - zlib, LZO, LZF, FastLZ and QuickLZ –, if they are installed on the same machine. The tool comes as a C++ library that is linked into the product that is supposed to use it, but there are several other bindings for it: Haskell, Java, Perl, Python, and Ruby.

Snappy has been open sourced under the New BSD License.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT