BT

Your opinion matters! Please fill in the InfoQ Survey!

Google's BigQuery Gaining Momentum

| by Michael Hausenblas  Followers on Feb 14, 2014. Estimated reading time: 1 minute |

A note to our readers: As per your request we have developed a set of features that allow you to reduce the noise, while not losing sight of anything that is important. Get email and web notifications by choosing the topics you are interested in.

Google's BigQuery, a SaaS query offering by Google, seems to gain more and more momentum. It allows to query large-scale columnar data structures in the cloud. Developers can load data into BigQuery via upload to Google Cloud Storage (the equivalent of Amazon's S3) or stream data into the platform, and then perform OLAP-style queries using a SQL-like query language.

Practitioners now increasingly share their hands-on experience with BigQuery. For example, Graham Polley of Shine Technologies reported:

We decided to run our own tests, laying down the gauntlet to BQ and using a data set with 1.5 billion rows. Things were about to get very interesting – could Google’s sales pitch of "being able to interactively analyse massive data sets with billions of rows" really do what it said it could? It could and we were impressed. In fact, very impressed. Even when not using cached results (cached results can be toggled on and off) we experienced consistent results in the 20-25 second range for grinding through our massive data set of 1.5 billion rows using relatively complex queries to aggregate the data.

BigQuery can be used stand-alone, but also provides integration with other services, such as Google Apps Script or Google Analytics. Concerning the latter, Jonathan Weber (Data Evangelist at LunaMetrics) wrote an informative piece where he states:

First, BigQuery export is available only for Google Analytics Premium customers. You can have the BigQuery export turned on through your Premium account manager. Note that there are costs for both data storage and processing in BigQuery, but GA Premium users get a $500/month credit to use toward those charges. In many cases, that $500 will take you a long way. For reference, I took a look at one of our Premium customers using BigQuery. Their site has about 6M visits and 50M pageviews per month. Data has been exporting since September, and this month their storage charges will be about $12.86.

While BigQuery is only available as a cloud-based solution, the underlying technology that powers BigQuery (Dremel) is the core of many Open Source SQL-in-Hadoop solutions such as Apache Drill or Impala.

Rate this Article

Adoption Stage
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Login to InfoQ to interact with what matters most to you.


Recover your password...

Follow

Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.

Like

More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.

Notifications

Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

BT