InfoQ

InfoQ

News

My Bookmarks

Login or Register to enable bookmarks for unlimited time.

The content has been bookmarked!

There was an error bookmarking this content! Please retry.

Presentation: Pipes and Y! Query Language

Posted by Abel Avram on Feb 27, 2009

Sections
Architecture & Design,
Enterprise Architecture,
Operations & Infrastructure
Topics
Web 2.0 ,
Cloud Computing ,
Architecture ,
Web Services
Tags
Yahoo!

In this presentation filmed during QCon SF 2008,  Jonathan Trevor presents two Yahoo! technologies: Pipes and Y! Query Language (YQL). Both technologies can be used to process data obtained from various sources, but while Pipes is limited to Yahoo web services, YQL can process many types of data.

Watch: Pipes and Y! Query Language (53 min.)

Yahoo Pipes uses a visual tool to specify a series of pipes through which input data flows and is filtered and processed in various ways. There are many data sources like personal data, CSV, feeds, web pages, services like Flick, Yahoo Search, and others. Processing is done through operators like: Filter, Loop, Regex, Sort, Union, and others. The result is shown in a web page, a feed, or given to an application. The service runs in Yahoo’s cloud, is free and does not offer security protection for sensitive data since anyone can copy and run a pipe made by someone else.

YQL is similar to Pipes but uses a textual textual language and can process both Yahoo web services data and any structured data with an URL. It has an SQL like syntax with three statements: SELECT, SHOW, DESC. This approach is more powerful and the applications created are protected.

Trevor explains both technologies, how they have been developed, their current status and shows demo examples.

No comments

Watch Thread Reply

Educational Content

10 tips on how to prevent business value risk

One category of risk that project teams need to ensure they address is business value failure – delivering a product that fails to provide value for the business investor.

Interview: Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives

InfoQ spoke to the authors of Software Systems Architecture on a couple of new topics, the System Context viewpoint and Agile, which have been added to the second edition.

Beauty Is in the Eye of the Beholder

Alex Papadimoulis discusses ugly code, where it comes from, how to avoid it, and how to get rid of it.

Architecting Visa for Massive Scale and Continuous Innovation

John Davies examines Visa’s architecture and shows how enterprises have architected complex integrations incorporating Hadoop, memcached, Ruby on Rails, and others to deliver innovative solutions.

Max Protect: Scalability and Caching at ESPN.com

Sean Comerford unveils ESPN.com’s architecture, what components are used and why, and the current changes the website goes through.

The Seven Deadly Sins of Enterprise Agile Adoption

Are there repeated patterns of failure on Enterprise Agile Enablement efforts? Sanjiv and Arlen discuss Seven Deadly Sins to avoid when adopting Agile in an enterprise.

Questions for an Enterprise Architect

Erik Dörnenburg answers: What is Enterprise and Evolutionary Architecture?, discussing 4 issues: Turning strategy into execution, Ensuring conformance, Where do the architects sit? Buying or building?

Wrap Your SQL Head Around Riak MapReduce

Sean Cribbs explains what Map-Reduce and Riak are, why and how to use Map-Reduce with Riak, and how to convert SQL queries into their Map-Reduce equivalents.