Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Presentations Stream Processing: Philosophy, Concepts, and Technologies

Stream Processing: Philosophy, Concepts, and Technologies



Dan Frank discusses stream data processing and introduces NSQ – Bitly’s open source queuing system – and other new technologies used for communication between streaming programs.


Dan Frank grew up in Brooklyn, with the mathematical distinction of occupying the zip code corresponding to the first five digits of the Fibonacci sequence. He earned a degree in Math from Yale, and has since been working as a developer at small startups that process large quantities of data. He is currently a member of the data science and infrastructure engineering teams at Bitly.

About the conference

The ETE Conference has established itself as the most diverse and interesting conference on the East Coast. Curated by developers, for developers, it brings together the brightest minds in software development. Visit we provide content throughout the year and you can subscribe to our Chariot TechCast podcast.

Recorded at:

Nov 10, 2013

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • There's so much prior art

    by peter lin,

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Why do people ignore prior art and research? Event processing, and stream processing are techniques used to build event based architectures. Not all event processing problems are sequential messages. Topics like temporal logic, event correlation, partially ordered sets and totally ordered sets are critical aspects of solving event related problems.

    The research goes back decades and lots of it came out of AI research. People should keep in mind that often times order affects the processing and analytics. Proper discussion of event processing should at minimum touch on these important aspects.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p