In 2006, The New York Times had 20 engineers, all located in a separate building off-site. Engineering and journalism were organized as completely separate entities, even ad-sales departments were separate. How do you change a culture like this into a culture where technology drives and supports journalism? This was the challenge Marc Frons had when he joined the New York Times as technology officer in 2006. During QCon New York 2013, Frons -- now CIO at The New York Times -- with his colleague Rajiv Pant -- their CTO -- gave a keynote about this challenge.
A lot changed at The New York Times since then: departments have been merged and journalists and engineers mingle. Events are organized to encourage technological development, including:
- Hack days, during which engineers and journalist together hack on new technology projects, preferably related to The New York Times, but not necessarily.
- 100% days, the answer to Google's 20% time, where engineers can spend a full day per month on whatever exciting technology project they come up with.
The New York Times has been hiring aggressively and managed to attract great talent, including people like Jeremy Ashkenas, the creator of CoffeeScript and Backbone.js. Yet, the company faces the same challenges as many other companies building software: build or buy? What software do you build in-house, and what software do you buy from other vendors? The company has experimented in many ways, ranging from building software using their own programming language, to buying complete off-the shelf system and putting them in production. Today, the company uses custom-built content-management system and data analysis tools, but runs much of its infrastructure on Amazon EC2, and does continuous integration with Hudson.
Another such recurring theme in modern web development is the web versus native debate on mobile. The New York Times decided not to bet on one horse, they are trying to deliver a good experience on all (mobile) platforms, both web and native.
The company is also opening up to external developers, organizing local events and open sourcing many of their technologies, including:
- Nimbul: an open-source, enterprise-oriented cloud management tool.
- CloudSource: a simple role-based SVN / bash deployment tool
- NYT Transformer: a flexible data processor that can be customized for various input and output types
- DBSlayer: a DB connection pooling layer that speaks HTTP and JSON
- XSLcache: a XSL caching extension for PHP to speed up XSL transforms
The New York Times also exposes much of its content via APIs, on its developer network, encouraging third parties to build applications using this data.
The New York Times went through a cultural shift that many publications are going through right now: the transition from paper to online, from low-tech to high-tech. Today's QCon keynote gave a good overview of the challenges these companies are facing today, and how, in many ways, they overlap with most other hi-tech companies.