Spark SQL, part of Apache Spark big data framework, is used for structured data processing and allows running SQL like queries on Spark data. In this article, Srini Penchikala discusses Spark SQL module and how it simplifies running data analytics using SQL interface. He also talks about the new features in Spark SQL, like DataFrames and JDBC data sources.
Lean Product Development takes an end-to-end focus on the flow of work through a system. Rather than focus on traditional measures such as capacity utilization, it proves more effective to focus on how work is moving through the system. This article discusses what impedes the flow of work, and how we manage impediments to the flow of work.
Software developers spend a tremendous amount of time and energy focused on how to build the best possible products. We obsess over what web framework to use or whether to go with a NoSQL or SQL database. While these questions are important, they often neglect to address an equally important aspect of software development: culture.
Bulk data is usually accessed via files & FTP. As collaboration migrates to APIs, what are the requirements for data APIs? Here we describe a meta-data driven architecture for bulk data ingestion.
We are at a crossroads in the agile-adoption narrative. Agile started spreading “bottom-up”, then shifted from teams to executives and recently to consultancy for large enterprises. What will be next?
In this article, authors summarize the information published in "Tools of the Trade" column series in IEEE Software magazine for last ten years.
This article about lean documentation gives you 6 practices to maximize information while minimizing the number of words. This makes it useful to read and at the same time easy to maintain. 3
Stopping after unit testing is like starting automobile production after testing the nuts and bolts. This article investigates some important techniques in integration testing.
With the emergence of the WebRTC API standard, developers without any experience in telecommunications technology are enabled to easily integrate real-time comms into their applications.
Culture plays an important role in organizational change. Successful agile adoption tends to depend on the ability to change culture by making the culture explicit and becoming more conscious of it.
The Cynefin model can be used in different ways - this article examines the situation where there is a need to make sense of situation as we have found that there are a large number of issues.
In this article, authors reflect on Credit Suisse's adoption of SOA principles and the organization's transition from mainframe programs to open SOA services with interface contracts and governance. 4