Databricks has recently announced a new record in the Daytona GraySort contest using the Spark processing engine. The Daytona GraySort contest is a 3rd party benchmark measuring how fast a system can sort 100 Terabytes of data. Databricks posted a throughput of 4.27 TB/min over a cluster of 206 machines for their official run.
When using Domain-Driven Design (DDD) separating the concerns of a large system into bounded contexts with each context using its own data store there is often a need to share some common data. One way of doing that is to let each context publish events about changes, events that others can listen to, Julie Lerman recently explained in MSDN Magazine.
This post describes the relation of Agility and Modularity. Why modularity is important and how can we use it is described in OSGi white paper.
A common context for Command-Query Responsibility Segregation (CQRS) is a collaborative domain with multiple users working on the same set of data, it is less suitable in domains with only a single user working on a piece of data, Udi Dahan explained when discussing CQRS in an interview.
Hortonworks Data Platform (HDP) version 2.2 with features based around Hadoop and YARN has better support for enterprise features such as security, compliance and so on as well.
Microsoft recently published a case study describing how a massively multiplayer online (MMO) game used Microsoft Azure to support tens of thousands of players in a single space battle. The case study looks at how architectural considerations like connectivity, latency, and scale can be addressed in an elastic cloud environment that must respond quickly to unexpected bursts in demand.
Architecture is looking at things in a bigger context, thinking ahead and making decisions. Using their technical and communication skills architects can play a significant role in this by establishing direct contacts with people at different levels in an organisation, Gregor Hohpe said when sharing his experiences from working for Google, and now as Chief IT Architect at Allianz.
This post describes the minimum viable architecture for developing minimum viable product. It includes the architecture specification in different phases of startup.
Chad Fowler, CTO at 6Wunderkinder, the company behind Wunderlist, describes how they went from a large monolithic Rails application and a large monolithic database to a system with many microservices, and the architecture they ended up with. Starting by adding new functionality as services and splitting the large database into smaller databases, they ended up doing a big rewrite of a new system.
Service-Oriented Architecture (SOA) is a very broad term and practically meaningless. Microservices is a subset of SOA with the value being that it allows us to put a label on this useful subset of SOA terminology, Martin Fowler stated in his keynote introducing Microservices when opening the GOTO Berlin Conference 2014.
Martin Fowler describes Sacrificial Architecture. This post highlights the need and benefits of sacrificial Architecture.
Layered systems are an architectural style used essentially to avoid coupling, the biggest enemy of software maintainability, with Ports and Adapters, or a Hexagonal Architecture, an example of such an architecture, Ian Cooper explains in a presentation about architecture styles, specifically the Hexagonal Architecture.
Microsoft recently announced new machine learning capabilities for Microsoft Azure platform. Developers can also create their own web services and publish them to Azure Marketplace. Microsoft also announced availability of Apache Storm for Azure. Azure Stream Analytics, Data Factory and Event Hubs for Azure were all announced in the past few weeks by Microsoft. In this article we explore moreabout
The success of the RICON conference is a testimony to the importance of big applications in industry today. InfoQ speaks to RICON host Basho Technologies about considerations in building distributed systems and technical lessons learned at the conference.
Domain-Driven Design (DDD) together with Onion Architecture is a combination that Wade Waldron believes has increased his code quality dramatically since he started using it a few years back. Using DDD was a kick-off but together with Onion architecture he found his code to be more readable and understandable, and far easier to maintain.