Netflix has developed an orchestration engine called “Conductor”, and has used it internally in production for the last year . During this time they executed some 2.6 million process workflows, starting with linear ones and ending with dynamic ones running over multiple days. Now they have open sourced Conductor, making it available to all those interested in workflow orchestration.
The key to success when working with a microservices based distributed system is to focus on the distributed process as a whole, not on the microservices themselves. The services are the least important part, Eric Ess claimed at the recent Microservices Conference in London, in his presentation on how to monitor distributed processes at jet.com.
Julien Nioche, director of DigitalPebble, PMC member and committer of the Apache Nutch web crawler project, talks about StormCrawler, a collection of reusable components to build distributed web crawlers based on the streaming framework Apache Storm. InfoQ interviewed Nioche, main contributor of the project, to find out more about StormCrawler and how it compares to other similar technologies.
Microsoft recently announced an addition to its Platform as a Service (PaaS) offering called Azure Functions. Initially launched as a preview service in March 2016, Azure Functions provide developers with an event-driven serverless compute platform that allow organizations to pay for only what they consume.
Julien Le Dem, the PMC chair of the Apache Arrow project, presented on Data Eng Conf NY on the future of column-oriented data processing. Apache Arrow is an open-source standard for columnar in-memory execution. InfoQ interviewed Le Dem to find out the differences between Arrow and Parquet.
Software security is a complex problem, and is becoming even more complex using Microservices where each service has to deal with security, David Borsos explained at the recent Microservices Conference in London, during his presentation evaluating four end-user authentication options within a microservice based systems.
At the recent AWS Re:Invent event, Amazon announced a new preview service, called AWS Batch. AWS Batch allows organizations to optimize their scheduling and workload execution across a cloud-based landscape. Amazon has built this service in response to many AWS customers building their own batch platforms using EC2 instances, containers and CloudWatch.
In our continued effort to showcase lessons learned by microservices practitioners, we look at an article Piotr Gankiewicz has recently written with his own tips and tricks. These include references to CQRS, asynchronous architectures, service discovery and how choosing the right database for each service is important.
A developer panel was held at Microsoft Connect() following the multiple annoucements of new features and releases. Microservices and containers are in the center of the discussion, along with Azure, serverless architecture and developer tooling.
Lawyer.com recently announced that they are adopting the HTTP/2 protocol. Gerald Gorman, tech entrepreneur, CEO, and co-founder of Lawyer.com, spoke to InfoQ about their technology implementation, their position on microservices and lightweight containers, their unique search engine, and their use of social media.
Microsoft recently announced the General Availability of Microsoft Flow, a cloud-based automation platform that provides workflow and connectivity capabilities across many popular online and on-premises services. Since the preview, in April 2016, 117,000 people from 61,000 organizations have used the service to automate their business processes.
Amazon has updated their AWS Well-Architected Framework (PDF) based on feedback from clients, adding a new pillar, Operational Excellence.
Microservices has a very long history, not as short as many believe. Neither was SOA invented in the 90s. We have been working with the core ideas behind services for five decades, Greg Young explained at the recent Microservices Conference in London, during his presentation on working with microservices.
When it comes to application security, we often include it as an afterthought. We have learnt how to add test into the development workflows, but with security we often assume someone else will come and fix it later on, Sam Newman claimed in his keynote at this year’s Microservices Conference in London.
Developers attracted to the Azure Service Bus Relay for its ability to expose web services on internal networks to Internet-facing consumers have had, until recently, only one way to build such services: WCF. Using a just-released public preview called Azure Relay Hybrid Connections, developers may now use any Web socket friendly platform to connect local services to this cloud-based broker.