Tracking change and innovation in the enterprise software development community
QCon London 2010!
March 10-12, 100
Speakers, 80 sessions,
"THE" team leads and,
architects conference.
Scaling Scrum
in the enterprise?
ScrumWorks Pro
can help.
Start a trial today
Free Book Offer
SOA Demystified
$40 Value
Posted by Niclas Nilsson on Aug 14, 2007
The future of computing is going to be concurrent. Even desktop CPUs are multicore nowadays, and when customers are buying more and more CPUs to their servers, they expect their applications to scale well to utilize their new investment. But that’s not going to happen with many software systems of today.
Handling concurrency is hard. The problem itself is hard, but it doesn’t help that the concurrency mechanisms in the widely used programming languages are low-level mechanisms compared to many other abstractions in the same languages. It’s a lot to deal with, but there are ways to make it easier.
Ralph Johnson wrote a blog post about Erlang being the next Java. Erlang has a fundamentally different view of inter-process communication, and Ralph argues that even though the Erlang style of concurrency can rather easily be added to other languages, they will have trouble competing.
I do not believe that other languages can catch up with Erlang anytime soon. It will be easy for them to add language features to be like Erlang. It will take a long time for them to build such a high-quality VM and the mature libraries for concurrency and reliability. So, Erlang is poised for success. If you want to build a multicore application in the next few years, you should look at Erlang.
And there may be more things to get as free side effects. Joe Armstrong, the creator of Erlang, wrote that the challenges of scalability, fault-tolerance and runtime upgradable systems share a solution. He said that even though these are different kinds of problems,
“a system that is fault-tolerant can easily be made scalable and easily made so that we can do in-service upgrade”.
Armstrong accredits all this to Erlang having a correct set of primitives for failure detection and for stable storage. Since Erlang was built with telecom systems in mind, all of these aspects are important, and Ericsson has built Erlang systems with an uptime of 9 nines. That translates to a downtime of 31 ms a year.
The Erlang take on concurrency differs from the mainstream languages by not having any shared state. The processes are very lightweight, and if a process wants to communicate with another process, it communicates by sending messages. This architecture makes it possible to scale and distribute systems in quite a different way than systems that uses shared memory for communication between processes or threads. To dive deeper into this way of thinking about concurrency, a good introduction of the concept (which requires no Erlang knowledge) was written by Slava Akhmechet.
Maybe it’s time to approach the concurrency problem in a different way? Maybe it’s even time to learn to new language?
Agile Development: A Manager's Roadmap for Success
Right Collaboration Architecture Drives Business Transformation
Consolidation and Virtualization Are NOT Enough: The Case for Non-x86
Hi Niclas,
There must be some alignment of the universe at work here :). I am currently looking into Erlang for my current engagement which lives in the space of producing official statistics for countries. The volume of data to be processed, real-time ETL requirements, and the proverbial concurrency issues led me to Erlang. The side effect for me was that tackling my problems from a functional perspective (as opposed to an OO perspective) may well yield a cleaner service model than what I originally conceived. Time will tell.
I also have to work with metadata that must be semantically correct and semantically interoperable (within the organisation and outside it as well). There's the case for Erlang being cleaner and easier to walk massive directed graphs (think OWL, RDF, etc) than, say, Java. For me the question is "Is Erlang the language for Web 3.0".
Regardless, Erlang should be learnt. Our obsession with object orientated analysis needs a timeout, and functional analysis is a good alternative. IMHO, Erlang is definitely easier to digest than Haskell.
Erlang's a very interesting language, and people should definitely do more with it. However, Scala is a next-generation JVM based language (think Groovy done right) and also encapsulates the concurrent message-passing actor pattern from Erlang, which makes it possible to use it in and with Java.
lamp.epfl.ch/~phaller/doc/ActorsTutorial.html
Hello Aslam,
It must be synchronicity.
I agree with you. Even though I'm a strong OO-advocate in many cases, there are definitely a lot of problems that just doesn't fit an object model very well. Us as an industry needs to expand our horizons to different ways of thinking more often than we do. We need to take the time; there is a lot of time to save by choosing the right tool for each job.
Alex: Interesting link. Scala is on my radar, but this looks like a good starting point for playing around with it -- thanks.
One of the problems I have with Erlang at the moment is its dependence on the GNU toolchain, i.e. you cannot build it from source without gcc/gmake. If Erlang is going to reach a wider audience it needs to eliminate the dependence on GNU specific extensions so that people can build it from source using, say, the Sun or Microsoft compilers.
>I agree with you. Even though I'm a strong OO-advocate in many cases,
> there are definitely a lot of problems that just doesn't fit an object
> model very well.
www.kirit.com/Blog:/2007-08-09/Erlang%20as%20an...
Since OO is about sending messages, I'd say Erlang has the best of both functional and OO worlds. Just because there's now dot syntax (foo.bar() for method invocation) doesn't mean OO concepts can't be applied.
Since OO is about sending messages, I'd say Erlang has the best of both functional and OO worlds. Just because there's now dot syntax (foo.bar() for method invocation) doesn't mean OO concepts can't be applied.
That was not what I meant. I wrote objectoriented code in C for a couple of years in the beginning of the 90's, so no objection there.
For single-CPUed parallel nodes, "share-nothing" is natural and most efficient, but why not "share something" when it comes natural and easy for CPUs/Cores in a same space? That should give Java more power than Erlang IMHO.
i share your surprise. message-passing is far from a new paradigm in parallel software development and the truth is that although it is clean and secure (if you discount the danger of deadlocks) it is far from convenient. being able to share memory/objects/state - which is what every multi-CPU or multi-core architecture gives you naturally - between concurrent threads is actually a major relieve over the stringency of having to communicate by messages only. in that sense i place more hope in the java.util.current features introduced in java 5 and to be enhanced further in java 7.
cheers,
gerald
www.gerald-loeffler.net
pardon my typo: that was meant to be "java.util.concurrent", of course...
@Compl Yue Still
Well - in *general* - as you scale to very large (>5,000) numbers of processes, in most applications the ratio of (a) communications between a small number of processes (say: 1 sender to 10 receivers) and (b) communications in a large group (say: 1,000 receivers) will become asymptotically close to one. Therefore implementations that explicitly share nothing will outperform those where messaging is implemented on top of shared memory.
And, if you do not know where the processes are distributed (across one CPU, or many cores, or over a network) then it is not obvious where to optimise for latency vs throughput, in a shared memory implementation. This further favours message passing approaches in being generally easier to apply successfully, and manageably if the system changes. You could call this property 'scale invariance'.
Finally, in a process centric language, shared memory is (logically) just a process. That is one way to implement a form of shared memory in Erlang.
@All
Personally I do not like these 'my language is bigger than your language' discussions. A good way to use messaging in Java/C# etc is to provide it as a low level service in the VM - like how people use remoting, it's a data communication layer. This could also work for Scala to deliver 'remote actors'.
We've found Erlang to be a good fit for implementing RabbitMQ which delivers messaging to Java and other languages, using the open 'AMQP' model for communication. This is not just for the reasons mentioned in the InfoQ article (good for concurrency and multicore) but also (1) because it is inherently extremely distributable without paying heavy performance costs and (2) it is very expressive so code complexity costs are lower e.g. the RabbitMQ codebase is *very* small.
Cheers
alexis
@Gerald
In fairness (no pun intended) this is less true in Erlang in which using messages is incredibly natural. For OO languages it may be more natural to use an object facade for messaging - clearly there are lots of ways to do this, and some of them look like shared memory.
@All
I recommend that people follow up on the Ralph Johnson article for more details. Another place to pick things up is on this forum.
I wrote objectoriented code in C for a couple of years in the beginning of the 90's, so no objection there.
How do you get C to do dynamic dispatch? Without dynamic dispatch I can't see how you call your code object oriented.
A long time ago, I did something like this using function pointers as entries in a table linked to from structs and invoked via macros.
Don't ask for details, it still makes my nose bleed to think about it.
Recent events with Skype suggest that people might indeed care ;-)
It's been said that a Fortran programmer can program Fortran in any language. I wonder whether the same applies to Erlang :-). It certainly seems to be getting a lot of attention from the OO community recently. The keynote talk at ECOOP this year was about Erlang, which is certainly a first!
We'll be having a talk from Joe Armstrong, the creator of Erlang at JAOO in Aarhus, Denmark this year. It's not too late to get the early registers discount if you want to come and tell him why Erlang is (or isn't) OO. We have Lex Spoon from the Scala group too, for comparison purposes :-).
I'm not sure Erlang is OO myself, but hey, just because we have those two letters in our conference name, doesn't mean we can't get some inspiration from other paradigms.
--
Aino Corry, Trifork
Program committee member, JAOO & QCon
@Aino
There is some discussion of your question regarding Erlang and the OO metaphor here
The author's conclusion is that "I have to agree with Ralph, Erlang is an object oriented language, but one where the objects themselves are implemented in a functional language".
Whether you agree with the author or not, it's also true that it would be tricky to 'program Erlang in any language' for lots of reasons, such as that every process has its own heap so that you can have *millions* of processes. It would be more accurate to say 'you can program the actor model in any language', but even that claim is a highly underdetermined :-)
@Dean
Just like Stefan said, you have to "roll your own" by making tricks with functions pointers.
For single-CPUed parallel nodes, "share-nothing" is natural and most efficient, but why not "share something" when it comes natural and easy for CPUs/Cores in a same space? That should give Java more power than Erlang IMHO.
Because of the von Neumann bottleneck. All of those processors accessing a single shared memory are going to have to go through some kind of bus. That bus has a finite bandwidth, which effectively limits the use of shared memory. There are tricks to lessen this effect, but I still haven't seen anyone get rid of it totally.
Therefore you need messaging. Since you don't want to expose the development team to the internal deployment architecture of a site, they should only use one of these paradigms.
Message passing scales better, therefore messaging is a better choice if you expect massive amounts of traffic.
I don't agree that this is not effective to do in Java. The mule framework is a high-level framework for building these kinds of applications.
On the other hand, at least some projects which expect "massive amounts of traffic" often base that more on wishful thinking than a realistic estimate of the future. This may land them in hot water - message passing has problems of its own, which are not easily solved. Sometimes, Newtons laws are easier to work with.
Geir
How does Erlang compare to Haskell? There seems to be more books published about Haskell. Does that mean that it is more mature and widely used? If one was going to invest the time and learn a functional language, which would be the better one?
So many languages and so little time...
I did the same in PL/1 on IBM mainframe at roughly the same time...
Likewise, it was fun at the time but I do not care to remember the details (;-)
Brian Marick takes us through a quick tour of the most important values and challenges to adopting Agile successfully (they aren't the typical challenges and values we hear in the community).
The line between development and architecture is tricky. Does it exist at all? Is an ivory tower actually needed? There's a balance in the middle, but how do you move from developer to architect?
The word 'authority' sometimes produces an allergic response in hard-line agilists. Freedom and authority – both are bad if misused and both are good if used in right spirit for a noble cause.
"Getting Started with Grails" brings you up to speed on this modern web framework. Companies as varied as LinkedIn, Wired, and Taco Bell are all using Grails. Are you ready to get started as well?
Those familiar with only ITIL V2 often scoff at the thought that ITIL could serve as a governance framework for SOA. With ITIL V3, the focus of the framework shifted towards service-orientation.
SpringSource CTO Adrian Colyer discusses AspectJ, SpringSource's dm Server and tc Server products, OSGi and Scrum.
Heroku's Adam Wiggins talks about Rails, Background Jobs, Add-Ons, Ruby, and how Heroku manages to work around Ruby's inefficiencies using Erlang and other languages.
For Grady Booch the foundation of a good architecture is patterns, SOA being just one of many patterns. In this Second Life presentation, Booch attempts to bring more clarity on what architecture is.
21 comments
Watch Thread Reply