DSLs bringing the end of single language development?

Posted by Sadek Drobi on Aug 07, 2007 11:18 AM

Community

Architecture

Topics

Domain Specific Languages ,

Programming

For many years, mainstream practice in enterprise software development has been to standardize on a single general purpose language on software projects, with Java and C# today being the mainstream choices. With the rise of interest in DSLs, we may be entering into a phase in which multiple languages on software projects becomes the norm, but not with the same problems of the 80’s and early 90’s.

Martin Fowler suggests that  we may be entering such a period in which:

...will see multiple languages used in projects with people choosing a language for what it can do in the same way that people choose frameworks now.

Fowler asserts that "larger frameworks like Hibernate, Struts, ADO present as much of a challenge to learn as a language even if you program them in a single host language." Moreover, expressing needs in the host language might be rather difficult and result in creation of awkward configuration files "which are effectively external DomainSpecificLanguages written in XML".

Embedding DSLs instead of using libraries could be a more appropriate solution. Martin makes the observatoin that "an API is like declaring a vocabulary, a DSL adds a grammar which allows to write coherent sentences." Hence, using DSLs instead of frameworks could be more expressive offering "better ways of manipulating our abstractions" and making it "easier to see what we've written and to reveal our intentions."
 
According to Piers Cawley the main characteristic of DSLs is not their readability but "their narrow focus on their domain" which allows them to be explicit. Illustrating his idea with an example, Cawley shows that DLS can be used not only for having “programs written in them read like domain experts talk” but can also be technical to represent a framework that is best manipulated using their syntax.
 
Neal Ford also believes that momentum is growing for what he refers to as Polyglot Programming. This new era of software development is increasingly characterized by embedding more languages, which allows to choose "the right tool for the job and to leverage it correctly". He gives an example of how difficult concurrency libraries are in java programming language compared to Haskell for example, a functional language that "eliminates side effects on variables " and makes it easier "to write thread-safe code". Ford emphasizes that Haskell implementations exist for both Java (Jaskell) and .NET ( Haskell.net ) .
 
One risk of move away from single language development could be a return to problems in the late 80’s and early 90’s where languages were completely separate platforms which did not interoperate and could not be used well together.  Martin Fowler points out that there will be an important difference this time:

In the late 80's it was hard to get languages to inter-operate closely. These days there's a lot of attention to making environments that allow different language to co-exist closely. Scripting languages have traditionally had an intimate relationship with C. There's much effort to inter-operation on the JVM and CLR platforms. Too much has been invested in libraries for a language to ignore them.

It may finally become a great asset for the industry and also for developers to learn and use multiple languages. According to Pragmatic Programmers, this helps to find new ways to solve problems as it affects the way one thinks about programming. 

What do you think? Will the next 5 years see us mixing languages, and using DSLs almos as frequently as libraries?

1. Back to top

   Are DSL or Polyglot programming a real solution?

   Aug 7, 2007 3:02 PM by Jean-Jacques Dubray

   Software development has become incredibly complex because the solutions we want to build (i.e. the requirements) have also increased in complexity (ergonomy, security, scalability, flexibility, business rules, data models, multi-channel, portals ...). This complexity prevents most projects to achieve a reasonable ROI and deliver solutions in a reasonable amount of time. Having been a NeXT developer for over 5 years in the 90s, I always felt that MDD was a natural way to develop certain aspects of an application which are better expressed in a declarative way. However, after architecting two model driven application frameworks, it became clear that code has to show up somewhere, because there are things that better described in an imperative style (validation rules in the presentation layer and most of the business object layer, down to the DAL), even languages such as BPEL are imperative languages disguised in DSLs. The other issue with DSL is that they are rather inflexible, if some requirement can't be expressed in the DSL, you have to spend considerable amounts of energy to go around the DSL. The problem I see today is that the industry is creating two die-hard camps by opposing DSL to Programming. The reality is that they need and must be blended to precisely avoid the emergence the "polyglot programming" model. The current thinking is that no single programming language can solve all problems efficiently. Even though Google's tour-de-force with GWT shows that if you are ready to spend enough energy you don't need new programming languages, the reality, is that new programming languages appear every day. This is not beneficial to IT because the more programming languages that are necessary to build a solution, the more costly it will be for IT to build and maintain it. Teams will need to be larger and skills will need to be carefully scheduled within each project because no single developer will be polyglot enough to make these project succeed. Today there is a need to develop a service oriented, process centric, model driven programming model that goes well beyond the traditional concepts of software construction: object orientation, MVC and ER. The polyglotism and the emergence of DSLs are simply the expression of the limitations of these concepts and OO in particular. I have tried to describe the beginning of a possible solution for blending both declarative and imperative styles in a single programming paradigm here: http://www.wsper.org/wsper_primer.pdf I'd love to have some feedback. JJ-

   Reply

2. Back to top

   Depends what you do

   Aug 7, 2007 3:49 PM by Mike Gale

   For some tasks the prepackaged tools are just great. If you're shovelling data into and out of a database, there's plenty of tools. If you're not particular about the code a lot of the code writing systems could be perfect, for you. If you like what RoR, ASP.NET (design surface mode) etc. do and they fit your needs they're great. If your way of looking at issues is all about multidimensional arrays. You don't want to plumb the array stuff just use it, then many of the conventional languages are just a barrier. If you're working in an area that you visualise as a system of equations, maybe some partial differential equations you're probably even more ill-served by the mass OO languages. With the ability to more easily use the huge libraries already created, people who always wanted to go further, now, maybe, have a chance. Those who didn't realise it before, may now start to use DSL type ideas. My expectation is that a part of the programmer community will expand their competence. With it the scope of what they do. This will spawn it's own problems. Management that is criminally ignorant of what is happening. Me too programmers who use DSL's without proper cause...

   Reply

3. Back to top

   If you want unreadable, unmaintainable code

   Aug 7, 2007 4:06 PM by cowardly dragon

   DSLs made up on the fly would be great for programmer lock-in. Every framework and XML config is a DSL of its own definition, and we all know what is necessary to get a programmer up to speed in those situations. Call it "overhead". How would you like to get hired onto a maintenance role, or conversion role, and find out the project was written in a combination of Java, LISP, Obfuscated Perl, Ruby, and BASIC. Now, remove the extensive documentation on those docs. That is a project with multiple custom DSLs.

   Reply

4. Back to top

   Re: If you want unreadable, unmaintainable code

   Aug 7, 2007 7:41 PM by Alex Popescu

   cowardly dragon I tend to agree with you, but only at theoretical level. A badly documented project will always be a maintenance nightmare. Now, I do think that in some situations having a DSL (and seeing it as a self descriptive language) may help. The real problem I am seeing is the complexity and the costs brought by mixing multiple languages. Instead of needing an expert for a "single" language you will need either an expert in N languages or N experts. Which I don't think is cost effective. This indeed doesn't apply for the cases where the DSLs are created with the same "project language" (see Ruby, Java + Groovy, etc.). And I think the cost analysis is one of the reasons why companies involved with big projects living for long periods (f.e. Google) are tempted to use a limited number of languages/environments. bests, ./alex -- .w( the_mindstorm )p. ________________________ Alexandru Popescu Senior Software Eng. InfoQ TechLead&CoFounder

   Reply

5. Back to top

   What language you would use to develop DSL?

   Aug 8, 2007 12:07 AM by Thamaraiselvan Poomalai

   Well, What language you would use to develop DSL? Multilanguage developemnt will never die. Its always going to be as diverse as our nature, our earth and our universe.

   Reply

6. Back to top

   Re: If you want unreadable, unmaintainable code

   Aug 8, 2007 9:16 AM by Sameer Wadkar

   DSL's are the proper direction to go. In my experience learning a new language takes time and effort but it pays of f during maintenance. On the other hand I have seen teams make their own frameworks to avoid cost of learning and pay heavily during maintenance. One needs to strike the right balance though. Sometimes a framework is simply too much overhead from a performance standpoint. If you need to eke out the last ounce of perfomance from an OR mapping layer you are better off writing your own and exploiting specific aspects of your application that Hibernate simply would not know. On the other hand writing your own rules based engine might be a serious overkill for most teams.

   Reply

7. Back to top

   An interesting take...

   Aug 8, 2007 9:44 AM by Jason Carreira

   ... from someone who hasn't been coding day-to-day in a decade. It's a fine bit of ivory tower pontificating, but not terribly practical. Of course, consultants are usually tasked with working on the development portion of projects, and don't stick around to see the devastation caused by "sexy" decisions when it comes to maintenance and operations.

   Reply

8. Back to top

   Just don't see the relevance ....

   Aug 8, 2007 11:04 AM by Ke Jin

   DSLs and single or multi-language developments have all been around for several decades. Many DSLs are merely designed to capture and express high level domain concepts/patterns/structures instead of as a way to define and create multiple host programming languages. I just don't see why DSLs would favor or weaken the single (host) language development and vice versa.

   Reply

9. Back to top

   Re: If you want unreadable, unmaintainable code

   Aug 8, 2007 4:19 PM by Alex Popescu

   DSL's are the proper direction to go. In my experience learning a new language takes time and effort but it pays of f during maintenance.

   Sameer, I am a bit confused by your comment. You are saying that learning a new language is better than learning a new framework in your language? How is that?

   On the other hand I have seen teams make their own frameworks to avoid cost of learning and pay heavily during maintenance.

   On this one I fully agree. And a DSL is a form of a new in-house framework.

   If you need to eke out the last ounce of perfomance from an OR mapping layer you are better off writing your own and exploiting specific aspects of your application that Hibernate simply would not know.

   Once again I am confused: and how do you deal with the rest of the app? Remember the 80/20 rule. I don't think it is a good advise to create your own ORM solution if you just need special treatment for those 20%. IMO, you should continue to use a "Hibernate" for the 80% and tweak it for the rest of 20%. That would be cost effective.

   On the other hand writing your own rules based engine might be a serious overkill for most teams.

   I am 100% sure that writing and maintaining something similar to Hibernate is overkill to most teams too. bests, ./alex -- .w( the_mindstorm )p. ________________________ Alexandru Popescu Senior Software Eng. InfoQ TechLead&CoFounder

   Reply

10. Back to top

    Re: An interesting take...

    Aug 8, 2007 4:23 PM by Alex Popescu

    This consultant vs full time maintainer seems to be a crevasse on our field. Aren't consultants joining a project responsible for the maintenance/operations too? In case they are not, I frankly fail to see how they can be useful to a project (I may probably need to exclude some small 1-time products). ./alex -- .w( the_mindstorm )p. ________________________ Alexandru Popescu Senior Software Eng. InfoQ TechLead&CoFounder

    Reply

11. Back to top

    Re: If you want unreadable, unmaintainable code

    Aug 8, 2007 4:42 PM by Sameer Wadkar

    Hi Alexandru, What I meant is it is more cost effective to learn a new language or a new framework within your own language than trying to reinvent the wheel using the language one knows. A lot of developers are afraid of doing this for a variety of reasons such as- 1. The new framework and language looks very complicated. However while re-inventing the wheel feels easy at first it quickly becomes a nightmare and its hard to go back. 2. What if the chosen framework cannot deliver? It is a risk which is best mitigated by prototyping. Mitigating it by creating your own is a ensuring failure for most teams (consulting at least)

    Once again I am confused: and how do you deal with the rest of the app? Remember the 80/20 rule. I don't think it is a good advise to create your own ORM solution if you just need special treatment for those 20%. IMO, you should continue to use a "Hibernate" for the 80% and tweak it for the rest of 20%. That would be cost effective.

    Maybe the ORM example was a bad one. But my last project was in .NET. It started in 2004 when I could not recommend using Hibernate.net as it was in beta or maybe alpha. It being client server and I being not conversant in .NET did not think ADO.net would work for reasons of heavy cost of serializing deep nested domain objects from client to server( maybe I am wrong here but there were about 10 tables where transactional CRUD was required so I could afford to build my own ORM). I developed my own ORM and it was a pain to maintain but eventually I could exploit several unique aspects of my application to do very focussed serializations ensuring a very high performance. I am sure I could do that using Hibernate in java but the point I am making is there are situations when you want to eke out the last ounce of performance and the only way to do that is to exploit specifics of your unique applications. It should be the last option though. All in all I really prefer using high quality frameworks and languages instead of building my own. Sameer

    Reply

12. Back to top

    Re: If you want unreadable, unmaintainable code

    Aug 8, 2007 5:34 PM by Alex Popescu

    Maybe the ORM example was a bad one. [...]

    Don't know if a bad one, but definitely misleading :-). Thanks for clarifications. Now it makes more sense.

    All in all I really prefer using high quality frameworks and languages instead of building my own.

    So, in fact we agree on this. Great! bests, ./alex -- .w( the_mindstorm )p. ________________________ Alexandru Popescu Senior Software Eng. InfoQ TechLead&CoFounder

    Reply

13. Back to top

    Re: If you want unreadable, unmaintainable code

    Aug 9, 2007 8:42 AM by Sadek Drobi

    One very important point about DSLs and special purpose languages that wasn't mentioned, is the way it abstracts the domain.Be it applied to a technical domain (an API) or a functional business one it is technology implementation neutral. Technology implementation will change over time, but dsl will become more and more mature in its domain. That makes it more and more expressive, maybe as expressive as SQL today for quering databases! Also it can become, in my opinion, an ubiquitouslanguage, that is a better way for communicating rules and concepts about the domain. Sadek Drobi

    Reply

14. Back to top

    Domain VS Context

    Aug 9, 2007 3:09 PM by Wavell Watson

    To me at some point work within *domains* is obviously reinventing the wheel and thats when DSLs are a good idea. A true domain, which would be a valid target for a domain specific language, is a target *field of study*. This would be knowledge that changes so slowly so as to be found in text books like chemistry or accounting. Also, a domain is vast enough so that one person could spend his whole career studying it (a field). This is confused with context-specific knowledge, which is the application of the domain. So you could say double-entry accounting, which has been around for 300+ years, is a domain. At the same time applying double entry accounting to Dell's accounting department, which is tightly integrated with other partner's accounting systems, is the context specifc part. With that being said, why in the world are we rewriting double-entry accounting system *terms* over and over? The terms within accounting should be an internal domain specific language, not found in some xml somewhere like as if they are going to change in the next 20 years! Also, if you can't understand the DSL in a mature domain, that means you haven't the knowledge to complete the system anyways. When an accountant says assets = liabilities + equity + (revenue − expenses) then when you don't understand those terms you can't blame that on your DSL implementation complexity, althought many would love to try. Now this is in contrast to a so-called DSL written to marshall data from some layer for updating into a database. That domain could be useful, but how long will it last?

    Reply

• Older >