CAST: Adding Spring Lowers the Quality of JEE Applications

by Abel Avram on Feb 14, 2013 |

A CAST report discloses that JEE enterprise software has lower quality when using Spring or Struts than using just Hibernate. Also, the quality degrades when Java is mixed with C or C++.

CAST Software has recently released the report Java Applications and Coffee: The Variations are Endless. The purpose of the CAST CRASH report is to understand the impact on structural quality –measured in reliability, security, performance, maintenance - of JEE applications when using various frameworks and when Java code is combined with other technologies. The report has analyzed in particular the impact of Hibernate, Spring and Struts on 496 JEE business applications (not open source) belonging mostly to Fortune 500 companies and totaling 152 millions lines of code.

The findings of the report are:

Around half of JEE applications do not use frameworks. The large number may be explained by the fact that the development of some of them started before any framework was available.
Applications without frameworks have a large variation in quality, between 2.7 and 3.7 on scale from 1 to 4. The report authors’ interpretation is that frameworks tend to raise the quality of applications by setting constraints and enforcing certain architectural patterns.
Applications using Struts or Spring besides Hibernate have lower quality levels then those having just Hibernate. One of the reasons is that such applications have a much larger codebase and are more complex. But the averages of all categories are spread closely around 3.25, as shown in the following chart. (The thin gray lines show how quality of all applications in a category is dispersed while the white horizontal bar in the middle represents the average for that category.)

Adding both Spring and Hibernate makes JEE applications 2-3 times larger than other variants of JEE apps: 498 KLoC compared to 184 KLoC for apps without any framework or 183 KLoC for those using Struts (shown in the picture below).

The codebase size of pure JEE application is up to 6 times smaller than of those using JEE and C++, as shown below.

Pure JEE applications (not mixed with non-Java technologies) have higher quality.
The quality of JEE applications tends to degrade when mixed with other technologies especially C/C++. Older/lower-level technologies seem to lower the quality of applications. The quality remains about the same when mixed with .NET technologies:

The report is based on source code analysis that CAST uses to evaluate the quality of many Fortune 500 enterprise software.

Related Editorial

From Killer Apps to App Killers: The Devastating Effect of Architecturally Complex Defects

Impact of Java EE Frameworks on the Structural Quality of Applications

Extend Your Java EE Apps with a Mobile Web Interface

Making the Shift from Relational to NoSQL

Beyond HTML5 and Native Development: Build Richer, More Functional Mobile Applications

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Community comments

Struts 1.x ... by Jens Hadlich Posted

Makes sense by Cameron Purdy Posted

Lies, damned lies and statistics by Pavel Grushetzky Posted

Really? by Albert Mendonca Posted

Re: Really? by Jonathan Allen Posted

Re: Really? by anshuman mehta Posted

Re: Really? by Albert Mendonca Posted

Re: Really? by John Doe Posted

Re: Really? by Albert Mendonca Posted

Re: Really? by John Doe Posted

Nice by Juan Pickselov Posted

Or maybe... by Richard Richter Posted

PHP vs J2EE by Vic Cvc Posted

Statistics? by Hugo Valk Posted

Spring vs JEE by raj n Posted

Re: Spring vs JEE by Fred Amiter Posted

Re: Spring vs JEE by First name Last name Posted

Torture numbers, and they’ll confess to anything by Ronald Miura Posted

Re: Torture numbers, and they’ll confess to anything by Abel Avram Posted

Re: Torture numbers, and they’ll confess to anything by Ronald Miura Posted

... and statistics by Mark N Posted

What if you add JUnit? by ANDREW CLIFFORD Posted

Closed metrics on a closed data set by Mathhew Miller Posted

Full report available by Alexandra Szynkarski Posted

TechDebt.org by Michael MULLER Posted

Struts 1.x ... by Jens Hadlich

and Spring 2.x or what? ;-)

Makes sense by Cameron Purdy

Complexity is the enemy of quality. In cases where Spring reduces complexity, my guess is that it generally increases quality. In cases where Spring increases complexity, my guess is that it generally decreases quality. Early on, Spring was excellent at reducing complexity, because J2EE (the early version of Java EE) was itself very complex, and needed simplification. Now that Java EE is far simpler, with Context & Dependency Injection built in, and an annotation-driven development model, Spring may end up adding complexity in many cases.

Peace,

Cameron Purdy | Oracle

Lies, damned lies and statistics by Pavel Grushetzky

These number don't exactly support the title of the article.

If CAST would find a bunch of Spring-less JEE apps that introduced Spring, and measure quality before and after, that could give a food for thought.
Or if it would let a bunch of teams build the same app with and without Spring...

With just numbers from this report it is easy to speculate
"Spring lowers quality of JEE apps"
"Pure JEE is for simplest and smallest apps"

The report supports both statements - and none at the same time.

Really? by Albert Mendonca

Perhaps they should also survey how many of these were correctly "Springified" and how many of these actually know to correctly use Spring.

Re: Really? by Jonathan Allen

If a low percentage of Spring users know how to use it correctly, well that's a pretty good indicator that its too hard to use correctly.

Nice by Juan Pickselov

"Applications using Struts or Spring besides Hibernate have lower quality levels then those having just Hibernate."

Then? So in code, do we have If Than Statements?

Or maybe... by Richard Richter

...it is used on bigger and more complex apps where it is not so easy to keep the quality high (not that they should not try). Also "no framework" has lower quality than Spring+Hibernate, so adding Spring rather raises quality a bit, hm? I "love" this kind of conclusions and I'm no big Spring guy. I'm rather Java EE (not JEE, BTW) guy, currently just working more and more with Spring. And like with everything, you have to know that stuff.

Good survey is a good thing, they collected some interesting data I guess, but those conclusions are just plain silly, comparing incomparable. Not to mention that with all the convergence in Java EE vs Spring space, Spring still can give a lot to Java EE and I'll gladly see how Java EE gets simpler. Good that Java EE started to prune things, Java SE libs would need that as well. Because on this front any lib can beat Java standards with its wannabe backwards compatibility.

PHP vs J2EE by Vic Cvc

You can see results in products, for example CMS. PHP developers tend not to use heavy frameworks and... their CMS are much more successful and iterated. Just simple (an overused word) library vs a framework. The more complex the app, the less framework you can use, for example MMO games ... all they use is an entity system.

Re: Really? by anshuman mehta

If a low percentage of Spring users know how to use it correctly, well that's a pretty good indicator of how easy it is to hook up and use. (albeit incorrectly when the 'concepts' are not given weight).

Statistics? by Hugo Valk

So what does this report tell us? I can at best draw these conclusions:

- The average quality when using Java frameworks is roughly the same
- Using Hibernate seems to raise quality vs. using JPA or even plain JDBC, but the sample size is too small to draw a definitive conclusion
- Using an extra web framework raises your LoC
- JEE combined with COBOL gives you highest average quality next to JEE only (probably because we are talking banking systems here)
- Having to integrate your JEE app with C/C++ systems seems to affect quality, but the sample size is too small to draw a definitive conclusion

There is nothing to conclude about the quality of Spring and Struts, since the samples are too small.

Re: Really? by John Doe

If SpringFramework needs to be "correctly" used to produce quality, I tend to say that it misses its goal by far !
SpringFramework is absolutely not a standard, and itself even doesn't respect the SOLID and KISS principles needed
by every thing that need to be big. Spring awesomeness touches its end ...

Spring vs JEE by raj n

Spring was great during J2EE times. Now with JEE it is just splitting hairs. I personally prefer JEE.

Re: Really? by Albert Mendonca

Those who have a difficulty in understanding Spring and in general concepts of DI should rather go do farming!

Re: Really? by Albert Mendonca

Wow what a comment!!! Would you agree you could take anything and use in a twisted manner? Hibernate even? Of course Spring is not a standard - but hey who told you i care about the standard more than i care about a good technology? I have worked on several "standard" JEE apps but there never came a time to test that "standard". On the other hand there are a whole bunch of technologies that are not standards but used to great effectiveness and productivity!

Re: Spring vs JEE by Fred Amiter

Spring is just unnecessary with JEE.

Re: Really? by John Doe

I'm not agree with your assertion. It's too easy to say "anything can be used in twisted manner". In Framework you have the word "frame".
I won't list all the painfull option springframework offers, and the lack of convention its imperative style plus xml brings.
Would you agree that saying "if a project have a poor quality using spring it's because people do not know the quintessence of spring"
is not an option. On top of that If you want develop a framework don't go with spring, an application yes but not a framework.
In the other hand, when I talk about standard, I did not say JEE is THE way. I am myself measured. SpringFramework and Oracle propose
enterprise solutions but I wont' take any of them. I mean one of the whole stack. Spring is too bloated and unmodular, JEE stack is not
developer friendly, I think bringing injection by the app server is a mistake.
My way, good little library with one purpose example : Guice, Shiro, reflections, Jersey many others. Plus some JSR 330, 310 , 303, ...
I take care of the integration.

Torture numbers, and they’ll confess to anything by Ronald Miura

Although this research ignores/omits lots of relevant information (size in functionality instead of lines of code; inherent vs accidental complexity), its conclusions are reasonable.

From the actual video:

"Conclusions:
- In addition to ease of development, frameworks do enable higher quality of applications
- Larger more complex applications tend to use a combination of frameworks in their architecture (Hibernate + Spring)"

That is, frameworks are good, and people tend to use more of them to solve large/complex problems.

But one thing must be noted: the conclusions of the research are not what was posted at InfoQ. I just don't know if is due Abel Avram's incapacity of reading the slides, 'pure JEE' fanboyism, or just desire to generate controversy (thus, more traffic to the site). From the InfoQ article:

"- Applications using Struts or Spring besides Hibernate have lower quality levels then those having just Hibernate. One of the reasons is that such applications have a much larger codebase and are more complex."

That is, Spring is (or frameworks in general besides Hibernate are) bad. He mistakes cause as consequence, affirming the complete inverse of the original conclusion.

meh

Re: Torture numbers, and they’ll confess to anything by Abel Avram

Ronald,
you may want to read the article again. My article is in accordance with the study.

JEE applications (no frameworks) have a quality spread over a larger interval of the scale compared to those using frameworks. (Frameworks vs. no frameworks) The explanation: frameworks introduce a number of usage patterns that force such applications keep their quality spread over a smaller interval.

Another conclusion is that Spring apps have lower quality than those using just Hibernate. (This is a comparison between apps using frameworks.) And the explanation is not that Spring lowers the quality by itself, but rather because apps using Spring are larger and more complex, being prone to more errors.

If Springs apps have a slight quality problem does not mean Spring is bad. Spring offers a number of features that bring a JEE app much faster to the market outweighing the slight dip in quality.

Re: Torture numbers, and they’ll confess to anything by Ronald Miura

And the explanation is not that Spring lowers the quality by itself, but rather because apps using Spring are larger and more complex, being prone to more errors.

You should read the title again.

If Springs apps have a slight quality problem does not mean Spring is bad. Spring offers a number of features that bring a JEE app much faster to the market outweighing the slight dip in quality.

Spring apps don't have a 'slight quality problem'! Large and complex applications have a 'quality problem', Spring, probably, is what made it 'slight'.

As i said, the study itself gets to a reasonable conclusion. Which doesn't include Spring bashing.

And, to those who didn't bother to watch the video, the author says that most 'No framework' apps in the study are old or very small apps. Not complex apps made simple by using the magic of EJB3. Although, personally, I would never classify an application that uses EJB3, CDI, or JPA as 'No framework', as they are just platform-provided frameworks.

... and statistics by Mark N

Too me, the numbers are too close to come to any useful conclusion. They also do not account for the human variable. I can, however, come to my own conclusion from my experience. Too many developers think a framework or no framework is a silver bullet. Neither is true. It takes both experience (time) and capability (you are born with it) to know when to do what or when not to. For the majority of applications I have worked on, using an ORM and DI/IoC framework have reduced the application's LOC and overall complexity. YMMV. Remember the good old days with COBOL when developers complained because "we" used procedure calls instead of just coding everything in on big procedure? Sure, it increases complexity on some level, but reduced overall complexity.

What if you add JUnit? by ANDREW CLIFFORD

Hey, JUnit is a framework too. If you add that into the mix does quality go up?

No comment on quality versus cost or time-to-market. Struts probably points to very old development that is mature and probably in an organization that doesn't make quality a priority. It doesn't say how Spring is used: Spring MVC? DI? JdbcTemplate? Quality seems to be lower on the web app tier. Does a JEE container mean just Tomcat/Jetty or does it have to have the full-blown EJB container too.

I could see how Hibernate might hurt performance quality if it's not tuned correctly. Struts, again, is a sign of other low quality practices. I can't say I understand how adding Spring could lower quality other than selecting it may mean a general loss of control on technical stewardship.

Still the numbers are interesting but I need to know what they mean by quality and how it is actionable.

Re: Spring vs JEE by First name Last name

Don't use stuff just because it is standard otherwise you will falling into j2ee trap again. Do what you need to solve your business problem it does not really matter if you use any framework in java or .net. As long as you are a decent programmer, it should not be difficult to switch between technologies.

Closed metrics on a closed data set by Mathhew Miller

All this proves is that, of the anonymous applications tested by a proprietary tool, the largest number of lines of code correlated with apps using hibernate and struts, and that these applications tend to perform marginally worse on the tool's metrics.

Funny, that more complex applications might use more frameworks and also have lower total quality than less complex applications.

Of course, lines of code has been a godawful metric since forever, so already we're also dealing with bozos. Are they counting each line feed in an XML file as a "line?" Are they analyzing code per or post cglib?

I don't have a horse in this race, having used all the technologies listed. But I do have a code quality suite I won't be buying.

Full report available by Alexandra Szynkarski

The full report from this webinar can actualy be found here: www.castsoftware.com/resources/resource/cast-re...

It gives more detail into the stats that were presented. Also a look at the types of violations associated to each framework.

TechDebt.org by Michael MULLER

If you're interested in software quality benchmarking, should take a look at TechDebt.org, a live & collaborative dashboard on Technical Debt & Software Quality measures. Feeded with +4600 real code projects, 895M lines of code analyzed.