InfoQ

News

JUnit Still Not Dead

Posted by Craig Wickesser on Aug 14, 2008

Community
Java
Topics
Unit Testing ,
Release
Tags
TestNG ,
JUnit
JUnit is a unit testing framework for Java which has been around for years. It was originally developed by Kent Beck and Erich Gamma and is now maintained by the community which has recently released JUnit 4.5. In the many years that JUnit has been around numerous projects in the xUnit family have been spawned including: Even with years of development effort and usage on countless software projects around the world some continue to wonder if JUnit's fate is nearing its end. Often times when someone (or some group) of people begin to doubt the long term existence of a software project they will rely on web page viewing statistics. Just this week Mike blogged about the declining "interest" in JUnit based upon search engine results. Although he concludes by stating he doesn't believe JUnit is dead there seems to be some inclination that perhaps JUnit is being surpassed.  One reason might be that as newer and better software practices evolve more and more developers catch on to them. Unit testing is certainly important and widely accepted, however, behavior driven development, mock objects and newer-featured unit testing frameworks are standing in the spotlight.

One framework which is often compared to JUnit is TestNG, a testing framework inspired by JUnit. Michael Minella provides a brief but useful comparison between JUnit and TestNG.

Feature JUnit TestNG
User Defined Life Cycle check check
Test Organization (groups, etc)   check
Distributed Test Execution   check
Parallel Test Execution   check
Data Driven Tests   check
Dependency Testing   check
IDE Integration check check
Ant Integration check check
Maven Integration check check
Domain Specific Extensions (Database, HTTP, etc) check  
Active Community check check
source


To summarize his thoughts, JUnit has been adopted and extended more so than TestNG, however, TestNG seems to offer more functionality and has the ability to invoke JUnit tests, making it a tempting choice in testing frameworks.

In conclusion, JUnit may not be in the spotlight as much as it was a year ago, but is it really walking closer towards the 'bright light'? What do you think, what does the future have in store for JUnit?

22 comments

Watch Thread Reply

I do think TestNG is better by Juven Xu Posted Aug 14, 2008 10:17 PM
*infamous*????? by Artur Karazniewicz Posted Aug 15, 2008 2:26 AM
Re: *infamous*????? by Michael Stachel Posted Aug 15, 2008 8:29 AM
Re: *infamous*????? by Mark Levison Posted Aug 15, 2008 10:05 AM
Re: *infamous*????? by Artur Karazniewicz Posted Aug 15, 2008 12:17 PM
Re: *infamous*????? by James Williams Posted Aug 19, 2008 6:15 PM
Re: *infamous*????? by Sean Kennedy Posted Aug 26, 2008 8:50 PM
Assumptions based on web statistics??? by Guillaume Bertrand Posted Aug 15, 2008 4:50 AM
JUnit is far from dead by Michael Minella Posted Aug 15, 2008 10:20 AM
Re: JUnit is far from dead by craig w Posted Aug 15, 2008 11:04 AM
Re: JUnit is far from dead by Kurt Christensen Posted Aug 15, 2008 11:57 AM
Re: JUnit is far from dead by Bernd Eckenfels Posted Aug 15, 2008 3:59 PM
Re: JUnit is far from dead by Sai Venkatakrishnan Posted Aug 17, 2008 9:18 AM
Using TestNG by Mileta Cekovic Posted Aug 15, 2008 3:46 PM
infamous? by Etienne L Posted Aug 18, 2008 7:01 AM
xUnit as "The Camel's Nose" by Paul Nelson Posted Aug 18, 2008 2:01 PM
TestNG by Joshua Partogi Posted Aug 19, 2008 5:31 AM
Re: TestNG by James Williams Posted Aug 19, 2008 6:24 PM
Re: TestNG by Morten Hattesen Posted Aug 19, 2008 7:05 PM
Re: TestNG by Robin de Silva Jayasinghe Posted Aug 21, 2008 3:38 AM
Junit ...infamous... by Amin Mohammed-Coleman Posted Aug 20, 2008 2:39 AM
Search statistics are not the way to measure a mature tool popularity by Fernando Lozano Posted Aug 20, 2008 7:11 AM
  1. Back to top

    I do think TestNG is better

    Aug 14, 2008 10:17 PM by Juven Xu

    I like the features of TestNG, but of course it's a little more complicated.

  2. Back to top

    *infamous*?????

    Aug 15, 2008 2:26 AM by Artur Karazniewicz

    "JUnit has recently released version 4.5 of the infamous Java testing framework"





    "*infamous*"? Did the Author smoke something Yesterday?.. Since when JUnit is infamous? Did I miss something?





    Also above comparison of JUnit and TestNG is little bit unfair, IMHO.





    IDE integration of JUnit is usually far, far ahead than TestNG. It usually comes with Your IDE, right there, build in, with all the goodies. I can simply pres my favurite keystroke and have test stub done for me. I can see reference for all my test, every coverage tool comes with support for JUnit, every CI server comes with integration fo JUnit...





    Second, there are *gazilions* of plugins and JUnit integrations basically for everything You could even think about.




    Last but not least - xUnit is supposed to be a *simple* tool. Just do the things it's supposed to do - Unit Test Your code. Lots of extra features TestNG has, for 80% of usages, are useless. Or at least I lived without them for last ten Years.





    Artur

  3. Back to top

    Assumptions based on web statistics???

    Aug 15, 2008 4:50 AM by Guillaume Bertrand

    Correct me if I am wrong, but the only assumptions to say that JUnit is declining is that less people are visiting the JUnit website than the TestNG one?



    JUnit is a mature framework, that many people use and that comes integrated in your IDE (as Artur rightly pointed out). I, for one, have been using JUnit for many years and I don't think I visited the website more than once (twice if you include today's visit for the sake of my argument); I just update my Eclipse regularly and JUnit gets updated along the way!



    If need be, I am happy to visit the JUnit website every time I run a test :)

  4. Back to top

    Re: *infamous*?????

    Aug 15, 2008 8:29 AM by Michael Stachel

    ... Lots of extra features TestNG has, for 80% of usages, are useless. Or at least I lived without them for last ten Years.


    This is the same kind of statement in the opposite direction as you did with asking whether the author did smoking something. So I answer you: "Have you lived behind a rock the last ten years or is your code being tested only "Hello World" stuff? :-)

    I recommend you take a look at TestNG and then come up with something more substantial. It is good that you defend your choice but without making claims which proves nothing.

    Michael

  5. Back to top

    Re: *infamous*?????

    Aug 15, 2008 10:05 AM by Mark Levison

    JUnit is a unit testing tool and it focuses on doing a good job of that.


    TestNG - wants to be a unit testing tool and a whole lot more.


    Neither is perfect, both make useful tools. Zealotry doesn't serve any of us well. I keep both tools in my arsenal and am agnostic as to which one I use.


    Also contrary to the belief of some Junit is still actively developed and evolving.


    Minor nit with the article: It can be argued that Dependency testing etc are not unit testing and hence the reason that its been left out of JUnit.


    Also the claims that Cedric has made with respect to his influence of JUnits adoption of the NUnit like attributes have turned off more than a few members of the community.

  6. Back to top

    JUnit is far from dead

    Aug 15, 2008 10:20 AM by Michael Minella

    Although in my blog post I said that I believe TestNG was a better framework, it's far from the leader regarding market share. JUnit still has the lion's share of the market. Between tool integration, developer familiarity and the fact that test driven development is still not as dominant as it should be, it will be a while before other tools gain enough traction to threaten JUnit's market dominance.

  7. Back to top

    Re: JUnit is far from dead

    Aug 15, 2008 11:04 AM by craig w

    @Artur
    I used "infamous" to indicate that JUnit is so widely known. Quite often if you ask a Java developer about unit testing JUnit is at least one of the tools they use or have heard of. And the comparison between JUnit and TestNG was meant to provide "brief" overview, not saying it was perfect or completely unbiased.

    It is true that there are lot of extensions (I did make a reference to it near the end of the article).

    @Mark
    I agree I thinking having both in your tool belt is a good idea, as a developer, we should be open minded and adopt whatever works best for the work we are doing.

  8. Back to top

    Re: JUnit is far from dead

    Aug 15, 2008 11:57 AM by Kurt Christensen

    Nitpick FYI: I think what Artur was picking on was that you meant to say "famous" and not "infamous", which many, many people do inadvertently. From dictionary.com:



    in·fa·mous

    –adjective

    1. having an extremely bad reputation: an infamous city.

    2. deserving of or causing an evil reputation; shamefully malign; detestable: an infamous deed.

  9. Back to top

    Re: *infamous*?????

    Aug 15, 2008 12:17 PM by Artur Karazniewicz

    Michael. To answer Your question. Yes I looked at TestNG - it's nice, neat and funny. But how did its existence made JUnit "*infamous*"? You know, writing unit tests is about testing Your code, not testing brand new testing framework itself.




    Artur

  10. Back to top

    Using TestNG

    Aug 15, 2008 3:46 PM by Mileta Cekovic

    I am using TestNG for several years. Enough said.

  11. Back to top

    Re: JUnit is far from dead

    Aug 15, 2008 3:59 PM by Bernd Eckenfels

    Dont spent too much time talking about testing - use it to write test cases.

  12. Back to top

    Re: JUnit is far from dead

    Aug 17, 2008 9:18 AM by Sai Venkatakrishnan

    Junit is a solid good framework. It delivers what is needed and everything else is an add on. I think Junit will be here to stay for a pretty long time. Junit is mainly used to drive the design so in that point of view I think that any tool which lets me do that job of driving design and lightweight is good. TestNG's feature rich but that mustn't divert us from the primary use of these tools.

  13. Back to top

    infamous?

    Aug 18, 2008 7:01 AM by Etienne L

    I don't see the need to call JUnit "infamous" to prove your point Wickesser. Btw, your analysis and "conclusion" is a little bit light for this kind of statement. Wouldn't it be better to compare TestNG to Junit and let the reader build their opinion? And I am not talking about a list of bullet points: a real case example would have better served your demonstration.

  14. Back to top

    xUnit as "The Camel's Nose"

    Aug 18, 2008 2:01 PM by Paul Nelson

    There is one major benefit that I think the article misses, though it is a companion benefit to one mentioned. I have been studying both Erlang and Objective-C lately in my spare time, and having an xUnit framework (EUnit and OCUnit respectively) provides a familiar tool with which to poke and prod these new (to me) territories.

    It's very cool having experience in a tool transfer so seamlessly to other languages, and has significantly reduced my learning curve for new programming languages, frameworks and libraries. I hope xUnit's ubiquity insures it's long-term success (I think it will)... at least until something else becomes as pervasive.

  15. Back to top

    TestNG

    Aug 19, 2008 5:31 AM by Joshua Partogi

    I like TestNG better because it is easier for me to exclude or include tests that I want to run by setting it up in testng.xml.

  16. Back to top

    Re: *infamous*?????

    Aug 19, 2008 6:15 PM by James Williams

    I couldn't agree more. I've used TestNG on a Seam project and found the IDE integration lacking. JUnit has served me well and with the use of annotations that came with version 4 I quickly untied my Seam tests from TestNG and used JUnit.

    James
    www.jameswilliams.com.au

  17. Back to top

    Re: TestNG

    Aug 19, 2008 6:24 PM by James Williams

    JUnit provides this too with the @Ignore annotation. TestNG does give you the ability to have different sets of tests (fast and slow for example) but it's tool support in Eclipse is shocking. The plugin does not reliably refresh and runs the tests based on a generated xml file that can get out of sync with the actual state of the code. I've spent time wondering why my changes don;t seem to make a difference to then realise I needed to clean my project. And don;t get me started about not being able to run a single test method in a test class. :-)


    James

    www.jameswilliams.com.au

  18. Back to top

    Re: TestNG

    Aug 19, 2008 7:05 PM by Morten Hattesen

    As of JUnit 4 it is quite easy to disable (exclude) tests, by adding the @Ignore annotation on @Test methods (or as of version 4.3, on entire JUnit classes).

    I find that easier than fiddling around in xml files.

  19. Back to top

    Junit ...infamous...

    Aug 20, 2008 2:39 AM by Amin Mohammed-Coleman

    I'm probably going to say the same that others have said. JUnit is the most popular Unit testing framework and I have worked in various companies and they all use JUnit. At the end of the day it's all about testing code that you have written and if JUnit works for you then thats ok. I personally don't care as long as there is a mature and solid framework that ensures proper testing for my code.

    By the way..JUnit is certainly not infamous...

  20. Has anyone wondered that maybe web searches and web site accesses for JUnit may be just the result of it being included by default with most IDEs? And the fact there are so many books and third-party articles that cover JUnit usage? People simply doesn't to "go to the source" for JUnit!

  21. Back to top

    Re: TestNG

    Aug 21, 2008 3:38 AM by Robin de Silva Jayasinghe

    Additionally JUnit has got nice extension points for manipulating the set of tests to be run. Just extend Filter or Sorter. Especially when you decorate your testcases with custom annotations they become very powerful. :)

  22. Back to top

    Re: *infamous*?????

    Aug 26, 2008 8:50 PM by Sean Kennedy

    "JUnit has recently released version 4.5 of the infamous Java testing framework"

    perhaps after this article JUnit 4.6 will become infamous by spawning a google search for JUnit every time a unit test passes

    then everyone will know how wonderful JUnit is

Educational Content

Rails in the Large: How Agility Allows Us to Build One Of the World's Biggest Rails Apps

Neal Ford shows what ThoughtWorks learned from scaling Rails development: infrastructure, testing, messaging, optimization, performance.

Stuart Halloway on Clojure and Functional Programming

Stuart Halloway discusses Clojure and functional programing on the JVM in depth, and touches on the uses of a number of other modern JVM languages including JRuby, Groovy, Scala and Haskell.

Orion Henry and Blake Mizerany on Heroku

Orion Henry and Blake Mizerany talk about the technology behind Heroku and the benefits of the new add-on system.

Security for the Services World

Chris Riley presents security issues threatening service based systems, examining security threats, presenting measures to reduce the risks, and mentioning available security frameworks.

Navigating The Rapids:Real-World Lessons in Adopting Agile

This talk investigates technical issues encountered when moving to an Agile process.

Codename "M": Language, Data, and Modeling, Oh My!

Don Box and Amanda Laucher present “M”, a declarative language for building data models, domain models or external DSLs. Don Box's demos show some of M’s features and latest changes of the language.

SOA Manifesto - 4 Months After

It is four months since the SOA manifesto was announced; InfoQ interviewed the original author’s to get insight into the motivations and the process behind the initiative.

Memory Barriers and JVM Concurrency

This article explains the impact memory barriers, or fences, have on the determinism of multi-threaded programs.