Unit Testing of Dates is often talked about problem – consider that one can’t depend on an absolute date in the test case. If the test passes, then the test will be considered successful only for the respective date. Instead, the standard advice is to use dependency injection (Adam Sroka, BigVisible):
In order to test a number of interesting dates you need to pass them into the calculation. Thus, the method that does the calculation should never get the date for itself – whether it is “now,” or from a database, or from user input, etc. You should obtain the date at a higher level and pass it in to your calculation.
It's easy to see how this works for Unit Tests – but the question remains: what do for Acceptance/System tests? Andreas Ebbert-Karroum faced just this problem and came up with a list of three alternatives:
- In your tests, make the expected result also date/time dependent.
- Change your application, so that it does not use the system current time, but a time service, that you can control remotely to use a different time than the current system time
- Change the system time remotely
Gerard Meszaros, author of xUnit Test Patterns, uses #2 aka the Virtual Clock Pattern:
We use an abstraction layer over top of the real time service (read: hide the calls to the system time API behind an interface) and then replace the implementation when testing with one controllable by the test scripts. By default, the real implementation is plugged in.
Pros: Can ensure that all test conditions are covered on every run. Can be used to test time race conditions.
George Dinwiddie, Software Development Coach, also notes that a single time source avoids subtle problems with systems that run across multiple machines with multiple time references i.e. system time on one machine vs. database time on another.
Ward Cunnigham (wikipedia) takes a different approach: “In the "swim" framework we tested cases that lasted weeks. We wrote SQL with a global substitution, $now, that would, in test, accumulate an expression that would have terms for every time advance, be it hours, days or weeks. This had the advantage of looking great when we traced the SQL.”
Mike Stockdale, Developer of FitSharp, points out that it supports relative date parsing, ex: today+2. Rick Mugridge, author of FitLibrary, notes it has: “a general date generator "DSL" which allows for the selection of relative dates in different time zones, supplied in different formats. It allows for dates to be selected at the end of a month, on a Monday, etc. This has been used extensively on a booking system where date selection in the future is critical, and where the tests have to make use of the data (and dates) that exist in the system under test.” While Martin Gijsen, test automation architect, uses ANTLR to solve the same problem.
So both Andreas’s first and second approaches are in wide use. Which approach have you used? What are their pros and cons?
Community comments
Its simple if your application ...
by Eirik Maus,
Re: Its simple if your application ...
by Mark Levison,
Re: Its simple if your application ...
by Eirik Maus,
Re: Its simple if your application ...
by Mark Levison,
Re: Its simple if your application ...
by Eirik Maus,
Re: Its simple if your application ...
by Ingo Boegemann,
Re: Its simple if your application ...
by Mark Levison,
Re: Its simple if your application ...
by Ingo Boegemann,
Re: Its simple if your application ...
by Eirik Maus,
Unclear on Option #3?
by Ted Young,
Re: Unclear on Option #3?
by Mark Levison,
Re: Unclear on Option #3?
by Ted Young,
Re: Unclear on Option #3?
by Andreas Ebbert-Karroum,
Re: Unclear on Option #3?
by Andreas Ebbert-Karroum,
Prefer #2
by John O'Hanley,
Re: Prefer #2
by Mark Levison,
What about adjusting the test data?
by Chuck van der Linden,
Re: What about adjusting the test data?
by Mark Levison,
One of the solutions is TimeMachine
by Konrad Kaminski,
Re: One of the solutions is TimeMachine
by Mark Levison,
Re: One of the solutions is TimeMachine
by Konrad Kaminski,
Date and Time Testing Tool - TimeShiftX
by Andreas D,
Re: Date and Time Testing Tool - TimeShiftX
by Ray Story,
Its simple if your application ...
by Eirik Maus,
Your message is awaiting moderation. Thank you for participating in the discussion.
... uses the (much too unknown, under-utilized, under-maintained....) library timeandmoney for date and time handling. You find it on sourceforge.
Applications uses a static method on a singleton clock with injectable "timeSource" to get the current time or date
(examples taken from our local fork so some trivial classes might be missing in the sourceforge version):
CalendarDate today = SystemClock.today();
For unit testing time- and progress-dependent functions, set an adjustable, fixed timesource:
// setup
AdjustableFixedTimeSource ts
= new AdjustableFixedTimeSource(SystemClock.now());
SystemClock.getInstance().setTimeSource(ts);
// run some function
myService.runSomeFunction();
// "let one hour pass"
ts.setTime(SystemClock.now().plus(Duration.minutes(60));
myService.checkTimeoutAlarmTrigger();
// validate correct behaviour
assertTrue(myService.alarmIsRaised());
Naturally we have several several TimeSource classes that can be used, including an OffsetTimeSource that follows the normal clock adjusted for some offset, which let you run integration tests with a normally progressing clock set to a specific date or time-of-day.
We have also discussed combining a TimeSource with the transactionManager in order to "freeze the time" at the start of a transaction, so transactions not only becomes wisible as a unit in the database but also contains timestamp data that are all identical to the start of the transaction. This will make virtually all aspects of the transaction appear atomic. (We haven´t done it, though, but it IS quite simple).
Re: Its simple if your application ...
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Interesting Eirik thanks for letting us know about another good library. Is this it: sourceforge.net/projects/timeandmoney/? With Eric Evans involved can I assume its well tested? Not just unit but also acceptance/api tests (even if they're written in JUnit)?
Cheers
Mark Levison
The Agile Consortium
Unclear on Option #3?
by Ted Young,
Your message is awaiting moderation. Thank you for participating in the discussion.
Could you elaborate on what option #3 (change system time remotely) means and how it would help in testing?
;ted
Re: Its simple if your application ...
by Eirik Maus,
Your message is awaiting moderation. Thank you for participating in the discussion.
Yes its that one.
Unlike java and joda it has the basic concepts right, which is an enormous benefit. For example is a CalendarDate only three immutable numbers (y,m,d), but the beginning and end can be mapped to (immutable) TimePoints using a calendar and a time zone. The 'money' part will give you little, but I can assure you it is the simplest and best library for time calculations around.
There are quite some tests but I've also found a bug or two.
We have been using a local copy for years.
Unfortunately, we get risk getting fired if we let a single line of source code escape the entrance door (no, I'm not kidding), so our fixes and additions have not been possible to integrate with the sourceforge project. The example above is prose, as you can see (no curly braces).
It is really unfortunate that we cannot contribute fixes and community to a library that is a core part of our systems. It is a security related promise to our customers (we do the payment and money transfers between banks - a sector seemingly built on security through secrecy) and also a way to prevent deverlopers from working on pet projects in their paid time. I hope someone else can.
Re: Its simple if your application ...
by Eirik Maus,
Your message is awaiting moderation. Thank you for participating in the discussion.
Well, we use the OffsetTimeSource in the program and have made a Servlet to adjust it from http get-urls.
This facilitates adjusting the time when the system runs inside a container. A small web page on top of the servlet lets human testers adjust the time in order to check behaviour in the easter, on new years eve, just around midnight etc. Such adjustments can also be scripted on *nix with wget or curl (and also in java using jwebunit or even just plain java.net.URL's).
It is also possible to create a TimeSource that reads the offset from a database table now and then, which is more convenient if you need to adjust the time for a cluster installation.
Otherwise, we try to run most tests in-process, embedding the webcontainer (jetty) as a library instead of the opposite. This is a whole lot faster and simpler, with tests typically running inside the IDE in 10 seconds or less. That way you can simply use the same approach as in the junit test example I provided.
Fitnesse tests use the same approach as junit, of course. Create a fixture that sets an adjustable time source and adjusts the time as specified in the test. It must probably return 'ok' so fitnesse has something to validate afterwards but that is just cosmetics.
Re: Its simple if your application ...
by Ingo Boegemann,
Your message is awaiting moderation. Thank you for participating in the discussion.
Time and money is marked as alpha and the files have been last updated in 2006?
Re: Unclear on Option #3?
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Ted - I'm not personally a fan of option #3 although apparently Andreas is. If I'm correct in understanding you set the time on the machine to be what ever you need for the test. The benefit being that there is no risk of bug in the dependency injection layer. Frankly it seems like a lot of work for a tiny benefit. In the past I've used both options 1 and 2. The bigger issue is trying to make an exhaustive list of all the edge cases you will encounter. Inevitably you miss something which is why I like as much as possible to use 3rd party libraries to hide the complications.
Cheers
Mark Levison
The Agile Consortium
Re: Its simple if your application ...
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Sounds like an excellent library.
Quite aside from the fact this seems silly and punitive - are you allowed to report bugs and make feature requests?
Cheers
Mark Levison
The Agile Consortium
Re: Its simple if your application ...
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Ingo - I just checked code was last committed ~600 days ago. So just under two years - Apr 2008? The projects authors are Eric Evans (of Domain Driven Design fame), Barry Hawkins (also a DDD guy) and Blaine Buxton. Based on the street cred of the authors and the fact that Eirik is using the library in production it seems like a good bet. In the worst case download it, try it and throw it away after a day if you don't like what you see.
BTW I will ask Barry Hawkins if he can jump in and tell us what he thinks of the current state of the library.
Cheers
Mark Levison
The Agile Consortium
Re: Unclear on Option #3?
by Ted Young,
Your message is awaiting moderation. Thank you for participating in the discussion.
Ah, OK. The word "remotely" was what confused me. I'm not a fan of changing the machine's system time either, because all sorts of weird operating system-related things start happening. If I were really that concerned, I'd probably put the system under test into a VM and use then change the system time. However, it seems like that would be more useful in a high-level integration test and not unit tests, since the whole point of unit tests is to isolate things not to change the world.
;ted
Re: Its simple if your application ...
by Ingo Boegemann,
Your message is awaiting moderation. Thank you for participating in the discussion.
That would be nice!
The web site quoted above (sourceforge.net/projects/timeandmoney/files/) shows the last release to 0.5.1 and the date for this release to be 2006-10-26
I looked at the library before and liked what it did - the above release number and date however makes it look like a dead project ....
Prefer #2
by John O'Hanley,
Your message is awaiting moderation. Thank you for participating in the discussion.
I prefer style #2.
I like how it gives me complete control of what time it is.
It's not hard to integrate the idea of a TimeSource interface into an application.
It's also possible to create a JDK Logger which uses such a time source.
Java Practices has a short example of that.
The web4j tool has such a TimeSource built in to it.
Re: Prefer #2
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Thanks for the references John.
Cheers
Mark Levison
The Agile Consortium
Re: Its simple if your application ...
by Eirik Maus,
Your message is awaiting moderation. Thank you for participating in the discussion.
It is dead as in: large organizations use it but are prohibited from providing anything back to the project.
Also, the lack of community is in this case not a large issue. After all, how many ways to add a week to a date will the future hold that we don´t already have today? Unlike more recently developed areas, Time is not a moving target.
Re: Unclear on Option #3?
by Andreas Ebbert-Karroum,
Your message is awaiting moderation. Thank you for participating in the discussion.
Hi,
option three means, that you ssh (or telnet) into your test machine and set the time and date to what you current automated acceptance test requires. even if you are in a mixed windows/linux environment this is easily achievable.
Andreas Ebbert-Karroum
Re: Unclear on Option #3?
by Andreas Ebbert-Karroum,
Your message is awaiting moderation. Thank you for participating in the discussion.
Hi,
exactly, something not useful for unit testing at all, but changing the time of your system under test from your test suite remotely is a nice option in your toolbox to have.
Andreas
What about adjusting the test data?
by Chuck van der Linden,
Your message is awaiting moderation. Thank you for participating in the discussion.
The way I see it there are three solutions
1) Adjust the time values in the expected results (#1 above) This works good if your results are in XLS format, and can contain date based formulas that get evaluated as the expected results are read into the system.
2) Adjust the Perceived 'current' time as seen by the test code(#2 #3 above both address this basically via different means)
3) Adjust the time values in the test data.
In my experience for systems that had to do any kind of 'aging' evaluation (finance stuff like wash sale rules, long-term vs short term gains, or 'stale' aging for items waiting for responses, workflow routing etc) #3 actually turned out to be an easier solution. We used a set of scripts that adjusted the test data according to the difference between when the data snapshot was taken, and the current time, so that offsets for aging calculations always yeilded the same results, something that was say 29 days old, would always be 29 days old. The system level tests commenced after the database had been restored and 'adjusted', and that approach worked great for stuff were accuracy was restricted to 'day' or 'hour' level of granularity. such as the examples given above.
One of the solutions is TimeMachine
by Konrad Kaminski,
Your message is awaiting moderation. Thank you for participating in the discussion.
There is an open source library TimeMachine which heavily simplifies creating these kinds of tests. It allows moving the current time of JVM to an arbitrary point of time in the future. At the same time it does not change the current system time (the change is only visible inside JVM).
The power of this library comes from the fact that you don't have to modify your existing code to use it as it enhaces the bytecode of loaded classes to modify the behaviour of System.currentTimeMillis/nanoTime, Object.wait and Thread.sleep methods. Therefore the various types of schedulers (java.util.Timer, Quartz, etc.) as well as embedded databases work right away.
You can find the examples of using it on the main page.
Konrad Kaminski
Re: One of the solutions is TimeMachine
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Thanks Konrad - this seems very interesting, have you used this in anger? Was it safe/stable?
Do you know of anything to help our friends the .NET developers?
Cheers
Mark Levison
The Agile Consortium
Re: What about adjusting the test data?
by Mark Levison,
Your message is awaiting moderation. Thank you for participating in the discussion.
Chuck - changing the data seems like an interesting approach, how much effort did you find you had to put into maintaining the scripts? How about the performance - was this suitable only for system tests?
Cheers
Mark Levison
The Agile Consortium
Re: One of the solutions is TimeMachine
by Konrad Kaminski,
Your message is awaiting moderation. Thank you for participating in the discussion.
We have used it to test long running processes in our workflow product and we didn't have any problems with that. The only obstacles so far came from the inappropriately written code in third-party libraries. An example of that is Bitronix transaction manager which periodically tries to do recovery of transactions. Unfortunately if it does this recovery at time T and finishes it at time T + Q, it then tries to do it at time T + some delay (instead of T + Q + some delay) even though it is in the past. So if we move to the future by a year then it tries to make up for all this time and fires lots of threads...
As for the .NET stuff I'm not aware of any kind of a tool like that. At the same time I'm not really a .NET developer anyway.
Regards
Konrad Kaminski
Date and Time Testing Tool - TimeShiftX
by Andreas D,
Your message is awaiting moderation. Thank you for participating in the discussion.
I'm not a fan of changing the system time due to the OS side effects. And overriding time functions or modifying data sets can add a lot of additional work and not cover other components in the stack.
When needed, I prefer a 3rd party tool to handle the needed time changes. Several exist, I've had much success with TimeShiftX
It's convenient as it syncs all components
Re: Date and Time Testing Tool - TimeShiftX
by Ray Story,
Your message is awaiting moderation. Thank you for participating in the discussion.
I've had good luck with Time Machine for the time shift testing, which isn't the same one as the Javart one mentiond by Konrad. It's from another company.
It does the same thing as the Time Shift X and is pretty robust. I just like the web-interface better.
Still syncs everything, though