System/Acceptance Testing with Time and Dates
- Share
-
- |
Read later
My Reading List
Unit Testing of Dates is often talked about problem – consider that one can’t depend on an absolute date in the test case. If the test passes, then the test will be considered successful only for the respective date. Instead, the standard advice is to use dependency injection (Adam Sroka, BigVisible):
In order to test a number of interesting dates you need to pass them into the calculation. Thus, the method that does the calculation should never get the date for itself – whether it is “now,” or from a database, or from user input, etc. You should obtain the date at a higher level and pass it in to your calculation.
It's easy to see how this works for Unit Tests – but the question remains: what do for Acceptance/System tests? Andreas Ebbert-Karroum faced just this problem and came up with a list of three alternatives:
- In your tests, make the expected result also date/time dependent.
- Change your application, so that it does not use the system current time, but a time service, that you can control remotely to use a different time than the current system time
- Change the system time remotely
Gerard Meszaros, author of xUnit Test Patterns, uses #2 aka the Virtual Clock Pattern:
We use an abstraction layer over top of the real time service (read: hide the calls to the system time API behind an interface) and then replace the implementation when testing with one controllable by the test scripts. By default, the real implementation is plugged in.
Pros: Can ensure that all test conditions are covered on every run. Can be used to test time race conditions.
George Dinwiddie, Software Development Coach, also notes that a single time source avoids subtle problems with systems that run across multiple machines with multiple time references i.e. system time on one machine vs. database time on another.
Ward Cunnigham (wikipedia) takes a different approach: “In the "swim" framework we tested cases that lasted weeks. We wrote SQL with a global substitution, $now, that would, in test, accumulate an expression that would have terms for every time advance, be it hours, days or weeks. This had the advantage of looking great when we traced the SQL.”
Mike Stockdale, Developer of FitSharp, points out that it supports relative date parsing, ex: today+2. Rick Mugridge, author of FitLibrary, notes it has: “a general date generator "DSL" which allows for the selection of relative dates in different time zones, supplied in different formats. It allows for dates to be selected at the end of a month, on a Monday, etc. This has been used extensively on a booking system where date selection in the future is critical, and where the tests have to make use of the data (and dates) that exist in the system under test.” While Martin Gijsen, test automation architect, uses ANTLR to solve the same problem.
So both Andreas’s first and second approaches are in wide use. Which approach have you used? What are their pros and cons?
Related Sponsor
Agile Project Management That Adapts With You. Provide your global development team a shared space that adapts to the way they work.
Its simple if your application ...
by
Eirik Maus
Applications uses a static method on a singleton clock with injectable "timeSource" to get the current time or date
(examples taken from our local fork so some trivial classes might be missing in the sourceforge version):
CalendarDate today = SystemClock.today();
For unit testing time- and progress-dependent functions, set an adjustable, fixed timesource:
// setup
AdjustableFixedTimeSource ts
= new AdjustableFixedTimeSource(SystemClock.now());
SystemClock.getInstance().setTimeSource(ts);
// run some function
myService.runSomeFunction();
// "let one hour pass"
ts.setTime(SystemClock.now().plus(Duration.minutes(60));
myService.checkTimeoutAlarmTrigger();
// validate correct behaviour
assertTrue(myService.alarmIsRaised());
Naturally we have several several TimeSource classes that can be used, including an OffsetTimeSource that follows the normal clock adjusted for some offset, which let you run integration tests with a normally progressing clock set to a specific date or time-of-day.
We have also discussed combining a TimeSource with the transactionManager in order to "freeze the time" at the start of a transaction, so transactions not only becomes wisible as a unit in the database but also contains timestamp data that are all identical to the start of the transaction. This will make virtually all aspects of the transaction appear atomic. (We haven´t done it, though, but it IS quite simple).
Re: Its simple if your application ...
by
Mark Levison
Cheers
Mark Levison
The Agile Consortium
Unclear on Option #3?
by
Ted Young
;ted
Re: Its simple if your application ...
by
Eirik Maus
Unlike java and joda it has the basic concepts right, which is an enormous benefit. For example is a CalendarDate only three immutable numbers (y,m,d), but the beginning and end can be mapped to (immutable) TimePoints using a calendar and a time zone. The 'money' part will give you little, but I can assure you it is the simplest and best library for time calculations around.
There are quite some tests but I've also found a bug or two.
We have been using a local copy for years.
Unfortunately, we get risk getting fired if we let a single line of source code escape the entrance door (no, I'm not kidding), so our fixes and additions have not been possible to integrate with the sourceforge project. The example above is prose, as you can see (no curly braces).
It is really unfortunate that we cannot contribute fixes and community to a library that is a core part of our systems. It is a security related promise to our customers (we do the payment and money transfers between banks - a sector seemingly built on security through secrecy) and also a way to prevent deverlopers from working on pet projects in their paid time. I hope someone else can.
Re: Its simple if your application ...
by
Eirik Maus
[...] Not just unit but also acceptance/api tests (even if they're written in JUnit)?
Well, we use the OffsetTimeSource in the program and have made a Servlet to adjust it from http get-urls.
This facilitates adjusting the time when the system runs inside a container. A small web page on top of the servlet lets human testers adjust the time in order to check behaviour in the easter, on new years eve, just around midnight etc. Such adjustments can also be scripted on *nix with wget or curl (and also in java using jwebunit or even just plain java.net.URL's).
It is also possible to create a TimeSource that reads the offset from a database table now and then, which is more convenient if you need to adjust the time for a cluster installation.
Otherwise, we try to run most tests in-process, embedding the webcontainer (jetty) as a library instead of the opposite. This is a whole lot faster and simpler, with tests typically running inside the IDE in 10 seconds or less. That way you can simply use the same approach as in the junit test example I provided.
Fitnesse tests use the same approach as junit, of course. Create a fixture that sets an adjustable time source and adjusts the time as specified in the test. It must probably return 'ok' so fitnesse has something to validate afterwards but that is just cosmetics.
Re: Its simple if your application ...
by
Ingo Boegemann
Re: Unclear on Option #3?
by
Mark Levison
Cheers
Mark Levison
The Agile Consortium
Re: Its simple if your application ...
by
Mark Levison
There are quite some tests but I've also found a bug or two.
We have been using a local copy for years.
Unfortunately, we get risk getting fired if we let a single line of source code escape the entrance door (no, I'm not kidding), so our fixes and additions have not been possible to integrate with the sourceforge project. The example above is prose, as you can see (no curly braces).
It is really unfortunate that we cannot contribute fixes and community to a library that is a core part of our systems. It is a security related promise to our customers (we do the payment and money transfers between banks - a sector seemingly built on security through secrecy) and also a way to prevent developers from working on pet projects in their paid time. I hope someone else can.
Quite aside from the fact this seems silly and punitive - are you allowed to report bugs and make feature requests?
Cheers
Mark Levison
The Agile Consortium
Re: Its simple if your application ...
by
Mark Levison
BTW I will ask Barry Hawkins if he can jump in and tell us what he thinks of the current state of the library.
Cheers
Mark Levison
The Agile Consortium
Re: Unclear on Option #3?
by
Ted Young
;ted
Re: Its simple if your application ...
by
Ingo Boegemann
The web site quoted above (sourceforge.net/projects/timeandmoney/files/) shows the last release to 0.5.1 and the date for this release to be 2006-10-26
I looked at the library before and liked what it did - the above release number and date however makes it look like a dead project ....
Prefer #2
by
John O
I like how it gives me complete control of what time it is.
It's not hard to integrate the idea of a TimeSource interface into an application.
It's also possible to create a JDK Logger which uses such a time source.
Java Practices has a short example of that.
The web4j tool has such a TimeSource built in to it.
Re: Its simple if your application ...
by
Eirik Maus
Also, the lack of community is in this case not a large issue. After all, how many ways to add a week to a date will the future hold that we don´t already have today? Unlike more recently developed areas, Time is not a moving target.
Re: Unclear on Option #3?
by
Andreas Ebbert-Karroum
option three means, that you ssh (or telnet) into your test machine and set the time and date to what you current automated acceptance test requires. even if you are in a mixed windows/linux environment this is easily achievable.
Andreas Ebbert-Karroum
Re: Unclear on Option #3?
by
Andreas Ebbert-Karroum
exactly, something not useful for unit testing at all, but changing the time of your system under test from your test suite remotely is a nice option in your toolbox to have.
Andreas
What about adjusting the test data?
by
Chuck van der Linden
1) Adjust the time values in the expected results (#1 above) This works good if your results are in XLS format, and can contain date based formulas that get evaluated as the expected results are read into the system.
2) Adjust the Perceived 'current' time as seen by the test code(#2 #3 above both address this basically via different means)
3) Adjust the time values in the test data.
In my experience for systems that had to do any kind of 'aging' evaluation (finance stuff like wash sale rules, long-term vs short term gains, or 'stale' aging for items waiting for responses, workflow routing etc) #3 actually turned out to be an easier solution. We used a set of scripts that adjusted the test data according to the difference between when the data snapshot was taken, and the current time, so that offsets for aging calculations always yeilded the same results, something that was say 29 days old, would always be 29 days old. The system level tests commenced after the database had been restored and 'adjusted', and that approach worked great for stuff were accuracy was restricted to 'day' or 'hour' level of granularity. such as the examples given above.
One of the solutions is TimeMachine
by
Konrad Kaminski
The power of this library comes from the fact that you don't have to modify your existing code to use it as it enhaces the bytecode of loaded classes to modify the behaviour of System.currentTimeMillis/nanoTime, Object.wait and Thread.sleep methods. Therefore the various types of schedulers (java.util.Timer, Quartz, etc.) as well as embedded databases work right away.
You can find the examples of using it on the main page.
Konrad Kaminski
Re: One of the solutions is TimeMachine
by
Mark Levison
Do you know of anything to help our friends the .NET developers?
Cheers
Mark Levison
The Agile Consortium
Re: What about adjusting the test data?
by
Mark Levison
Cheers
Mark Levison
The Agile Consortium
Re: One of the solutions is TimeMachine
by
Konrad Kaminski
As for the .NET stuff I'm not aware of any kind of a tool like that. At the same time I'm not really a .NET developer anyway.
Regards
Konrad Kaminski
Educational Content
Large-Scale Continuous Testing in the Cloud
John Penix May 24, 2013
.jpg)
Managing Build Jobs for Continuous Delivery
Martin Peston May 24, 2013
Clojure in the Field
Stuart Halloway May 23, 2013
Hello stranger!
You need to Register an InfoQ account or Login to post comments. But there's so much more behind being registered.Get the most out of the InfoQ experience.
Tell us what you think