Testing Techniques for Applications With Zero Tests

Recently, a whole module of a legacy web application written in PHP4 around 10 years ago (and constantly "maintained" since) was needed to have new features.

We're talking about 1000s of lines of code within one function, or even case of a switch case.

Most of the time, people don't refactor maintained legacy applications, as somebody told me "the first rule of support development is: don't change anything other than requested, just add your stuff."

I haven't been able to track the application back to its beginnings but its imminent to me that if-else branches don't grow to 600 lines by themselves, without human intervention. Somebody has to mess these up, and someone has to have such kind of thinking. This is pretty general in enterprise programmng:
- most of the tasks are about legacy applications
- people fear to clean up things
- it's not about development, but adding requested features and fixing bugs.

Also, PHP is a dynamic language, and therefore formal refactoring tools are usually unavailable. For example, PHP refactoring support in netbeans is basically non-existing.

So, what would you do here?

I decided that a system's answer is dependent only from its input and its context. This seems pretty straightforward

System ( input, context ) -> output

OK, what is the input of a web application? Of course, its HTTP request! In PHP, it's hard to think about any other input.

What's the context? Context is given by two components basically: the underlying platform, whatever it is (no matter you have a framework or just common libraries, we call these platform together), and the persistent data layer. So:

Web app (request, persistent data) -> answer

What's the answer? First, it's an HTML (or XML, JSON, etc) output. We didn't have to care about it in this particular case. The other output is: changes to the persistence layer. It's unusual for web applications to change anything other than their database and cache layers.So:

Web app(request, persistent data) -> (written-out response, persistent data' )

OK, what to do? We have an old system and we want to refactor it to a system, and the question was: are they equal in functionality?

Question is: Web app == Web app' ?

Let's see what I did:
- Ask a manual tester to go through every possible combination on the user interface
- Recorded these into files (serialize($_REQUEST)), or, even better, (serialize($GLOBALS))
- Ask the DB layer to NOT write anything to the DB (ugly global variable hack, if it is present, only select queries are executed), this way, we ensure that we keep a consistet state
- Record every writing operation (so, instead of executing them, take note of them)
- An algorhythm:
1) load the serialized request,
2) start recording db,
3) run the original controller,
4) collect db recordings,
5) re-load request (in case it was modified by the original controller - we could never know)
6) run the new controller
7) collect db recordings
8) see if the two are equal

This way, I could be sure, that in all of the scenarios a manual tester could come up with, both of the controllers behave the same way.

After the original recordings, I did two last points:
9) re-load the request again
10) enable db writing
11) run the new controller
12) display result.

This way I could create an - albeit slower - but seemingly normally functioning version of the software, which did everything it did previously, and it was verified that functionality haven't changed with the new controller.

I called this blackbox-harness test.

What do you think?

This is nonsense.

You can't just throw the blame on the team, and the legacy code, and then look for a magical solution that will give you proper test coverage and find your bugs.

Instead, I suggest you start becoming more responsible for your monstrous creation - refactor, unit test, and improve your code bit by bit.

Manual tests and auto-generated tests will lead you nowhere.

There is no blame here - not on the development team (who didn't know any better) and not on the managers (who don't understand any of this to begin with); Just a solution to a problem.

As for your suggestion - you're right, but before you can refactor, you need some kind of safety net, otherwise you don't know if your refactorings will work, or break the code; this is exactly what he did.

Finally, manual tests are a must; not everything can be automated. And generated tests are much better than none.