The Case for Evolvable Software

Just because changing random parts of a program causes 5-10 tests not to fail doesn't mean you haven't crippled the code in some way; this doesn't imply robustness. It's more than likely you just don't have a complete enough suite of tests to notice the change.

Well,

If production code evolves like this you probably mean 'not readable' with 'crippled'. Correct?

The question is if that matters. As long as you can understand your the written tests, and are able to write new test-cases, than knowing exactly what to change in the production code might be less relevant; the computer might be able to figure that out.

Cheers,

No, I suppose he means crippled in the sense of broken, i.e. the code has new bugs the tests did not check for. If you really start to make effectively random code changes that are not reviewed by humans, you would need 100% test coverage for the whole code. That is, thousands of tests. Unfortunately I have yet to see a project with that kind of test coverage. :-}

Hi Hans-Peter,

I understand what you mean. Now just for the sake of following the proposal of evolving software: suppose that the job of a software engineer would be to add new tests only and -not- touch the production code anymore.

Would the software engineer be able to steer the evolution of the productioncode with newly written unittests? Could we add new features by writing unit tests that do not pass (yet!)?

Personally, I think we can't, because if we would add a new test, we are never sure if it is a 180 with a previous test that was written by somebody else. In other words, evolving software would then never be able to pass all tests...simply because new ones could be totally the opposite of existing tests.

Cheers,

What you are describing is actually a well known practice called test driven development for writing high quality software. First you write (so far failing) tests for new/modified functionality, and then adapt the code until the tests pass. You are right: If you have many tests, you will usually have to fix bugs in the tests as well.
When writing software, you usually have two somewhat othogonal "safety nets" to ensure correctness: first, the developer thinks carefully about the code when changing it, and second, one writes tests to verify the whole thing. If you remove the first safety net by evolving the software automatically, you need to strengthen the second net considerably. It is not clear to me whether this actually saves human effort in the end.
Perhaps there are special areas where it is very easy to write and maintain many tests, or where there is some kind of "training area" for the evolving system, or where bugs are inconsequential and can be fixed automatically right away. Maybe the evolving system serves as a special kind of machine learning.