Testing
Yesterday, I listened to Travis Doherty’s presentation on testing at the PHP Quebec monthly meeting. Half the presentation was on general testing concepts and the rest of it was a demonstration of Selenium. The presentation would have been great if only I didn’t hate the topic so much. Seriously. I hate testing.
It’s not that I don’t like the concept, it’s just that I hate having to think about test cases. Especially since it means they will have to be typed in at some point. Sure it’s nice to have a large test suite proving that you did not break anything during the last modifications, but when you actually want to break a behavior, all those tests need to be updated. Maintaining code is not fun. Writing tests is not fun. Maintaining tests is about as low as it can get. Oh, you can certainly be maintaining someone else’s test suite.
The other problem is that the topic is too broad. You could have a test suite with 100% code coverage and yet have an application that does not work at all. All you need is a developer to write the test cases based on the behavior of the code written. I have the feeling that the best individuals to write tests are the final users. They don’t have to think about test cases. All they have to do is think about the task they are trying to accomplish. On the other hand, functional testing is all they can provide. There is still a gap to bridge to reach the low level implementation. I’m not too certain those low level tests are any useful unless they are to test a widely used API, but that’s a different discussion.
That’s my other issue with testing. It never ends. No matter how many tests you write, it will never prove that the given piece of software is bug free. Of course, that is unless the given piece of software is absurdly simple. One of the reasons I measure the size of the projects I work on and make estimates is to know how I am progressing. I don’t see any such reference when writing tests.
The latest strategy I used on this front was to defer the actual writing of test cases out of my scope. I don’t have any results on this at the moment, but I certainly hope it will work. The idea is very simple. Write the test software taking generic input from an external file. Make sure the file is easy to understand by a non-developer user (which does not mean non-technical). The data format must allow to build data sets and attempt operations on it, with expected results. I used simple XML files, which are easy enough to understand by anyone who read HTML before.
For any date input, I run the input through strtotime(), which allows to enter relative dates like “next week”, “3 days ago” or such. This way, values like expiration dates are a lot easier to test.
Going back to the presentation and Selenium, it looks like a fine product to test user interfaces, although writing tests appears to take even more time and running them is even more time consuming.