Sorting out the data
Obtaining an estimate from collected data is easy enough. The FORECAST() function in OpenOffice.org will perform linear regression on a set of entries, which should be good enough if you data has any kind of correlation. All grouped together, it might not be that easy to find a correlation between the different projects, especially if they are of different nature or written in different languages. This entry will present techniques I used to divide my projects to find correlations.
The International Software Benchmarking Standard Group (ISBSG, pronounced ice bag) determined that the most determinant factors affecting the total project costs per function point are:
- The programming language
- The platform
- The size of the development team
While they have some data to support these elements, this is far from rocket science. Everyone knows that PHP is faster to write than C++, development cycles are slower on a mainframe than on PC and communication issues slow down teams. The only elements that you should care about in this list are the two first ones. As the technique I used is based on a personal process, team effort is not considered. The only way it could reflect the data is in the noise it generates in the non-task time.
You probably don’t want to mix handheld device application development with web development. Even among web applications, an intranet application can probably not be compared to a public application visible by thousand of users. Even if the technology is the same, the expected quality attributes simply cannot be compared. A public application requires much more attention to interface details, security issues, performance and so on, while the intranet application used by a dozen of trusted users can cut short on security (as long as error-proof validation is done) and performance issues. Even the user interface does not require as much attention as you can provide training as needed.
I know what you’re thinking. You could use a database to store your project information and use tags to assign the quality attributes and use these to select the projects that are going to be used as part of the estimation data! Stop! You don’t need anything complex. Chances are your business is evolving in a niche market or is working on similar types of products. If 80% of your projects fit into a similar category, there is no real need to split projects in dozens of subcategories. In my case, most of my development fits into the intranet type of application. I decided to ignore that other 20%. I simply don’t measure it. I have 80% of my projects in a single group, and the other 20% is divided into multiple small groups where only one or two projects can fit. Measuring these projects simply feel like a waste of time since I will probably never use the data.
If among very similar projects you still can’t reach a correlation, I can think of 3 different reasons. There are probably more.
- You are not consistent in the counting technique
- The counting technique is too simplistic or not well adapted
- Your output is simply not regular
The two first issues can be solved with time. Try counting on more projects, try with different techniques. It does take time, but what good can you do on a Friday afternoon anyway? The last element might happen if you are new to the development environment. There is nothing more you can do about it. While measuring is a good training for the future, it won’t help you at all. You simply need to learn more about the environment, get used to the behaviours, error types, and all those things that make the difference between a junior and a senior programmer. If this is your case, you are probably better off measuring the defects you generate and detect, along with their types. While it does not appear to add any value, it will help you identify your weak points and help determine where to focus efforts and what to look out for when problems occur.
The COSMIC-FFP method discussed in a previous entry indicated that function point additions and modifications should be counted together. I do the opposite. Both my estimation table and my time entry sheet have distinct columns to track additions and modifications. Modifications tend to be much shorter than additions (by a factor of 3). In a perfect world where all my projects would have the same ratio of additions and modifications, I would not have to make a distinction. Some of the projects are modifications only of existing features while others are almost completely new development, with a few integration aspects (which I count as modification).
I found this separation after a few projects. At some point, I was tracking 3 columns: addition, modification and integration. I merged the last two as the performance indicators were very similar. You can find information like this by carefully looking at the data and comparing it with other projects. When you have 4-5 projects, you can do this easily. Pay attention to the distance between your final value and the estimated one as a percentage and ask yourself why it’s different between two projects. It may be because you wasted half a day on a single problem, or it might be that the two projects are of different nature, such as containing more modifications of existing features.
I also noticed that the estimates tend to be more accurate when the project size is significant. Small projects that only span on 1-3 days tend to be inaccurate. Performance is a lot more about human issues than technology. Projects that span on short periods do not get your average performance, they get your performance on a specific day. If you suffered from insomnia over the night, it will affect your performance on that day. A two day project is likely to be delayed significantly by a single bad day. On the other hand, a two week project will easily absorb that bad day.
The downside of estimates is that you only know if they were good at the end of the development, but once you got a few estimates within 5-10%, you can feel more confident about them. I did say 5%. It does sound unrealistic when all those books mention terrible fiasco stories where the final product doubled the initial schedules, but it’s a completely different story. What I do here is programmer-centric micromanagement. I plan a few weeks ahead at most. I don’t deal with contractual issues, external relations, communication among developers, poor requirements, and all those things that could get a project to fail. The estimates are very accurate because the data is very specific. It’s not about general PHP performance, it’s about my own performance.