L-P Huberdeau

When working on a project, there are hundreds of possibilities for reuse of existing components. On first sight, some of them seem to do all we need or to do it partially. However, we rarely know the details. Without looking at the code, there is no way to know how it does things or what the priorities were. In the best cases, good documentation with usage samples will be available (API documentation is useless in this case). We don’t know how flexible it is or what the limitations are. There is also the whole question of quality. There is nothing more annoying that reusing code only to realize it’s full of bugs, then go through the code and realize the documentation was wrong.

The question really is to reuse or to rewrite. Writing it ourself is a very safe route without too many surprises, but it’s a long one. When you write it yourself, you know the details. The implementation is usually light an tailored to your needs. It’s easy to extend in the future, because you know the details and the vision. Choosing the reuse route is accepting a packaged deal. The implementation does what it does. Evaluation will tell you it does what you need. It will also come with a whole lot of elements you don’t really need. It may end up being good, because you may need them in the future, but on the short term, it clutters your vision. Reusable code weights a lot more than custom-built code. It contains more validations and verifications to deal with all sorts of environments and to produce better errors for the implementor (if you produce a reusable library that is “lightweight” and does not contain all of this stuff, what you produced is a custom piece of code you decided to give out). All this code has a value. When reusing code, you have a peace of mind. If you migrate to a different platform, or browser technology evolves, you won’t have to deal with the problems yourself. A simple update will do (never considered reusing an unmaintained component).

However, when you develop your software and need something that does not quite fit with what it does, you’re in trouble. You can work around the problem, which is likely to be innefficient, or you can fork the library and deal with the trouble of updating (only works with Open Source, but you want to use Open Source to have this option). The latter is completely undesirable. The ideal solution would be to contact the authors and see if you can get your modifications, or at least the required hooks, to be part of the core. Good solution, but they might refuse, and you still need to go through their code and understand it.

Reuse works great as long as you don’t push the limits. Low initial cost, high cost to maintain if you go beyond what it does.

To summarize:

• Writing a quick solution can be faster than reusing an existing component
• The quick solution will be easier to extend
• The quick solution will have a lot of maintenance to do if the environment changes
• Reuse makes migration easy
• Reuse will require more effort if you need to push the limits

The worst thing you can do is to decide to write it, publish it and maintain it. Down the road, you will have a clone of the solution you discarded for being “too heavy”.

So, if you want to reuse a component, the one thing you want to avoid is having to go beyond it’s limits, or expect those limits to be moved further away by the time you need it. It basically means you know the library is maintained, will be in the future, and has a good vision of where it’s going. This is a lot of trust required. Why do I prefer Zend Framework over CakePHP or CodeIgniter? Mostly because I met with the project leaders early on in the project. I discussed with them and figured out the project could be trusted. I put my trust in the people behind it. Problem is, not everyone gets to travel around and meet project leaders (not that it’s hard, just that most don’t think they can). So if you can’t have a face to face discussion with project leaders (which may give a positive bias anyway), what can be relied on? You want to make sure there is a community of people invested in the component and that the project will survive. One man shows working on hobby projects don’t really qualify.

Reusing a component adds weight to your project managment. Dependencies have to be maintained. You need to monitor releases and security advisory. It may call for an update that will require a few minutes or a few hours of work, and that call may not fit in your project’s schedule. You need to trust the project to be responsible about this too. If they reuse components themselves, do they follow the security advisory, or will you have to do that too?

One thing for sure, when you decide to reuse a component, you need to abstract it away. There are two reasons to this. One is self defense, so you can change to something else without too much effort if required, the other is more subtle and often forgotten: to preserve the aesthetics of your project. Each program is developed with a certain style. In a good program, you can guess how functions will be named and how objects need to be interacted with. The high quality components you reuse are probably like that too, but it’s not the sample style. Reusing without abstracting will create something that looks like patchwork, even when done by talented people.

These are the rational aspects of code reuse. There are also the irrational ones, which mainly cause to reject reuse.

• Misunderstanding
• Wrong expectations
• Focus on other aspects

I’ve seen people discard solutions for having high hopes, and I probably did it in the past. The laziness that makes great programmers can turn bad. When looking for a reusable solution to their problem, they hope that it will do everything they need. A bit like the VB expectation of RunTheProgramINeedForAssignmentNumber6(). Sometimes, you need to accept that the library won’t do everything and making a feature request for it is ridiculous. If you want to reuse a data storage layer, it may not do the remote synchronization you need, and it’s right not to do it, but you should not discard the storage layer because it does not do everything you need.

No matter what I have done wrong in this world, my punishment has a name: Newark. It seems like I can’t think about going there without something wrong happening. A few months ago, it was a missed connection caused by the US border security system going down, causing extremely long waiting lines. Today, I get to spend a lot more quality time in this sterile environment called Newark. At least, this time, it’s possible to sit somewhere to eat.

My flight was supposed to leave Montreal at 17:10. As I got to the airport, it was cancelled. No problem. They booked me on the 14:30 flight, which was delayed until 18:30. One more hour in Montreal, one less in New Jersey. Felt like a good deal. My connection there was planned to be slightly over 3 hours anyway.

Bad luck. They got a green light to take off at 16:30. That’s one more hour in New Jersey.

After a terribly bumpy ride in a regional jet under strong winds, in which every passenger’s face was blank (except for those laughing nervously, clearly having lost control), I was greeted in Newark by an announcement: My next flight was delayed by 3 hours.

Awesome.

That does leave plenty of time to write documentation.

Update: 4 hours delay.

Update: 4.5 hours delay.

Software quality is a bit like world peace. Everyone wants it, but when issues get close to you, there are always excuses. Schedule pressure, complexity, beliefs, blaming it on someone else, you name it. Small deviations in the course of action to handle exceptional situations may not seem to matter, but they do. In the end, you get what you pay for and you can’t really expect to have quality software without paying attention to quality.

The base theory is really simple. Defects get injected in the software as part of the development effort. They are caused by human mistakes and oversights, and that won’t change any time soon. The goal is to remove them to make a “bug free” application. Every activity for quality will remove a certain amount of these injected defects. If the amount injected was known, it would be really easy to keep looking until it goes down to 0. The truth is, we never know how many were injected and it’s impossible to prove the software is completely defect free. All you can do is reach a certain confidence level. Sadly, raising the confidence level has exponential costs and unless you’re working on code for a nuclear central, you probably won’t be able to justify the costs.

Some people believe that the developers are responsible for the code quality. It’s true in a certain way, but soldiers are not responsible for wars. Leaders are. On an ethical point of view, the soldiers should refuse to engage in conflicts that kill innocent people, but they have to follow orders. In the same way, developers should stand their ground and request to be granted the time to do things right, but many will fear loosing their source of income.

Being meticulous and careful is one way to reduce the amount of defects injected, but you can’t hope that it will solve the quality issue. A client asked me if they should be using unit testing or checklists when performing QA. To me, the answer was obviously both, but it does not seem to obvious to everyone.

Let’s move on to a different analogy. If you cook a meal for guests, you might review the recipe, make sure you have all the ingredients, validate that their quality is good and be very meticulous while performing the procedure. The fact that you did everything well at the start certainly does not mean that you should not taste the resulting meal before serving it. On the other side, skipping the verification on the ingredients does not make sense either. If you taste in the end and realize the milk use was not fresh, you just made one hell of a waste. It might also be very difficult from the end result to understand why the food tastes so terrible. You would have to be one confident chef to serve a meal without ever tasting any of the components.

Being meticulous in development is like lining up the ingredients, verifying them and paying attention to every step you make. Unit testing is like tasting the sauce while it’s still cooking. QA is the final verification before you serve. Peer review does not really apply to cooking a meal, but having someone watch over you and make sure you don’t make mistakes isn’t a terrible idea in the first place. Prototyping is like accepting to waste some food to practice on a complex procedure.

A programmer cooking for himself alone, a mother cooking for her children and preparing a Christmas dinner for your whole family all require different levels of quality. The time it takes is widely different. The three examples here would be around 20 minutes, an hour and a full day. Notice how exponential it is? What affects the quality level required is the cost of failure. When I cook for myself alone, I don’t really care if some details are not completely right. The consequence is basically not enjoying my meal as much or a few dollars if I have to redo it. Think about the consequences for the two other scenarios. Is the time taken justified? Certainly is. It’s natural and everyone understands it.

The exact same thing is true with software. However, it’s abstract and invisible, so people don’t see it so quickly.

A lot of the techniques used to obtain higher confidence in software quality are about piling up redundant checks. Quality levels can generally be improved incrementally, but each check you add has a cost, and the cost is not only time. Final QA requires that you have some sort of specification or use cases. Peer review works a lot better if you have strong conventions and an agreement on what is OK and what’s not. Really not that easy to get. Unit testing is the most controversial because it affects how the software is designed. Code has to be written in a way that supports testability. It changes the design. It may have performance impacts. It adds complexity to the design and forces to put priorities on the quality attributes that are required.

Design decisions are a sensitive issue, but I think any reasoned decision is better than no decision at all. You can’t have all the quality attributes in your application. You need to prioritize what you need and live with those decisions. Having worked on projects for long periods, I tend to put maintainability and testability really high. Others may prefer flexibility and performance. It’s very hard to obtain a consensus.

Consequences have to be evaluated at the beginning and the quality objectives have to be determined from the start. Decisions have to be made. Impacts have to be know. Recovering from insufficient quality is hard and expensive. Failing can be catastrophic. Developers are accountable for quality, but so is the rest of the organization, including marketing, sales and management.