L-P Huberdeau's blog

Every improvement process begins with a good idea. When looking at new tools and methodologies, management always has a good objective. Whichever tool or methodology they choose is probably the best one they could have selected in the context. Why is it that down the hierarchy, after the implementation, no one fully agrees with the decision? Of course there is resistance to change, but I think the main issue is that the goals are not communicated well enough. At the bottom, workers are given a set of instructions and procedures to follow, most of which have no meaning to them.

I have been trying to get them involved from the beginning of the process, but it does not seem to give the expected results, at least, not with everyone. From what I observed, there seem to be a clear distinction between those who are willing to improve the quality of their work and those who don’t. When asked for input, some get excited and give ideas you could have never thought of. Others simply see more procedures along the road. Most of the time, those who get excited already have spreadsheets they maintain to attain the objective you are trying to reach. They build themselves tool and are miles ahead of you, but those tools could not scale up beyond their desk and are quite a burden to maintain. Of course, if you propose them a better, more efficient way to do what they already do, they will join the effort. They will not only join it, they will lead it and become champions for the project.

With the other group, you get a flat 0. The difference between both groups is simply the culture to improvement. If given freedom and responsibilities, individuals will take ownership of their work and try to improve their situations. When given orders, procedures and fear, they will become cynical and do their work 9 to 5. They develop a very bad relationship with anything called improvement, even if it’s for their own good.

Getting out of the cynical loop is hard. It requires to work with individuals one by one and can be very time consuming. To be done, trust must be established. Sadly, managers are in a very bad position to do this. By cynics, management is seen as authority, not as someone who can support. I am not management. I am just a software developer. Once in a while, some forget that I work for management and come ask me questions about the purpose of things. I wish I had more time for it, but I always try to get back to the root objective and make sure they are understood. I know I was successful when they propose improvements.

Procedures should never been seen as something static. They should be seen as a baseline to improve from. It should reflect what is done and not the other way around.

I have procedures for myself. No one else uses them, but I still maintain them. They come from hard experience. At some point, I was making large changes to the database structure. Releases were incremental, so I often had to update the production database. I had to figure out a way to make it as painless as possible. The first step was to create a query log containing the changes applied during the development cycle. After a few errors when moving to production, I added a test step where I would import the production database to the development server, apply the patch and perform a diff of the structure before going to production. After a change that corrupted the backup process, I added a backup verification after each database modification.

Why do I maintain a procedure for this? Because a few months after the problems, I would forget some of the steps. In a paragraph like the one above, it’s fairly easy to understand the purpose. When burned in a formal procedure, it would look like this:

Development database update

1. Execute the structure changing query on the database
2. Add the query to the structure_patch.sql file

Database deployment

1. Import the latest nightly backup in the test database
2. Apply the structure_patch.sql file
3. Export the structure of the test database and the development database
4. Compare the exported structures of test and development using a text diff tool
5. Apply the patch on the production server
6. Run the backup script
7. Verify that the backup is completed
8. Overwrite the development database with the verification backup

Cold isn’t it? If I hadn’t explained the purpose of each step, would you be inclined to follow them all? What about the last step? What is it for? It was not described above. What is it for? Well, this one was added because over time, some minor changes appear with no effect on the database but create noise in the text diff, such as index key ordering in the tables.

I think attaching the story behind the steps of a procedures could help their acceptance. If there is no story to it, why is it required anyway? Of course not everyone would read all stories, but when they fall on a step they find annoying, they can find the purpose. Take the final step. Although I wrote it, I always found it annoying because it destroyed my test data. Since I wrote it, I know that it’s not really a required step. The database can go through quite a few cycles before those minor changes become annoying. Knowing the purpose, I can decide to skip it when I feel it necessary.

In a perfect world, procedures should be written by those using them. They should be written by peers for peers. Software developers (I am guilty too) tend to make way too many assumptions about how things are made or should be made. The solution to this probably to make sure at least one person in each workgroup has the capacity to develop custom tools. This does not mean that every group should contain a professional software developer, it simply means someone in the group should know just enough programming to write scripts to automate part of their work in whichever tool they are using, write basic macros in spreadsheets or such. All teams I was with off the chart productivity actually had those capabilities.

If needed, once the team has working prototypes of the tools they need and a rough process defined, a software developer can build a real application out of the requirements developed by the end user. Getting users to define their tools. This really sounds like the objective behind of agile processes.