L-P Huberdeau's blog

The idea first came to me over the summer while I was in Strasbourg discussing the future changes to TikiWiki. It all started because we decided to release more often. A lot more often. As it was not something done frequently before, the tools to do it were hard and tedious to use. Some manual steps are long and annoying. Packaging the releases was one of them. All those scripts were rewritten to be easier to use and to regroup multiple others. That part is good now. One of the other issues was the changelog. TikiWiki is a very active project. Having over 1000 commits per month is not uncommon. Many of them are really small, but it’s still 1000 entries in the changelog. Ideally, we’d like the changelog to be meaningful to people updating, but we just had no way to go through all the changelog before the release.

One solution that came up was to use a wiki page to hold the changelog, get new items to be appended, and then people could go though it and translate the commit messages from developer English to user English, filter the irrelevant ones and come up with something useful to someone. Well, we had no direct way to append the commit messages to the end of a page, so this was not done over the 6 month period and we’re now getting dangerously close to the next release. Looks like the next changelog will be in developer English.

Now, what does this have to do with code review? Not so much. I was reading “Best Kept Secrets of Peer Code Review (which they mail you for free because it’s publicity for their software, but is still worth reading because it still contains valid information) and figured TikiWiki could be a good platform to do code reviews on if only we could link it with Subversion in some way. After all, TikiWiki is already a good platform to collaborate over software development as it was developed for it’s own internal purposes for so long. When you dogfood a CMS for a long time, it tends to become good for software development (also tends to have complex UIs intuitive to developers though). It contains all you need to write quality documentation, track issues, and much more by just enabling features and setting it up.

Moreover, code review is done by the community over the Subversion mailing list. The only issue is that we don’t really know what was reviewed and what was not. I personally think a mail client is far from being the best environment to review changes in. Too often, I’d like to see just a couple lines more above to fully check the change or verify something in an other related file. The alternative at this time is to use subversion commands afterwards and open up files in VI. I wish it required less steps.

Wiki pages are great, because they allow you do do what you need and solve unanticipated issues on the spot. Specialized tools on the other hand are great at doing what they were meant to do, but they are often draconian in their ways, and forcing someone to do something they don’t want to never really works. I’ve made those errors in the past designing applications, and I felt Code Collaborator made the same ones. The book mentioned above contains a chapter on a very large case study where no code could be committed unless reviewed first. Result: few pages were spend explaining how they had to filter data for those cases the code was not really reviewed and people only validated it within seconds. I’d rather know it was not reviewed than get false positives.

Anyway, I started thinking of the many ways TikiWiki could allow to perform code reviews. The most simple layout I could think of was this one:

• One wiki page per commit, grouping all the changes together. Reviewers can then simple edit the page to add their comments (or use the page comments feature, but I think adding to the page is more flexible) and add links to other relevant information to help others in the review.
• A custom plugin to create links to other files at a given revision, just to make it easier to type. This could actually be a plugin alias to something else. No need for additional code.
• A custom plugin to display a diff, allow to display full diff instead and link to full file versions for left and right.
• An on-commit plugin for Suversion to make the link.

With commits linked to other related commits, to related documentation and discussions, other side features like wiki mind map and semantic links will surely prove to be insightful.

Then I went a little further and figured trackers could be used to log issues and perform stats on in the future. Checklists could be maintained in the wiki as well and displayed as modules on the side, always visible during the review. If issues are also maintained in a tracker, they could be closed as part of the commit process by analyzing the commit message. However, this is mostly an extra as I feel there is enough value in just having the review information publicly available. The great part of using a vast system is that all the features are already there. The solution can be adapted and improved as required without requiring complete new developments.

Now, the only real show stopper in this was that there is no direct way of creating all this from a subversion plugin. TikiWiki does not have a webservice-accessible API to create these things and is unlikely to have one any time soon. The script could load the internal libraries and call them if they were on the same server, but that’s unlikely to be the case. A custom script could be written to receive the call, but then it would not be generic, so hard to include in the core. As we don’t like to maintain things off the core (part of the fundamental philosophy making the project what it is), it’s not a good solution. With this and the changelog idea before, I felt there was still a need for something like this. I’m a software engineer, so I think with the tools I use, but I’m certain there are other cases out there that could use a back door to create various things.

To keep the story short, I started finding too many resemblances to the profiles feature. Basically, profiles YAML files containing descriptions of items to create. They are to be used from the administration panel to allow to install configurations on remote repositories, to configure the application faster based on common usage profiles. Most of the time, they are only used to create trackers, set preferences and the such. However, they contain a few handles to create pages, tracker items and some other elements to create sample data and instruction pages. If they could be ran multiple times, be called by non-administrator users, contain a few more options to handle data a little better (like being able to append to pages), they could pretty much do anything that’s required in here, and a lot more.

An other problem was that profile repositories are also TikiWiki instances. They require quite a few special configurations, like opening up some export features, using categories and such. I wouldn’t want all this stuff just to receive a commit notification, and I wouldn’t want to execute a remote configuration without supervision from the administrator. More changes were required to better handle access to local pages.

Still, those were still minor changes. A few hours later, Data Channels were born. What are these? It’s so simple it’s almost stupid. A name, a profile URI and list of user groups who can execute it from an HTTP call.

Next steps toward using TikiWiki as a code review tool:

• Discuss this with the community
• Write a profile to create the base setup and the base data channel profiles
• Write a subversion plugin to call the channels
• Write the plugins to display code differences

Interested in something like this for your project or company? Just drop me a line.

Seems like this is the time for retrospectives.

• In January, I was coming back to my normal life after my last internship at Autodesk, moderately busy organizing CUSEC and starting what could have been my last semester of school if only the last courses I have were not precisely in the same time slot. To keep things light, there was also a CodeFest along the way.
• February turned out to be a very important. It’s the month in which I decided to stop thinking about multilingual wikis and just do it. Proof of concept ready within a day. Few more days to bring it feature complete, and then spent the remaining weeks documenting it and explaining how it works. Most still don’t get it, but I live OK with that.
• At the begining of March was the Blitzweekend in which we made a first beta release of TikiWiki 1.10. I then spend time preparing my presentation (with CS Games in between) for PHP Quebec that month, which turned out to be my best presentation ever. I spent most of the second half of the month writing reports for school, which was well worth it because I won a prize for it multiple months later. It might have been time better spent if I had actually worked for clients during that time, but it wouldn’t have been as fun. Closing out on 4 year old ideas feels great.
• Although I was mostly done writing for school, it was far from over. In April, I wrote the paper that would be presented in Porto with Alain Désilets and Sébastien Paquet. These academic conferences get you to work before you are even accepted. Tech conferences are so much easier to handle. By the end, I figured my writing skills were good enough. It was also the month where I went on vaccation for the first time in a very long while.
• Coming back from Cuba early May, I spent a few days not eating or doing anything because it required too much effort based on my new standards. Luckily, starvation got me to start doing stuff again. Other than an other CodeFest, May was mostly quiet.
• In June, I mostly worked on Profiles, a configuration delivery systems for TikiWiki. Other than that, usual work.
• I went to Strasbourg in July for TikiFest during which 2.0 saw it’s first RC and major changes in the community. It was my first trip in Europe, so I spent some time visiting Strasbourg and Paris. While I was there, I had the confirmation that my paper was accepted, so I had to handle the reviews and submit the final version while I was away from home. I was really glad I had co-authors to handle most of it.
• August was quiet, other than the fact that I officially completed my studies. We held a small TikiFest in Montreal. I had a lot to cover in TikiWiki after the meetings in France, so I spent quite a lot of time finalizing the 2.0 release (which was made that month).
• I then went to Porto in September for WikiSym and a TikiFest, living in the WikiHouse for nearly two weeks with 9 other people. It was a lot of fun. Porto was a great city. It turns out the part where I presented my paper was only a minor aspect of the trip. Surprising. While I was there, I learned I was accepted for php|works in November. It seemed like every time I traveled, I would need to travel more.
• In October, I really had to catch up on work, so other than a CodeFest and preparing my presentation, I just worked.
• November was a rush. Right after php|works in Atlanta, I had to be back in Montreal because the largest TikiFest ever was happening at the same time with people from all over the world.
• Nothing happened in December, except a huge pile of work.

It was a lot of fun.