Have you ever been in a situation where a new hire in a company is having a walk-through of the projects and the structures, and the mentor can’t really explain how it became such a mess? Mentions technologies that were once promising and revolutionary, only to be left now as shameful legacy?

Only time can test new technologies. What may look promising based on demos and samples may simply not scale to larger applications or cause maintenance burdens on the long term. I grew to be conservative when it comes to technologies. I still use PHP daily after all. I know it has flaws, but I also know it won’t fail me. Starting a new green field project is challenging. There are tons of decisions to be made. Tons of new and exciting toys to play with. However, trying to be too innovative hurts most of the time. Bleeding edge is a very well coined term. New technologies mean new problems to solve, which can be fun early on, but when you need to deliver and you start to hit limitations you were not aware of, waste of time starts eating away the benefits.

Immature products do not come with a huge body of knowledge and clear guidelines. You can use a great technology in a wrong way and create horrors. We have all seen some.

Of course, new technologies need to be adopted. In the long run, stable becomes obsolete. While I believe relational databases are not going away any time soon, those new data stores that used to be in the academic world will eventually mature and become mainstream. It started with all of the start-ups in the world using MongoDB or Cassandra or CouchDB, but this is not mainstream. Early adopters at best. I do try out new frameworks and databases on a regular basis, and enjoy it. However, I keep them outside of the critical path until I am confident enough that I understand the technology well enough.

There are plenty of places to experiment around projects. Perhaps a report needs to be built and SQL is not too suitable for it. A prototype for a new feature can be a good place to experiment as well. If it is to go straight into the main application, I take extra precautions. I prepare a contingency plan. I make sure there are good abstractions in place that allow me to replace it if anything goes wrong. If the technology does not allow me to abstract it away, it’s probably not a design elegant enough for me to use it anyway. I always place maintainability above my desire to try new things, which can be hard.

Experiments are supposed to fail once in a while. If you end up in a situation where everything you try is wonderful and you end up using, there is something wrong with the evaluation process. Even more so if you experiment on the bleeding edge, with technologies out for a couple of weeks. Failures are not a bad thing. Most of the time, new technologies come as a whole package that you are supposed to either take or discard. However, most of the time, they are based on ideas that are simply less common. Ideas that you can take away and use to influence your designs.

Trying to please project managers rather than filling the actual needs is a bad engineering practice, even though it creates successful projects on paper. Just like the vast majority will not remember who finished second, a pile of project reports finished on time and on schedule is a manager’s pride. Reports don’t often state that the definition of complete was distorted. Trying to track the invisible causes dysfunction. At some point, you need to trust the people working on the project.

Estimates are hardly ever accurate on day 0. Until some work is done, it’s just impossible to know how long something is going to take. You can have a wild guess. Having worked on similar problems before helps assessing the risks and adjusting the estimate, but when it’s brand new work integrating with unknown systems, starting to code is the only way to feel the resistance the system will offer. The risks need to be assessed. Unstable or undocumented APIs, incoherent data, undefined business logic and dozens of other factors can affect how long something will take. If you have committed to a tight deadline and the developers know it, knowing precisely how long tasks are going to take won’t save you. Letting them work might.

When the primary risks are tackled and something is in place, management becomes possible. Making a list of changes that deviate from the current state, the baseline, to reach a target is quite simple. With the primary risks out of the way, it should be possible to break down the list of tasks into fairly even sizes and manage by tracking velocity. Managing becomes what it should be: looking at the time available and prioritizing. Trying to manage by tracking velocity  during the inception, which ends when the primary risks are tackled is just a waste of time. It leads to panic because velocity is not high enough early on until for a magical reason velocity sharply increases at some point in the project, which is when the actual construction starts. That is, if panic decisions did not screw up with the project.

You can plan all you want, but software has its own agenda. The problem space defines how long inception and elaboration will be, not a schedule on the wall.

The main issue that that those additional cases cannot be validated right away. Even if you contact people directly, it’s unlikely that they will get a development version to test with. Validation will have to wait until the software is released, and they might not even test as soon as the software comes out. With a 6-month release schedule for major changes, that means that the use case validation will take 6 to 12 months.

When the feedback finally arrives, changes are often needed. It’s not usually very large changes. Small changes to the user interface to include existing capabilities, minor bug fixes or other issues that take less than 2 hours to resolve. Some say that figuring out the problem is half of the job. In this case, finding the issue consumes 99% of the schedule. However, fixing it is not the end of it. For re-validation, a release still needs to happen. It might be in a minor release depending on the moment of the fix, which may be a month away in the best cases. Still, the story is not over as yet more issues may be found.

The reason it takes so long is that development is made for preemptive needs rather than immediate needs. They are nice to have features, but not having them is not a show stopper or they would not be using the software. Alternatively, it may be a show stopper, in which case they are not using the software at all and use something else in the mean time.

This is still in the best of cases, as some people will just try it and declare it broken, stick to their old ways and never signal an issue. In their minds, the feature remains broken forever and they will stay away from it. They might come back much later once the feature has matured. Because they have a work-around, they won’t ever feel the urge to transition, and the longer it takes, the harder it will be as the work-around probably uses some techniques that are not as clean and slightly corrupt the data structure.

Assuming the feature is useful enough for a critical mass to try it and report issues, it can easily take 2 years for a feature to go from functional to mature and broadly usable. It is a long time. This is for a feature that really worked from the start, had known behaviors, documentation, unit testing and all of what you would expect from production-ready code. It still takes years.

The only way to speed-up the process is to find some other users with critical needs that will have a detailed case to resolve. Most of the time, they will not even know they can hook into some existing functionality. Getting a handful of those users who will be brave enough to install a development version and actively test for their use case can cut down the maturation process in half. Every time an issue is resolved (in a good way, not a dirty hack), it unlocks many more use cases and allows for more improvements. That’s when the feature becomes first-class.

Faster iteration is the key. If your organization uses a waterfall process or has a distant QA team that does not work closely to the developers, the same issue is likely to hit you. If you can’t live with the long maturation process the way an open source project can, you need to plan for it and manage it as a risk. Don’t wait until the week before the release to tie-up loose ends. Make sure the code is more than just a proof of concept early.

The open source model is widely different from the typical business plan. There used to be a time when contributors were volunteers, working for passion and love of the project in their free time. These days, I feel most of the contributors to projects make a living out of it. I do it. There is nothing wrong with that. In fact, it’s a good thing. It’s a lot more sustainable. Everyone needs to make a living, so odds are you loose a contributor because the company he works for just bought out (or is on the verge to be) and needs to work 70 hour weeks and burns out are much lower. Contribution can become a top priority for a reasonable amount of work hours per week. The Linux kernel was probably one of the first sample where all the top contributors ended up being paid by companies to do it.

As an individual, I find it makes a lot of sense to focus on open source software. There is nothing that I hate doing more than writing the same code twice. Building from an open platform, and contributing back, allows me to avoid writing the same thing twice and prevents me from ever writing some code. I’m mostly a developer. I don’t do much of the applying for contracts and filling specific needs. Tried it before. Not a happy place for me, even if basing it on open solutions. I’d rather leave the pressure to deliver on someone else. Turns out many companies out there see open source as a business opportunity. They can build on an existing product to deliver more value fast. I get hired to take them further while they handle the day to day problems. It’s a good deal for both sides.

There is however a major difference between a company serving real tangible clients and the open source world. After working for several clients, I can see a major difference between the successful ones and those that barely stay above waters. It turns out it probably differentiates successful from unsuccessful in any field. It’s called vision. The good companies see ahead. They anticipate problems and make sure they are resolved before the client meets them, at least the fundamental ones. Unsuccessful companies just keep fighting fires and push down the problems and hope for someone to resolve them right away.

A while back, I read one of Joel’s articles on micromanagement. Apart from the conference organizer’s inside jokes about terrible WIFI access in conference centers (and the great advice to make sure they give it for free if it does not handle the load — which is always), the following passage made me smile:

At the top of every company, there’s at least one person who really cares and really wants the product and the customer experience to be great. That’s you, and me, and Ryan. Below that person, there are layers of people, many of whom are equally dedicated and equally talented.

But at some point as you work your way through an organization, you find pockets of people who don’t care that much.

Having spent most of my time as either an employee (or technically an intern, as I never really had a full time job otherwise), I spent most of my time as a consultant working in fairly small of organizations and have been kept closer to developers than the management-types. I can say without an hesitation that the people down the ladder mostly blame high level management for getting in their way and preventing them from doing their job. Both are probably right. However, I still find Joel’s wording a bit harsh.

To function correctly with open source and make the relationship efficient, you need to embrace it. Companies trying to make it a one way relationship ended up failing. In open source, the project extends beyond the company. Every line of code you don’t contribute back is one line you will have to maintain yourself. At some point, it will simply break. Upgrading to obtain the later versions will become harder. At which point, you better hope you have killer traceability to go back to the original issue, because you will have to implement it over again. With a high turnover, it might just kill the company, and the community (and consultants part of it) might not be motivated in helping you out if you did not contribute back when times were good.

Before leaving for Boston to tackle the issue, I had gone through the code and cleaned up parts of it, making conditions more explicit and code slightly more expressive. The objective was not as much to improve the code as to understand the raw design underneath it. I think all software as a design, only too often, it is unintentional and hidden. In those cases, refactoring is initially about making the current design explicit. Once that is done, it can be refactored further and improved to match new requirements. When initially setting the goals for the week, saying we’re not going to fix any issues, add new features or otherwise solve everyone’s own favorite issue is a tough sale.

A successful refactoring sprint is about discipline. The group must stay away from distractions and concentrate on the task. Our task was to extract the field rendering and input logic into cohesive units. The initial input was composed of a few files between 1KLOC and 6KLOC, containing around 40 field types being handled, all mashed together. Some lines were between 500 and 700 characters long. Some parts were duplicated in multiple locations, with the mandatory differences that make them hard to reconcile. Removing that duplication was one of the primary objective. It’s challenging. I don’t think anyone thought it was possible when we began, but it had to be done.

Figuring out where to begin is not easy. Initially, you can’t even get everyone working. Even if the problem had a natural separation with the field types, the code would fight back when too many people worked on it. At first, an initial interface had to be defined as the target to reach. Then it had to be plugged. Essentially, it comes down to if we have a handler to do it, use it, otherwise, revert back to whatever was there before. Those kind of hooks had to be deployed in many places, but we began with one. Working on a few handlers to see how it worked out, learning about the design.

As more handlers got to be created, more hooks were added in other places, leading to revisiting the previous ones. It’s a highly iterative process. I made the first iteration alone and others were introduced gradually. Everyone’s first handler took a whole day. It was much more than my most pessimistic estimations. There was a lot to learn. However, the pace then accelerated. As each of us understood the design of the code and the patterns to be found, the pace accelerated. We could see those huge files melting. Each step of the way, it became easier. Anyway, that was the feeling.

Then someone asked how far were we. I pulled out a white board and made a list of the field types that were still to be done. The initial list came as a disappointment. The list was still long. We were only half way and way past the week’s mid-mark. However, past the initial disappointment, having the list visible ended up being a motivator, because each one that was completed made the list shorter. It encouraged to fully complete the handlers rather than leaving dangling issues.

We ended up completing on the last evening. This was a one week burn. The last few hours were hard for everyone. After spending a week working long hours on challenging code, I don’t think we could have accomplished more than we did. However, there was great satisfaction. The refactoring process is not completed. One of the issues was tackled, but there are other areas of the code that need to be worked on. However, the bulk of the job was done as a team effort, and now there are stronger grounds to build from. No one could have done it alone.

It should be noted that the week was not only hard work. It was also a social event where non-coding contributors of the community and users were welcome to stop by and chat. There were late night discussions around beers, leading to even less sleep, and the whole week was a great team building experience. While we were shuffling thousands of lines of code around, the documentation team also re-organized the structure of the documentation.

• Architecture is more about form than structure. Implementation details are not in the scope. Interfaces, classes and design paradigms are not even considered. Only the general form of the final solution is.
• Architecture is more about compression than abstraction. Abstractions are in code. They are detailed. The architecture part of the work is about larger scale partitioning into well named patterns, which may have multiple implementations.
• Much architecture is not about solving user problems. While I don’t fully agree with this one, it’s true that most users will not see the changes right away.

These are high level concepts that have a huge impact on the code. The partitioning that results determines how additions are made to the code and how they will be. There is a direct relationship to the software design and the API available to implementers.

When I look at code in software designed using good techniques, there is typically a clear distinction between some core managing the general process and the extensions following interfaces that are called at the appropriate time. When you look at code inside the core, it really does not seem to do much. There are usually a few strange incantations to call the extension points efficiently and massage the information sent through. The code is not really pretty, but the structure it represents is clean. The glue has to go somewhere.

The leafs, or extension points, are typically a complete jungle. Contortions must be made to fit the interface as mapping is done to an external system. Some pieces were written quickly to serve a pressing matter, fell into a technical debt backlog and eventually out of sight. Code is duplicated around, taken from the older generations to the new ones, evolving over time, except that the ancestors stay around and never get the improvements from the new generations. Quality varies widely as does the implementer’s abilities and experience, but all of the components are isolated and do not cause trouble… most of the time.

Seeing how code rots in controlled environments, I’m always a bit scared when I see a developer searching the web and grabbing random plugins for an open source platform and including them in the code. Disregarding the license issues that are almost never studied, that practice is plain dangerous. There are security implications. Most developers publishing plugins are not malicious, they are simply ignorant of the flaws they introduce.

jQuery is probably the flagship in the category of quality core containing arcade incantations and the jungle of plugins. Surely, having 50,000 plugins may seem like a selling point, but when you consider most of them are lesser duplicates of other ones. Code quality is appallingly low. In most cases, it takes less than 30 seconds to realize they were written by people (self-proclaimed experts) who knew nothing of jQuery, just enough JavaScript to smash together a piece of functionality and branded it as a jQuery plugin for popularity’s sake while following a tutorial. Never use a plugin without auditing the code first.

Even if good care is taken to control the leafs, ugly code will appear all over. There are no other solutions than to go back and add the missing abstractions. Provide the additional tools needed to handle the frequent problems that were duplicated all around. No amount of planning will predict the detailed needs of those extension points. What allows architecture to work is compression, to be able to skip details so the system can be understood as a whole. The job is not done when the core is in place. Some time must be allocated to watching the emergence of patterns and to respond to them, either by modifying the core or providing use-at-will components. It can be made in multiple steps too.

Recently, I was asked to do a lot of high level refactoring in Tiki. Major components had systemic flaws known for a while and many determined they had to be attacked after the release of the long term support version. High level work has several impacts, but sometimes, just providing low level tools can improve the platform significantly. Cleaner code will make the high level changes easier to perform. It only takes a few hours to run through several thousands lines of code and identify commonalities that could be extracted. Extract them, deploy it around. Iterate. Automated tests to support those changes would be nice, but most of the time, those changes are so low level, it’s almost impossible to get wrong.

The documentation also contains hints like this one:

[3] One might wonder why pop() returns void, instead of value_type. That is, why must one use front() and pop() to examine and remove the element at the front of the queue, instead of combining the two in a single member function? In fact, there is a good reason for this design. If pop() returned the front element, it would have to return by value rather than by reference: return by reference would create a dangling pointer. Return by value, however, is inefficient: it involves at least one redundant copy constructor call. Since it is impossible for pop() to return a value in such a way as to be both efficient and correct, it is more sensible for it to return no value at all and to require clients to use front() to inspect the value at the front of the queue.

Not all libraries in the world are so careful. Take this snippet from the Zend Search Lucene documentation:

Java Lucene uses the ‘contents’ field as a default field to search. Zend_Search_Lucene searches through all fields by default, but the behavior is configurable. See the “Default search field” chapter for details.

When I came across this reading the documentation to build a fairly complex index, I thought it was a reasonable default to search all fields. It’s very convenient. I could store my content in the field they belong, make sure they are searchable by default and still allow for finer-grained search when required. Fantastic. I went ahead with it, write the code to collect the information and index it properly and the query abstraction at the same time in a good old TDD fashion. Everything worked. Of course, I was testing on very small data sets.

I then went ahead to test with larger data sets. I had an old back-up from a site installed with around 2000 documents in it. It felt like a decent test. I expected the indexing to be around half an hour for that type of data based on what I had read online. The search component had not been selected for speed, it had been selected for portability. Speed was one of those sacrificed attributes as long as it was not too terrible. Of course, the initial indexing took longer than expected, but only by a factor of 2, and I knew some places were not fully optimized yet (it’s now down to 20-25 minutes).

The really big surprise came as I attempted to make a simple search in the index. It timed out. After 60 seconds. Initial attempts at profiling failed as it was getting late in the afternoon on a Friday night. I closed up shop and had a bit of trouble getting it out of my head that night.

When I got back to it, I took out the time limit, started a profiling session on it and enjoyed my coffee for a little while. The results indicated that the search spent pretty much all of it’s time optimizing the search query. It was making tens of thousands of of calls to some functions eventually making reads on disk. There was not much more reporting in there to help me. I started adding some var_dumps in the code to see what was going on. Well, it turns out that “search all fields by default” was not such a great idea. It actually made it search through all the fields and basically expand the query. Because of how I interpreted the API and documentation, I had built my index to be quite expanded and it contained a few dozens of fields, not all of which existed for all documents. It was a mistake. There was one valid reason why the Java implementation did not behave that way: it was not possible to do it efficiently.

I ended up modifying the index generation to put all content in a contents field, duplicating the content you would actually want to search independently in their own fields and search contents by default. Indexation time wasn’t altered by much, the code changes were actually very minor and easy due to the array of tests available and search speed went up dramatically. It’s not as fast as sphinx for sure, but it does offer decent performance and can run on pretty much any kind of cheap hosting, which is a good feature for an open source CMS. It still needs to be investigated, but it’s also likely to be a smooth upgrade path to using Solr for larger installations. Abstractions around the indexing and searching will also allow to quickly move to other index engines as needed.

Asymptotic analysis is one really boring part of the computer science curriculum, but it’s really something to consider when building libraries that are going to be used with large numbers of documents. The API needs to reflect the limitations and the documentation must explain them clearly.

Knowing the ins and outs of a project has some advantages. Changes can be made really quickly because you know exactly where to look at. When you see a piece of code, you can know from the style who wrote it and ask the right questions. You know the history of how it came to be and why it’s like that. Sadly, you also know why it won’t change anytime soon. Or can it? Given some amount of effort, anything about software can change. We’re only too busy with our daily work to realize it.

The same effect applies at different levels. People arriving in a new project have the gift of ignorance. They don’t see the reasons. They see what is wrong to them and how it should be for them. Of course, this often comes out in a harsh way for those who spent a lot of effort making it as good as it is, with all the flaws it has. Anyone who has been working on a project for a significant amount of time and demonstrates it knows that reality can be hard. Still, for all the good advice newcomers can have, what they don’t have is the trust of the community and the knowledge required to know all the repercussions a change can have. Still, they do tend to see major issues that no one had intentions to solve.

At a much smaller level, it’s not much different than closing a coding session at the end of a day in the middle of trying to resolve an impossible bug. The next morning, the issue just magically appear. This can be associated to being well rested and sharper, but it might also just be about letting the time go. Deciding to stop searching is a tough call. You’re always 15 minutes away from finding the solution when debugging, until you realize the hypothesis was wrong. It takes a fair amount of self-confidence to know you can take the night off and that you will still be able to fix the issue in time for a deadline. Sure, getting back into it will take some time. It requires to rebuild the mental model of the problem, but rebuilding might just be what’s needed when you are on the wrong track.

This is nothing new. It has been written many times that code review are more effective when done a few days later or by someone else entirely. In this case, Wiegers indicates that when the work is too fresh in your memory, your recite it based on what it should be rather than what it is, hence you don’t really review it. What if when you search for a problem, whether a localized bug or a systemic issue that requires major work, unless you had taken enough distance to clear your assumptions, there is no way you could find it outside of blind luck?

Make sure you can work without them

I actually got to this one by accident, but it happened to be a good one. When I joined the team, there were plenty of services available. However, the entry point I needed to get to be able to use any of them was not present at all. I had a vague idea of what the data would look like, so I started building static structures that would be sent off to the views to render the pages. I had a bit of heads up before the integrators joined the team and certainly did not want them to wait around for weeks until the service was delivered.

At some point, the said service actually entered in someone’s iteration, which means it would be delivered in a near future. Fairly quickly, a contract was made for the exact data format that would be provided. Although I was not wrong on the values that would be provided, the format was entirely different. My initial format then became an intermediate format for the view, providing the strict minimum required, and a layer was added in the system to translate the service’s format to our own. The service was not yet available, so the service was just stubbed out. The conversion could be unit tested based on the static data in the service stub. Plugging the service when it arrived was a charm and except for a few environment configurations, it was transparent to integrators.

During the entire development, this fake data ended up being very useful.

• Whenever the services would change, there was an easy way to compare the expectations to what we actually received, allowing to update the stub data and the intermediate layer.
• When something would go terribly wrong and services would just fail, it was always possible to revert to the fake data and keep working.
• It allowed to reach code paths that would not normally be reached, like error handling.

Expect them to fail

One of the holy grails with SOA is that you can replace services on the fly and adjust capacity when needed. This may be partially true, but it also means that while your component works fine, the neighbor may be completely unreachable during a maintenance or while restarting. If you happen to need it, you might as well be down in many cases. While one would hope services won’t crash in production, they happen to crash in development fairly often. To live with this, there is one simple rule: expect every single call to fail, and make a conscious decision about what to do about it. For one, it will make your system fault tolerant. If the only call that failed is fetching some information that is only used being rendered, it’s very likely that you don’t need to die with a 500 internal error. However, if you didn’t expect and handle the failure, that’s what will happen.

Adding this level of error handling does add a significant cost to the development. It’s a lot of code and a lot of reflexions needed. Live with it or reconsider your SOA strategy.

Early on, adding the try/catch blocks wasn’t much of a reflex. After all, you can write code that works without it and PHP sure won’t indicate you that you forgot one. When the first crashes occurred in development, we still had few services integrated. Service interruptions just became worst as we integrated more. What really pushed towards adding more granularity in catching exceptions is not really the non-gracefulness in which they would break the system, it’s the waste of time. With a few team members pushing features in a system, a 15 minutes interruption may not seem like much, but it’s enough to break the flow, which is hard enough to get into in an office environment. Especially when the service that breaks has nothing to do with the task you’re on at the moment.

It does not take much.

• When fetching information, have a fall-back for missing information.
• When processing an information, make sure you give a proper notification when something goes wrong.
• Log all failures and have at least a subtle indication for the end-user that something went wrong and that logs should be verified before bothering someone else.

Build for testability

Services live independently in their own memory space with their own states. They just won’t reset when your new unit test begins, making them a pain to test. However, that is far from being an excuse for not automating at least some tests. Every shortcut made will come back to hunt you, or someone else on the team. It’s very likely that you won’t be able to get the remote system in every possible state to test all combination. Even if you could, the suite would most likely take very long to execute, leading to poor feedback. Mocks and stubs can get you a long way just making sure your code makes the correct calls in sequence (when it matters), passing the right values and stopping correctly when an error occurs. That alone should give some confidence.

To be able to check all calls made, we ended up with an interface defining all possible remote calls providing the exact same parameters and return values as the remote systems. There was a lot of refactoring to get to the solution. Essentially, every single time an attempt was made to regroup some calls because they were called at the same time and shared parameters, or because it was too much data to stub out for just those 2 tiny values, it had to be redone. Some error would happen with the real services because the very few lines of code that were not tested with the real service happened to contain errors, or something would come up and suddenly, those calls were no longer regrouped.

As far as calling the real services go, smoke testing is about the only thing I could really do. Making a basic call and checking if the output seems to be in the appropriate format. In the best of worlds, the service implementers would also provide a stub in which the internal state can be modified, and maintain the stub to reflect the contract made by the real service. It could have solved some issues with the fact that some services are simply impossible to run in a development environment. Sticking to the contract is the only thing that can really be done in an automated fashion for development. I first encountered that type of environment a few years back where running a test actually implied walking to a room, possibly climbing a ladder, switching wires and getting back to the workstation to check.

Have an independent QA team

It might not be miserably, but chances you fail are fairly high when a lot of components need to talk to each other and there is no way you can replicate all of it at once. A good QA team testing in an environment that maps to the production environment will find the most hallucinating issues. In most cases, they are caused by a mismatch between the understanding of the interface between the implementor and the client. When you have a clear log pointing out the exact source of the problem and all your expectations documented in tests and stubs, it does not take a very long discussion to find the source of the issue. Fixing it just becomes adjusting the stubs, and fixing the broken tests.

If you’re lucky enough, there might not be issues left when it goes to production. Seriously, don’t over-do SOA. It’s not as magic as the vendors or “enterprise architects” say it is.

In my last post, I mentioned a lot of wiki-related events happening over the summer and that I would attend them all. It turns out it was an exhausting stretch. Too many interesting people to meet, not enough time — even in days that never seem to end in Poland. As always, I was in a constant dilemma between attending sessions, the open space or just creating spontaneous hallway discussions. There was plenty of space for discussion. The city of Gdansk being not so large, at least not the touristic area in which everyone stayed, entering just about any bar or restaurant, at any time of the day, would lead to sitting with an other group of conference attendees. WikiMania did not end before the plane landed in Munich, which apparently was the connection city everyone used, at which point I had to run to catch my slightly tight connection to Barcelona.

I know, there are worst ways to spend par of the summer than having to go work in Barcelona.

I came to a few conclusions during WikiSym/WikiMania:

• Sociotechnical is the chosen word by academics to discuss what the rest of us call the social web or web 2.0.
• Adding a graph does not make a presentation look any more researched. It most likely exposes the flaws.
• Wikipedia is much larger than I knew, and they still have a lot of ambitions.
• Some people behind the scenes really enjoy office politics, which most likely creates a barrier with the rest of us.
• One would think open source and academic research have close objectives, but collaboration remains hard.
• The analysis performed leads to fascinating results.
• The community is very diverse, and Truth in Numbers is a very good demonstration of it for those who could not be there.

As I came back home, I had a few days to wrap up projects before getting to work for a new client. All of which had to happen while fighting jet lag. I still did not get time to catch-up with the people I met, but I still plan on it.

One of the very nice surprises I had a few days ago is the recent formation of Montréal Ouvert (the site is also partially available in English), which held it’s first meeting last week. The meeting appeared like a success to me. I’m very bad at counting crowds, but it seemed to be somewhere between 40 and 50 people attending. Participants were from various professions and included some city representatives, which is very promising. However, the next steps are still a little fuzzy and how one may get involved is unclear. The organizers seemed to have matters well in hand. There will likely be some sort of hack fest in the coming weeks or months to build prototypes and show the case for open data. I don’t know how related this was to Make Web Not War a few months prior. It may just be one of those idea whose time has come.

I also got to spend a little time in Ottawa to meet with the BigBlueButton team and discuss further integration with Tiki. At this time, the integration is minimal because very few features are fully exposed. Discussions were fruitful and a lot more should be possible with the now in development version 0.8. Discussing the various use cases indicated that we did not approach the integration using the same metaphor, partially because it is not quite explicit in the API. The integration in Tiki is based on the concept of rooms as a permanent entity that you can reserve through alternate mechanisms, which maps quite closely to how meeting rooms work in physical spaces. The intended integration was mostly built around the concept of meetings happening at a specific moment in time. Detailed documentation cannot always explain the larger picture.