Working on larger projects come with the annoyances of project management. Every stakeholder sends in someone to check on the project status and make sure it is on track. It’s not really possible to blame them to keep an eye on where their money is going, but it can be very counter productive. Software projects are not like every other project. It’s not a production line and it is very hard to measure. Complexity is hidden, and so can progress be. A well run project will tackle the bigger risks first to avoid uncertainty near the end. This means that early on, progress may just be invisible. Outsiders like to see visible progress, but focusing on visual details just builds pretty prototypes. It looks complete, but it’s not.
Trying to please project managers rather than filling the actual needs is a bad engineering practice, even though it creates successful projects on paper. Just like the vast majority will not remember who finished second, a pile of project reports finished on time and on schedule is a manager’s pride. Reports don’t often state that the definition of complete was distorted. Trying to track the invisible causes dysfunction. At some point, you need to trust the people working on the project.
Estimates are hardly ever accurate on day 0. Until some work is done, it’s just impossible to know how long something is going to take. You can have a wild guess. Having worked on similar problems before helps assessing the risks and adjusting the estimate, but when it’s brand new work integrating with unknown systems, starting to code is the only way to feel the resistance the system will offer. The risks need to be assessed. Unstable or undocumented APIs, incoherent data, undefined business logic and dozens of other factors can affect how long something will take. If you have committed to a tight deadline and the developers know it, knowing precisely how long tasks are going to take won’t save you. Letting them work might.
When the primary risks are tackled and something is in place, management becomes possible. Making a list of changes that deviate from the current state, the baseline, to reach a target is quite simple. With the primary risks out of the way, it should be possible to break down the list of tasks into fairly even sizes and manage by tracking velocity. Managing becomes what it should be: looking at the time available and prioritizing. Trying to manage by tracking velocity during the inception, which ends when the primary risks are tackled is just a waste of time. It leads to panic because velocity is not high enough early on until for a magical reason velocity sharply increases at some point in the project, which is when the actual construction starts. That is, if panic decisions did not screw up with the project.
You can plan all you want, but software has its own agenda. The problem space defines how long inception and elaboration will be, not a schedule on the wall.