tl;dr: Scheduling tasks is hard
- We assume everything will go well, but we’re actually crap at estimating
- We confuse progress with effort
- Because we’re crap at estimating, managers are also crap at estimating
- Progress is poorly monitored
- If we fall behind, natural reaction is to add more people
Overly optimistic:
Three stages of creation: Idea, implementation, interaction
Ideas are easy, implementation is harder (and interaction is from the end user). But our ideas are flawed.
We approach a task as a monolithic chunk, but in reality it’s many small pieces.
If we use probability and say that we have a 5% chance of issues, we would budget 5% because it’s one monolithic thing.
But the real situation is that each of the small tasks has a 5% probability of being delayed. Thus, 0.05^n
Oh, and our ideas being flawed? Yeah… virtually certainty that the 5% will be used.
Progress vs effort:
Wherein the man-month fallacy is discussed. It comes down to:
- Adding people works only if they’re completely independent and autonomous. No interaction means assign a smaller portion, which equates to being done faster
- If you can’t split it up tasks, it’s going to take a fixed amount of time. Example in this case is childbearing – you can’t shard the child across multiple mothers. (“Unpartitionable task”)
- Partitionable w/ some communication overhead – pretty much the best you can do in Software. You incur a penalty when adding new people (training time!) and a communication overhead (making sure everyone knows what is happening)
- Partitionable w/ complex interactions – Significantly greater communication overhead
Estimation issues:
Testing is usually ignored/the common victim of schedule slippage. But it’s frequently the time most needed because you’re finding & fixing issues with your ideas.
Recommended time division is 1/3 planning, 1/6 coding, 1/4 unit tests, 1/4 integration tests & live tests (I modified the naming)
Without checkins, if the schedule slips, people only know towards the end of the schedule, when the product is almost due. This is bad because a) people are preparing for the new thing, and b) the business has invested on getting code out that day (purchasing & spinning up new servers, etc)
Better estimates:
When a schedule slips, we can either extend the schedule, or force stuff to be done to the original timeframe (crunch time!) Like an omelette, devs could increase intensity, but that rarely works out well
It’s common to schedule to an end-user’s desired date, rather than going on historical figures.
Managers need to push back against schedules done this way, going instead for at least somewhat data-based hunches instead of wishes
Fixing a schedule:
Going back to progress vs effort.
You have a delayed job. You could add more people, but that rarely turns out well. Overhead of training new people & communication takes it’s toll, and you end up taking more time than if you just stuck with the original team
Recommended thing is to reschedule; and add sufficient time to make sure that testing can be done.
Comes down to the maximum number of people depends on the number of independent subtasks. You will need time, but you can get away with fewer people.
The central idea, that one can’t substitute people for months is pretty true. I’ve read things that say it takes around 3 months to get fully integrated into a job, and I’ve found that to be largely true for me. (It’s one of my goals for future co-op terms to try and get that lowered.)
The concept of partitioning tasks makes sense, and again it comes back to services. If services are small, 1 or 2 person teams could easily take care of things, and with minimal overhead. When teams start spiraling larger, you have to be better at breaking things down into small tasks, so you can assign them easily, and integrate them (hopefully) easily. It seems a bit random, but source control helps a lot here.
Estimation is tricky for me, and will continue to be, since it only comes from experience – various ‘rules of hand’ that I’ve heard include take the time that you’ll think you’ll need, double it, then double it again.
But it’s a knock-on effect – I estimate badly, tell my manager, he has bad data, so he estimates wrongly as well… I’ve heard stories that managers pad estimates. That makes sense, especially for new people. I know estimates of my tasks have been wildly off. Things that I thought would take days end up taking a morning. Other things like changing a URL expose a recently broken dependency, and then you have to fix that entire thing… yeah. 5 minute fix became afternoon+. One thing which I’ll try to start doing is noting down how ling I expect things to take me, and then compare it at the end to see whether or not I was accurate. Right now it’s a very handwavey “Oh I took longer/shorter than I wanted, hopefully I’ll remember that next time!”
Which, sadly, I usually don’t.