March 6, 2017
Start With A Goal. The Backlog Will Follow.The little pairing project I'm doing with my 'apprentice' Will at the moment started with a useful reminder of just how powerful it can be to start development with goals, instead of asking for a list of features.
As usual, it was going to be some kind of community video library (it's always a community video library, when will you learn!!!), and - with my customer role-playing hat on - I envisaged the usual video library-ish features: borrowing videos, returning videos, donating videos, and so on.
But, at this point in my mentoring, I'm keen for Will to get some experience working in a wider perspective, so I insisted we started with a business goal.
I stipulated that the aim of the video library was to enable cash-strapped movie lovers to watch a different movie every day for a total cost of less than £100 a year. We fired up Excel and ran some numbers, and figured out that - in a group with similar tastes (e.g,, sci-fi, romantic comedies, etc) - you might need only 40 people to club together to achieve this.
This reframed the whole exercise. A movie club with 40 members could run their library out of a garden shed, using pencil and paper to keep basic records of who has what on loan. They could run an online poll to decide what titles to buy each month. They didn't really need software tools for managing their library.
The hard part, it seemed to us, would be finding people in your local area with similar tastes in movies. So the focus of our project shifted from managing a collection of DVDs to connecting with other movie lovers to form local clubs.
Out of that goal, a small feature list almost wrote itself. This is how planning should work; not with backlogs of feature requests, but with customers and developers closely collaborating to achieve goals.
It's similar in many ways to how TDD should work - in fact, arguably, it is TDD (except we start with business tests). When I'm showing teams how to do TDD, I advise them not to think of a design and then start writing unit tests for all the classes and methods and getters and setters they think they're gloing to need. Start with a customer test, and drive your internal design directly from that. Classes and methods and getters and setters will naturally emerge as and when they're needed.
When I run the Codemanship Agile Software Development workshop, we do it backwards to illustrate this point. Teams are tasked with coming up with internal designs, and then I ask them to write some customer tests afterwards. Inevitably, they realise their internal design isn't what they really needed. Stepping further back, I ask them to describe the business goals of their software, and at least half the teams realise they're building the wrong thing.
So, my advice... Ditch the backlog and start with a goal. The rest will follow.
December 28, 2016
The Best Software Developers Know It's Not All 'Hacking'A social media debate that appears to have been raging over the Xmas break was triggered by some tech CEO claiming that the "best developers" would be hacking over the holidays.
Putting aside just how laden with cultural assumptions that tweet was (and, to be fair, many of the angry responses to it), there is a wider question of what makes a software developer more effective.
Consider the same tweet but in a different context: the "best screenwriters" will spend their holiday "hacking" screenplays. There's an assumption there that writing is all there is to it. Look at what happens, though, when screenwriters become very successful. Their life experiences change. They tend to socialise with other movie people. They move out of their poky little downtown apartments and move into more luxurious surroundings. They exchange their beat-up old Nissan for a brand new Mercedes. They become totally immersed in the world of the movie business. And then we wonder why there are so many movies about screenwriters...
The best screenwriters start with great stories, and tell their stories in interesting and authentic voices. To write a compelling movie about firefighters, they need to spend a lot of time with firefighters, listening to their stories and internalising their way of telling them.
First and foremost, great software developers solve real problems. They spend time with people, listen to their stories, and create working solutions to their problems told in the end user's authentic voice.
What happens when developers withdraw from the outside world, and devote all of their time to "hacking" solutions, is the equivalent of the slew of unoriginal and unimaginative blockbuster special effects movies coming out of Hollywood in recent years. They're not really about anything except making money. They're cinema for cinema's sake. And rehashes of movies we've already seen, in one form or another. because the people who make them are immersed in their own world: movie making.
Ironically, if a screenwriter actually switched off their laptop and devoted Xmas to spending time with the folks, helping out with Xmas dinner, taking Grandma for a walk in her wheelchair, etc, they would probably get more good material out of not writing for a few days.
Being immersed in the real world of people and their problems is essential for a software developer. It's where the ideas come from, and if there's one thing that's in short supply in our industry, it's good ideas.
I've worked with VCs and sat through pitches, and they all have a Decline Of The Hollywood Machine feel to them. "It's like Uber meets Moonpig" is our equivalent of the kind of "It's like Iron-Man meets When Harry Met Sally" pitches studio executives have to endure.
As a coach, when I meet organisations and see how they create software, as wel as the usual technical advice about shortening feedback cycles and automating builds and deployment etc, increasingly I find myself urging teams to spend much more time with end users, seeing how the business works, listening to their stories, and internalising their voices.
As a result, many teams realise they've been solving the wrong problems and ultimately building the wrong software. The effect can be profound.
I'll end with a quick illustration: my apprentice and I are working on a simple exercise to build a community video library. As soon as we saw those words "video library", we immediately envisaged features for borrowing and returning videos, and that sort of library-esque thing.
But hang on a moment! When I articulated the goal of the video library - to allow people to watch a movie a day for just 25p per movie - and we broke that down into a business model where 40+ people in the same local area who like the same genres of movies club together - it became apparent that what was really needed was a way for these people to find each other and form clubs.
Our business model (25p to watch one movie) precluded the possibility of using the mail to send and return DVDs. So these people would have to be local to wherever the movies were kept. That could just be a shed in someone's garden. No real need for a sophistated computer system to make titles. They would just need some shelves, and maybe a book to log loans and returns.
So the features we finally came up with had nothing to do with lending and returning DVDs, but was instead about forming local movie clubs.
I doubt we'd have come to that realisation without first articulating the problem. It's not rocket science. But still, too few teams approach development in this way.
So, instead of the "It's like GitHub, but for cats" ideas that start from a solution, take some time out to live in the real world, and keep your eyes and ears open for when someone says "You know what's really annoys me?", because an idea for the Next Big Thing might follow.
December 8, 2016
What Do I Think of "Scaled Agile"?People are increasingly asking me for my thoughts on "scaled agile", so I though I'd take a quiet moment to collect my thoughts in one place.
Ever since that fateful meeting in Snowbird, Utah in 2001, some commercially-minded folk have sought to "scale up" the Agile brand so it can be applied to large organisations.
I'll give you an example of the kind of organisation we're talking about: a couple of years ago I was invited into a business that had about 150 teams of developers all effectively working on the same system (or different versions of the same system). I was asked to put together a report and some recommendations on how TDD could be adopted across the organisation.
The business in question was peppered throughout with Agile consultants, Scrum Masters, Lean experts, Kanban experts, and all manner of Agile flora and fauna.
Teams all used user stories, all had Scrum or Kanban boards, all did daily stand-ups, and all the paraphernalia we associate with Agile Software Development.
But if there was one thing they most definitely were not, it was agile. Change was slow and expensive. There was absolutely no sense of overall direction or control, or of an overall picture of progress. And the layers of "scaled Agile" the managers had piled on top of all that mess was just making things worse.
It's just a fact of life. Software development doesn't scale. Once software projects go above a certain size (~$1 million), chaos is inevitable, and the best you can hope for is an illusion of control.
And that, in my considerably wide experience of organisations of all sizes attempting to apply agile principles and practices, is all that the "scaled agile" methods can offer.
I've seen some quite well-known case studies of organisations that claim to be doing agile at scale with my own eyes, and they just aren't. 150 teams doing scaled agile, it turns out, is just 150 teams doing their own thing, and on a surface level making it look like they're all following a common process. But they're still dogged by all the same problems that any organisation trying to do software development at scale is dogged by. You can't fix nature.
Instead, you have to acknowledge the true nature of development at scale; that these are highly complex systems, not conducive to overall top-down control, from which outcomes organically emerge, planned or not.
Insect colonies do not follow top-down processes. A beehive isn't command and control, even if it might look to the casual observer that there's a co-ordinated plan they're all following. What's actually happening is that individual bees are responding to broadcast messages ("goals") about where the pollen, or the threat, can be found, and then they respond according to a set of simple internalised rules to that message, co-operating at a local level so as not to bump into each other.
In software development teams, the internalised rules - often unspoken and frequently at odds with the spoken or written rules - and the interactions at a local level determine what the outcomes the system will produce. We call these internalised rules "culture". Culture is often simple, but buried so deep that changing the culture can take a long time.
In particular, culture round the way we communicate and collaborate tends to steer the ship in particular directions, regardless of which direction you point the rudder. This is a property of complex adaptive systems called "strange attractors".
Complex systems have a property called "homeostatis" - a tendency, when disturbed, to iteratively revert back to their original dynamic state, as determined by their strange attractors. Hence, a heartbeat can rise to more than 150 bpm, but will eventually return to a resting heart rate of about 70-80 bpm.
We can apply external stimuli to a system to try and change the way it performs, but the intrinsic properties of the agents within that system, and particularly their interactions, will ultimately determine the outcome.
Methods like SAFe, LeSS and DAD are attempts to exert top-down control on highly complex adaptive organisations. As such, in my opinion and in the examples I've witnessed, they - at best - create the illusion of control. And illusions of control aren't to be sniffed at. They've been keeping the management consulting industry in clover for decades.
The promise of scaled agile lies in telling managers what they want to hear: you can have greater control. You can have greater predictability. You can achieve economies of scale. Acknowledging the real risks puts you at a disadvantage when you're bidding for business.
But if you really want to make a practical difference in a large software development organisation, the best results I've seen have come from focusing on the culture: what do people really value? What do people really believe? What are people's real habits? What do they really do under pressure?
You build big, complex products out of small, simple parts. The key is not in trying to exert control over the internal workings of each part, but to focus on how the parts - and the small, simple teams who make them - interact. Each part does a job. Each part will depend on some of the other parts. An overall architecture can emerge by instilling a set of good, practical organising principles across the teams - a design culture, like we have in the architecture of buildings, for example. The teams negotiate with each other to resolve potential conflicts, like motorists on our complex road systems trying to get where they need to go without bumping into each other.
Another word for this is "anarchy". I advice you not to use it in client meetings. But that is what it is.
I think it's very telling that so many of the original signatories of the Agile Manifesto have voiced scepticism - indeed, in some cases been very scathing - of "scaled agile". The way I see it, it's the precise opposite of what they were trying to tell us at Snowbird.
This is why, as a professional, I've invested so much time in training and coaching developers and teams, rather than in management consulting. I certainly engage with bosses, but when they ask about "scaled agile" I tell them what I personally think, which is that it's a mirage.
November 13, 2016
Think About Workflows Before You Think About FeaturesSunday morning's often my time for doing admin and logistical-type stuff, like booking flights or trains or hotels.
Today I searched for a hotel near a client's offices, then booked through the chain's website. After that I copied and pasted the hotel's address to search on Google Maps for a decent restaurant nearby, and then pasted the restaurant's name into Trip Advisor to see reviews. Satisfied, I then pasted the hotel's postcode into an app for booking a taxi to the client site on the first morning because it was 2 miles away (and I find walking to strange places in strange towns a bit risky on the first day).
Finally, I searched on Trainline and booked reserved seats, because it's way cheaper to do in advance.
In all, I used 6 different websites:
1. Google Maps (to find a hotel near client offices)
2. Hotel website to book a room
3. Google Maps again to find a restaurant nearby
4. Trip Advisor to see reviews of the restaurant
5. Restaurant website to reverse a table for dinner
6. Taxi booking site
7. Trainline to find and reserve seats for a return journey
And it occurred to me that there was a lot of browser tabs and copying and pasting involved, creating the impression of some kind of workflow.
This workflow sits above the use cases of Google Maps, the hotel's website, Trip Advisor, the restaurant's website, the taxi booking site, and Trainline.
We can visualise how this workflow maps on to the 6 different software systems involved, using a simple technique borrowed from the book Business Modeling with UML (and even as I type this, I'm copying and pasting the URL to the book, part of yet another higher-level workflow):
The reason I'm mentioning all of this is three-fold: firstly, if I were designing these systems, I would find this kind of context very useful. Right now, to move from one step in the workflow to the next, I have to open a new browser tab, go to the right web page, and paste in data from a previous step. Clunky! Studying these real world workflows can help us to smooth out these kinks. For example, if we found that 50% of people who book hotel rooms on our site then go on to search for nearby restaurants, we could add a link that will take us straight to Google Maps and fill in the data for us.
Secondly, and more importantly, when software design's done right, we start by thinking about business goals and about business workflows, not system features. This technique helps us to identify system use cases from the outside in, starting with the business context. That's as it should be.
And thirdly, it makes good business sense, when we're designing applications, to ask ourselves "where are the user's coming from?" Have they just booked a hotel? Did they just book a flight? Work your way backwards and build relationships with the developers of software that might - indirectly via search engines, perhaps - be sending users our way, so we can work with third parties to smooth that path. And, likewise, smooth the user's path to downstream activities in their workflow. What are they likely to want to do next? In sales and marketing, it's essential to understand what triggers buyers' needs, so you can pitch your tent right outside that triggering event, so to speak. There's a reason why late-night petrol stations sell flowers and firelighters, and it has absolutely nothing to do with selling petrol.
Arguably, this is how the web should work now. Not links to content, but controls for triggering actions in other people's systems - an event-based (rather than content-based) model. But it doesn't. So we'll have to hand-wire that sort of thing ourselves.
And this isn't just relevant to web-based workflows, of course. An end-to-end sales scenario in your company many involve multiple workflows, enacted using multiple systems. Work to understand those workflows, and smooth the user's path from one step to the next. See them copying and pasting data from one system to another? That's a hint that there's more automation to be done.
November 8, 2016
Business Benefits of Continuous Delivery: We Need Hard DataSomething that's been bugging me for a while is our apparent lack of attention to the proclaimed business benefits of Continuous Delivery.
I'm not going to argue for one second that CD doesn't have business benefits; I'm a firm believer in the practice myself. But that's just it... I'm a believer in the business benefits of Continuous Delivery. And it's a belief based on personal and anecdotal experience, not on a good, solid body of hard evidence.
I had naturally assumed that such evidence existed, given that the primary motivation for CD, mentioned over and over again in the literature, is the reduced lead times on delivering feature and change requests. It is, after all, the main point of CD.
But where is the data that supports reduced lead times? I've looked, but not found it. I've found surveys about adopting CD. I've found proposed metrics, but no data. I've found largely qualitative studies of one or two organisations. But no smoking gun, as yet.
There's a mountain of data that backs up the benefits of defect prevention, but the case for CI currently rests on little more than smoke.
This, I reckon, we need to fix. It's a pillar on which so much of software craftsmanship and Agile rests; delivering working software sooner (and for longer).
Anything that supports the case for Continuous Delivery indirectly supports the case for Continuous Integration, TDD, refactoring, automation, and a bunch of other stuff we believe is good for business. And as such, I think we need that pillar to unassailably strong.
We need good data - not from surveys and opinion polls - on lead times that we can chart against CD practices so we can build a picture of what real, customer-visible impact these practices have.
To be genuinely useful and compelling, it would need to come from hundreds of places and cover the full spectrum of Continuous Delivery from infrequent manual builds with infrequent testing and no automation, to completely automated Continuous Deployment several times a day with high confidence.
One thing that would of particular interest to Agile mindsets would be how the lead times change over time. As the software grows, do lead times get longer? What difference does, say, automated developer testing make to the shape of the curve?
Going beyond that, can we understand what impact shorter lead times can have on a business? Shorter lead times, in of themselves have no value. The value is in what they enable a business to do - specifically, to learn faster. But what, in real terms, are the business benefits of learning faster? How would we detect them? Are businesses that do CD outperforming competitors who don't in some way? Are they better at achieving their goals?
Much to ponder on.
April 30, 2016
Goals vs. ConstraintsA classic source of tension and dysfunction in software teams - well, probably all kinds of teams, really - is the relativity between goals and constraints.
Teams often mistake constraints for goals. A common example is when teams treat a design specification as a goal, and lose sight of where that design came from in the first place.
A software design is a constraint. There may be countless ways of solving a problem, but we chose this one. That's the very definition of constraining.
On a larger scale, I've seen many tech start-ups lose sight of why they're doing what they're doing, and degenerate into 100% focusing on raising or making the money to keep doing whatever it is they're doing. This is pretty common. Think of these charities who started out with a clear aim to "save the cat" or whatever, but fast-forward a few years and most - if not all - of the charities' efforts end up being dedicated to raising the funds to pay everybody and keep the charity going.
Now, you could argue that a business's goal is to make money, and that they make money in exchange for helping customers to satisfy their goals. A restaurant's goal is to make money. A diner's goal is to be fed. I give you money. You stop me from being hungry.
Which is why - if your organisation's whole raison d'être is to make a profit - it's vitally important to have a good, deep understanding of your customer's goals or needs.
That's quite a 19th century view of business, though. But even back then, some more progressive industrialists saw aims above and beyond just making a profit. At their best, businesses can provide meaning and purpose for employees, enrich their lives, enrich communities and generally add to the overall spiffiness of life in their vicinity.
But I digress. Where was I? Oh yes. Goals vs. constraints.
Imagine you're planning a trip from your home in Los Angeles to San Francisco. Your goal is to visit SF. A constraint might be that, if you're going to drive, you'll need enough gasoline for the journey.
So you set out raising money for gas. You start a lemonade stall in your front yard. It goes well. People like your lemonade, and thanks to the convenient location of your home, there are lots of passers-by with thirsts that need quenching. Soon you have more than enough money for gas. But things are going so well on your lemonade stall that you've been too busy thinking about that, and not about San Francisco. You make plans to branch out into freshly squeezed orange juice, and even smoothies. You get a bigger table. You hire an assistant, because there's just so much to be done. You buy a bigger house on the same street, with a bigger yard and more storage space. Then you start delivering your drinks to local restaurants, where they go down a storm with diners. 10 years later, you own a chain of lemonade stalls spanning the entire city.
Meanwhile, you have never been to San Francisco. In fact, you're so busy now, you may never go.
Now, if you're a hard-headed capitalist, you may argue "so what?" Surely your lemonade business is ample compensation for missing out on that trip?
Well, maybe it is, and maybe it isn't. As I get older, I find myself more and more questioning "Why am I doing this?" I know too many people who got distracted by "success" and never took those trips, never tried those experiences, never built that home recording studio, never learned that foreign language, and all the other things that were on their list.
For most of us - individuals and businesses alike - earning money is a means to an end. It's a constraint that can enable or prevent us from achieving our goals.
As teams, too, we can too easily get bogged down in the details and lose sight of why we're creating the software and systems that we do in the first place.
So, I think, a balance needs to be struck here. We have to take care of the constraints to achieve our goals, but losing sight of those goals potentially makes all our efforts meaningless.
Getting bogged down in constraints can also make it less likely that we'll achieve our goals at all.
Constraints constrain. That's sort of how that works. If we constrain ourselves to a specific route from LA to San Francisco, for example, and then discover half way that the road is out, we need other options to reach the destination.
Countless times, I've watched teams bang their heads against the brick wall trying to deliver on a spec that can't - for whatever reason - be done. It's powerful voodoo to be able to step back and remind ourselves of where we're really headed, and ask "is there another way?" I've seen $multi-million projects fail because there was no other way - deliver to the spec, or fail. It had to be Oracle. It had to be a web service. It had to be Java.
No. No it didn't. Most constraints we run into are actually choices that someone made - maybe even choices that we made for ourselves - and then forgot that it was a choice.
Yes, try to make it work. But don't mistake choices for goals.
April 23, 2016
Does Your Tech Idea Pass The Future Dystopia Test?One thing that at times fascinates and at times appals me is the social effect that web applications can have on us.
Human beings learn fast, but evolve slowly. Hence we can learn to program a video recorder, but living a life that revolves around video recorders can be toxic to us. For all our high-tech savvy, we are still basically hominids, adapted to run from predators and pick fleas off of each other, but not adapted for Facebook or Instagram or Soundcloud.
But the effects of online socialisation are now felt in the Real World - you know, the one we used to live in? People who, just 3-4 years ago, were confined to expressing their opinions on YouTube are now expressing them on my television and making pots of real money.
Tweets are building (and ending) careers. Soundcloud tracks are selling out tours. Facebook viral posts are winning elections. MySpace users are... well, okay, maybe not MySpace users.
For decades, architects and planners obsessed over the design of the physical spaces we live and work in. The design of a school building, they theorise, can make a difference to the life chances of the students who learn in it. The design of a public park can increase or decrease the chances of being attacked in it. Pedestrianisation of a high street can breath new life into local shops, and an out-of-town shopping mall can suck the life out of a town centre.
Architects must actively consider the impact of buildings on residents, on surrounding communities, on businesses, on the environment, when they create and test their designs. Be it for a 1-bed starter home, or for a giant office complex, they have to think about these things. It's the law.
What thought, then, do software developers give to the social, economic and environmental impact of their application designs?
Having worked on "Web 2.0" sites of all shapes and sizes, I have yet to see teams and management go out of their way to consider such things. Indeed, I've seen many occasions when management have proposed features of such breath-taking insensitivity to wider issues, that it's easy to believe that we don't really think much about it at all. That is, until it all goes wrong, and the media are baying for our blood, and we're forced to change to keep our share price from crashing.
This is about more than reliability (though reliability would be a start).
Half-jokingly, I've suggested that teams put feature requests through a Future Dystopia Test; can we imagine a dark, dystopian, Philip K Dick-style future in which our feature has caused immense harm to society? Indeed, whole start-up premises fail this test sometimes. Just hearing some elevator pitches conjures up Blade Runner-esque and Logan's Run-ish images.
I do think, though, that we might all benefit from devoting a little time to considering the potential negative effects of what we're creating before we create it, as well as closely monitoring those effects once it's out there. Don't wait for that hysterical headline "AcmeChat Ate My Hamster" to appear before asking yourself if the fun hamster-swallowing feature the product owner suggested might not be such a good thing after all.
This blog post is gluten free and was not tested on animals
April 20, 2016
A* - A Truly Iterative Development ProcessMuch to my chagrin, having promoted the idea for so many years, software development still hasn't caught on to the idea that what we ought to be doing is iterating towards goals.
NOT working through a queue of tasks. NOT working through a queue of features.
Working towards a goal. A testable goal.
We, as an industry, have many names for working through queues: Agile, Scrum, Kanban, Feature-driven Development, the Unified Process, DSDM... All names for "working through a prioritised list of stuff that needs to be done or delivered". Of course, the list is allowed to change depending on feedback. But the goal is usually missing. Without the goal, what are we iterating towards?
Ironically, working through a queue of items to be delivered isn't iterating - something I always understood to be the whole point of Agile. But, really, iterating means repeating a process, feeding back the results of each cycle, until we reach some goal. Reaching the goal is when we're done.
What name do we give to "iterating towards a testable goal"? So far, we have none. Buzzword Bingo hasn't graced the door of true iterative development yet.
Uncatchy names like goal-driven development and competitive engineering do exist, but haven't caught on. Most teams still don't even have even a vague idea of the goals of their project or product. They're just working through a list that somebody - a customer, a product owner, a business analyst - dreamed up. Everyone's assuming that somebody else knows what the goal is. NEWSFLASH: They don't.
The Codemanship way compels us to ditch the list. There is no release plan. Only business/user goals and progress. Features and change requests only come into focus for the very near future. The question that starts every rapid iteration is "where are we today, and what's the least we could do today to get closer to where we need to be?" Think of development as a graph algorithm: we're looking for the shortest path from where we are to some destination. There are many roads we could go down, but we're particularly interested in exploring those that bring us closer to our destination.
Now imagine a shortest-path algorithm that has no concept of destination. It's just a route map, a plan - an arbitrary sequence of directions that some product owner came up with that we hope will take us somewhere good, wherever that might be. Yup It just wouldn't work, would it? We'd have to be incredibly lucky to end up somewhere good - somewhere of value.
And so it is - in my quest for a one-word name to describe "iteratively seeking the shortest (cheapest) path to a testable goal", I propose simply A*
"What method are we following on this project?"
Of course, there are prioritised lists in my A* method: but they are short and only concern themselves with what we're doing next to TRY to bring us closer to our goal. Teams meet every few days (or every day, if you're really keen), assess progress made since last meeting, and come up with a very short plan, the results of which will be assessed at the next meeting. And rinse and repeat.
In A*, the product owner has no vision of the solution, only a vision of the problem, and a clear idea of how we'll know when that problem's been solved. Their primary role is to tell us if we're getting warmer or colder with each short cycle, and to help us identify where to aim next.
They don't describe a software product, they describe the world around that product, and how it will be changed by what we deliver. We ain't done until we see that change.
This puts a whole different spin on software development. We don't set out with a product vision and work our way through a list of features, even if that list is allowed to change. We work towards a destination - accepting that some avenues will turn out to be dead-ends - and all our focus is on finding the cheapest way to get there.
And, on top of all that, we embrace the notion that the destination itself may be a moving target. And that's why we don't waste time and effort mapping out the whole route beyond the near future. Any plan that tries to look beyond a few days ends up being an expensive fiction that we become all too easily wedded to.
August 1, 2015
My First, Last & Only Blog Post About #NoEstimatesI've been keeping one eye on the whole #NoEstimates debate on Twitter, and folk have asked me my opinion quite a few times. So here it is.
I believe, very firmly, that the problem with estimation stems from us asking the wrong question.
In fact, this is where many big problems in software development arise; by asking the customer "What software would you like us to build?"
This naturally leads to a shopping list of features, and then a request to know "How much will all that cost and how long will it take?"
If we asked instead "What problem are we trying to solve, and how will we know when we've solved it?" - together with accompanying questions like "When do you need this solution?", "What is a solution worth to you?" and "How much money do you have to invest in solving it?" - we can set out on a different journey.
I believe software development needs to be firmly grounded in reality, and the reality is that it's R&D. At the start, the honest answer to questions like "What features are needed?", "How much will it cost?" and "How long will it take?" is I Don't Know.
Pretending to know the unknowable is what lands us in hot water in the first place. We don't know if we can solve the problem with the budget and the time available.
In the management quest for accounting certainties, though, nobody wants to hear that, and no developer with a mortgage to pay wants to admit it. So we go with the fairy tale instead.
Once we're in the fairy tale - where we know if we deliver this list of features, it will solve the customer's problem, and we can predict how long and how much it will take - it's almost impossible to get out of it. Budgets are committed. Deadlines are agreed. Necks are on chopping blocks.
So, what we do instead, is we wait for the reality to unfold, and then when it no longer matches the fairy tale, there's a major shitstorm of blame and recrimination. Typically, the finger is pointed at everyone and everything except that first mistake; the original sin of software projects: pretending to know the future.
After getting their fingers burned once, the customer's and manager's instinct is to "fix" the problem by "improving" estimates next time around. This is fixing the fairy tale by inventing an even more elaborate fairy tale, to try and disguise the fact that's it's fantasy. This is the management equivalent of sacrificing virgins to make it rain.
The only way out of the estimating nightmare is to call "bullshit" on it, and publicly accept - indeed, embrace - the uncertainty that's inherent in what we're doing.
Yes, you might lose the business if you start out saying "I don't know", but consider that the business you're losing is the same old Death March teams have been suffering for decades. That's not work. That's just passing the time for money.
By all means offer a guess, so the customer can budget realistically. But you must be absolutely 100% crystal clear with them that, at the end of the day, we don't know. We just don't know. It's a punt.
Sell yourself on what you do know. What's your track record as a team? What have you delivered in the past? How much did that cost? How long did that take? And - most importantly, but regrettably least asked - did it work?
When a movie studio hires a director, the director makes no guarantees that this new film will be a commercial success, or that it will cost no more than budgeted, or be completed dead on time. The history of cinema is littered with amazingly good, and often very successful, movies that cost more and took longer than planned. But somehow, James Cameron seems to have no trouble getting movies off the ground. That's because of his track record, not his ability to accurately predict production costs and schedules.
Studios gamble with huge sums of money, and - yes - they do ask for estimates, and things do get hairy when schedules slip and costs overrun, but fundamentally they know what game they're in.
It's time we did, too.
April 15, 2015
Reality-Driven Development - Do We Need To Train Ourselves To Tune Into Real-World Problems?The world is full of problems.
That is to say, the world is full of people who are hindered in their efforts to achieve their goals, no matter how small and everyday those goals might be.
I'm no different to anyone else in that respect; every day I hit barriers, overcome obstacles, and occasionally give up trying. It might be something as prosaic as being unable to find an item in the shops when I'm in town, or wondering if there's a café or a pub nearby that has seating available at that specific moment in time. Or it could be something as astronomically important as wondering if aliens have ever visited our solar system, and have they left any hardware behind.
Problems, problems, problems. The real world's full of interesting problems.
And so it comes as some surprise that when my apprentice and I take a moment to think about ideas for a software project, we struggle to think of any real-world problems that we could have a go at solving.
It struck me in that moment that maybe, as a species, we software bods are a bit crap at noticing problems that have nothing to do with developing software. Typically, when I ask developers to think about ideas for a project, the suggestions have a distinctly technical bent. Maybe a new testing tool, or a plug-in for Visual Studio, or Yet Another MVC Framework, or something that does something to do with builds (and so on.)
A lifetime of minor frustrations, some of which software might be able to help with, pops clean out of our heads. And I ask: are we just not tuned in to real-world problems?
There are people who walk into a room, and notice that the scatter cushions compliment the carpet. I don't. I'm just not tuned in to that sort of thing. And if you give them some money, and say "improve this room", they'll have all sorts of ideas about furniture and bookshelves and paintings and curtains and pot plants, while I would buy a Massive F***ing TVTM and a Playstation 4. I'm tuned into technology in ways they're not, and they're tuned into home furnishings in ways I'm not.
Do we need to mindfully practice noticing everyday problems where software might help? Do we need to train ourselves to tune in to real-world problems when we're thinking about code and what we can potentially do with it? Do we need to work at noticing the colour of the scatter cushions when we walk into a room?
I've suggested that Will and I keep notes over the next two weeks about problems we came up against, so when we next pair, there might be some inspiration for us to draw from.
Which gives me an idea for an app....