November 27, 2016

Learn TDD with Codemanship

Software Craftsmanship is a Requirements Discipline

After the smoke and thunder of the late noughties software craftsmanship movement had cleared, with all its talk of masters and apprentices and "beautiful code", we got to see what code craft was really all about.

At its heart, crafting high quality code is about leaving it open to change. In essence, software craftsmanship is a requirements discipline; specifically, enabling us to keep responding to new requirements, so that customers can learn from using our software and the product can be continually improved.

Try as we might to build the right thing first time, by far the most valuable thing we can do for our customers is allow them to change their minds. Iterating is the ultimate requirements discipline. So much value lies in empirical feedback, as opposed to the untested hypotheses of requirements specifications.

Crafting code to minimise barriers to change helps us keep feedback cycles short, which maximises customer learning. And it helps us to maintain the pace of innovation for longer, effectively giving the customer more "throws of the dice" at the same price before the game is over.

It just so happens that things that make code harder to change also tend to make it less reliable (easier to break) - code that's harder to understand, code that's more complex, code that's full of duplication, code that's highly interdependent, code that can't be re-tested quickly and cheaply, etc.

And it just so happens that writing code that's easy to change - to a point (that most teams never reach) - is also typically quicker and cheaper.

Software craftsmanship can create a virtuous circle: code that's more open to change, which also makes it more reliable, and helps us deliver value sooner and for longer. It's a choice that sounds like a no-brainer. And, couched in those terms, it should be an easy sell. Why would a team choose to deliver buggier software, later, at a higher cost, with less opportunity to improve it in the next release?

The answer is: skills and investment. But that's a whole other blog post!

November 23, 2016

Learn TDD with Codemanship

101 TDD Tips - #1 - #40

We're now up to #40 of my 101 TDD Tips (follow for daily additions).

And you can get a monthly digest of all the tips posted so far from

November 13, 2016

Learn TDD with Codemanship

Think About Workflows Before You Think About Features

Sunday morning's often my time for doing admin and logistical-type stuff, like booking flights or trains or hotels.

Today I searched for a hotel near a client's offices, then booked through the chain's website. After that I copied and pasted the hotel's address to search on Google Maps for a decent restaurant nearby, and then pasted the restaurant's name into Trip Advisor to see reviews. Satisfied, I then pasted the hotel's postcode into an app for booking a taxi to the client site on the first morning because it was 2 miles away (and I find walking to strange places in strange towns a bit risky on the first day).

Finally, I searched on Trainline and booked reserved seats, because it's way cheaper to do in advance.

In all, I used 6 different websites:

1. Google Maps (to find a hotel near client offices)
2. Hotel website to book a room
3. Google Maps again to find a restaurant nearby
4. Trip Advisor to see reviews of the restaurant
5. Restaurant website to reverse a table for dinner
6. Taxi booking site
7. Trainline to find and reserve seats for a return journey

And it occurred to me that there was a lot of browser tabs and copying and pasting involved, creating the impression of some kind of workflow.

This workflow sits above the use cases of Google Maps, the hotel's website, Trip Advisor, the restaurant's website, the taxi booking site, and Trainline.

We can visualise how this workflow maps on to the 6 different software systems involved, using a simple technique borrowed from the book Business Modeling with UML (and even as I type this, I'm copying and pasting the URL to the book, part of yet another higher-level workflow):

The reason I'm mentioning all of this is three-fold: firstly, if I were designing these systems, I would find this kind of context very useful. Right now, to move from one step in the workflow to the next, I have to open a new browser tab, go to the right web page, and paste in data from a previous step. Clunky! Studying these real world workflows can help us to smooth out these kinks. For example, if we found that 50% of people who book hotel rooms on our site then go on to search for nearby restaurants, we could add a link that will take us straight to Google Maps and fill in the data for us.

Secondly, and more importantly, when software design's done right, we start by thinking about business goals and about business workflows, not system features. This technique helps us to identify system use cases from the outside in, starting with the business context. That's as it should be.

And thirdly, it makes good business sense, when we're designing applications, to ask ourselves "where are the user's coming from?" Have they just booked a hotel? Did they just book a flight? Work your way backwards and build relationships with the developers of software that might - indirectly via search engines, perhaps - be sending users our way, so we can work with third parties to smooth that path. And, likewise, smooth the user's path to downstream activities in their workflow. What are they likely to want to do next? In sales and marketing, it's essential to understand what triggers buyers' needs, so you can pitch your tent right outside that triggering event, so to speak. There's a reason why late-night petrol stations sell flowers and firelighters, and it has absolutely nothing to do with selling petrol.

Arguably, this is how the web should work now. Not links to content, but controls for triggering actions in other people's systems - an event-based (rather than content-based) model. But it doesn't. So we'll have to hand-wire that sort of thing ourselves.

And this isn't just relevant to web-based workflows, of course. An end-to-end sales scenario in your company many involve multiple workflows, enacted using multiple systems. Work to understand those workflows, and smooth the user's path from one step to the next. See them copying and pasting data from one system to another? That's a hint that there's more automation to be done.

November 8, 2016

Learn TDD with Codemanship

Business Benefits of Continuous Delivery: We Need Hard Data

Something that's been bugging me for a while is our apparent lack of attention to the proclaimed business benefits of Continuous Delivery.

I'm not going to argue for one second that CD doesn't have business benefits; I'm a firm believer in the practice myself. But that's just it... I'm a believer in the business benefits of Continuous Delivery. And it's a belief based on personal and anecdotal experience, not on a good, solid body of hard evidence.

I had naturally assumed that such evidence existed, given that the primary motivation for CD, mentioned over and over again in the literature, is the reduced lead times on delivering feature and change requests. It is, after all, the main point of CD.

But where is the data that supports reduced lead times? I've looked, but not found it. I've found surveys about adopting CD. I've found proposed metrics, but no data. I've found largely qualitative studies of one or two organisations. But no smoking gun, as yet.

There's a mountain of data that backs up the benefits of defect prevention, but the case for CI currently rests on little more than smoke.

This, I reckon, we need to fix. It's a pillar on which so much of software craftsmanship and Agile rests; delivering working software sooner (and for longer).

Anything that supports the case for Continuous Delivery indirectly supports the case for Continuous Integration, TDD, refactoring, automation, and a bunch of other stuff we believe is good for business. And as such, I think we need that pillar to unassailably strong.

We need good data - not from surveys and opinion polls - on lead times that we can chart against CD practices so we can build a picture of what real, customer-visible impact these practices have.

To be genuinely useful and compelling, it would need to come from hundreds of places and cover the full spectrum of Continuous Delivery from infrequent manual builds with infrequent testing and no automation, to completely automated Continuous Deployment several times a day with high confidence.

One thing that would of particular interest to Agile mindsets would be how the lead times change over time. As the software grows, do lead times get longer? What difference does, say, automated developer testing make to the shape of the curve?

Going beyond that, can we understand what impact shorter lead times can have on a business? Shorter lead times, in of themselves have no value. The value is in what they enable a business to do - specifically, to learn faster. But what, in real terms, are the business benefits of learning faster? How would we detect them? Are businesses that do CD outperforming competitors who don't in some way? Are they better at achieving their goals?

Much to ponder on.

November 6, 2016

Learn TDD with Codemanship

Real Immersion is the Key to Understanding Your Customer's Needs

I've long been a believer that the key to producing good solutions is understanding the problem.

This is something that developers will nod their heads in agreement at. But I'm not sure they fully grasp what I mean by "understanding the problem".

I have a background in model-driven approaches to requirements analysis and solution design, and am no stranger to the meetings, documents and diagrams dev teams use to describe problem domains. I would even go so far as to claim to be something of a dab hand (in a previous life).

And that's how I know that traditional approaches to domain analysis are usually entirely insufficient to the task. To illustrate what I mean, imagine a non-developer sat and listened to you explain how you do software development. They write copious notes. They draw copious diagrams with boxes and arrows on them. And at the end of all that, do they really understand how you create software? Is an explanation sufficient?

And if they came back and said "we've come up with a better way of developing software", would you find that credible?

Now let's turn the tables back the usual way. You're asked to write some software for, say, an estate agent. You sit in meeting rooms while they explain estate agency to you. You take copious notes. You draw copious diagrams. And then you come back to them and say "we have solved your problem". Credible?

What gets missed in all those meetings and all those documents and diagrams is the complexity and the nuance. Submerged beneath the explanations is an iceberg of tacit knowledge. That's the iceberg that sinks many projects.

My own experience has taught me that teams have come up with much better solutions after they've seen things for themselves, and - even better - tried things for themselves. If an estate could write software, they'd probably write much better solutions for estate agents than we could.

What developers need to do is become the end user, even if it's just for a while. Shadow domain experts and watch them do their work. Try key tasks for yourself, if that's possible. Books on software analysis and design talk about "immersing" ourselves in the problem domain. But I do not think that words means what they think it means. To me, it means becoming the customer - walking a mile or three in their shoes.

Some domains, of course are very complicated. Walking a mile in a brain surgeon's shoes is not practical. It would take years to understand things as well as a brain surgeon does. In these specialisms, we should probably approach things from the other direction: some brain surgeons should learn to write software.

But most problem domains - most jobs - aren't as complex as yet. It maybe takes a day or two to learn how to operate a supermarket checkout. It maybe takes a week or two to learn how to schedule classes in a school.

In the majority of cases, it's practical for developers to spend time on the "shop floor", observing what goes on in the business activities the software will be used in, and learning how to execute those jobs themselves to at least a basic level.

Not only can it clear up a lot of potential misunderstandings - I've seen months of business analysis blown away by just a few hours spent on the shop floor - but it can also help us to empathise with our end users, as well as build relationships with them that will help us when we need their input and feedback. It may also make us think twice about creating software that will affect these people adversely.

The meetings and the diagrams happen as a consequence of this real immersion. Discussions about what can be improved happen a lot faster when every dog has seen the rabbit.

Of course, there'll be lots of devs still nodding in agreement. It is good form to care about the end users and about solving real problems that make their lives easier.

But, although the results of real immersion can be spectacular, but you may have to drag teams kicking and screaming to experience it. We're highly educated professionals, not supermarket checkout operators, goddamit!

In XP, I've found that this is the first step to an on-team customer: making yourself the customer, if just for a while. Climbing inside their heads requires us to start by experiencing first-hand what it's like to be them. For anything but the most rudimentary of tasks - and, as programmers who automate stuff, we should know by now that there's no such thing as a "rudimentary task" - we can no more build a true understanding of the problem through explanations than we could build a true understanding of the colour orange from reading books.

Now, I get on this hobby horse every once in a while, and developers nod their heads and mutter words of support, and nothing changes. I can't stress this enough, though: this is some powerful voodoo. It makes an enormous difference.

You will no doubt agree in principle, but will always find an excuse why this is not possible or desirable in your team. The hidden objection, once you strip away all the excuses is simply: "we don't want to do this".

Try it. I dare you.

October 29, 2016

Learn TDD with Codemanship

How Do I Know a "Software Developer" When I See One?

More thoughts on what a software developer is; this time on how we know one when we see one.

Everyone has their own story, and software developers come by many routes. My personal journey, like so many, started with the home computing boom of the early 1980s, but didn't really flourish into what I'd now call "software development" until more than a decade later, when I was introduced to practices with names like "requirements engineering", "software configuration management", "architecture", "modeling", "unit testing", "usability", "formal specification", and all the rest.

20 years ago, when I was relatively new to a lot of these things, I believed in The One True WayTM. Today, I've experienced first-hand several One True Ways - Fusion, Catalysis, the Unified Process, Select Perspective (and a myriad other UML-y, component-y methods, even designing a few myself), and Extreme Programming. The way I practice XP draws on previous experience with Formal Methods, model-driven development, plus ideas pinched from books on Cleanroom Software Engineering, the Personal and Team Software Processes, Feature-Driven Development, business strategy, complexity theory, and anywhere else I stumbled across good ideas that could be applied to writing software.

Having absorbed so many ideas - or, more accurately, so many different interpretations of many of the same good ideas (e.g., iterate towards testable goals, test early & often, build software out of simple swappable parts, etc) - I feel I have a decent handle on what software development is, and have practical experience across a spectrum of usually cosmetically different approaches to it.

And I know a lot of software developers. Their journeys may be different to mine, but they all seem to have one thing in common with me: diversity of experience and ideas. They're not people who just learned Extreme Programming, just work in Java, just do TDD, just use Git, just write web applications, and so on.

They've typically been around long enough to have had careers that span multiple "movements" in software development: the OO/patterns movement of the early-mid-nineties, the UML/UP/component-based development movement of the late 90's. The early agile/XP movement of the early noughties. The post-agile/craftsmanship movement of the late noughties. And, yes, I am saying that if you're my idea of a "software developer", chances are you've been in this game - remaining at the code face to some degree - for 20+ years.

But not necessarily. Someone who reads a diverse range of literature on software development, and tries the ideas for real, could develop the breadth and depth of knowledge and experience faster. But we're still talking a decade or more, realistically. Nobody learns that fast!

And the experience is important. So much of what we do relies on tacit knowledge. It's not written down anywhere. You can't cram that.

So - to recap and summarise - if you asked me "is that person a 'software developer'?", I'd look for experience of all the key disciplines of software development (including some team management, hiring, finance/admin, etc), and I'd look for at least 2 examples of each:

  • 2+ approaches to development (e.g., Unified Process and DSDM)

  • 2+ programming languages & paradigms (e.g., Java and F#)

  • 2+ technology stacks (e.g., LAMP and Android)

  • 2+ VCS technologies & workflows (e.g., Git and SVN)

  • 2+ build & deployment technologies & approaches (e.g., Maven+Jenkins, Make+Bash)

  • 2+ kinds of visual modeling/sketching of architecture & design (e.g., UML and ERD)

  • 2+ database technologies & paradigms (e.g, MySQL and Neo4j)

  • 2+ kinds of testing (e.g., Web GUI with Selenium, and customer tests with FitNesse)

  • 2+ roles they've played (e.g., programmer and architect, or tester and business analyst) - though, really, a "software developer" could wear any hat on a small project

  • 2+ business domains, 2+ different scales of project/programme

You get the general idea, I'm sure.

On top of all this, I'd expect a software developer to have contributed to the wider developer community: blogging (at a minimum), teaching, mentoring, writing, speaking/presenting, creating or contributing to useful development tools and standards. You know... paying it forward. I would not be what I consider a software developer if other people hadn't done these things.

October 28, 2016

Learn TDD with Codemanship

What is a "Software Developer"?

Over the last few months, I've been thinking a lot about what a software developer is, and about how employers might know one when they see one.

I've long had my own idea: that a software developer - as opposed to a programmer, or a web developer, or a tester, or user experience designer, or any other specialised role - is a bit like a building contractor.

My Dad was a builder for many years, and worked in and then took over the family business. He studied at college and worked on building sites, but he wasn't a tradesman. Not a bricklayer, or a carpenter, or an electrician, or a plumber, or a steelworker, or an architect, or a quantity surveyor, or a structural engineer.

As a building contractor - the person in charge of delivering working buildings - he had to be a bit of all those things, and a bit more. He could do bricklaying, who could do carpentry, he could do plumbing. And he could design buildings, as well as read blueprints, cost the work, calculate how deep the foundations would need to be, how big the boiler would need to be, how thick the loft insulation would need to be and so on.

To me, a software developer is a bit like that. We're responsible for delivering working software, and as such we need to have a practical appreciation of everything that's required to make that happen. We're not bricklayers, but we can lay bricks. We're not plumbers, but we can plumb if we need to. We're not electricians, but we could do wiring to a safe standard if necessary.

A software developer isn't a database expert, but can design, implement and administer databases if needed. A software developer isn't a Java programmer, or a C# programmer, or Swift programmer, but could do the coding to a high enough standard if needed. A software developer isn't a UX expert, but could design a usable GUI if required. A software developer isn't a software architect, but could create and maintain a workable architecture, along with any necessary visual models, if called for.

And a software developer isn't a project manager, but they could manage a project without tying themselves in knots, keeping track of time and resources, hiring teams, getting all the space and equipment the team needs, liaising with project stakeholders, and being a bit of a sales person for the team when selling is needed.

Nor is a software developer a requirements or a business analyst, but they're capable of working directly with customers and other stakeholders, identifying real requirements, and articulating requirements in a way that can lead to working software that meets the customer's needs.

For me, a software developer can be trusted to work alone - on projects of a certain size, and on certain technology stacks, in certain problem domains - to produce working software that's fit for purpose. Just as my dad, perhaps with the help of one labourer, could build a kitchen extension or a conservatory.

And on more ambitious projects he'd know who to hire. He knows a good bricklayer when he sees one. He knows a good chippy when he sees one. He knows what he wants from a plumber, from an electrician, from a roofer, from a landscape gardener.

He could build you a kitchen extension by himself. And he could hire and manage a team that could build you a factory unit, or a housing estate.

In my mind, that's what a software developer is. One of Scott Ambler's "generalising specialists", who has a hands-on, practical understanding of all the key disciplines in software development - management, strategy, requirements, security, infrastructure, architecture, design, programming, testing, configuration & release management, support, maintenance, decommissioning and transition - and knows who to hire when the job is too much for one developer.

The main difference between my vision and what a building contractor does is that, in my vision, everyone on the development team is a developer, just with particular specialisms. So, going back to the building metaphor, a building contractor who specialise in brick and block work, a building contractor who specialises in plumbing, and so on.

This gives great flexibility. Specialists who only know their specialism can quickly become bottlenecks. I've lost count of the days I've wasted waiting for the UX designer to come and help me on a user story. And then there was that time the DBA went on a month-long holiday and had locked everyone else out of making changes to the database...

So, not so much "collective code ownership" as collective everything ownership.

Well, that's my idea of what a software developer is. You may now throw the furniture around.

October 26, 2016

Learn TDD with Codemanship

101 TDD Tips Digest

I've been posting a TDD tip every weekday on the @codemanship Twitter feed, and we're now up to #20.

Folk have asked where they can get all the tips so far, in order (Twitter's not good at that sort of thing), so I've pasted them into a TDD Tips PDF digest (via this page) for your convenience, which I'll update every 20 tips.

Keep your eyes peeled on the Twitter account for new tips.

October 15, 2016

Learn TDD with Codemanship

If Your Code Was Broken, Would You Know?

I've been running a little straw poll among friends and clients, as well as on social media, to get a feel for what percentage of development teams routinely (or continuously) measure the level of assurance their automated regression tests give them.

For me, it's a fundamental question: if my code was broken, would I know?

The straw poll suggests that about 90% of teams don't ask that question often, and 80% don't ask it at all.

The whole point of automated tests is to give us early, cheap detection of new bugs that we might have introduced as we change the code. So profound is their impact, potentially, that Michael Feathers - in his book Working Effectively With Legacy Code - defines "legacy code" as code for which we have no automated tests.

I've witnessed first-hand the impact automating regression tests can have on delivery schedules and development costs. Which is why that question is often on my mind.

The best techniques I know for "testing your tests" are:

1. A combination of the "golden rule" of Test-Driven Development (only write source code if a failing test requires it, so all the code is executed by tests), and running tests to make sure their assertions fail when the result is wrong.

2. Mutation testing - deliberately introducing programming errors to see if the tests catch them

I put considerable emphasis on the first practice. As far as I'm concerned, it's fundamental to TDD, and a habit test-driven developers need to get into. Before you write the simplest code to pass a test, make sure it's a good test. If the answer was wrong, would this test fail?

The second practice, mutation testing, is rarely applied by teams. Which is a shame, because it's a very powerful technique. Code coverage tools only tell what code definitely isn't being executed in tests. Mutation testing tells us what code isn't being meaningfully tested, even if it is being executed by tests. It specifically asks "If I broke this line of code, would any tests fail?"

The tools for automated mutation testing have improved greatly in recent years, and support across programming languages is growing. If you genuinely want to know how much assurance your tests can give you - i.e., how much confidence you can have that the code really works - then you need to give mutation testing a proper look.

Here are some mutation testing tools that might be worth having a play with:

Java - PIT

C# - VisualMutator

C/C++ - Plextest

Ruby - Mutant

Python - Cosmic Ray

PHP - Humbug

October 14, 2016

Learn TDD with Codemanship

101 TDD Tips, via @Codemanship

If you're on the Twitters, you can follow my "101 TDD Tips" series through the hashtag #101TddTips