December 2, 2018

...Learn TDD with Codemanship

Architecture: The Belated Return of Big Picture Thinking

A question that's been stalking me is "When does architecture happen in TDD?"

I see a lot of code (a LOT of code) and if there's a trend I've noticed in recent years it's an increasing lack of - what's the word I'm looking for? - rationality in software designs as they grow.

When I watch dev teams produce working software (well, the ones who do produce software that works, at least), I find myself focusing more and more on when the design decisions get made.

In TDD, we can make design decisions during four distinct phases of the red-green-refactor cycle:

1. Planning - decisions we make before we write any code (e.g., a rough sequence diagram that realises a customer test scenario)

2. Specifying- decisions we make while we're writing a failing test (e.g., calling a function to do what you need done for the test, and then declaring it in the solution code)

3. Implementing - decisions we make when we're writing the code to pass the test (e.g., using a loop to search through a list)

4. Refactoring - decisions we make after we've passed the test according to our set of organising principles (e.g., consolidating duplicate code into a reusable method)

If you're a fan of Continuous Delivery like me, then a central goal of the way you write software is that it should be (almost) always shippable. Since 2 and 3 imply not-working code, that suggests we'd spend as little time as possible thinking about design while we're specifying and implementing. While the tests are green (1 and 4), we can consider design at our leisure.

I can break down refactoring even further, into:

4a. Thinking about refactoring

4b. Performing refactorings

Again, if your goal is always-shippable code, you'd spend as little time as possible executing each refactoring.

Put more bluntly, we should be applying the least thought into design while we're editing code.

(In my training workshops, I talk about Little Red Riding Hood and the advice her mother gave her to stay on the path and not wander off into the deep dark forest, where dangers like Big Bad Wolves lurk. Think of working code as the path, and not-working code as the deep dark forest. I encourage developers to always keep at least one foot on the path. When they step off to edit code, they need to step straight back on as quickly as possible.)

Personally - and I've roughly measured this - I make about two-thirds of design decisions during refactoring. That is, roughly 60-70% of the "things" in my code - classes, methods, fields, variables, interfaces etc - appear during refactoring:

* Extracting methods, constants and such to more clearly document what code does

* Extracting methods and classes to consolidate duplicate code

* Extracting classes to eliminate Primitive Obsession (e.g., IF statements that hinge on what is obviously an object identity represented by a literal vaue)

* Extracting and moving methods to eliminate Feature Envy in blocks of code and expressions

* Extracting methods and classes to split up units of code that have > 1 reason to change

* Exctracting methods to decompose complex conditionals

* Extracting client-specific interfaces

* Introducing parameters to make dependencies swappable

And so on and so on.

By this process, my code tends to grow and divide like cells with each new test. A complex order emerges from simple organising principles about readabililty, complexity, duplication and dependencies being applied iteratively over and over again. (This is perfectly illustrated in Joshua Kerievky's Refactoring to Patterns.)

I think of red-green-refactor as the inner loop of software architecture. And lots of developers do this. (Although, let's be honest, too many devs skimp on the refactoring.)

But there's architecture at higher levels of code organisation, too: components, services, systems, systems of systems. And they, too, have their organising principles and patterns, and need their outer feedback loops.

This is where I see a lot of teams falling short. Too little attention is paid to the emerging bigger picture. Few teams, for example, routinely visualise their components and the dependencies between them. Few teams regularly collaborate with other teams on managing the overall architecture. Few devs have a clear perspective on where their work fits in the grand scheme of things.

Buildings need carpentry and plumbing. Roads need tarmaccing. Sewers need digging. Power lines need routing.

But towns need planning. Someone needs to keep an eye on how the buildings and the roads and the sewers and the power lines fit together into a coherent whole that serves the people who live and work there.

Now, I come from a Big ArchitectureTM background. And, for all the badness that we wrought in the pre-XP days, one upside is that I'm a bit more Big Picture-aware than a lot of younger developers seem to be these days.

After focusing almost exclusively on the inner loop of software architecture for the last decade, starting in 2019 I'm going to be trying to help teams build a bit of Big Picture awareness and bring more emphasis on the outer feedback loops and associated principles, patterns and techniques.

The goal here is not to bring back the bad old days, or to ressurect the role of the Big Architect. And it's definitely not to try to reanimate the corpse of Big Design Up-Front.

This is simply about nurturing some Big Picture awareness among developers and hopefully reincorporating the outer feedback loops into today's methodologies, which we misguidedly threw out with the bathwater during the Agile Purges.

And, yes, there may even be a bit of UML. But just enough, mind you.

Posted 1 week, 6 days ago on December 2, 2018