December 2, 2018
Architecture: The Belated Return of Big Picture ThinkingA question that's been stalking me is "When does architecture happen in TDD?"
I see a lot of code (a LOT of code) and if there's a trend I've noticed in recent years it's an increasing lack of - what's the word I'm looking for? - rationality in software designs as they grow.
When I watch dev teams produce working software (well, the ones who do produce software that works, at least), I find myself focusing more and more on when the design decisions get made.
In TDD, we can make design decisions during four distinct phases of the red-green-refactor cycle:
1. Planning - decisions we make before we write any code (e.g., a rough sequence diagram that realises a customer test scenario)
2. Specifying- decisions we make while we're writing a failing test (e.g., calling a function to do what you need done for the test, and then declaring it in the solution code)
3. Implementing - decisions we make when we're writing the code to pass the test (e.g., using a loop to search through a list)
4. Refactoring - decisions we make after we've passed the test according to our set of organising principles (e.g., consolidating duplicate code into a reusable method)
If you're a fan of Continuous Delivery like me, then a central goal of the way you write software is that it should be (almost) always shippable. Since 2 and 3 imply not-working code, that suggests we'd spend as little time as possible thinking about design while we're specifying and implementing. While the tests are green (1 and 4), we can consider design at our leisure.
I can break down refactoring even further, into:
4a. Thinking about refactoring
4b. Performing refactorings
Again, if your goal is always-shippable code, you'd spend as little time as possible executing each refactoring.
Put more bluntly, we should be applying the least thought into design while we're editing code.
(In my training workshops, I talk about Little Red Riding Hood and the advice her mother gave her to stay on the path and not wander off into the deep dark forest, where dangers like Big Bad Wolves lurk. Think of working code as the path, and not-working code as the deep dark forest. I encourage developers to always keep at least one foot on the path. When they step off to edit code, they need to step straight back on as quickly as possible.)
Personally - and I've roughly measured this - I make about two-thirds of design decisions during refactoring. That is, roughly 60-70% of the "things" in my code - classes, methods, fields, variables, interfaces etc - appear during refactoring:
* Extracting methods, constants and such to more clearly document what code does
* Extracting methods and classes to consolidate duplicate code
* Extracting classes to eliminate Primitive Obsession (e.g., IF statements that hinge on what is obviously an object identity represented by a literal vaue)
* Extracting and moving methods to eliminate Feature Envy in blocks of code and expressions
* Extracting methods and classes to split up units of code that have > 1 reason to change
* Exctracting methods to decompose complex conditionals
* Extracting client-specific interfaces
* Introducing parameters to make dependencies swappable
And so on and so on.
By this process, my code tends to grow and divide like cells with each new test. A complex order emerges from simple organising principles about readabililty, complexity, duplication and dependencies being applied iteratively over and over again. (This is perfectly illustrated in Joshua Kerievky's Refactoring to Patterns.)
I think of red-green-refactor as the inner loop of software architecture. And lots of developers do this. (Although, let's be honest, too many devs skimp on the refactoring.)
But there's architecture at higher levels of code organisation, too: components, services, systems, systems of systems. And they, too, have their organising principles and patterns, and need their outer feedback loops.
This is where I see a lot of teams falling short. Too little attention is paid to the emerging bigger picture. Few teams, for example, routinely visualise their components and the dependencies between them. Few teams regularly collaborate with other teams on managing the overall architecture. Few devs have a clear perspective on where their work fits in the grand scheme of things.
Buildings need carpentry and plumbing. Roads need tarmaccing. Sewers need digging. Power lines need routing.
But towns need planning. Someone needs to keep an eye on how the buildings and the roads and the sewers and the power lines fit together into a coherent whole that serves the people who live and work there.
Now, I come from a Big ArchitectureTM background. And, for all the badness that we wrought in the pre-XP days, one upside is that I'm a bit more Big Picture-aware than a lot of younger developers seem to be these days.
After focusing almost exclusively on the inner loop of software architecture for the last decade, starting in 2019 I'm going to be trying to help teams build a bit of Big Picture awareness and bring more emphasis on the outer feedback loops and associated principles, patterns and techniques.
The goal here is not to bring back the bad old days, or to ressurect the role of the Big Architect. And it's definitely not to try to reanimate the corpse of Big Design Up-Front.
This is simply about nurturing some Big Picture awareness among developers and hopefully reincorporating the outer feedback loops into today's methodologies, which we misguidedly threw out with the bathwater during the Agile Purges.
And, yes, there may even be a bit of UML. But just enough, mind you.
October 12, 2018
TDD Training - Part I (Classic TDD), London, Sat Dec 1stMy flagship Codemanship TDD training course returns in a series of 3 standalone Saturday workshops aimed at self-funding learners.
It's the exact same highly popular training we've delivered to more than 2,000 developers since 2009, with 100% hands-on learning reinforced by our jam-packed 200-page TDD course book.
Part 1 is on Saturday Dec 1st in central London, and it's amazingly good value at just £99.
Part I goes in-depth on "classic" TDD, the super-important refactoring discipline, and software design principles that you can apply to your code as it grows and evolves to keep it easy to change so you can maintain the pace of development.
- Why do TDD?
- An introduction to TDD
- Red, Green, Refactor
- The Golden Rule
- Working backwards from assertions
- Testing your tests
- One reason to fail
- Writing self-explanatory tests
- Speaking the customer's language
- Triangulating designs
- The Refactoring discipline
- Software Design Principles
- Simple Design
- Tell, Don’t Ask
The average price of a public 1-day dev training course, per person, is around £600-800. This is fine if your company is picking up the tab.
But we've learned over the years that many devs get no training paid for by their employer, so we appreciate that many of you are self-funding your professional development. Our Saturday workshops are priced to be accessible to professional developers.
In return, developers who've attended our weekend workshops have recommended us to employers and colleagues, and most of the full-price client-site training and coaching we do comes via these referrals.
Please be advised that we do not allow corporate bookings on our workshops for self-funders. Group bookings are limited to a maximum of 4 people. If you would like TDD training for your team(s), please contact me at email@example.com to discuss on-site training.
Find out more at the Eventbrite course page
October 6, 2018
Be The Code You Want To See In The WorldIt's no big secret that I'm very much from the "Just Do It" school of thought on how to apply good practices to software development. I meet teams all the time who complain that they've been forbidden to do, say, TDD by their managers. My answer is always "Next time, don't ask".
After 25 years doing this for a living, much of that devoted to mentoring teams in the developer arts , I've learned two important lessons:
1. It's very difficult to change someone's mind once it's made up. I wasted a lot of time "selling" the benefits of technical practices like unit testing and refactoring to people for whom no amount of evidence or logic was ever going to make them try it. It's one of the reasons I don't do much conference speaking these days.
2. The best strategies rely on things within our control. Indeed, strategies that rely on things beyond our control aren't really strategies at all. They're just wishful thinking.
The upshot of all this is an approach to working that has two core tenets:
1. Don't seek permission
2. Do what you can do
Easy to say, right? It does imply that, as a professional, you have control over how you work.
Here's the thing: as a professional, you have control over how you work. It's not so much a matter of getting that control, as recognising that - in reality - because you're the one writing the code, you already have that control. Your boss is very welcome to write the code themselves if they want it done their way
Of course, with great power comes great responsibility. You want control? Take control. But be sure to be acting in the best interests of your customer and other stakeholders, including the other developers on your team. Code is something you inflict on people. Do it with kindness.
And so there you have it. A mini philosophy. Don't rant and rave about how code should be done. Just do it. Be the code you want to see in the world.
Plenty of developers talk a good game, but their software tells a different story. It's often the case that the great and worthy and noble ideas you see presented in books and at conferences bear little resemblence to how their proponents really work. I've been learning, through Codemanship, that it's more effective to show teams what you do. Talk is cheap. That's why my flagship TDD workshop doesn't have any slides. Every idea is illustrated with real code, every practice is demonstrated right in front of you.
And there isn't a single practice in any Codemanship course I haven't applied many times on real software for real businesses. It's all real, and it all really works in the real world.
What typically prevents teams from applying them isn't their practicality, or how difficult they are to learn. (Although don't underestimate the learning curves.) The obstacles are normally whether they have the will to give it a proper try, and tied up in that, whether they're allowed to try it.
My advice is simple: learn to do it under the radar, in the background, under the bedsheets with a torch, and then the decision to apply it on real software in real teams for real customers will be entirely yours.
October 1, 2018
50% Off Codemanship Training for Start-ups and CharitiesOne of the most fun aspects of running a dev training company is watching start-ups I helped a few years ago go from strength to strength.
The best part is seeing how some customers are transforming their markets (I don't use the "d" word), and reaping the long-term benefits of being able to better sustain the pace of innovation through good code craft.
I want to do more to help new businesses, so I've decided that - as of today - start-ups less than 5 years old, with less than 50 employees, will be able to buy Codemanship code craft training half-price.
I'm also extending that offer to non-profits. Registered charities will also be able to buy Codemanship training for just 50% of the normal price.
September 28, 2018
Micro-cycles & Developing Your Inner Egg TimerWhen I'm coaching developers in TDD and refactoring, I find it important to stress the benefits of keeping one foot on the path of working code at all times.
I talk about Little Red Riding Hood, and how she was warned not to stray off the path into the deep dark forest. Bad things happen in the deep dark forest. Similarly, I warn devs to stay on that path of code that works - code that's shippable - and not go wandering off into the deep dark forest of code that's broken.
Of course, in practice, we can't change code without breaking it. So the real skill is in learning how to make the changes we need to make by briefly stepping off the path and stepping straight back on again.
This requires developers to build a kind of internal egg timer that nudges them when they haven't seen their tests pass for too long.
An exercise I've used to develop my internal egg timer uses a real egg timer (or the timer on my smartphone). When I'm mindfully practicing refactoring, for example, I'll set a timer to countdown for 60 seconds, and start it the moment I edit any code.
The moment a source file goes "dirty" - no longer compiles or no longer passes the tests - the countdown starts. I have to get back to passing tests before the sands run out (or the alarm goes off).
I'll do that for maybe 10-15 minutes, then I'll drop the countdown to 50 seconds and do another 10-15 minutes. Then 40 seconds. Then 30. Always trying, as best I can, to get what I need to do done and get back to passing tests before the countdown ends.
I did this every day for about 45-60 minutes for several months, and what I found at the end was that I'd grown a sort of internal countdown. Now, when I haven't seen the tests pass for a few minutes, I get a little knot in my stomach. It makes me genuinely uncomfortable.
I do a similar exercise with TDD, but the countdowns apply the moment I have a failing test. I have 60 seconds to make the test pass. Then 50. Then 40. Then 30. This encourages me to take smaller steps, in tighter micro-cycles.
If my test requires me to take too big a leap, I have to scale back or break it down to simpler steps to get me where I want to go.
The skill is in making progress with one foot firmly on the path of working code at all times. Your inner egg timer is the key.
June 10, 2018
Only This Week - Save Up To 65% On Codemanship Training
For one week only, we’re offering a veritable picnic of on-site code craft training at never-to-be repeated prices.
Save up to 65%, and train your developers in key skills like TDD, refactoring and OO design for as little as £40 per person per day. That’s full, action-packed hands-on days of code craft training.
Book any Codemanship training course before June 17th and save a whopping 50%. Book all four of our courses and save 65%. That’s a massive £12,000.
Find out more by visiting www.codemanship.com
June 8, 2018
The Entire Codemanship TDD Course Book - Absolutely FreeChanges are afoot with my code craft training and coaching company, Codemanship, and as part of that, I'm making my 222-page TDD course book available to download as a spiffy full-colour PDF for free.
It covers everything from the basics of Red-Green-Refactor, through software design principles to apply to your growing code, all the way up to advanced topics other TDD books and courses don't reach, like mutation testing, property-based and data-driven testing and Continuous Inspection. Many people who've read the book have commented on how straightforward and to-the-point it is. Shorter than most TDD/code craft books, but covers more, all in practical detail.
Of course, to get the best from the book, you should try the exercises.
Better still, try the exercises with the guy who wrote the book in the room to guide you.
April 28, 2018
8 Rules of Maintainable Code: A Handy Cut-Out-And-Keep ChartIf you've been on the Codemanship TDD training course, you may vaguely recall the first afternoon when we discuss design principles and how they can shape our code as it emerges.
I posit 8 principles that I ask participants to apply to the exercises, drawing from Simple Design, "Tell, Don't Ask" and S.O.L.I.D. These 8 factors are interrelated, and form a kind of virtuous - if somewhat complex - virtuous circle.
Code that's easier to change tends to be easier to test quickly. Fast-running tests make refactoring easier. Which helps us make our code easier to change. And around we go.
We don't do slides on the course (hoorah!), but I'm trying this morning to visualise these 8 principles and how they relate to each other in a single graphic.
There's the simple version:
And this is my latest iteration, to print off and hang on your toilet wall or put on a spiffy t-shirt. All non-profit uses are fine.
Going beyond maintainability, there's also a relationship between Clean code and reliability. Code that can be tested very quickly tends to have far fewer bugs. And code that's simpler and easier to understand is likely to get broken when we change it. So, it's more of a virtuous triangle, really.
April 6, 2018
Could Refactoring (& Refuctoring) Help Us Test Claims About Benefits of Clean CodeOne of the more frustrating things about teaching developers about code craft and "Clean Code" is the lack of credible hard evidence from respectable sources about the claimed benefits of it.
Not only does this make code craft a tougher sell to skeptics - and there was a time when I was one of them, decades ago - but it also calls into question whether the alleged benefits are real.
The biggest barrier to doing research in this area has been twofold:
1. The lack of data points. Most software engineering academic studies take data from a handful of projects. If this were, say, medical research, we'd never get our medicines on to the market.
2. The problem of comparing apples with apples. There are so many factors in software development that it's pretty much impossible to isolate one and rule out all others. Studies into the effects of adopting TDD can't account for the variations in experience and ability, for example. Teams new to TDD tend to have to deal with a steep learning curve before they become productive again.
When I consider some of the theories about what makes code harder to change - the central plank of the code craft thesis - some we have strong evidence to back them up, others... not so much.
I've had a bit of a brainwave in this area that might help researchers. Take a code base, then specifically vary it along a single dimension. e.g., refactor to remove duplication, or "refuctor" to introduce duplication (by inlining functions and modules). The resulting variants should all be functionally equivalent, but you could fine-grain the levels of variation. Then ask developers to make changes to the logic, and measure how much code had to be edited to achieve those changes. Automated acceptance tests would ensure that every change was logically equivalent.
I can easily envisage how refactoring (and it's evil twin, refuctoring) could be used to vary readability, complexity, duplication, coupling and cohesion (e.g., by moving methods between classes to introduce or eliminate feature envy), "swabbability" (e.g., by introducing dependency injection, or by reversing the dependency inversion by using explicit references to concrete implementations of interfaces) and a range of other code qualities. Automated tests could ensure that every variant still works exactly the same way on the outside.
And the tests themselves could be varied. For example, you could manipulate test suite execution time so that in some cases developers had to wait an hour for feedback, while others only need wait seconds for the same feedback.
I think I might be on to something. What do you think?
March 24, 2018
Code Craft: What Is It, And Why Do You Need It?One of my missions at the moment is to spread the word about the importance of code craft to organisations of all shapes and sizes.
The software craftsmanship (now "software crafters") movement may have left some observers with the impression that a bunch of prima donna programmers were throwing our toys out of the pram over "beautiful code".
For me, nothing could be further from the truth. It's always been clear in my mind - and I've tried to be clear when talking about craft - that it's not about "beautiful code", or about "masters and apprentices". It has always been about delivering software that works - does what end users need - and that can be easily changed to solve new problems.
I learned early on that iterating our designs was the ultimate requirements discipline. Any solution of any appreciable complexity is something we're unlikely to get right first time. That would be the proverbial "hole in one". We should expect to need multiple passes at it, each pass getting it less wrong.
Iterating software designs requires us to be able to keep changing the code over and over. If the code's difficult to change, then we get less throws of the dice. So there's a simple business truth here: the harder our code is to change, the less likely we are to deliver a good working solution. And, as times goes on, the less able we are to keep our working solution working, as the problem itself changes.
For me, code craft's about delivering the right thing in the short-to-medium term, and about sustaining the pace of innovation to keep our solution working in the long term.
The factors involved here are well-understood.
1. The longer it takes us to re-test our software, the bigger the cost of fixing anything we broke. This is supported by a mountain of evidence collected from thousands of projects over several decades. The cost of fixing bugs rises exponentially the longer they go undetected. So a comprehensive suite of good fast-running automated tests is an essential ingredient in minimising the cost of changing code. I see it being a major bottleneck for many organisations, and see the devastating effect long testing feedback loops can have on a business.
2. The harder it is to understand the code, the more likely it is we'll break it if we change it.
3. The more complex our code is, the harder it is to understand and the easier it is to break. More ways for it to be wrong, basically.
4. Duplication in our code multiplies the cost of changing common logic.
5. The more the different units* in our software depend on each other, the wider the potential impact of changing one unit on other units. (The "ripple effect").
6. When units aren't easily swappable, the impact of changing one unit can break other modules that interact with it.
* Where a "unit" could be a function, a module, a component, or a service. A unit of reusable code, essentially.
So, six key factors determine the cost of changing code:
* Test Assurance & Execution Time
* Abstraction of Dependencies
Add to these, a few other factors can make a big difference.
Firstly, the amount of "friction" in the delivery pipeline. I'd classify "friction" here as "steps in releasing or deploying working software into production that take a long time and/or have a high cost". Manually testing the software before a release would be one example of high friction. Manually deploying the executable files would be another.
The longer it takes, the more it costs and the more error-prone the delivery process is, the less often we can deliver. When we deliver less often, we're iterating more slowly. When we iterate more slowly, we're back to my "less throws of the dice" metaphor.
Frequency of releases is directly related also to the size of each release. Releasing changes in big batches has other drawbacks, too. Most importantly - because software either works as a whole or it doesn't - big releases incorporating many changes present us with an all-or-nothing choice. If change X is wrong, we now have to carefully rework that one thing with all the other changes still in place. So much easier to do a single release for change X by itself, and if it doesn't work, roll it back.
Another aside factor to consider is how easy it is to undo mistakes if necessary. If my big refactoring goes awry, can I easily get back to the last good state of the code? If a release goes pear-shaped, can we easily roll it back to a working version, with minimal disruption to our end customer?
Small releases help a lot in this respect, as does Version Control and Continuous Integration. VCS and CI is like seatbelts for programmers. It can significantly reduce lost time if we have a little accident.
So, I add:
* Small & Frequent Releases
* Frictionless Delivery Processes (build-test-deploy automation)
* Version Control
* Continuous Integration
To my working definition of "code craft".
Noted that there's more to delivering software than these things. There's requirements, there's UX, there's InfoSec, there's data management, and a heap of other considerations. Which is why I'm clear to disambiguate code craft and software development.
Organisations who depend on software need code that works and that can change and stay working. My belief is that anyone writing software for a living needs to get to grips with code craft.
As software continues to "eat the world", this need will grow. I've watched $multi-billion on their knees because their software and systems couldn't change fast enough. As the influence of code spreads into every facet of life, our ability to change code becomes more and more a limiting factor on what we can achieve.
To borrow from Peter McBreen's original book on software craftsmanship, there's a code craft imperative.