April 11, 2018
The Foundation of a Dev Profession Should Be MentoringWhat makes something like engineering or law or medicine a "profession"? Ask me 20 years ago, I'd have said it was standards and ethics, policed by some kind of professional body and/or the law. There are certain things, say, an electronic engineer isn't supposed to do, certain things you can't ask your doctor for, certain things a lawyer would end up in jail for doing.
Ask me today, and my answer would be this: a profession is a community of people following a vocation - like writing software or teaching children - that professes how it works to people who want to learn how to do it.
Experienced school teachers help people learning to be school teachers how to teach. They pass on the benefit of their experience, including all the stuff an even more experienced teacher passed on to them.
I still very much believe that standards and ethics must be part of a profession of software development. But I'm increasingly convinced that the bedrock of any such profession would be mentoring. I think of all the time I wasted in my early years of programming, and all the things that would have helped enormously to know back then. Even programming for fun in my teenage bedroom would have been made easier with some basic code craft like unit testing and rudimentary version control.
I was very lucky to be exposed to much more experienced "software engineers" who nudged me firmly in the direction of rigorous user-centred iterative software development, mentioning books I should read, newsgroups I should visit, courses I should go on, and showing me with their day-to-day examples techniques I still apply - and teach - today.
I make it my business today to pass on the benefits of the mentoring I received.And that, to my mind, should be the basis for a profession of software development.
For that to work, though, it's necessary that developers stay developers. "Use it or lose it" has never been more true than in software. I see developers I coached 10 years ago get promoted into management roles - sheesh, I know a lot of CTOs, according to LinkedIn - and quickly lose their coding abilities and fall behind with the technology. Their experience might be invaluable to someone starting out, but it's hard to lead by example if the last programming you did was in Visual C++ 6.0 and your junior devs are working in F#.
So, another pillar of this professional foundation must necessarily be parallel career progression - up to CTO equivalent - for developers. Looking for work for the first time in a decade has left me in little doubt that - with a handful of glorious exceptions that I'm exploring - many employers don't want older (i.e., more expensive) developers, and even the most senior dev roles typically pay a lot less than management equivalents. I meet a lot of senior managers who are reluctantly in this roles because they have big mortgages and school fees to pay. They'd much rather have stayed hands-on. If the best potential mentors are disappearing into meeting rooms all day, it will always be impossible to square this circle.
The idea's been floated before - including by me - but I think it's finally time to start a software developer's guild, with a specific purpose of championing long-term mentoring and parallel career progression for devs who want to stay devs.
Who's with me?
April 6, 2018
Could Refactoring (& Refuctoring) Help Us Test Claims About Benefits of Clean CodeOne of the more frustrating things about teaching developers about code craft and "Clean Code" is the lack of credible hard evidence from respectable sources about the claimed benefits of it.
Not only does this make code craft a tougher sell to skeptics - and there was a time when I was one of them, decades ago - but it also calls into question whether the alleged benefits are real.
The biggest barrier to doing research in this area has been twofold:
1. The lack of data points. Most software engineering academic studies take data from a handful of projects. If this were, say, medical research, we'd never get our medicines on to the market.
2. The problem of comparing apples with apples. There are so many factors in software development that it's pretty much impossible to isolate one and rule out all others. Studies into the effects of adopting TDD can't account for the variations in experience and ability, for example. Teams new to TDD tend to have to deal with a steep learning curve before they become productive again.
When I consider some of the theories about what makes code harder to change - the central plank of the code craft thesis - some we have strong evidence to back them up, others... not so much.
I've had a bit of a brainwave in this area that might help researchers. Take a code base, then specifically vary it along a single dimension. e.g., refactor to remove duplication, or "refuctor" to introduce duplication (by inlining functions and modules). The resulting variants should all be functionally equivalent, but you could fine-grain the levels of variation. Then ask developers to make changes to the logic, and measure how much code had to be edited to achieve those changes. Automated acceptance tests would ensure that every change was logically equivalent.
I can easily envisage how refactoring (and it's evil twin, refuctoring) could be used to vary readability, complexity, duplication, coupling and cohesion (e.g., by moving methods between classes to introduce or eliminate feature envy), "swabbability" (e.g., by introducing dependency injection, or by reversing the dependency inversion by using explicit references to concrete implementations of interfaces) and a range of other code qualities. Automated tests could ensure that every variant still works exactly the same way on the outside.
And the tests themselves could be varied. For example, you could manipulate test suite execution time so that in some cases developers had to wait an hour for feedback, while others only need wait seconds for the same feedback.
I think I might be on to something. What do you think?
March 24, 2018
Code Craft: What Is It, And Why Do You Need It?One of my missions at the moment is to spread the word about the importance of code craft to organisations of all shapes and sizes.
The software craftsmanship (now "software crafters") movement may have left some observers with the impression that a bunch of prima donna programmers were throwing our toys out of the pram over "beautiful code".
For me, nothing could be further from the truth. It's always been clear in my mind - and I've tried to be clear when talking about craft - that it's not about "beautiful code", or about "masters and apprentices". It has always been about delivering software that works - does what end users need - and that can be easily changed to solve new problems.
I learned early on that iterating our designs was the ultimate requirements discipline. Any solution of any appreciable complexity is something we're unlikely to get right first time. That would be the proverbial "hole in one". We should expect to need multiple passes at it, each pass getting it less wrong.
Iterating software designs requires us to be able to keep changing the code over and over. If the code's difficult to change, then we get less throws of the dice. So there's a simple business truth here: the harder our code is to change, the less likely we are to deliver a good working solution. And, as times goes on, the less able we are to keep our working solution working, as the problem itself changes.
For me, code craft's about delivering the right thing in the short-to-medium term, and about sustaining the pace of innovation to keep our solution working in the long term.
The factors involved here are well-understood.
1. The longer it takes us to re-test our software, the bigger the cost of fixing anything we broke. This is supported by a mountain of evidence collected from thousands of projects over several decades. The cost of fixing bugs rises exponentially the longer they go undetected. So a comprehensive suite of good fast-running automated tests is an essential ingredient in minimising the cost of changing code. I see it being a major bottleneck for many organisations, and see the devastating effect long testing feedback loops can have on a business.
2. The harder it is to understand the code, the more likely it is we'll break it if we change it.
3. The more complex our code is, the harder it is to understand and the easier it is to break. More ways for it to be wrong, basically.
4. Duplication in our code multiplies the cost of changing common logic.
5. The more the different units* in our software depend on each other, the wider the potential impact of changing one unit on other units. (The "ripple effect").
6. When units aren't easily swappable, the impact of changing one unit can break other modules that interact with it.
* Where a "unit" could be a function, a module, a component, or a service. A unit of reusable code, essentially.
So, six key factors determine the cost of changing code:
* Test Assurance & Execution Time
* Abstraction of Dependencies
Add to these, a few other factors can make a big difference.
Firstly, the amount of "friction" in the delivery pipeline. I'd classify "friction" here as "steps in releasing or deploying working software into production that take a long time and/or have a high cost". Manually testing the software before a release would be one example of high friction. Manually deploying the executable files would be another.
The longer it takes, the more it costs and the more error-prone the delivery process is, the less often we can deliver. When we deliver less often, we're iterating more slowly. When we iterate more slowly, we're back to my "less throws of the dice" metaphor.
Frequency of releases is directly related also to the size of each release. Releasing changes in big batches has other drawbacks, too. Most importantly - because software either works as a whole or it doesn't - big releases incorporating many changes present us with an all-or-nothing choice. If change X is wrong, we now have to carefully rework that one thing with all the other changes still in place. So much easier to do a single release for change X by itself, and if it doesn't work, roll it back.
Another aside factor to consider is how easy it is to undo mistakes if necessary. If my big refactoring goes awry, can I easily get back to the last good state of the code? If a release goes pear-shaped, can we easily roll it back to a working version, with minimal disruption to our end customer?
Small releases help a lot in this respect, as does Version Control and Continuous Integration. VCS and CI is like seatbelts for programmers. It can significantly reduce lost time if we have a little accident.
So, I add:
* Small & Frequent Releases
* Frictionless Delivery Processes (build-test-deploy automation)
* Version Control
* Continuous Integration
To my working definition of "code craft".
Noted that there's more to delivering software than these things. There's requirements, there's UX, there's InfoSec, there's data management, and a heap of other considerations. Which is why I'm clear to disambiguate code craft and software development.
Organisations who depend on software need code that works and that can change and stay working. My belief is that anyone writing software for a living needs to get to grips with code craft.
As software continues to "eat the world", this need will grow. I've watched $multi-billion on their knees because their software and systems couldn't change fast enough. As the influence of code spreads into every facet of life, our ability to change code becomes more and more a limiting factor on what we can achieve.
To borrow from Peter McBreen's original book on software craftsmanship, there's a code craft imperative.
March 20, 2018
Why I Won't Take Automated "Hacker Tests" To Get Job InterviewsI'm back on the contract market - give me a shout if you're in the London area (or looking for remote-workers) and could use a very experience Java and/or C# bod - and it's been a looong time since I looked for regular work.
Much seems to be the same as it was when I was last contracting: the junior recruiters randomly filtering out candidates because their CV doesn't specifically mention that version of Spring, the dispiriting job ads that effectively say "It's a shitstorm here, but you get foosball and free breakfast!", the banks who make us wait 6 weeks for an interview date, the ever-growing lists of languages, tools and frameworks we're expected to have 500 years experience of. Yep. It's all as I left it.
But there's something new among all this. More and more of us are apparently being asked to take some kind of automated online coding test before the employer will even consider speaking to us. I was asked this week to take a hackerrank test that lasted 90 minutes. The recruiter said the client was "very positive" about my CV. But, it turns out, this step in the recruitment process was non-negotiable.
I have no problem with being asked to demonstrate technical competence. I kind of do it for a living. I code in front of other developers on training workshops, at conferences, on YouTube, and via my Github account. I'm not hiding anything. If you want to pair program with me on a problem to see the cut of my jib, I'm okay with that.
But I draw the line at these online timed tests. The focus on them is necessarily very narrow, for a start. Maths puzzles and algorithms and "stuff about programming languages". That sort of thing. Is this the new "whiteboard interview"? (Flashbacks to interviews where someone wrote some Java on a board and asked "Will that compile?" I'm sorry, I wasn't aware we'd be compiling this software in our heads.)
I think the focus has to be narrow, because there's a limit to what can be scored automatically. Basically, "this is what we know how to measure". I understand that a lot of these tests focus on algorithms. You're asked to solve a problem, and then scored on passing acceptance tests (easy to automate) and execution time (again, easy to automate).
While I agree that passing acceptance tests is kind of important, I worry about the next biggest factor being Big O-style algorithmic efficiency. Maybe my solution is slower, but easier to understand, for example. And it's sneaky, too. If there are performance criteria, we should bee told what they are up-front. I'm not in the business of making code faster than it needs to be just as a matter of course.
I also worry about the competition element of some of these tests, especially given the narrow focus. I do not rank "hackers" by their ability to create efficient algorithms alone, or by their in-depth knowledge of Java syntax. Let's measure something else to illustrate what I mean; in my Team Dojo, developers have to work together to solve a set of non-trivial problems. They also score points for passing acceptance tests. And what I've learned from watching hundreds of teams take this test is that individual technical ability is a poor predictor of team performance. Teams of coding ninjas are routinely outperformed by teams of average devs who just worked together better. It's quite inspiring to watch.
My other objection to taking these tests is the time candidates are expected to invest speculatively, just to be considered for interview. If you're on the market for work, you may be making multiple applications every week. What if they all ask you to take one of these tests, just to be considered? This creates a big overhead for candidates. If you're coming to the end of a contract, have young kids at home, or are caring for a relative, or have other time commitments, where are you going to find 90 minutes in your day just to prove that you know LINQ. Every. Single. Time. You. Apply?
I would be in favour of a website where devs can once-and-for-all demonstrate their competence in something. Not every time an employer says "dance for me!" I thought this site was called "Github", but that shows what I know.
But I'm not in favour of this cookie-cutter-one-size-fits-all approach to filtering. I guess my real gripe about being asked by a well-known Agile consultancy to take a hackerrank test is not that they asked, but that there was simply no other way of demonstrating my coding chops that they'd consider.
In discussing this with other developers, it seems as if there's a "horses for courses" situation here. Not everyone codes in their spare time, not everyone has a portfolio of stuff (e.g., on Github) they can point to. Not everyone shines when they're put on the spot. Not everyone likes to take stuff away and work alone. There's no one single way that will give every developer a chance to show us what they can really do.
Perhaps what I'm saying here is that we should let the candidate choose how they demonstrate technical competence. I might say "take a look at my screencasts" or "let me fire up Zoom.us and pair with one of your devs" or "how about I come in and run a little hands-on workshop?" Someone else might want to do the hackerrank test, perhaps because they lack job experience and need to demonstrate some raw ability, or maybe they just get nervous with new people. Someone else might want to do a - gulp - whiteboard interview because they worry they'll mess up coding in front of other people, but can demonstrate how much they've learned.
The point is that I can tell shit from shinola any of these ways. If you suck, and you have a portfolio, I'll know it from looking at that. If you suck and we pair, I'll know soon enough. If you suck and take a hackerrank test... I'll still want to see the code. But eventually, I'll know. (So might as well look at your Github.) And if you suck and we get around a whiteboard, I'll get it from that, too.
It seems to me that these automated coding tests are an attempt to remove the "it takes one to know one" element from filtering candidates. My contention is that you can't. That kind of machine intelligence is still decades way. Meanwhile, we're stuck with people assessing other people. And it helps enormously if those people know what they're looking at (and what they're looking for.) That's what needs fixing here.
There's no economy of scale in software development. Why would we believe there's economy of scale in software developer recruitment? That's the problem these online tests claim to solve, but - evidently - they haven't. They just filter out experienced candidates like the many developers I've spoken to.
So we might as well let candidates put their best foot forward and let them decide which foot that is.
Otherwise the end result is you filter out a lot of good people who'd be great additions to your team, but who just don't fit in your recruiting process.
Perhaps we need a Dev Recruitement Manifesto?
March 16, 2018
Lamenting the Golden Age of High-Integrity Software That Never CameWhen I was a much younger programmer, I read a paper that had a big impact on the way I thought about software integrity.
Up to then, I - like so many - believed that "software has bugs". It seemed inevitable. Because all the software I'd seen had bugs. And all the software I'd written had bugs. We just have to live with it, right?
And then along came this paper on a thing called Cleanroom Software Engineering, and my mind was blown.
IBM wrote a COBOL pre-compiler that had about 85,000 lines of code and zero bugs reported in production. Not one. Ever. And what really struck me is that - bearing in mind how primitive dev tools were in the 1980s - it only took a team of six, achieving an average dev productivity that was measurably higher than the industry average. Also, the cost of maintaining the product - typically a lot higher than the cost of initial development - was relatively low; just one developer-year per year. Because nobody was bug fixing.
Now, of course, compared to software today 85 KLOC isn't much. But it's not insignificant, statistically. Maybe an equivalent product today would have 20x as much code. But what's 20x zero?
A single paper turned my whole worldview about software integrity (vs. productivity) upside-down. I've been lucky enough to experience this kind of approach - not specifically Cleanroom, but along similar lines - since, and seen the results for myself. Seeing is believing, and - praise Knuth! - I'm a believer!
So you can probably imagine my frustration to see how, 20 years later, the "software has bugs" paradigm still dominates. Who out there is producing very high-integrity code? Vanishingly few. I've waited and waited for high-integrity development techniques to catch on. I've even stirred the pot a few times myself with attempts at training products and talks with various publishers about a book that updates the ideas for the hipster Agile generation. To no avail. Still, vanishingly few are interested.
It's not as if there isn't a compelling business case. More reliable code, for little to no extra cost (you might even save time and money)? Lower maintenance costs? Happier customers? A world of digital stuff we can rely on? What's not to like? It's not as if these techniques are incompatible with Agile, either. I've done both at the same time, for real.
But for every person like me out there selling the dream, there are 10 more actively briefing against it. "Quick and dirty". "Move fast and break stuff". "Perfection is the enemy of good enough." Etc etc etc.
It's an easy sell to managers who don't understand the relationship between quality, time and cost. Cut some corners, get there sooner, save some money. A much harder proposition is "take more care, get there sooner, save some money". Bosses don't believe it. Heck, most devs don't believe it, despite the mountain of strong evidence to back it up.
I still live in hope that - one day - high-integrity software will go mainstream. The tools and techniques are not, despite what you may have heard, rocket science. Most devs are smart, and most devs could learn to do this. I did, so it can't be that difficult.
March 11, 2018
Proposing The xUnit "Meta-Kata"A while back I ruminated on refactoring old-fashioned "test it all with a main method" test code (like we did back in the day) into the xUnit unit test framework pattern.
It occurred to me that this might make a good code kata. TDD a well-known kata (e.g., FizzBuzz, Bowling Game, "Rock, Paper, Scissors"), but starting without a unit testing framework doing it all in a single main method.
As the code evolves, refactor the test code to remove code smells like methods testing more than one thing, classes testing more than one "unit" or "feature" (depending on how you roll), high-level modules that depend directly on low-level test fixtures, multiple tests being different examples of the same test, and so on.
It could be an interesting exercise in discovering frameworks. Perhaps it'll take multiple katas for a complete xUnit framework to reveal itself, just as so many great and useful frameworks don't really take shape until they've been reused a few times on other problems.
It might also be an exercise in applying the TDD discipline on two problems simultaneously; a test of your Craft Fu.
Really, it would be a kata within a kata: a meta-kata, if you like. And as such, I think it could be really interesting and rather challenging. I'll hopefully be giving it a go - well, probably a few goes - when I pair with my "apprentice" Will Price soon. When I think we've cracked it, I'll post a screencast and the code (with version history, so you can play it back).
March 9, 2018
S.O.L.I.D. C# - Online Training Event, Sat Apr 14thDetails of another upcoming 1-day live interactive training workshop for C# developers looking to take their design skills to the next level.
I'll be helping you get to grips with S.O.L.I.D. and much more besides with practical hands-on tutorials and exercises.
Places are limited. You can find out more and grab your place at https://www.eventbrite.co.uk/e/solid-c-tickets-44018827498
March 7, 2018
C# Refactoring with Resharper - Live Online Training Workshop, March 17thJust a quick note to plug a live online training event that's happening on Saturday March 17th. If you're a C# developer wants to get to grips with the refactoring discipline and the Resharper plug-in for Visual Studio, this is the perfect kick start.
Refactoring's an essential skill for a code crafter, but still something a lot of even quite experienced developers have yet to wrap their heads around. Take your code craft to the next level.
There's a morning and afternoon session, each lasting 3 hours. Places are limited to 8 in each session, to ensure you get more attention from your trainer (me) while you tackle the exercises.
Find out more and book your place at https://www.eventbrite.co.uk/e/c-refactoring-with-resharper-tickets-43923282721
February 4, 2018
Don't Bake In Yesterday's Business Model With Unmaintainable CodeI'm running a little poll on the Codemanship Twitter account asking whether code craft skills should be something every professional developer should have.
Every professional software developer should be able to write good unit tests, to use version control & do Continuous Integration, to apply design principles and to refactor code safely. Do you...— Codemanship (@codemanship) February 3, 2018
I've always seen these skills as foundational for a career as a developer. Once we've learned to write code that kind of works, the next step in our learning should be to develop the skills needed to write reliable and maintainable code. The responses so far suggest that about 95% of us agree (more than 70% of us strongly).
Some enlightened employers recognise the need for these skills, and address the lack of them when taking on new graduates. Those new hires are the lucky ones, though. Most employers offer no training in unit testing, TDD, refactoring, Continuous Integration or design principles at all. They also often have nobody more experienced who could mentor developers in those things. It's still sadly very much the case that many software developers go through their careers without ever being exposed to code craft.
This translates into a majority of code being less reliable and less maintainable, which has a knock-on effect in the wider economy caused by the dramatically higher cost of changing that code. It's not the actual £ cost that has the impact, of course. It's the "drag factor" that hard-to-change code has on the pace of innovation. Bosses routinely cite IT as being a major factor in impeding progress. I'm sure we can all think of businesses that were held back by their inability to change their software and their systems.
For all our talk of "business agility", only a small percentage of organisations come anywhere close. It's not because they haven't bought into the idea of being agile. The management magazines are now full of chatter about agility. No shortage of companies that aspire to be more responsive to change. They just can't respond fast enough when things change. The code that helped them scale up their operations simultaneously bakes in a status quo, making it much harder to evolve the way they do business. Software giveth, and software taketh away. I see many businesses now achieving ever greater efficiencies at doing things the way they needed to be done 5, 10 or 20 years ago, but unable to adapt to the way things are today and might be tomorrow.
I see this is finance, in retail, in media, in telecoms, in law, in all manner of private sector organisations. And I see it in the public sector, too. "IT delays" is increasingly the reason why government policies are massively delayed or fail to be rolled out altogether. It's a pincer movement: we can't do X at the scale we need to without code, and we can't change the code to do X+1 for a rapidly changing business landscape.
I've always maintained that code craft is a business imperative. I might even go as far as to say a societal imperative, as software seeps into every nook and cranny of our lives. If we don't address issues like how easy to change our code is, we risk baking in the past, relying on inflexible and unreliable systems that are as anachronistic to the way things need to be in the future as our tired old and no-longer-fit-for-purpose systems of governance. An even bigger risk is that other countries will steal a march on us, in much the same way that more agile tech start-ups can steam ahead of established market players simply because they're not dragging millions of lines of legacy code behind them.
While the fashion today is for "digital transformations", encoding all our core operations in software, we must be mindful that legacy code = legacy business model.
So what is your company doing to improve their code craft?
February 1, 2018
BDD & Specification By Example - Where Did We Go Wrong?I've been saving this post up for a while, but with a bit of pre-dinner free time I wanted to put it out there now.
I meet a lot of teams, and one thing many of them tell me is that the "customer tests" they've been driving their designs from are actually written by the developers, not the customer.
DEV TEAMS who do BDD/ATDD: who writes your Cucumber/FitNesse/RSpec etc tests?— Codemanship (@codemanship) July 18, 2016
Sure, they're written using a "Behaviour-Driven Development" or "Acceptance Testing" tool like Cucumber or Fitnesse. But just because you've built a "granny annex" on your house, if there's no granny living in it, it's just an "annex".
We've dropped the ball on this. The CHAOS report, published every year by the Standish Group, consistently cites lack of customer involvement as the number one factor in project failure. A tool won't fix that.
Especially when that tool wasn't designed with customer collaboration in mind. When your "Getting Started" guide begins "First, install Visual Studio..." or requires your customer to learn a mark-up language or to use version control, arguably you're bound to have a hard time getting them to engage in the process.
Increasingly, I work with teams who want to somehow connect the way their customer actually prefers to capture examples with the way devs like to automate tests. 90% of the time, that means pulling data out of Excel spreadsheets - still the most widely used tool in both communities - into unit tests. Some unit testing frameworks even have that facility built in (e.g., MSTest for .NET). But reading data from spreadsheets is child's play for most developers. With OLD DB or JDBC, for example, a spreadsheet's just a database.
But, regardless of the tools, the problem most teams need to solve is a people problem. I've found that close customer involvement is so critical to the chances of a team succeeding at solving the customer's problems that I actually stop development until they engage at the level we need them to. No play? No code.
The mistake many of us make is to give them a choice. "Would you like to spend a lot of time with us discussing requirements and playing with candidate releases and giving us feedback?" "No thanks, ta very much. See you in a year's time."
We made a rod for our backs by allowing them to be absentee partners and trying to figure out what they want and need for them. Specification By Example presents us with an opportunity to make the relationship clearer. The customer has to be "trained" to understand that if they haven't agreed a test for it, they ain't gonna get it.