October 18, 2017

Learn TDD with Codemanship

12 Things a Professional Computer Programmer Needs to Learn

The last few years has seen an explosion of great learning resources for people interesting in getting into computer programming.

But alongside that, I've noticed a growing number of people, who have ambitions to work in the industry as programmers, being bamboozled into believing all it takes is a few weeks of self-paced JavaScript tutorials to reach a professional level.

Nothing could be further from the truth, though. Programming languages are just one small aspect of writing software as a professional (albeit a crucial one).

When learners ask me "What else do I need to know how to do?", I'm typically unprepared to answer. Unhelpfully, I might just say "Loads!"

Here, I'm going to attempt to structure some thoughts on this.

1. Learn to code. Well, obviously. This is your starter for 10. You need to be able to make computers do stuff to order. There's no getting around that, I'm afraid. If you want to do it for a living, you're probably best off learning programming languages that are in demand. As unhip and uncool as they may be these days, languages like Java and C# are still very much in demand. And JavaScript is at the top of the list. To become an in-demand "full-stack" software developer, you're going to need to learn several languages, including JavaScript. Research the kinds of applications you want to work on. Find out what technologies are used to create them. Those are the languages you need to learn.

2. Learn to use Version Control. Version Control Systems (VCSs) are seatbelts for programmers. If your code has a nasty accident, you want to be able to easily go back to a versin of it that worked. And most professional developers collaborate with other developers on the same source code, so to do it for a living you'll want to know how to use VCSs like Git and Mercurial to effectively manage collaborating without tripping over each other.

3. Learn to work with customers. Typically, when we're learning to code, we tackle our own projects, so - in essence - we are the customer. It gets a bit more complicated when we're creating software for someone else. We need to get a clear understanding of their requirements, and so it's important to learn some simple techniques for exploring and capturing those requirements. Look into use cases and user stories to get you started. Then learn about Specification by Example.

4. Learn to test software. There's more to making sure our code works than running the application and randomly clicking buttons. You'll need to understand how to turn requirement specifications into structured test scripts that really give the code a proper, in-depth workout. How do make sure every requirement is satisfied? How do make sure every line of code is put through its paces? How do we identify combinations of inputs that the code can't handle?

5. Learn to write automated tests. Automated tests are commonly used in Specification by Example to really nail down exactly what the customer wants. They are also crucial to maintaining our code as it grows. Without a decent set of fast-running automated tests, changing code becomes a very risky and expensive business. We're likely to break it and not find out for a long time. Learn how to write automated unit tests for your code, and how to automate other kinds of tests (like system tests that check the whole thing through the user interface or API, and integration tests that check system components work together).

6. Learn to write code that's easy to change. On average, software costs 7-10x as much to maintain over its lifetime as it did to write in the first place. And if there's one thing we've learned from 70 years of writing software, it's that it'll need to change. But, even though we call it "software" - as opposed to "hardware" - because it's easier to change than the design of, say, a printed circuit board, it can still be pretty hard to change code without breaking it. You'll need to learn what kind of things we can do in code tend to make it harder to change and easier to break, and how to avoid doing them. Learn about writing code that's easy to read. Learn about simple design. Learn how to avoid writing "spaghetti code", where the logic gets complicated and tangled. Learn how to shield modules in your code from knowing too much about each other, creating a dense web of dependencies in which even the smallest changes can have catastrophic impact. Learn how to use abstractions to make it easier to swap out different parts of the code when they need to be replaced or extended.

7. Learn to improve the code without breaking it. We call this skill "refactoring", and it's really, really important. Good programmers can write code that works. Great programmers can improve the code - to make it easier to understand and easier to change - in ways that ensure it still works. A function getting too complicated to understand? Refactor it into smaller functions. A module doing too much? Refactor it into multiple modules that do one job. This skill is very closely connected to #5 and #6. You need to know bad code when you see it, and know how to make it better. And you need to be able to re-test the code quickly to make sure you haven't broken anything. Automated Tests + Design Smarts + Refactoring form a Golden Circle for code that works today and can be easily changed tomorrow to meet new customer requirements.

8. Learn to automate donkeywork like building and deploying the software. Good software developers don't manually copy and paste files to production servers, run database scripts, and all of that repetitive stuff, when they want to create test or production builds of their systems and deploy them to a live environment. They program computers to do it. Learn how to automate builds, to do Continuous Integration, and automate your deployments, so that whole delivery process can become as easy and as frictionless as possible.

9. Learn about software architecture. Is your application a mobile app? A website? A Cloud service? Does it need huge amounts of data to be stored? Does it need to be super-secure? Will some features be used by millions of users every day? Will it have a GUI? An API? Is the data really sensitive (e.g., medical records)? We have 7 decades of knowledge - accumulated through trial and error - about how to design software and systems. We have principles for software architecture and the different qualities we might need our software to have: availability, speed, scalability, security, and many more. And there are hundreds of architectural patterns we can learn about that encapsulate much of this knowledge.

10. Learn to manage your time (and yourself). You might enjoy the occasional late night working on your own projects as a beginner, but a professional programmer's in this for lomg haul. So you need to learn to work at a sustainable pace, and to prioritise effectively so that the important stuff gets done. You need to learn what kinds of environments you work best in, and how to change your working environment to maximise your productive time. For example, I tend to work best in the morning, so I like to get an early start. And I rarely spend more than 7-8 hours in a day programming. Learn to manage your time and get the best out of yourself, and to avoid burning out. Work smarter, not harder, and pace yourself. Writing software's a marathon, not a sprint.

11. Learn to collaborate effectively. Typically, writing software is a team sport. Teams that work well together get more done. I've seen teams made up of programmers who are all individually great, but who couldn't work together. They couldn't make decisions, or reach a consensus, and stuff didn't get done because they were too busy arguing and treading on each others' toes. And I've seen teams where everyone was individually technically average, but as a single unit they absolutely shone. Arguably, this is the hardest skill, and the one that takes the longest to master. You may think code's hard. But people are way harder. Much has been written about managing software teams over the decades, but one author I highly recommend is Tom DeMarco (author of "Peopleware"). In practice, this is something you can really only learn from lots and lots of experience. And increasingly important is your ability to work well with diverse teams. The days when computer programming was predominanly a pursuit for western, white, middle class heterosexual men are thankfully changing. If you're one of those people who thinks "girls can't code", or that people from third-world countries are probably not as educated as you, or that people with disabilties probably aren't as smart, then I heartily recommend a different career.

12. Learn to learn. For computer programmers, there's 70 years of learning to catch up on, and a technology landscape that's constantly evolving. This is not a profession where you can let the grass grow under your feet. People with busy lives and limited time have to be good at making the most of their learning opportunities. If you thought you'd learned to learn at college... oh boy, are you in for shock? So many professional programmers I know said they learned more in the first 6 months doing it for a living than they did in 3-4 years of full-time study. But this is one of the reasons I love this job. It never gets boring, and there's always something more to learn. But I've had to work hard to improve how I learn over the years. So will you. Hopefully, the more you learn, the clearer the gaps that need filling will become.

So, there are my twelve things I think you need to learn to be a professional computer programmer. Agree? What would be on your list? You can tweet your thoughts to @jasongorman.






October 17, 2017

Learn TDD with Codemanship

Manual Refactoring : Convert Static Method To Instance Method

In the previous post, I demonstrated how to introduce dependency injection to make a hard-coded dependency swappable.

This relies on the method(s) our client wants to invoke being instance methods. But what if they're not? Before we can introduce dependency injection, we may need to convert a static method (or a function) to an instance method.

Consider this Ruby example. What's stopping us from stubbing video ratings is that we're getting them via a static fetchRatings() method.



Converting it to an instance method - from where we can refactor to dependency inject - is straightforward, and requires two steps.

1. Find and replace ImdbRatings.fetchRating( with ImdbRatings.new().fetchRating( whereever the static method is called.

2. Change the declaration of fetchRating() to make it an instance method. (In Ruby, static method names are preceded by self. - which strikes me as rather counterintuitive, but there you go.)



NOW RUN THE TESTS!

If fetchRating() was just a function (for those of us working in languages that support them), we'd have to do a little more.

1. Find and replace fetchRating( with ImdbRatings.new().fetchRating( wherever that function is called.

2. Surround the declaration of fetchRating() with a declaring class ImdbRatings, making it an instance method.

(AND RUN THE TESTS!)

Now, for completeness, it woul make sense to demonstrate how to convert an instance method back into a static method or function. But, you know what? I'm not going to.

When I think about refactoring, I'm thinking about solving code maintainability issues, and I can't think of a single maintainability issue that's solved by introducing non-swappable dependencies.

When folk protest "Oh, but if the method's stateless, shouldn't we make it static by default?" I'm afraid I disagree. That's kind of missing the whole point. Swappability is the key to managing dependencies, so I preserve that by default.

And anyway, I'm sure you can figure out how to do it, if you absolutely insist ;)



October 16, 2017

Learn TDD with Codemanship

Manual Refactoring : Dependency Injection



One of the most foundational object oriented design patterns is dependency injection. Yes, dependency injection is a design pattern. (Not a framework or an architectural philosophy.)

DI is how we can make dependencies easily swappable, so that a client doesn't know what specific type of object it's collaborating with.

When a dependency isn't swappable, we lose flexibility. Consider this Ruby example where we have some code that prices video rentals based on their IMDB rating, charging a premium for highly-rated titles and knocking a quid off for poorly-rated ones.



What if we wanted to write a fast-running unit test for VideoPricer? The code as it is doesn't enable this, because we can't swap the imdbRatings dependency - which always connects to the IMDB API - with a stub that pretends to.

What if we wanted to get video ratings from another source, like Rotten Tomatoes? Again, we'd have to rewrite VideoPricer every time we wanted to change the source. Allowing a choice of ratings source at runtime would be impossible.

This dependency needs to be injected so the calling code can decide what kind of ratings source to use.

This refactoring's pretty straightforward. First of all, let's introduce a field for imdbRatings and initialise it in a constructor.



NOW RUN THE TESTS!

Next, introduce a parameter for the expression ImdbRatings.new().



So the calling code decides which kind of ratings source to instantiate.



AND RUN THE TESTS!

Now, technically, this is all we need to do in a language with duck typing like Ruby to make it swappable. In a language like, say, C# or C++ we'd have to go a bit further and introduce an abstraction for the ratings source that VideoPricer would bind to.

Some, myself included, favour introducing such abstractions even in duck-typed languages to make it absolutely clear what methods a ratings source requires, and help the readability of the code.

Let's extract a superclass from ImdbRatings and make the superclass and the fetchRating() method abstract. (Okay, so in C# or Java, this would be an interface. Same thing; an abstract class with only abstract methods.)



DON'T FORGET TO RUN THE TESTS!


One variation on this is when the dependency is on a method that isn't an instance method (e.g., a static method). In the next post, we'll talk about converting between instance and static methods (and functions).



October 13, 2017

Learn TDD with Codemanship

Time for the Chief Software Developer?

A blog post by Extreme Programming co-creator Kent Beck about ageism in software development has set me thinking again about this whole issue.

As 50 looms on the horizon, I'm becoming only too aware of this predilection employers have for younger - cheaper - developers. "Do you think they might be a little old?" is a question I routinely hear from hiring managers in start-ups in particular.

The net effect of this - aside from throwing great talent and experience on the scrapheap - is we're a profession of perpetual beginners. Young people entering software development are very lucky if they're exposed to industry veterans in any practical way.

So insights about refactoring, say, that date back to the 1980s have to be rediscovered and reinvented year after year. The upshot is that most developers can't refactor code. So most code suffers from a severe lack of maintainability. By the time developers have learned to write better code, our industry dictates that they move away from the code-face. And around we go.

My solution to the "How come you're not a CTO yet?" question that inevitably comes up with someone of my vintage has been to sidestep it. As a developer, I'm getting too old and too expensive for most employers. But as a trainer and coach, 25 years of professional experience is a distinct advantage. So I code every day, and work with other developers on actual software, instead of sitting in meetings or being a PowerPoint jockey.

But sidestepping the job shouldn't be necessary. There should be a viable career path for people who want to stay hands-on. They're both necessary and valuable, especially to technology companies.

I maintain that there should be such a thing as a Chief Software Developer, who sits on the board like a CTO or CIO, but writes code on a day-to-day basis. They would be there to defend the reality of writing software from the uninformed "optimism" of bosses who think it's easy, and who just want everything yesterday, at half the price, and hang the consequences for the business.

I can think of many, many start-ups who could have avoided The Hell of A Thousand Complete Rewrites had such a person had one hand on the wheel. Indeed, I can think of a few very large companies who'd still be trading today if someone with genuine authority had said "No, let's do this right".

I'm a realist, though. I appreciate that history teaches us such authority is rarely given freely. Development teams need to take it. Perhaps one way teams could pave the way would be to identify who in their team would be a Chief Software Developer, and then just start calling them that. After a few years of many teams in many businesses doing it, the drip-drip-drip effect might just make it an officially acknowledged reality.

Who would be your Chief Software Developer?




Learn TDD with Codemanship

Manual Refactoring : Extract Superclass



Consider a situation where we have two or more classes that share a good deal of common features and implementation.



One way of removing the duplication is to extract a superclass - a common base class that puts all the commonality in one place. (A discussion about the merits or otherwise if inheritance vs. composition we'll save for another day. Just go with me for today.)

First of all, let's declare this base class.



NOW RUN THE TESTS!

Now, before we paste in the implementation from one of the other classes, we need to think about what would be pulled up into this base class and what would remain in the subclasses. i.e., What's the same and what's different in each class?

They're almost identical, except that a Dvd has a director, a Book has an author, and an Album has an artist.

We could leave the implementation of summarise() in each class. But that would mean that the bulk of those implementations would be duplicate code. Really, we just want to leave the specialised parts. So let's generalise this method and extract methods for the specialisms.

RUN THE TESTS AFTER EXTRACTING EACH METHOD!



Next, let's copy and paste the implementation of Dvd into Content.



AND RUN THE TESTS!

The plan is to specialise creator() in the subclasses, so let's make that abstract (how you do this will be language-specific.)



RUN THE TESTS!

A class that has one or more abstract methods should not be directly instantiate-able, so we need to make the class abstract, too, using the ABCMeta Python library.



RUN THE TESTS!

Next, we need to remove the specialised code about director in the constructor, leaving subclasses to add those.



TESTS!

So now we have a complete abstract base class that contains all of the duplicated code. Next, we want our 3 specialised classes to extend this base class, which will require us to remove the duplicated code from each one, and invoke the base class's constructor.

Do one subclass at a time, and RUN THE TESTS after each one.






October 12, 2017

Learn TDD with Codemanship

Manual Refactoring : Extract Class



A core principle of software design is that modules (I'll leave you to interpret that word for your own tech stack) should have a single distinct responsibility.

There are two good reasons for this: firstly we need to separate code that's likely to change at different times for different reasons, so we can make one change without touching the other code. And it provides us with much greater flexibility about how we can compose systems to do new things reusing existing modules.

Consider this simple example.



Arguably, this Python class is doing two jobs. I can easily imagine needing to change how movie ratings work independently of how movie summaries work.

To refactor this code into classes that each have a distinct single responsibility, I can apply the Extract Class refactoring.

First, we need a new class Ratings to move the ratings fields and methods to.



NOW RUN THE TESTS!

Next, paste in the features of Movie we want to move to Ratings.



AND RUN THE TESTS!

Now, we need to substitute inside Movie, delegating ratings methods to a field instance of the new class Ratings.



RUN THE TESTS!

Okay, so - technically - that's Extract Method completed. We now have two classes, each with a distinct responsibility. But I think we can clean this up a bit more.

First of all, another core principle of software design is that dependencies should be swappable. Let's introduce a parameter for ratings in Movie's constructor.



We can now vary the implementation of ratings - e.g., to mock or stub it for testing - without changing any code in Movie.

If Movie was part of a public API, we'd leave those delegate methods rate() and average_rating() on it's interface. But let's imagine that it's not. Could we cut out this middle man and have clients interact directly with Ratings?

Let's refactor the test code to speak to Ratings directly.



AND RUN THE TESTS!

Now, arguably, the first two tests belong in their own test fixture. Let's extract a new test class.



RUN THE TESTS!

Then we can remove the now unused delegate methods from Movie.



DON'T FORGET TO RUN THE TESTS!

And, to finish off, put each class (and test fixture) in its own .py file.

AND RUN THE TESTS!

This was a fairly straightforward refactoring, because the methods we wanted to move to the new class accessed a different field to the remaining methods. Sometimes, though, we need to split methods that access the same fields.



If I extract a new class for generating HTML, it will need to access the data of the Movie object it's rendering. One choice is to pass the Movie in as a parameter of to_html().



This has necessarily introduced Feature Envy in HtmlFormatter for Movie, but this may be a justifiable trade-off so that we can render movies in other kinds of formats (e.g., JSON, XML) without changing Movie. Here we trade higher coupling for greater flexibility.

In this refactored design, Movie doesn't need to know anything about HtmlFormatter.

Whether or not that's the right solution will depend on the specific context of your code, of course.



October 10, 2017

Learn TDD with Codemanship

Manual Refactoring - Summary

Due to the increasing popularity of dynamically-typed languages like Python and Ruby, as well as a growing trend for programming in stripped-down editors like Atom, Vim and VS Code that lack support for automated refactoring, I'm putting together a series of How-To blog posts for script kiddies that need to refactor their code the old-fashioned way - i.e., by hand.

The most important message is that manual refactoring requires extra discipline. (I often find when I'm refactoring by hand that things can get a bit sloppy, and I'm sure if I watched it back, the code would be broken for much longer periods of time.)

So far, I've done 12 posts covering some key refactoring basics:

Rename

Introduce Local Variable

Introduce Field

Inline Variable & Simple Method

Inline Complex Method

Introduce Parameter

Extract Method

Move Instance Method

Extract Class

Extract Superclass

Dependency Injection

Convert Static Method to Instance Method

In coming days, I'll be adding to this list, as well as putting together my definitive guide for manual refactoring, which may become an e-book, or a commemorative plate, or something like that.



Learn TDD with Codemanship

Manual Refactoring : Move Instance Method


I good deal of the refactoring I do involves shifting responsibilities between the classes in my code to try and minimise coupling between them.

When I see an example of Feature Envy - when a method of one class relies heavily on features of another class - I consider moving that method to where it arguably belongs.

Moving instance methods can be simple, or it can get complicated. Let's start with a simple example.



Here, the area() method of CarpetQuote only uses features of Room. It's only used internally within CarpetQuote, so if I moved it to Room no client code would break. Also, area() uses no features of CarpetQuote, so there'd be no need to create a dependency pointing back the other way.

To move an instance method, there needs to be an instance to move it to. This becomes the new target of any invocations. (i.e., self.area() becomes self.room.area() ).

First, I cut the area() method code and paste it into the Room class.



Then I change references to Room's getters to point at the equivalent internal fields of Room.



Then I change any invocations of area() in CarpetQuote to the new target instance, which i this case is the room field.



NOW RUN THE TESTS!

The accessors get_width() and get_length() are no longer being used, so I can inline these, moving our design into a more "Tell, Don't Ask" style.

As well as being able to move instance methods to fields, we can move them to method parameters. Imagine a slightly different version of our original example, where the Room is passed in to the area() method.



This time, when we move area() to the Room class, the target instance is the parameter value self.room. The invocation is turned around and we remove that parameter, so self.area(self.room) becomes self.room.area().



AND RUN THE TESTS!

Now let's make things a little more complicated. What if area() is called outside of CarpetQuote?

There are three potential solutions here:

1. Leave area() on CarpetQuote as a delegate to room.area()



(RUN THE TESTS!)

The advantage of this approach is when we need to preserve a public API. It leaves the client code untouched, though arguably is less clean as it creates obtuse indirection in our code. If this wasn't a public API, there might be a better way.

2. Expose the room field so the calling code can access it.



(AND RUN THE TESTS!)

Yuck. This is my least favourite solution, although it does work.

3. Switch invocation targets in the client code.



(RUN THE TESTS!)

This, in my opinion, is the cleanest solution. It requires a bit more work, but gets the client talking directly to the object it wants to talk to, and removes the unnecessary "middle man", making the code simpler.

Finally, we need to consider what to do when a method we want to move has internal dependencies.

Imagine when we calculate the area of carpet required, we need to subtract an uncarpeted border that the customer has asked for (i.e., not wall-to-wall carpet).



If we wanted to move area() to Room now, we could go about resolving the references to border in two ways.

1. Pass a reference to this instance of CarpetQuote as a parameter of area()



So Room can now resolve that reference.



(RUN THE TESTS!)

But this bi-directional dependency is less than ideal, and could create problems for us later.

2. Resolve the reference in CarpetQuote and pass the result.



(RUN THE TESTS!)

So that Room doesn't need to depend at all on CarpetQuote.






October 8, 2017

Learn TDD with Codemanship

Manual Refactoring : Extract Method



Another commonly used refactoring is Extract Method. We may wish to extract a block of code or an expression that's repeated multiple times into a shared method, for example. Or break down a long method into a composed method, or document code by using a method name to explain what it does.



In this example, I have some test set-up code that's repeated three times. I'd like to extract this into a shared method which I can parameterise to make it reusable in all three instances.

This example is fairly straightforward. I just need to cut the repeated code from the first test and replace it with a call to a new method that I've yet to declare.



Then I declare the method create_quote(), and paste in the code I cut from the test. Because the object carpet_quote is referenced further down (in the test assertion), it needs to be a return value of this new method.



NOW RUN THE TESTS!

Now I'm free to introduce parameters for price per square metre and room width and length to make it reusable in the other tests.

In a slightly more nuanced example, the code we want to extract into a method references variables that are declared outside of that code.



Here, we'll need to introduce these parameters from the start. Again, cut the code we want to extract and replace it with a method call, this time passing in those variables.



Now declare the new method - with parameters - and paste in the extracted code.



AND RUN THE TESTS!

So, the process is quite straightforward.

1. First, cut the code you want to extract into a new method. Note that this code must executable in its own right - a statement, a block of code or an expression.

2. Identify any inputs - that is, references to variables declared before the extracted code.

3. Replace with a call to the new method, complete with any required input parameters.

4. Declare the new method and paste in the extracted code. If a variable is referenced after the extracted code, it will need to be made a return value.

5. Run your tests!

You've probably figured out that this won't work if there's more than one return value, so that would be a pre-condition to the Extract Method refactoring. In those cases, we can find other ways to act on a variable without passing it back (e.g., make it a field, or pass it by reference if the language allows that).




Learn TDD with Codemanship

Manual Refactoring : Introduce Parameter


Oftentimes I find myself extracting methods to remove duplication that start with hardcoded literals that need to be parameterised to make the new method reusable.

To introduce a parameter we need to do it in a single step, before we can run the tests again.



In this simple example, I want to parameterise the newly extracted factory method create_quote so that I can reuse it on the other two tests.

First, add a parameter to the method signature for the price per square metre.



Then cut the literal value for price inside the method and replace it with a reference to the new parameter.



Now paste the literal value we replaced as a parameter value in the method call. (If the method is being called in multiple places, you'll need to repeat this everywhere it's called.)



NOW RUN THE TESTS!

Rinse and repeat for room width and length, running the tests after completing each refactoring.



For an extra flourish, let's inline the local variable carpet_quote inside that method to simplify.



AND RUN THE TESTS!

Now we can reuse this method in the other two tests.