Continuous Integration is NOT the key

I’ve been reading this book for several months and finally I finished it; hurray!. It took so long because in the meantime I’ve been learning other programming languages and strategies and… well, honestly, the book was quite boring. Not only for the contents, that I’ll explain in detail below, but also because its poor style: long paragraphs, very few examples… I honestly find Robert Martins or Steve McConnells much more entertaining, because they spreadly use examples, short sentences, short paragraphs, experience stories, and they write in such a way that it’s like they talk directly to you.

Anyway, I started this book reading carefully every single line because I had really good feelings about it. By reading the summary I found lots of interesting topics related to releasing, evolving and maintaining a software product. But as time went by I started to skim pages one after another, very fast, in my last summer vacations. And, when I finished it I felt relieved.

Why do I criticize this book? Well, the issue is that the book was thirdly edited in 2010 and it’s currently obsolete. The content is pointless because it talks about Continuous Integration as the only and best key to achieve success in a project development. And, well, I have a completely different opinion about that. The thing is that when they talk about the ads of continuous integration what I see is just its weaknesses:

  • Check in frequently. How could developers checkin frequently on the mainline if it’s certainly possible to break the build and lots of fingers will point on their direction, demanding to fix the problem within minutes?? How could they even use meaningful comments in their checkins if they only dare to checkin when they’re done with their work? Most of the people will just type “task done”, as all the changes have been checked in together… if they could checkin the changes separatedly by using meaningful blocks they probably would include comments such as: “refactor in class XXX done”, “fixed a problem in the service provider”, or “New form added to enter new users”. The key is identify small tasks, but even small tasks will certainly require more than one bunch of changes.
  • If something goes wrong, stop the process until it’s fixed, or revert the changes! We were about increasing productivity, weren’t we? Then why stopping all the people until the build is fixed? That breaks the flow, increases stress and makes that the developers don’t checkin frequently, that is for sure.
  • Tests should run as fast as possible, ideally within seconds; 10 minutes at most. That’s not a real necessity, but it’s imposed to solve a Continuous Integration weakness: since the developers commit to the mainline all the time then you need to test every single changeset created. In order to achieve that you need that your tests run fast. What could you test in less than 10 minutes? Including compilation and building the installers, of course… let’s see… probably just (hopefully) well designed unit tests.

Kent Beck probably would say that you can test whatever you want just with unit tests, or that unit tests gives you a high confidence about your changes. Right, but when it’s all about complex software systems, we all know that we need to test a GUI, we have acceptance tests, non-functional requirements (such as performance), and so on, and unit tests cannot cover those issues (this is by definition; I don’t say that; Kent Beck does). So, what confidence you get about the existence of new freshly-introduced bugs in the code?

If only developers could work on isolated environments they could take the time to run as many tests as possible; this way, when the code is integrated in the mainline there’s a strongly certainty that they don’t introduce new bugs. Take into account that I’m talking about tests that should be run in a build, not local tests that are intended to be run by a developer. These last ones should be certainly fast.

I understand the main intention: keep the product releasable all the time. That is good, indeed and I agree with that. But there must be a better option, right?

The problem is that those authors that strongly defend the Continuous Integration pattern are not very open-minded to new, better approaches, Martin Fowler among them. They even state that branching is a bad solution, when they actually mean to say: merging is a nightmare if we don’t have the right tools to perform it. But things are not quite like that nowadays.

If only we had the right tools to branch our code, work on such isolated environments without disturbing the rest of the team and then merge our changes on the mainline… we wouldn’t have to worry about checkin it without introducing bugs, since the code wouldn’t need to even compile until we are done with it! We could checkin our code as frequently as we want, create sets of changes (aka changesets) that really express intention, and, assuming that intention is important in our job (writing code and tests, writing comments in our checkins), that sounds good, isn’t it?

On the other hand, you can find the book useful because it covers interesting topics related to development: testing strategies, infrastructures, some hints… but always oriented to continuous integration, so for the better please keep that in mind and forget about the continuous integration point of view.

Still, the book mentions two key concepts that I absolutely agree with: frequency and automation. It’s very important to release as frequently as possible (I do everyday) and it’s very, very important to do it as automatedly as possible, to avoid human errors and to trace where something went wrong, if that eventually happens.

There are other useful hints, too: the release process should be constantly improved, there should be a responsible to lead the whole process, aware of improving not only the process of building new releases, but also introducing new tests, new oportunities of testing and new methodologies: coverage tests, performance, cloud computing and virtualization, cyclomatic complexity and other metrics, evolving tests, mutating tests, and so forth.

To get to a conclusion,  I’ll explain in the next article what is a good solution to achieve success in a software project development by means of rapid development, high quality, control, and reduced stress, which inevitably leads to satisfaction. This is from my own experience, working for almost four years in a company that uses a different pattern (let’s say Continuous Integration redefined to the perfection ;-)) and leading the release management during the last two years (almost).

Please, don’t misunderstand me: continuous integration is not an evil; it’s better than nothing, but it’s not the best solution to take. If you are concerned about taking control on your development and getting better quality and satisfaction, I’d just keep Continuous Integration aside and first read the next article.