What you could learn from ‘How Google Tests Software’ by James Whittaker (2012, 235 pages)

Posted by Max on November 17, 2017 in Book reviews | 1 Comment

Sofware is being built of scope and scale never before seen.  For example.  300 hours of video are uploaded to youtube every minute and over 50% of the world’s population is online.

Google is one of the companies that are building and maintaining software at scale.  While many of Google’s applications are not life-critical, the world certainly notices any disruption.  These disruptions are surprisingly rare, given the number of products, the quantity of code and the sheer number of developers.

How does Google maintain such good code quality on such a scale?  The book ‘How Google Tests Software’ provides many of the answers.

The book is divided into several main sections.  The first few explain the various test roles in Google and then the book explores what Google is doing to improve testing.

My key takeways from the book are.

• Developers own quality.  This is the most important point the book makes.  It is very easy for developers to ‘throw’ code at developers and then focus on the next task (this is a problem for everyone, not just developers, we all hate in-progress tasks and have a sense of satisfaction when tasks are complete).  Creating an ‘ownership’ mindset is very difficult and mean that testing changes from a step in the process, to an enabler.
• Status of testers.  Recognition and compensation need to equal developers, otherwise, there will be an unhealthy power dynamic, testers will become more frustrated and leave.
• Testing must not slow down innovation.  Testing should support the business achieving its aims, and not impede discovery and exploration.   This is very difficult, especially in high-risk or heavily regulated industries.   There are several ways to manage this:
  • Be careful when you start to involve full-time testers.  Unless you have a legal or regulatory reason, try to keep testing lightweight until you can demonstrate that a proof of concept has market traction.  This is not to say do not involve testers, but just to ensure that testing the product in the market is more important.
  • Take a risk-based approach.    Google has several simple tools which they use to prioritise code to test.  On a 2×2 matrix, they rate features for the frequency of failure and impact of failure.  The highest scoring features are focused on first, and then the teams work down the list.
• Scarcity brings clarity.  Having limited resources means that you have to prioritise work.  The process of prioritisation (if done well!) exposes all current and future work and forces you to make choices and tradeoffs.  Limited resources also mean that testers are less likely to be used for mundane tasks, which other are in a better position to do.
• Separate builds.  Google has up to five builds of their main software.  For example, Android has Dev, Test, Beta and Release.  This creates huge complexity and integration issues, which they have had to create a lot of special tooling for.  However, the advantage is that users can be involved earlier using, for example, the Canary build of Chrome.  User testing earlier massively reduces project and product risk by picking ups bugs, missing features and unnecessary features much earlier.
• Small, medium and large tests.  There are multiple definitions of tests (smoke, integration, regression, end-to-end).  Simply this to small, medium and large (and recently added enormously), make it very simple and clear.  It delineates who should be responsible for each test (dev work on small, testers on everything larger)
• Code readability.   When your developer team increase in size, it is more difficult to share knowledge.  Developers and testers will have to move teams and work on different areas of code.  Having code style guides and common easy to use tools makes collaboration easier and increase dev an tester flexibility.

While this book is helpful and provides a new and alternative view, this approach will not work for everyone.   Google have the scale and money to invest in changing their culture and build the tooling and infrastructure, most companies do not have this luxury.

From this book, you can learn how Google tests software, the challenges of changeing developer culture and an alternative approach to testing.

You can find ‘How Google Tests Software’ here on Amazon UK.