Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Spark the Change Winner - Mission-critical: a case study of GCHQ’s culture of innovation

Spark the Change Winner - Mission-critical: a case study of GCHQ’s culture of innovation

In July 2014, Spark the Change offered an Award for organisations trying to change the way they worked. We were looking for creative innovation and responsiveness, and a commitment to building happier workplaces. From 67 applicants, the prize was awarded to GCHQ. The acronym stands for Government Communications Head Quarters and it is one of the UK’s three intelligence services (MI5 and MI6 are the other two). In this article, Spark judge and writer, Helen Walton, interviews the Intelligence Agency to hear ‘non-classified’ details of an exceptional innovation culture.

Inside the Ring of Secrecy

GCHQ’s famous building in Cheltenham is affectionately known to both insiders and outsiders as ‘the doughnut’.

An intelligence agency obviously has to be careful about sensitive information. And so, when I arrived to interview several GCHQ staff, I was only permitted into the outermost part of the ring. I was not able to walk around the workspace and see the kudos wall, doodle tables, collaborative spaces or innovation boards for myself. Nor was I given the last names of the GCHQ employees to whom I spoke (part of keeping their staff safe). And yet – sitting with seven staff members from across the organisation and hearing about the many ways GCHQ was committed to innovation in its work, management structures and governance was a compelling confirmation of why GCHQ had won the Spark Award.

For an organisation whose mission requires it keeps much of its capabilities secret, the agreement to apply for and take full part in the Spark Award – indicates an openness which may surprise those who know of GCHQ only from TV dramas or recent press coverage.

GCHQ staff commemorate the First World War, creating a huge red poppy in the iconic 'doughnut' building with the Royal British Legion. ©GCHQ 2014

The best thinking requires diversity

Popular culture has a rather schizophrenic tendency to oscillate between painting GCHQ as filled with omniscient, near-omnipotent Spooks and a collection of out-dated Oxbridge bureaucrats who cannot hope to compete with companies such as Google or Microsoft. The GCHQ staff I spoke to were too polite to say so, but I got the impression that they found the latter stereotype particularly galling.

In fact, of course, GCHQ has a distinguished history as a home for the brightest minds and eccentric innovators, no matter what their backgrounds. Alan Turing may have been a Cambridge mathematician, but the genius behind the first computer was Tommy Flowers, a Post-Office engineer who earned his degree at night classes. During World War 2, 75% of Bletchley Park’s workforce was female, and women mathematicians and linguists continued to work as codebreakers after the war.

Today GCHQ has an impressive focus on recruiting women and has a target to improve women’s representation at the most senior levels (at 25%, while still not equal, it is doing better than the FTSE 250 figure of 13%). GCHQ also makes significant efforts to recruit those from differing backgrounds, offering between 40-80 apprenticeships to applicants without degrees and running an impressive schools programme encouraging STEM study. GCHQ also encourages a ‘neuro-diverse’ workforce – recognising the special talents those with dyslexia or dyspraxia may bring.

The reason is essentially pragmatic. Tom, an innovation lead who works in counter-terrorism, explained: ‘given the diversity of the threats that we face, we need an equal diversity of views to fight them. You don’t get ground-breaking thinking if everyone has the same background or outlook.’

Exponential Challenge – exponential creativity

‘There was a time,’ Tim the Deputy CTO explained, ‘when the organisation was simply being snowed under by the demands of our operations. Terrorist threats were proliferating and there were military operations that we needed to support. At the same time, technology was changing so fast that keeping up was a challenge – the investment needed in major ICT work was increasing all the time. We were working all out, all the time, and we were feeling the strain.’

For most organisations, the imperatives of delivery would have meant shutting down on any work not immediately operation-focused. Interestingly, GCHQ took the opposite approach. They launched a ‘time to innovate’ programme which meant not just those in dedicated research teams - could spend up to 10% of their working time on ‘innovation’ projects.

‘Creativity and innovation are not incidental to what we do,’ Tim went on. ‘They are part of the solution. They are not something we have to fit around day-to-day work because they are part of how we stop being so busy. They are not an extra expense – they are how we deliver better value for taxpayers.’

An Innovation Culture: formal and informal

All the GCHQ staff I spoke to were clear that the Time for Innovation programme was not a magic fix in itself. Competing priorities continued to create pressure, while obstacles could range from a struggle to create the right environment to internal resistance to change. While successes helped build support and attracted more resources and a sense of confidence, they experienced failures as well.

GCHQ has followed Agile methodologies for many years and so the concept of failing fast and failing cheaply was well embedded. But they also worked to create a culture that was supportive of failure. ‘The idea,’ Tim said, ‘was to change frustration into energy; to move from “why can’t I do x” to “I’m going to do x”.

Reid, one of the original Innovation Leaders, explained further: ‘We set up innovation boards where people could suggest ideas and attract others to spend their 10% innovation time collaborating. We have innovation leads who mentor and encourage individuals and teams – helping them gain access to resources or establishing lessons from failure. We even wrote an internal Innovation Guide with hints and tips to help keep innovating.’

Part of this involved thinking about how people communicated and collaborated together. Worried by how often ideas got shot down at an early stage, one of the developers, Lambert, handed everyone on the team a little toy alligator. ‘We all have a “gater brain”’ he explained, ‘the flight or fight response that is triggered whenever we come across something new and which makes us dismiss or attack ideas. We needed to “tame the gater”. Now, if someone starts knocking an idea down rather than building on it – we chuck a toy ‘gater across the room. It’s fun, but it’s a serious reminder about how we communicate.’

A more formal programme was the ‘Data Science Lab’, led by Reid and run twice a year as a 6-week programme. Up to 30 people take 6 weeks out of their normal work to collaborate on a project, split into 3 or 4 smaller teams. While the organisers set a ‘theme’ connected to immediate and long-term challenges, the individuals have a large say in shaping the work, deciding who they will work with and appointing any team leaders.

The Labs have no set hierarchy (although they still adhere to necessary governance) and they bring people together from across the organisation. They have proved themselves not only in the software delivered, but also in broader objectives, helping to break down silos and foster further collaboration. ‘People develop personally as well as professionally over the 6 weeks,’ Reid commented enthusiastically. ‘It’s a chance to devote all of your skills to doing something that really matters – and that helps re-connect us to why we work at GCHQ rather than anywhere else.’

Autonomy and Motivation – Spiderman style

Letting people try out ideas is fundamental both to their motivation and the GCHQ innovation strategy. Lambert explained how he’d decided that his team needed to reconfigure the workspace to improve their collaboration. He received encouragement – not only to go ahead and make the change – but to spread that change across the whole office floor. His original vision that he drew up is shown here:

(Click on the image to enlarge it)

A visual sketch of the ideal workspace by a GCHQ Innovation Lead. ©GCHQ 2014

The final plan, developed in a 2-day off-site event called ‘Engineering Tomorrow’s World’, involved finding ways to accommodate the specific needs of a leading-edge innovation department, what Lambert calls, the Five Cs:

  • co-location – a space to bring everyone together including special desk configurations and shared whiteboard ‘doodle’ tables
  • concentration – time or places for people to be quiet without interruptions to avoid breaking flow
  • collaboration – consider end-to-end flow to connect analysts, developers and others.
  • correct IT – tailor the security system to cope with changing requirements, including internet or mobile access, personalisation and new hardware and software opportunities
  • community – a Hub to draw together all of a community - ‘software devs’, for example - and encourage serendipitous exchanges.

‘The point,’ Lambert added, ‘is that I was a developer. The fact that I could be encouraged to run with an idea of changing the whole office workspace just blew my mind. How many companies would have offered me that opportunity?’

Reid jumped in with another example. ‘A recent graduate suggested an overnight codejam. He was given full autonomy – that meant finding and talking to people from facilities, security, HR, IT services in order to make the event work. Those are relationships that will stand him in really good stead in his career.’ Reid grinned: ‘we’re just like spiderman – with great power, comes great responsibility.’

The most striking example of this is an intrapreneurship scheme that GCHQ have labelled their ‘kickstarter’. By delegating budgets to much more junior levels than normal, the organisation is now letting those who have a particular problem or interest invest in ideas that might solve it. Anyone can add an idea seeking funding. Not only does this broaden and balance the innovation portfolio, but it offers a real sense of ownership and responsibility. As Tim said ‘for the first time we’re giving the same people who have the problem the money and power to solve it’.

It’s an adaptation that offers the public sector the benefits seen by some of the most innovative companies in the world – including 3M and HP. There’s no question that it remains a radical practice – but GCHQ has plans to expand the idea as they evaluate its success.

Overcoming Obstacles – achieving an external focus

GCHQ is deeply invested in harnessing the creativity of its employees, but is also well aware of the need to look externally as well.

‘We can’t solve everything by ourselves,’ Tim admitted cheerfully. ‘We’ve always worked with a few – fully security cleared – private companies, but we know there is excellent innovation coming out of SMEs and start-ups. We want to work with far more diverse companies and academics – but how do we do it without creating a security risk?’

‘And we want to help contribute to open-source software projects and give back to the development community in a broader sense,’ Lambert put in. ‘We’d like to be able to work in other places or experience external codejams. We need to test out new types of technology. But – of course – we are very aware of potential risks.’

Rather than dismissing the problem as ‘too hard’ (something many corporations have done), GCHQ instituted a series of controlled experiments.

One of the most unexpected was to launch a public call for bids. Initially run jointly with MI5, it invited companies and start-ups to bid on several broad problems – including how to verify people’s online identity; how to use open-source data to predict events and how to work securely in an insecure environment. As an experiment it has proved remarkably successful, bringing in a raft of new ideas and new relationships.

A key obstacle experienced by many organisations is the need to balance security with experimentation and collaboration. GCHQ has made big strides in being able to create an internal technology landscape that lets employees use new technologies with confidence. They were unable to share the details, but unlike many commercial organisations I have dealt with, still tied to desk-tops and proprietary systems, the team offered heartfelt praise to the IT support functions. ‘IT Services really rose to the challenge,’ Lambert said.

The real mission

Perhaps it’s unsurprising that an organisation needs to be innovative in managing a workforce largely made up of brilliant and diverse people. Rather endearingly, Tom remarked: ‘Part of what’s so amazing about working here is who you work with. I can’t remember the last time I was the brightest person in the room’.

Quite apart from the sense of history, people, experiments and innovations that characterised it, I had a strong sense that GCHQ’s culture was more than just a sum of its parts. From choosing between competing priorities to reminding themselves of why they worked here rather than somewhere offering higher pay, each of the 5 main speakers referred frequently to the organisation’s mission and how it underpinned their actions: keeping the UK safe.

Many readers in the UK may be taking part in the important debate about exactly how GCHQ goes about doing that – but no-one could deny that ‘keeping the UK safe’ offers a compelling and meaningful purpose that few corporations could match.

Naturally I had been impressed by GCHQ’s winning submission to the Spark Award. But I was even more impressed by my visit. The organisation’s commitment and energy was striking. Not only were they trying out radical management innovations and offering genuine autonomy to workers, but they seemed to have no concept of ‘good enough’, only of ‘better’.

I had always felt it a shame that recognition of the brilliance of the GCHQ engineers, Ellis, Cocks and Williamson, who had discovered public key cryptography in the 1970s, had been so long delayed. As I left the ‘ring of secrecy’, I had a similar feeling over GCHQ’s innovations in management and employee engagement. Their experiments and cultural attitude deserve to be more widely known.

For those who wish to hear about the GCHQ innovation programme directly, pre-book a ticket to Spark 2015 which will feature a talk from some of those leading the innovation programme at GCHQ.

About the Author

Helen Walton is co-founder of Gamevy, an employee-owned tech start-up with no bosses. They also run Spark the Change - a conference designed to help other companies consider management innovation and radical methods of working. Sparkthe Change runs in London and Toronto - follow @SparkConf.  Helen is a marketer who has worked on brands in make-up, skincare, fine art publishing and financial services. She is also a professional writer with eclectic interests, meaning she has authored several books on Agile Software Management, as well as puppet and radio plays... She's always happy to debate these topics or anything else on twitter @helenislovely.

Rate this Article