The spying game: when has espionage changed the course of history?

Espionage, Christopher Andrew reminds us, is the second oldest profession. The two converged when Moses’s successor Joshua sent a couple of agents to spy out ancient Jericho. There they were sheltered by the madam of the local brothel. All three are heroes in Israel today.

Generals and politicians have always needed secret information to track and outmanoeuvre their foreign and domestic enemies. So they place spies, suborn traitors, eavesdrop, decipher other people’s messages, subvert their governments, assassinate their servants and sabotage their property. The technology has changed massively over the centuries; the aims and the basic methods have not.

During the 20th century, thanks partly to the works of talented British novelists, ‘secret intelligence’ acquired a mystique among the public. Intelligence agencies found that flattering, even useful. But they still had to keep their operations secret. The British government went further. Until the late 1980s it maintained the absurd fiction that it had no secret service at all.

Christopher Andrew is one of our most distinguished and prolific intelligence historians. He believes that the historical role of intelligence is still insufficiently understood: the professionals make mistakes because they forget the achievements of their predecessors; historians fail to pay sufficient attention to the influence of intelligence on events. The Secret World is an ambitious under-taking, intended to restore what Andrew calls ‘the lost history of global intelligence’ and to demonstrate ‘the continued relevance of long-term experience to intelligence operations in the 21st century’.

He starts with the ancient Israelites, the Indians and the Chinese, to whom he attributes formidable intelligence skills, despite the skimpy evidence. By contrast, he believes, the Greeks and Romans foolishly relied on oracles rather than intelligence. Here he is too hard. The historian Thucydides criticises his friend Nicias for losing his army and his life because he was ‘too much inclined to divination and the like’. But he also shows that the Greeks regularly and successfully deployed spies, intercepted messages and brought down cities by subversion. The intoxicated ravings of the priestess were not the only attraction of the oracle at Delphi — which was also the Cold War Berlin of the ancient world, an intelligence bazaar where hard men gathered to bargain information.

Andrew’s narrative thickens when he gets to the better documented Renaissance. Venice, the Vatican and Spain all had effective networks of spies. Elizabeth I’s ruthless spymaster Francis Walsingham successfully brought down Mary Queen of Scots. Wellington ran his own highly successful intelligence networks, though even he was humbugged by Napoleon on the eve of Waterloo.

The story gets into its full stride with the 20th century. All the participants in both world wars and in the Cold War which followed had their intelligence successes and failures. The British would have been hard put to win the Battle of the Atlantic had they not broken the German codes. They read the German military signals before the battle of Crete too; but they still managed to snatch defeat from victory. The Germans deciphered British plans for the desert war almost in real time; but the British nevertheless chased them out of Africa.

As for the Russians, they were always good at spying. But they were sometimes too clever by half. They regularly broke foreign cyphers and penetrated foreign governments. Tsar Nicholas II and Stalin read British telegrams with pleasure and profit. But Nicholas’s secret policemen set up puppet trades unions which, alas, promoted rather than curbed revolutionary unrest. They recruited Yevno Azef to spy on terrorists. He betrayed the terrorists to the police; helped the terrorists assassinate senior officials; evaded the vengeance of both and died peacefully in his bed. Stalin’s purges of alleged spies destroyed some of his country’s most talented people. The Soviet operation against the Americans’ nuclear programme, the Manhattan Project, was one of the most successful in history; but the intelligence it produced was helpful rather than decisive in the Russians’ development of their own bomb.

Andrew’s story is full of intriguing facts and pleasing anecdotes, though sometimes burdened with confusing detail. But he does not quite engage with the broader questions he himself raises. How much influence did intelligence really have on the course of history? How much do you distort the historical record if you omit the secret world? What are the perennial roots of intelligence failure?

There are no good answers to the first two questions. Good intelligence is often essential, not least to military victory. But some intelligence is inaccurate, untimely or irrelevant. How far it helped keep the peace during the Cold War, as Andrew claims, is debatable. You have to judge rigorously in every case. It is not easy; but there is no longer much excuse for not trying. Until the last decades of the 20th century even official historians were allowed only restricted access to the documents of intelligence agencies. Knowledge of British success in breaking German codes became available to ordinary historians only in the 1970s. Things are better now. The agencies will always need to preserve their operational secrets, but thanks to Andrew and others, they — even the Russians — are more willing to publish selected documents. Intelligence studies have at last become a respectable academic discipline.

How useful history can be to policy-makers is a question that goes well beyond the secret world. A knowledge of history will not enable you to predict the future. But it can tell you what traps your predecessors fell into; what has been tried in the past and what didn’t work. There are lessons to be drawn, but policymakers and practitioners regularly ignore them. Most are not stupid — some even have degrees in history — but they are easily seduced by the argument that ‘this time it will be different’. Above all, they rarely have time for reflection; they are always under the pressure for what Churchill called ‘action this day’.

‘Knowing your enemy’ — the object of all intelligence — means more than just piling up facts about him. You need to understand his fears, ambitions and intentions, and a knowledge of history can help you do that. But political and military leaders are tempted to believe whatever bolsters their preconceptions. Stalin’s spies gave him ample warning of the impending German attack in June 1941. It was not what he wanted to hear. He refused to allow his army to prepare, and it was nearly destroyed when the Germans invaded.

It’s called confirmation bias, and it’s endemic in all human endeavour. Intelligence analysts try to aim off it, but the pressures are often too great. Ronald Reagan suffered from it too. He came to power convinced that everything the Russians did was evil. But he had the gift of empathy. He realised that, incredible as he found it, the Russians really did fear an unprovoked American attack. If that was paranoia, as Andrew suggests, there was plenty of it on both sides. Leaked American strategic documents, excitable policy discussion in Washington, deliberate American provocations around the Soviet border: all gave the Russians plausible cause for alarm. Reagan was equally disconcerted by the nuclear war plans of his own military;he called them ‘crazy’. But, unlike Stalin, he saw the real danger and changed his mind. He reached out to Mikhail Gorbachev. Between them, they brought the nuclear confrontation to an end.

And there is another question. Why do governments abandon capabilities that they have already found useful? Part of the answer is that an effective intelligence organisation requires the coordination and continuity of records, methodologies and networks. As long as it depends on brilliant individuals, such as Walsingham, it is vulnerable: the man changes; the organisation decays. So you need a permanent professional bureaucracy. But these did not emerge, in Europe at least, much before the 18th century.

That still does not explain why the Americans, for example, ran down their infant intelligence agencies after the first world war. Money had something to do with it. But Henry L. Stimson, the secretary of state, is said to have found spying ‘ungentlemanly’. That is a frivolous objection, which does not answer governments’ need for reliable intelligence. Ethical issues do, however, arise. In 1844 the British closed down their highly successful decyphering branch following a passionate public debate about the conflict between the needs of national security and the rights of true-born Englishmen.

The debate is legitimate, and it continues today, as we try to find the balance between preserving our democratic freedoms and letting the government use sophisticated systems of electronic surveillance to protect us from terrorists.

Despite its length, The Secret World does not adequately tackle such matters of interpretation and judgment. It is not the book one hoped for: a chronicle rather than a critical history. That is a pity: the subject is important and perennially fascinating. It awaits a more thoughtful treatment.