He is maligned by Moscow and enmeshed in scandal in London, but when Boris Johnson visited Kyiv this month, Ukraine’s president Volodymyr Zelensky could barely restrain his enthusiasm for the UK prime minister.

With Britain playing a key role in the western response to Russia’s invasion of Ukraine, Zelensky thanked Johnson for “your leadership, your assistance and the weapons, and for the very clear and specific position of your wonderful and powerful country”.

The contrast with Johnson’s domestic troubles is stark. This month he became Britain’s first serving prime minister to be sanctioned for breaking the law over the so-called partygate scandal. Two years ago, in another scandal, he made his billionaire friend Evgeny Lebedev, the son of a former KGB agent, a member of the House of Lords.

How the UK became a Russia hawk is not just a story about a beleaguered prime minister taking up a noble cause. It is also about how long it took Britain’s military and intelligence services, who were among the first to understand Russian president Vladimir Putin’s dark ambitions, to convince UK politicians to end their gladhanding of the Kremlin.

That repositioning took nearly a decade, according to more than half a dozen serving and former UK security officials. It happens to now mesh with post-Brexit Britain’s need to reset relations with the EU.

Johnson’s arms-to-Ukraine policy also represents a sea change from the 2000s when the then prime minister Tony Blair courted the Russian president, the Queen hosted Putin at Buckingham Palace and western nations believed the fall of the Berlin Wall had drawn a line under the cold war.

“There was genuine hope. Many people here wanted the end of history too,” said Alex Younger, former head of Britain’s overseas intelligence service MI6, in a reference to the best-selling 1992 book by US political scientist Francis Fukuyama, which argued that liberal democracy was the final form of government for all nations.

At the turn of the century, the Blair government was focused on combating terrorism rather than the threat posed by potentially hostile states such as Russia. UK troops were fighting al-Qaeda and the Taliban in Afghanistan following the September 11 terror attacks.

The Foreign Office was meanwhile concentrated on a “prosperity agenda” that emphasised trade promotion and foreign investment, including the money that Russian oligarchs and companies channelled through London.

“By buying and trading Russian stocks and bonds, the City of London helped Putin rearm and modernise his army,” said a senior UK defence adviser.

Even the Ministry of Defence was thinking about other things. “The general attitude about Russia was ‘the cold war is over, we won, move on’,” said Carl Scott, UK defence attaché in Moscow between 2011 and 2016.

“There was no coherent constituency that saw Russia as a threat. There was a lot of inertia, a lot of vested interests — and a lot of Russian money in London.”

One indicator of UK priorities was how little time the country’s intelligence agencies spent on Russia. In 2009, MI5, the domestic security service, devoted only 3 per cent of its resources to state-related threats, versus one-fifth in 2000, according to parliament’s intelligence and security committee.

GCHQ, which provides signals intelligence, devoted 4 per cent on former Soviet bloc countries in 2006, versus 70 per cent in the cold war. MI6 activity on Russia had also declined.

“There had been a real hollowing out of British expertise on Russia, because Russia wasn’t seen as a problem,” said the UK defence adviser. “British politicians also didn’t realise how different their Russian counterparts were — they’re intelligence officials not politicians as the west understands them.”

The UK began to reassess its attitude in 2014 after Russia annexed Crimea from Ukraine and embarked on military operations in Syria the following year.

The British government, then led by David Cameron, joined the US and EU in placing limited sanctions on Russia, while maintaining an open dialogue with Moscow.

The UK also began training Ukraine’s army alongside the US and Canada, and almost doubled spending on the intelligence services. The joint security fund, worth more than £1.5bn a year by 2020, was in part to rebuild the agencies’ Russian capacity.

In mainland Europe, Germany was more concerned about Russian energy supplies, while Emmanuel Macron, France’s president, sought a “trust-building dialogue” with Putin.

But UK membership of the Five Eyes intelligence alliance — which consists of the US, Australia, Britain, Canada and New Zealand — meant that London had access to information that Europe lacked about Moscow’s inner workings and, crucially, the KGB mindset of Putin’s team.

“As the only European member of Five Eyes, it was inevitable we became cheerleaders for a hawkish Russian position,” said a former UK security official.

Even so, the UK persisted in its engagement with Russia and in 2017 Johnson, then foreign secretary, visited Moscow. He joked that he was a “committed Russophile” and the first UK foreign minister “called Boris”.

It was the poisoning of double agent Sergei Skripal in Salisbury in 2018 that led UK politicians, including then prime minister Theresa May, to see the full scale of Moscow’s threat. She subsequently named two Russian military intelligence officers as prime suspects in Skripal’s attempted murder.

“A violent process of disillusionment set in,” said Younger. “Moscow has such a hooligan mindset that it didn’t realise how strategically alienating that action was.”

In solidarity, the US and EU nations threw out Russian spies, but Brexit had soured UK relations with Brussels and there was little political follow-through.

A similar pattern played out in Washington. The then US president Donald Trump, who had an affinity for Putin, told May he did not believe British intelligence’s assessment that Moscow had poisoned Skripal, according to the Washington Post.

Despite the Skripal attack, the UK did not adopt a position of outright hostility to Russia. “It took a long while for much of government to realise that the Russian technocrats and economists they spent their time talking to were not running the country,” said a senior UK diplomat.

As late as July 2020, parliament’s intelligence and security committee lambasted the government for “taking its eye off the ball on Russia”. “There was dogged resistance in some quarters to viewing Russia as a threat,” said Keir Giles, a senior consulting fellow at the Chatham House think-tank.

It was only in March last year that the government’s integrated security review identified Russia as an “acute direct threat to the UK”. Putin subsequently began to amass military forces on Ukraine’s border.

Paris and Berlin still struggled to make what one UK official described as the “cognitive leap” about Putin’s intentions. But Britain’s hardening attitude gelled with 10 Russo sceptic Baltic and Nordic countries that are members of the UK military-led Joint Expeditionary Force, and with US president Joe Biden, a longtime Putin hawk. “It’s easier to navigate a path when you have strategic consistency from the US,” said one Downing Street adviser.

It took a long time for the UK to realise the true threat of Putin. But with the worst fears of British military and intelligence services now realised, it would take a seismic event to change policy again.

A Number 10 spokesperson said: “The United Kingdom has been front and centre of the international response to Putin’s barbaric invasion of Ukraine, building an international coalition which continues to provide unprecedented financial, military and diplomatic support to President Zelensky and the Ukrainian people.”

Additional reporting by John Reed in Kyiv