Judging the success of the UK’s online security strategy is difficult, a government agency has reported – and it’s no surprise given it’s using debunked statistics.

The National Audit Office (NAO) has today released a report examining the government’s£650m cybersecurity strategy, looking to judge whether or not it’s working and offers good value for money.

That is a tough task, as there’s no easy way to calculate the cost of cybercrime — a fact the agency admits.  ”The NAO recognises, in particular, that there are some challenges in establishing the value for money of the cybersecurity strategy,” the agency said. “There is the conceptual problem that, if cyber-attacks do not occur, it will be difficult to establish the extent to which that was down to the success of the strategy.”

Those challenges are worsened by the NAO’s own use of bad data, and the misquoting of reports within its own analysis.

The NAO repeatedly cites the cost of cybercrime as between £18bn and £27bn — two figures that are respectively inaccurate and thoroughly debunked.

The £27bn figure is from a 2011 Detica report commissioned by the Cabinet Office, which has been widely dismissed as “scaremongering” and a “sales exercise” — Detica is owned by BAE Systems, a security and defence firm.

The £27bn figure is from a 2011 Detica report commissioned by the Cabinet Office, which has been widely dismissed as “scaremongering” and a“sales exercise”

The second figure, of £18bn, comes from a University of Cambridge report that was commissioned by the government purely to verify the accuracy of the Detica report. That figure is also inaccurate and misleading for two reasons. First, the Cambridge report shows the figures in dollars, so in the very least it should be $18bn, not £18bn. Second, the report’s authors specifically advise against adding up all of the numbers it provides, saying it would be “entirely misleading to provide totals lest they be quoted out of context, without all the caveats and caution that we have provided”.

And that, of course, is exactly what has happened.

The $18bn figure is made up of four parts: transactional crime, such as card fraud and “loss of consumer confidence” makes up more than $3bn; cost of infrastructure and protections such as antivirus accounts for $1.2bn; more traditional crime, such as tax and benefit fraud, shifting to the internet adds more than $14bn; “genuine” cybercrime, such as online banking fraud and botnets, only costs the UK an estimated $164m an year – less than consumers spend on antivirus software.

Getting the numbers straight is key, so we and the government don’t end up spending more on security protection than it actually costs us. For the current strategy, the government has laid out £650m in funding to 2015 to secure networks, as well as to educate the public and help UK businesses grab a slice of the “growing market in cybersecurity”.

Reports such as these that keep using artificially inflated numbers — from security firms — don’t help the education remit, but they certainly might help the latter.