However, with the spiralling number of surveys appearing in the media with each passing year, having a certain amount of knowledge about statistics has come to my aid on numerous occasions. Because it turns out that even intelligent people don’t really understand statistics at all.

Here, I’d simply like to address the number one, burning misunderstanding people have about statistics: the sample size has to be similar in number to the total population in a study.

No. Honestly, no. I realise it’s not wholly obvious, but no.

Take the recent example of the survey of Microsoft employees via Glassdoor.com, which showed that around half of them were unhappy with Steve Ballmer’s performance.

Because the sample size – that is, the number of people surveyed – was around 1,000, and the overall number of Microsoft employees is around 80,000, one leading Microsoft blogger sent a tweet saying: “Those surveyed for that report equates to about 0.625% of Microsoft employees… again, hardly representative at all. Seems very flawed.”

The 0.625% stems from the 500 Microsoft employees who weren’t happy with Ballmer.

But it’s not flawed at all. To explain why, I’ll analyse that survey backwards.

We’ll assume that Microsoft has precisely 80,000 employees, and that precisely 50% of them don’t believe Ballmer is doing a good job.

If we repeatedly surveyed 383 people (randomly chosen each time), then statistics show that 19 times out of 20 the results – that is, 95% of the time – we’d get a result showing between 45% and 55% of them didn’t believe Ballmer was doing a good job.

To switch that into statistical speak, that’s a 95% confidence level with a 5% margin of error (that is, 50% plus or minus 5%).

I know what you’re thinking: 95% confidence level isn’t enough. So let’s go for 99%. Assuming the same conditions – 80,000 employees, 50% unhappy – then we’d need a sample size of 659 people.

To put that into plain English, with a sample size of 659 precisely 99 out of 100 surveys would show a result of between 45% and 55% being unhappy with Ballmer’s performance.

9,985 out of 10,000 surveys would show 45% to 55% of Microsoft employees were Ballmer-unhappy

What happens if we up the sample size to 1,000? The confidence level increases to 99.85%, so 9,985 out of 10,000 surveys would show 45% to 55% of Microsoft employees were anti-Ballmer.

In fact, the Glassdoor.com ratings are based on a 1,119 sample size, giving a confidence level of 99.92%. Pretty strong.

One final point: the biggest problem with any survey is finding a truly random sample. Glassdoor.com doesn’t appear to vet its survey respondents (other than for insults, trade secrets or defamation), so you or I could contribute our own reviews should we so wish.

You could also argue that, as a recruitment-orientated website, it will be biased towards current Microsoft employees looking to leave, or ex-employees with a grudge.

But neither of those potential flaws would explain why Oracle’s CEO gets such a high approval rating when his company rating is actually lower than Microsoft’s. As such, we can all have confidence in Glassdoor.com’s sample-gathering techniques, while Steve Ballmer should think about how he can change his employees’ minds.

As you can see, across various tests the red bar is three, four, even six times as tall as the green one. But hold on — because that’s not an accurate reflection of relative performance.

You’ve probably already spotted the reason why: the Y axis doesn’t start at zero! Instead, it originates at a rather arbitrary 0.8, greatly exaggerating the difference in scale between the green and red bars. A more neutral representation of the same figures would see the red team still win, but by a decidedly less jaw-dropping margin:

Of course, skipping over part of an axis can sometimes be justified. If you’re charting small changes in large numbers, it makes sense to zoom in a little, just for the sake of clarity. But here the graph isn’t intended to illustrate a trend: it’s supposed to convey, at a glance, just how much bigger one set of numbers is than another. And that’s precisely what it doesn’t do.

Don’t think I’m picking on any one company here: this type of spin is part and parcel of marketing, in the IT business and beyond. And to be honest, I rather enjoy the mental work-out of decoding official PR messages to get to the truth. It just irks me that they think we’re that gullible.

What’s the most shameless marketing claim you’ve come across?