Politics

Statistricks, part 3: how they lie to you with polls

Opinion polls are way off with their predictions too often to be of any use. So why are they such big business?

Are you diving into the data, or is the data diving into you?

• Part 1: ‘Trade with the EU is declining’ (no, it isn’t)
• Part 2: ‘We send the EU £350m a week’ (no, we don’t)
• Part 4: ‘The UK is the fastest-growing economy in the G7’ (no, it isn’t)

Not so long ago, you could go years without coming across a survey. A few folks were dimly aware of a company called Gallup, thanks to Top of the Pops, for which they compiled the charts, but otherwise polling companies were shy little leprechauns that only popped out once every four years to sound out the populace before each general election.

Now you’re lucky if you can go four minutes without seeing a snapshot of public opinion. Twitter polls, website polls, newspaper polls, polls by phone and email and WhatsApp; polls on everything from support for the death penalty to your preferred shade of toast.

What’s my tribe?

The appeal to us plebs is obvious. We can’t get enough of other people’s opinions, whether our response is to nod sagaciously or spit out our tea.

Interestingly, our egos are so devious, it doesn’t much matter whether most people agree with us or not. Because if, according to any given poll, ours is the majority view, we tend to sit back and smirk: “Well, naturally my opinion is the right opinion.” If, on the other hand, we’re in the minority, our response is usually “Gosh, I’m so clever, unlike all these sheep!”

Either way, polls are reassuring because they reinforce our place in the world. Our tribal, hierarchical nature, our teat-seeking need to belong, compels us to constantly reaffirm our sense of identity, and polls give us that in a neat package.

Views as news gets views

If polls are a novelty gift for the hoi polloi, they’re a godsend for newspapers, struggling as they are with dwindling resources, and for rolling news channels with endless airtime to fill. No time-consuming investigation, photography, writing or planning required – the pollsters take care of it all for free, right down to the covering press release with its own ready-made headline finding. And the public lap it up.

The pollsters, of course, are laughing all the way to the bank. Their services are in greater demand than ever before; the polling industry in the UK currently employs 42,500 people – four times as many as fishing.

So, surveys are win, win, win, right? People get entertainment, journos get clicks, pollsters get rich. What’s the problem? Because there’s always a problem with you, isn’t there, Bodle?

Blunt tools

As a matter of fact, there are two. The first is that polls are low-quality information.

Despite having been around for almost 200 years, and despite huge advances in methodology and technology, gauging popular opinion is still an inexact science. For proof, look no further than the wild differences between any two surveys carried out at the same time on the same issue.

In the week prior to the 2017 UK general election, for example, Scotland’s Herald newspaper had the Tories winning by 13%, while Wired predicted a 2% win for Labour. (In the event, May’s lot won by 2.5%; picking a figure somewhere in the middle of the outliers is usually a safeish bet.)

The main headache for canvassers has always been choosing the right people to canvas. If you conducted a poll about general election voting intentions solely in Liverpool Walton, or took a snapshot of views on the likely longevity of the EU from 4,000 Daily Express readers (which the Express continues to do on a regular basis), then presented the results as a reflection of the national picture, you would rightly be laughed out of Pollville.

The key to a meaningful survey is to find a sample of people that is representative of the whole population. Your best hope of this is to make the sample as large as possible and as random as possible, for example by diversifying the means by which the poll is conducted (because market researchers wielding clipboards on the high street aren’t going to capture the sentiment of many office workers or the housebound, while online polls overlook the views of everyone without broadband), and sourcing participants from a wide area.

Even then, you have to legislate for the fact that pollees are, to a large degree, self-selecting. For one thing, people who are approached by pollsters must, ipso facto, be people who are easily contactable, whether in the flesh, by phone or online, which rules out a swathe of potentials; and for another, they’re likely to have more free time and less money (many polls still offer a fee).

Tories, trolls and tergiversators

Even if you do somehow manage to round up the perfect microcosm of humanity, there are further obstacles.

For one thing, people are unreliable. The “shy Tory factor” is well documented; people don’t always answer truthfully if they think their choice might be socially unacceptable. You can mitigate this problem somewhat by conducting your survey anonymously, which most pollsters now do.

Anonymity, however, only exacerbates a different problem. As anyone who has spent five minutes on social media will know, there are plenty of people around who just lie for kicks (or money). And if the questions aren’t being put to you in person, and your name isn’t at the top of the questionnaire, there’s even less pressure on you to tell the truth.

Furthermore, people don’t always know their own minds. If you’re faced with difficult questions in an area where your knowledge is sketchy, like trans rights or Northern Ireland, your honest answer to most questions would be “Don’t know”. But you’d feel dumb if you ticked “Don’t know” every time. Isn’t there a temptation to fake a little conviction?

And (Brexiters and Remainers notwithstanding), people’s opinions are not set in stone. Someone may genuinely be planning to vote Green when surveyed, then change their mind on the day.

Then there’s the issue of framing. Every facet of a survey, from its title to the introductory text, from the phrasing of the questions to the range of available answers, can unwittingly steer waverers towards certain choices.

Let’s say you want to study views on asylum seekers. If you ask 2,000 people “Do you agree that Britain should help families fleeing war and famine?”, you’re likely to get significantly different results than if you ask them, “Do you think Britain should allow in and pay for the upkeep of thousands of mostly young, mostly male, mostly Middle Eastern and African migrants?” (If this seems like an extreme example, I’ve seen some equally awful leading questions.)

Sometimes the questions don’t legislate for the full spectrum of possibilities. If no “don’t know” option is included, for example, people may be forced into expressing a preference that they don’t have.

Finally, the presentation of a poll’s results can make a huge difference. Few people have the patience to read through polls in their entirety, so what happens? Pollsters create a press release featuring the edited highlights – the highlights according to them.

When you consider all these pitfalls, suddenly it’s not so hard to see why pollsters’ predictions often fly so wide of the mark. But … so what if polls are inaccurate? They’re just a bit of fun!

This brings us to my second, more serious concern.

Market intelligence

While they’re passable diversions for punters and convenient space-fillers for papers and news channels, no one ever went on hunger strike to demand more polls. This constant drizzle of percentages and pie charts has not been delivered by popular demand. It’s a supply-side increase, driven by the people who really benefit from it.

Businesses live or die by their market research: the information they gather from the general populace. If you’re a food manufacturer launching a new bollock-shaped savoury snack, for example, it helps to know how many consumers are likely to buy it. But firms are also greatly dependent on their marketing – the information they send back into the community. And one of the best things they can do to promote their product is to generate the impression that by golly, people love Cheesicles!

We simply don’t have the time to do all the research required to formulate our own independent view on every imaginable issue. So what do we do? We take our cue from others: friends, or experts, or people we otherwise trust.

Hence the myriad adverts featuring glowing testimonials from chuffed customers. Hence celebrities being paid astronomical fees for sponsorship deals. Hence the very existence of “influencers”. Like it or not, our opinions are based, in large part, on other people’s opinions.

The only thing more likely to cause a stampede for Cheesicles than the endorsement of a random punter or celebrity is the endorsement of everyone. Why else would a certain pet food manufacturer spend 20-odd years telling everyone that eight out of 10 cats preferred it (until they were forced to water down their claim)? Aren’t you more tempted to give Squid Game a chance because everyone’s raving about it?

Even though some people quite like being classed among the minority – the brave rebels, the “counterculture” – those people are, ironically, in a minority. Most of us still feel safer sticking with the herd. So it’s in manufacturers’ interests to publish information that suggests their product is de rigueur.

(If you’re in doubt about the susceptibility of some people to third-party influence, look up the Solomon Asch line length test. As part of an experiment in 1951, test subjects – along with a number of paid plants – were shown visual diagrams of lines of different lengths and told to identify the longest one. The correct answer in each image was clear, but the stooges were briefed to vocally pick, and justify, the (same) wrong answer – and a surprisingly high proportion of the subjects changed their decision to match the wrong answer given by their peers. Later variations on the same study furnished less clear-cut results, but the phenomenon is real.)

And this is where all those flaws in polling methodology suddenly become friends. Polls can be inaccurate and misleading by accident – but they can also be misleading by design.

When businesses conduct a poll, they can (and have, and still do) use all the above loopholes to nudge the results in the “right” direction. They can select a skewed sample of people. They can select a meaninglessly small sample of people (still the most common tactic). They can ask leading questions, leave out inconvenient answers, present the results in a flattering way – or just conduct poll after poll after poll, discard the inconvenient results, and publish only those in which Cheesicles emerge triumphant.

Woop-de-doo, so businesses tell statistical white lies! Hardly front-page news, or the end of the world. If I’m duped into shelling out 75p for one bag of minging gorgonzola-flavoured corn gonads, well, I just won’t repeat my mistake.

True. But it’s a different story when the other main commissioners of polls play the same tricks.

Offices of state

It cannot have escaped your notice that the worlds of business and politics have been growing ever more closely intertwined. There’s now so much overlap of personnel between Downing Street, big business and the City (the incumbent chancellor, who arrived via Goldman Sachs and hedge funds, is just one of dozens of MPs and ministers with a background in finance), such astronomical sums pouring into the Tory party from industry barons, and so many Tories moonlighting as business consultants, that you might be forgiven for thinking that the two spheres had merged.

And as the association has deepened, so politicians (and other political operators like thinktanks and lobbying groups) have borrowed more tactics from their corporate pals. Public services are run like private enterprises; short-term profits and savings for the few are constantly prioritised over the long-term interests of the many; government communications departments have been transformed into slick, sleazy PR outfits. And one of the tools they’ve most warmly embraced is the poll.

While businesses carry out market research to gauge the viability of their products and services, political parties do so (largely through focus groups) to find out which policies and slogans will go down well. But whereas businesses only publicise polls to create the illusion of popularity, the practice has wider and scarier applications in the political sphere.

“Opinion polls are a device for influencing public opinion, not a device for measuring it. Crack that, and it all makes sense”
Peter Hitchens, The Broken Compass (2009)

Loath as I am to quote the aggressively self-aggrandizing Hitchens, on this occasion, he may have stumbled across a point. A number of studies (pdf) have looked into this phenomenon (pdf), and while the findings aren’t conclusive, they all point in the same direction: people can be swayed by opinion polls.

There are several mechanisms at play. First, if there’s a perception that one candidate in an election has an unassailable lead, some undecideds will back the likely winner, because they think the majority must be right (the “bandwagon effect”); a few will switch to backing the loser out of sympathy (the “underdog effect”); some of those who favoured the projected winner might not bother voting because it’s in the bag, while some of those who favoured one of the “doomed” candidates might give up for the same reason.

Conversely, if polls suggest a contest is close, turnout tends to increase. Even if your preferred candidate isn’t one of the two vying for top spot, you might be moved to vote tactically, to keep out the candidate you like least.

Polling also has an indirect effect via the media. When surveys are reporting good figures for a candidate, broadcasters and publishers tend to give them more airtime and column inches, thus increasing their exposure, and, consequently, their popularity.

However these effects ultimately balance out, it’s clear that the ability to manipulate polling information could give you enormous political power. “But that’s absurd!” you cry. “I’ve never had my mind changed by anything as frivolous as a poll!”

Really? Can you be absolutely sure of that? Even if you’re immune, can’t lesser mortals be affected? If it works in the advertising world, there’s no reason why shouldn’t it work in the political sphere.

You might object at this point that pollsters are legitimate enterprises that have nothing to gain from putting out false information. To which I would counter-object: polling companies are businesses too. They exist not as some sort of public service, but to make money for their clients. And their clients’ interests do not always align with the public good.

A brief look at the ownership and management of the pollsters does little to alleviate these fears.

Savanta ComRes (formerly ComRes)

Retained pollster for ITV and the Daily Mail. Founded by Andrew Hawkins, Christian Conservative and contributor to the Daily Telegraph with a clear pro-Brexit stance. This year, Hawkins launched DemocracyThree, a “campaigning platform” that helps businesses and other interest groups raise funds and build support – ie influence public opinion.

“Democracy 3.0 helps you build a support base, raise the funds you need for your campaign to take off, and then we work with you to appoint professional campaigners – such as lobbyists and PR experts – who can bring your campaign to life.”
DemocracyThree website

ICM

Co-founded in 1989 by Nick Sparrow, a fundraising consultant who worked as a private pollster for the Tories from 1995-2004. Now part of “human understanding agency” Walnut Unlimited, which is in turn part of UNLIMITED – a “fully integrated agency group with human understanding at the heart”.

The sales pitches for these firms include the following quotes:

“Our team are experts in public opinion, behavioural change, communication, consultation and participation, policy and strategy, reputation, and user experience.”
ICM website

“We help brands connect with people, by understanding people … Blending neuroscience, behavioural science and data science, we uncover the truth behind our human experiences … Our mission is to create genuine business advantage for clients … by uncovering behaviour-led insights from our Human Understanding Lab.”
Walnut Unlimited website

Populus

Official pollster for the Times newspaper, co-founded by Tory peer Andrew Cooper and Michael Simmonds, a former adviser to the Tory party now married to Tory MP Nick Gibb, who has recently been added to the interview panel to choose the next head of media regulator Ofcom.

YouGov

Founded by Nadhim Zadawi, the incumbent Tory health secretary, and Stephan Shakespeare, former owner of the ConservativeHome website and former associate of diehard Brexiters Iain Dale, Tim Montgomerie and Claire Fox.

Survation

Founded by Damian Lyons Lowe, who during the EU referendum campaign set up, at the request of Ukip’s Nigel Farage, a separate “polling” company, Constituency Polling Ltd, based in the Bristol office of Arron Banks’s Eldon Insurance. But its remit seems to have been less about asking questions and more about micro-targeting voters. “Interviews with several people familiar with Survation’s operations show that in addition to measuring public opinion, the firm’s executives also helped shape it.”

(I was unable to find any evidence of strong political affiliation among the leadership of Ipsos MORI or Qriously, and Kantar has changed ownership and CEO so frequently of late as for any such investigation to be meaningless. As a side note, there seems to have been a recent flurry of activity in this sector, with many companies being gobbled up into ever larger, faceless global marketing conglomerates, whale sharks hoovering up data, with ever more sinister specialisms: “consumer insights”, “market intelligence”, “human understanding”.)

I don’t know about you, but I’d expect the people who founded and run companies that were nominally about gathering and analysing data to be statistics nerds – people with an interest in objective truth – not, by an overwhelming majority, people with the same strong political leanings. Put it this way: CEOs of polling firms have final approval over which surveys are released. If you were married to a Tory MP, would you really sign off on a poll that was damaging to your husband’s party?

Someone of a more cynical bent might start wondering whether the hard right, having secured control of most of the UK’s print media and with its tendrils burrowing ever deeper into the BBC, was stealthily trying to establish a monopoly on data.

So maybe they’re not all angels. But surely they can’t just pump rubbish into the public domain willy-nilly? In a stable(ish) 21^st-century democracy like Britain, there must be checks and balances in place.

Well, here’s the thing. Businesses are prevented from publishing grossly misleading adverts by the Advertising Standards Authority, but there’s no such independent regulator for the polling industry. They police themselves, through a voluntary body called the British Polling Council, staffed entirely by industry members.

So, polls are bad information, they can influence people’s votes, the pollsters’ motives are questionable, and they’re accountable to no one. But what about journalists? Isn’t it their job to pick up on this sort of thing?

It is, but as I mentioned above, journalistic resources are so depleted now, and the pressure to get stories up fast so great, that they can ill afford to look gift stories in the mouth. And as I mentioned in my last post, journalism and broadcasting aren’t exactly brimming with Carol Vordermans. Even if they had the time and the inclination to carry out due diligence, they wouldn’t necessarily know how.

The bald fact is, when you look at a poll, whether it’s reached you through a newspaper, a website, a meme or a leaflet, you have no guarantee whatsoever that it’s been subjected to even rudimentary checks.

What can we do?

Surveys are – or were, originally – designed to present a snapshot of the popular mood. But even the most fair-minded, honourably intentioned, statistically savvy pollster, using the best possible methodology, can produce a poll that is complete and utter Cheesicles.

But judging by the vast amounts of money pouring into the industry, the political leanings of its ownership and management, and their alarming transformation from simple question-setters to behavioural change specialists, there’s a very real possibility that honourable intentions are an endangered species in the polling industry.

Polls aren’t going away any time soon. Businesses and politicos will always want to gauge which way the wind is blowing. But when it comes to the data they’re pumping back in the public domain – a tiny fraction of what they’re amassing – the rest of us don’t have to play along.

To journalists, I would say: please stop treating polls as an easy way of filling column inches. (Employees of the Daily Express, Mail, Sun and Telegraph, I’m not talking to you. I said “journalists”.)

This is the opposite of speaking truth to power; it’s speaking garbage to those who aren’t in power. It’s 1980s women’s magazine journalism, clickbait, guff, and you’ve repeatedly proven yourselves incapable of discerning good information from bad.

If you must run an article on a poll, then ensure that, at the very minimum, you ask, and get satisfactory answers to, these questions:

Who commissioned the poll?
Who carried out the poll?
What was the sample size? If it’s much less than 2,000 people, ignore it.
What’s the relative standard error? (A measure of the confidence in the accuracy of the survey. If Labour are leading the Tories in a poll by 36% to 35% and the RSE is over 2% – as it is on samples of less than 2,000 – then they may not be leading at all.)
What were the questions?
What was the methodology?

Then, when you publish the story, include all this information so that readers can draw their own conclusions about the poll’s reliability. Above all, include a link to the poll. If you don’t take all these steps, your story is worthless.

To the public, my advice would be: ignore polls. If you must read them, treat them as meaningless fun, fodder for a throwaway social media gag, and don’t for one second fall into the trap of thinking they’re conveying any sort of truth.

If you’re ever approached to participate in a poll, ask yourself: do you really want to be handing over your data to people who are likely to be using that data against you and enriching themselves in the process?

Finally, to the pollsters, I would say: we’ve got your number.

Bregrets, I’ve found a few

It takes courage to admit you were wrong. As Leave’s lies unravel, more and more Brexit voters – 310 and counting – are showing it

On 23rd June 2016, 17,410,742 people voted for the UK to end its 43-year membership of the European Union. They did so after a Leave campaign chock full of lies, distortions and scare tactics, many of which have been exposed as such in the days since the referendum.

Many stand by their vote. It’s hard to admit you made a mistake. But as it becomes clearer that both the alleged sins of the EU and the advantages of membership were grievously misrepresented, that no one fully explained the extent to which Britain’s economy is integrated with and dependent on the 27 other EU nations, and as more meat is added to the bones of stories about illegal cooperation between the Leave campaigns, suspicious donations to pro-Brexit groups and interference in the vote by malevolent foreign actors, more and more Brexiters are seeing the error of their ways. I’ll be keeping track of them here.

Bit harsh on yourself there, Leila – you’re far from alone. Many others have struggled with the idea that so many tabloid newspaper journalists and politicians could lie so brazenly, and so clearly contrary to the interests of the country, but it’s becoming increasingly obvious that this is what’s been happening.

Welcome aboard the good ship Remain, Peter’s friend.

Like a reformed smoker, Michael’s now quite the virulent anti-Brexit campaigner.

Good on you, Hugh.

Tim only went and wrote a bloody blog post about it.

The Brexit headbangers would have us believe that support for Remain is waning. That really doesn’t seem to be the picture these tweets are painting.

There’s no need to be afraid of mockery from Remain voters if you’re thinking of admitting your Leave vote might have been a mistake. We all voted on the basis of very poor-quality information.

Don’t kick yourself too hard, Tre. There are plenty of billionaire disaster capitalists queueing up to do that for you.

I agree. I voted out, but have definitely changed my mind. The logistics are just too difficult. https://t.co/3106fEDTr7

— Gordon McKay (@gmckay_51) August 28, 2017

Not exactly a full-throated recantation from Gordon, but it’s another vote in the bag. (Tweet has since vanished – I think Gordon runs one of those apps that automatically deletes all tweets more than a month old. The reply below, however, survives. Because of eventualities like this, I’m going through this post replacing all embedded tweets with screenshots.)

Aaand two more recruits, courtesy of Kristian.

It was almost certainly in your top three, Beth, but I won’t press the point.

This one came courtesy of @StuartBudd1 on Twitter.

Another Bregretter wedded to the old ways is MG:

This Leave voter is pretty upfront about her reasons for flip-flopping. And I imagine a fair few of these 4 million fellows might well vote differently in a second referendum – or at the very least abstain.

According to this study, the number of people who regretted voting Leave was already greater than the margin of victory for Leave – and that was in October 2016. As the scale of the task facing the UK government and its rank unfitness to undertake it become ever clearer, that number can only have risen.

There are doubtless hundreds, possibly thousands more Bregretters – it only took me an hour to collect the examples above. Feel free to send me any more admissions of error you may find (after thanking them for their courage and honesty, natch). The case for a second referendum – or, preferably, a simple retraction of article 50 – grows stronger by the day.

Bregretters found since 20/9/17

I may have made a rod for my own back here. Mind you, I’ll take a rod up the tradesman’s if it stops Brexit.

(A revealing little exchange, that one. Despite the wailing of Liam Fox and various shady corporate-sponsored thinktanks, there has been much speculation that the BBC has given far too much prominence to advocates of Brexit – and especially of hard Brexit.)

(That last post was retweeted by the @BrexityRegrets account, which had, as of 18/12/2017, winkled out 95 more people who have had second thoughts since 23/6/16.)

Inspired by a broadside of hatred and scorn from hardened leavers (and doubtless a few Mercer trolls) on 18 December, I decided to broaden my search and started finding more Bregretters in unexpected places like Mumsnet:

And here’s one from the Student Room:

Ooh, just found a Facebook group that should turn up a few more. Here’s one:

And another Facebook post:

Six more were interviewed here for the Huffington Post. Another wrote of her change of heart for the Telegraph. This caller to James O’Brien’s LBC radio show admitted he’d “made a mistake” and voted against the interest of his French partner of 15 years. This one made a similar admission, and O’Brien namechecks another (and implies many more) in this video. Another delicious radio moment came when caller David informed Nigel Farage that he would now vote Remain. Three more people seem to be admitting to a change of heart in this Channel 4 News clip. There are four more backtrackers in this separate Huffington Post article. This series of videos features 22 more. There’s one more reformed Remainer (not already covered elsewhere), Le Boy El Pablo, in this Time article. MSN spoke to another Bregretter here. For those who missed it, here’s a representative of an entire family who voted Leave and thought better of it the very next day – “when the facts started coming in”. Bill Walton facepalms about his vote in this article in the Sunderland Echo. Dorian Lynskey’s excellent piece on Leavers’ remorse provides us with a further six (one’s already included above). This Twitter thread found another eight flip-floppers not already covered above. And Lynda Smith uploaded this rather fitting video to YouTube to demonstrate how she now felt about her vote.

I’m reserving a special mention for this guy, who I found only because of the dedication of frothing Brexadi @JohnWebbWindsor on Twitter. Thanks, John!

This is proving to be quite an inexact science – there are possibilities of duplications, of course, and there’s no telling whether all the people concerned are telling the truth (although it’s hard to see why someone would make something like this up). Take this tweet thread, for example:

In the interests of fairness, I’ll only count this as one more Remainer. (PS: thanks for your bravery and honesty, Simon!) In any case, this is only supposed to give a general picture of the momentum building against a hasty and calamitous exit from the EU.

310 down. Only 598,690 to go.

Footnote, 12/02/18

Having hit the 300 mark, which I think is enough to make my point, and having realised that this page now takes a solid hour to scroll through, I’m going to be a little less rigorous with the updates, although I will continue to post links below every now and then.

Why I’ve changed my mind on Brexit

Support grows for second vote in Britain

Another Bregretter on Twitter

It’s a game to them. And you’re just a piece

How does it feel to be written off by a vampire onion with a fake CV?

Something pinged when I watched this interview. A connection was made where before there had only been a fuzzy proximity. And in that moment, one of the fundamental and perennial problems of politics crystallised.

“Low-value people”? Who defines people in terms of their value? As if it were some predetermined, unchangeable quantity? People who don’t see people as people, that’s who; people who see people as pieces in a game. Their game.

Look at yourself through Iain Duncan Smith’s eyes for a second. What are you? A pawn? A bishop? A king? Are you worth sacrificing for guaranteed mate in four?

Perhaps Risk is a better analogy. Up to six can play, diplomacy and chance have a greater role, and troops can be replenished. (There’s probably an even better one in the world of video games, but since I’ve been out of that world for a good 15 years now, I’ll leave that to the fancy of the digitally minded.)

The metaphorical link between politics and games has always been strong. Games started out, after all, as simplified simulations of life. Military commanders have long used counters on boards to represent troops. Game theory, under the right conditions and parameters, has been revealed as one of the better approximators of human behaviour. And increasing computer processing power means that the gap between simulations and reality is fast dwindling to nothing. It should come as no surprise that the opposite transformation sometimes occurs.

Iain Duncan Cunt — “And then I ate him. Artists are a bit … stringy”

But when it comes to real policies, which affect real people – people you know, people you love – is it really acceptable to think, and legislate, in terms of pebbles or pieces of plastic? How does it feel when a vampire onion with a fake CV writes you off on the basis of a report drawn up by a prematurely balding double-barrelled nanny’s boy straight out of Oxford via Harrow? Who made this fucking loser God?

I can’t shake the image of Duncan Smith, and sundry shadowy halitotic sepulchraves like Dominic Cummings, releasing silent farts in their Soho club, cradling a brandy and sniggering as they dispatch five infantry and two cavalry from Japan into Kamchatka. And then your disability benefits are stopped. Kaboom.

I like to think, if more people kept this image in their skulls as they walked into the polling booth, that our governments would look very different from the way they do today.

The art of the smear: how the far right destroyed public discourse

Online debates these days unfold with all the dignity and decorum of a stag do at a Reading Wetherspoons

“Whoever first hurled an insult at his enemy instead of a spear was the founder of civilization” – Sigmund Freud

Insults are nothing new. From the 4th century BC ding-dong between Demosthenes and Aeschines, to Shakespeare’s “You starveling, you elf-skin, you dried neat’s-tongue, you bull’s-pizzle, you stock-fish!”, to Noel Gallagher’s laceration of Robbie Williams as “the fat dancer from Take That”, incivility is as old as civilisation. We start calling each other names in the playground and, if the Leave voters I’ve encountered online are any guide, there’s no upper age limit on abuse.

But suddenly, it’s Ragnarök out there. Get involved in an online discussion about anything from topiary to the Tweenies and the chances are you’ll be set upon within minutes. Political “attack ads” were once frowned upon; now they’re the norm. There’s hardly been a single positive message in the UK general election campaign thus far; it’s just been smear after slur after slight. Most modern debates – usually online, but increasingly in real life – unfold with all the dignity and decorum of a stag do in a Reading Wetherspoons.

It’s all rather odd, because as I’m sure you already know, attacking a person in this way, instead of the substance of what they’re saying (or in the case of political debate, their policies), is a fallacy: a failure of reasoning that renders a claim invalid. In this case – the argumentum ad hominem fallacy – abuse is invalid because the truth of a person’s statement has nothing to do with their character, past behaviour or friends. Just because it’s Tony Blair saying, “Two plus two equals four”, that doesn’t mean the real answer is five. Much as it pains me to quote Margaret Thatcher, on this point, she had it right: “If they attack one personally, it means they have not a single political argument left.”

Civilisation is supposed to be advancing. We are devising ever better theories to explain the world, sending more kids to university, sharing more information, faster, than ever before. The use of logical fallacies should be declining, not increasing.

So why the recent explosion of invective? Some blame the global climate of fear since 9/11 and the ensuing jihadi atrocities. Some say it’s deindividuation: when we’re online, or part of a large group, we enjoy anonymity; our actions have fewer repercussions for us, either because we are untraceable, or we because we share responsibility with others. When we are at one remove from the consequences, our actions tend to be less inhibited.

Others point to the bubble effect, the fact that people tend to assort themselves into groups of like-minded people. If we spend too long in these echo chambers, hearing only what we want to hear, then we are likely to react more aggressively when someone attacks them.

These theories all have merit, but I don’t believe they are enough to explain the extent of the poisoning of discourse, or the speed with which it’s happened. Echo chambers have always been with us – we’re a tribal species – and abuse is less effective online than in the real world, because while we may be anonymous, our opponents often are too, so we don’t have as much informational ammunition.

There’s something else going on. I have a theory of my own, but before we go into that, it would be helpful to look at abuse in more detail. First, a key distinction. There are two broad classes of ad hominem, which serve different purposes. The first is relatively innocuous and is largely accepted as a legitimate debating tactic.

“You, sir, are drunk.” “And you, madam, are ugly … but at least I shall be sober in the morning.”

Winston Churchill’s zinger to Lady Astor is a regular chart-topper on “Top 10 Putdowns” lists – but like all ad hominems, it’s a fallacy. From a purely logical perspective, Lady Astor is in the right. Churchill’s comeback is fallacious; as well as being offensive, it’s irrelevant. He’s also drawn a false equivalence. While he is quite capable of controlling his level of inebriation, poor Lady Astor has no such dominion over the shape of her face.

But if Astor wins on substance, Churchill wins on style. This is a class of ad hominem that you might call a joust. It’s usually found in the context of the cut and thrust of repartee, and its purpose is to throw the opponent off balance. If your interlocutor has the upper hand in a discussion, a droll, well-targeted personal attack can move the debate to surer ground. Since people’s natural reaction to being attacked is to defend themselves, they’ll often attempt to defend the slight instead of driving home their advantage.

Jousts take place in real life, in real time, in the real world. If they hit the mark, they earn kudos for the wielder. Churchill’s comment may have been mean, but it was quick, it was (presumably at least partially) accurate, and it took some courage – presumably Dutch – because the target of his tongue-lashing was standing right next to him. Above all, it was funny.

Many of the greatest literary minds in history have been able practitioners of the joust. Chaucer, Swift, Pope, Voltaire, Johnson, Twain, Byron (“Posterity will ne’er survey/A nobler grave than this:/Here lie the bones of Castlereagh:/Stop, traveller, and piss”), Wodehouse (“She’s got brains enough for two, which is the exact quantity the girl who marries you will need”), Wilde (“I never saw anybody take so long to dress, and with such little result”) and Orwell (“He is simply a hole in the air”) all earned their stripes partly by tearing others off a strip. We excuse these slights, applaud them even, because they show qualities that we aspire to: intelligence, originality, speed of thought. Besides, there’s often a sense that the target had it coming. Who are you going to side with – the guy who won the second world war, or some judgmental fuddy-duddy?

The second class of ad hominem is the smear. It too is rapidly becoming an ingrained part of public discourse, but it is, I will try to argue, far more dangerous, and has no place there. Smears come in four main varieties.

1) Character assassination

There’s really only one contender for the title of World’s Bigliest Trash-Talker. As of March 2017, Donald John Trump was estimated to have slagged off no fewer than 320 different people, organisations and … items of furniture. Yeah, he dissed a table.

Trump’s gibes have the same drawback as jousts – they’re irrelevant to the discussion – and none of the merits. They’re not funny. They’re inane, unoriginal, and, as often as not, false. They do not signal a quick wit, because for the most part Trump delivers them via Twitter, or in prepared speeches to adoring crowds. And they are anything but brave, because Trump’s opponents are rarely in the same room when he vilifies them. To compare the US president’s infantile taunts to Shakespeare’s eloquent excoriations is to stick a Lego staircase next to the Taj Mahal.

Trump tweet about reporter — The “reporter who no one has heard of” was David Cay Johnston, winner of the Pulitzer prize for journalism.

And yet here he is, signing poorly drafted executive orders like baseball cards, merrily topping up the swamp with the contents of every septic tank in America, and hooking his drives ever closer to the nuclear bunker. How so?

When you look at the Mango Mussolini’s modus operandi, all becomes clear. Shower your enemies with insults; see which gains the most traction (ie likes and retweets); then use it again and again, until it becomes irretrievably associated with the target. So Hillary Clinton becomes “Crooked Hillary” at every mention; the New York Times “the failing NYT”; and rival for the Republican nomination Ted Cruz “Lyin’ Ted”.

These remarks are not designed to artfully throw Trump’s opponents off balance or to enhance his comic credentials. They’re not really directed at his opponents at all; they’re aimed at the wider world. Trump is not trying to undermine what his rivals are saying now, but everything they have ever said. These are cynical, systematic smear campaigns. And incredibly, through sheer force of repetition, they worked.

Most people are smart enough to realise that, just because a hay clump stapled to an overripe satsuma repeats something over and over again, that doesn’t mean it’s true. Unfortunately, it seems there are just enough credulous idiots out there for Trump’s bullying tactics to pay off. By means of a million gutless, leaden, charmless, bogus backstabs, Trump managed to destroy the credibility of everyone who stood in his way.

TL;DR: Your argument is rubbish because you are rubbish.

2) Tu quoque

“L’hypocrisie est un hommage que le vice rend a la vertu.” – François VI, Duc de La Rochefoucauld

The tu quoque – Latin for “you also” – is a special case of ad hominem also known as the appeal to hypocrisy. Here, instead of trying to undermine the target’s argument by maligning their character, you are attempting to do so by pointing out things they have said or done in the past that contradict their current position.

A tu quoque often feels somehow more cutting than a basic ad hominem, because, well, no one likes a hypocrite. However, it is just as invalid as a criticism, because it too fails to disprove the premise. Whether your past or current actions are 100% consistent with your view or not is completely immaterial. Let’s take a recent example.

On March 22 this year, the UK’s esteemed prime minister, Theresa May, stood up in parliament and delivered a broadside against Labour frontbenchers who had sent their children to grammar and private schools. The aim, of course, was to expose the hypocrisy of Labour, who as a party oppose the creation of more grammar schools. But as well as being unfair, May’s point is entirely without merit.

We live in a world where personal and public interests do not always align. (This is why we need governments; to balance private freedoms against the general good.) Politicians aren’t just politicians. They are also, despite appearances, human beings, and many of them are parents.

When you’ve got your MP’s hat on, grammar schools are a bad thing. Most studies have concluded that while they might benefit the few who attend them, they have a deleterious effect on other schools; they suck in all the best pupils and the best teachers, and regular schools suffer as a result.

But now look at the question from the viewpoint of the MP as parent. Grammar schools exist. The kids who go there do better. Given the choice between sending them to a grammar or to a comprehensive, which way do you swing? The effect of your decision on other people is infinitesimal – but it could make a huge difference to your child. You’re technically a hypocrite if you choose the grammar, but it doesn’t mean your opposition to the principle of grammars is wrong.

When Lily Allen and Gary Lineker spoke out in defence of refugees last year, they were bombarded with incandescent messages along the lines of “Well, how many are you taking into your mansion?”

Like May’s tirade, the detractors were missing the point. It isn’t Lily Allen’s job to look after refugees. There are lots of childless couples who’d probably be more than happy to take in a Syrian orphan, for example. The alt-right are constantly frothing at the mouth about the atrocities committed by Isis – but how many of them are donning combat fatigues and heading off to the Levant? If Allen and Lineker are hypocrites, so are they.

Few of us have the time, the equipment or the expertise to devote our lives to solving all the world’s problems. There are others better placed to tackle these matters: police, governments, charities. All we can do is flag them up, perhaps donate some cash, or if we’re lucky enough to have a free weekend, organise a trip to Calais to hand out clothes and food.

The fact is, we’re all hypocrites on a regular basis. Have you ever sat in gridlock and moaned about the traffic? You’re part of the problem! Ever merrily sucked on a fag while gravely warning a young relative never to take up the habit? Pot, meet kettle! Worried about overpopulation – and you have two kids? You’re a fine one to talk!

TL;DR: Your argument is rubbish because not every single thing you have ever said and done in your entire life has been 100% consistent with it.

3) Circumstantial ad hominem

The implication that a person has taken a position purely because it suits their agenda; they cannot be trusted on a particular issue because they have a horse in the race, or their judgment is otherwise clouded.

There’s sometimes something to this one. Most prisoners on death row insist that they are innocent, because it’s hugely to their advantage for others to believe them. However, by strict logical criteria, their claim is not automatically untrue simply because it is in their interest. We just need to take it with a healthy pinch of salt.

TL;DR: Your argument is rubbish because you are biased.

4) Guilt by association

Fetch the Nurofen. We are now entering the realms of spectacularly twisted logic, where the cognitive gymnastics can induce migraines in the unprepared.

4) is essentially 3) on steroids. The association fallacy is the most egregious and toxic of ad hominems, since it attempts to invalidate someone’s point by smearing them on the basis of her membership of, or tenuous association with, a particular group.

It’s become depressingly common practice, in online discussions, for your opponent to try to dig up dirt on you. They’ll read your profile, scan your previous tweets or comments, even Google you in their quest for incriminating evidence. Failing that, they’ll use whatever they find to try to pigeonhole you.

Why? Because then they can write off your opinion on the basis that they have already written off the opinions of everyone in your group. “Aha, you live in London! Of course you’d parrot pro-EU propaganda – membership benefits the metropolitan elite!” (Forty per cent of Londoners voted Leave.)

As well as committing the same logical misstep as the other smears – implying that a person’s identity is somehow relevant to their point – the association fallacy asks us to accept two further false propositions: a) that the entire group’s views and values are without merit; and b) that everyone in that group thinks and acts in exactly the same way.

Loath as I am to cut the hate-fuelled shitrag that is the Daily Mail any sort of slack, the continued attacks over its historic sympathy for Hitler and the Blackshirts make no sense. For one thing, everyone involved with the paper in 1934 is dead. New office, new owner, new staff. The words “the Daily Mail” now refer to a different entity; there is no immortal “Daily Mail soul” that inhabits everyone who sets foot in its offices. (True, the current team is starting to look more and more like its historic incarnation, but from a logical standpoint, there is no reason why this should be so.)

The guilt by association fallacy effectively means that no one can ever be right, because it holds that your view is worthless if you, or any group you have ever been associated with, have ever done anything wrong.

Telltale phrases: “Typical Leaver”, “You liberals are all the same”, “Just what I’d expect from a Muslim”.

TL;DR: your argument is rubbish because I consider (erroneously) that everyone in the group I have assigned you to (erroneously) is wrong about everything.

I used to think that humanity was slowly waking up to the preposterousness of generalisations like “All men are bastards”, “All Jews are stingy” and “All Gypsies are thieves”. But suddenly, writing off entire cross-sections of society at a stroke is enjoying a renaissance.

I picked the Daily Mail example for a reason. Of all the smear campaigns mounted in the last few years, one of the most sustained and successful has been the coordinated effort to discredit the entirety of the world’s journalists – or, to use the dismissive alt-right term, the “MSM” (mainstream media).

Organisations like the Mail, Express and Fox News haven’t helped the cause with their rabidly partisan headlines and editorials, and it’s true that the Independent, for example, has a strongly pro-EU slant. But the far right demagogues would have you believe that every journalist in every media outlet in the world is part of some huge conspiracy to deceive you, to keep a boot on the throat of the working classes and maintain the status quo.

The alt-right pursue this narrative by seizing on any and every mistake, oversight or lapse of judgment from any publication or broadcaster and magnifying it to ludicrous proportions. So, because the Guardian once ran an erroneous story about Jeremy Corbyn sitting on the floor of a train, we now cannot believe any Guardian story we read. Because Buzzfeed runs listicles and fluff pieces on its site, its investigative journalism is worthless. Because the BBC once published a news item about the negative effects of Brexit, it is irredeemably biased in all matters. The fact that Nigel Farage – an MEP for a party with no representation in parliament who only bothers turning up for work to sour the UK’s foreign relations – gets more airtime than any other politician in the UK is neither here nor there.

Yes, the mainstream media have made mistakes. And yes, some lean in a particular political direction. But on the whole, they try to give a reasonably balanced view of things. They are bound by ethical standards and libel laws, and subject to the oversight of a regulator. When they get things wrong, they apologise, they retract, and they print corrections.

Again, most people don’t buy into the alt-right’s absurd narrative. But a small, significant section of society have swallowed the lie, and now routinely reject any facts from mainstream sources as “biased” or “fake news”. To them, all of the media – from the Independent to the Guardian to the Mail to the BBC to ITV to CNN to the Times to Al-Jazeera to Buzzfeed to the South China Morning Post to the Dumfries & Galloway Standard – are all part of some vast conspiracy to prop up the metropolitan elite. We have now reached a point where a Brexiter, in a discussion about Brexit, will demand that you provide evidence to back up your point – and then, when you do, airily dismiss it because it happened to be published in the Sunday Times.

So if no conventional news sources can be trusted, who are these people listening to? Who has stepped into the information vacuum to expose The Real Truth? Blow me down with a feather, if it isn’t those selfsame alt-right demagogues! The alt-right demagogues, whose newsgathering resources generally stretch to a Twitter and YouTube account and a camcorder in their mum’s basement. The roving-reporter alt-right demagogues, like Paul Joseph Watson, who openly admits that he hardly ever leaves his Battersea flat, Julian Assange, who has been trapped in the Ecuadorian embassy for five years, and Katie Hopkins, who broke her world exclusive that there were quite a few Asian shops in London by going to the shops.

(This post is plenty long enough without a diversion into the shameless mendacity of the alt-right, but I’ve posted a few examples here, and there are plenty more at Snopes, FactCheck.org and FullFact.)

It wasn’t enough, of course, to smear the media. They are, after all, just mirrors to majority opinion. If a new order is to be imposed, all the traditional institutions must be undermined.

Dominic Cummings’ “Take back control” was very clever. The NHS promise on the bus probably did it for some people. The Breaking Point poster, even though immigration from outside Europe had precisely jack shit to do with the EU, undoubtedly won over a few racists. But for me, the real stroke of genius from the Leave contingent, and the turning point of the whole campaign, was Michael Gove’s “People have had enough of experts.”

Never mind that he was told to say that by one of the exact same metropolitan elite experts he was maligning. Never mind that he apologised abjectly for the comment the next day and retracted it altogether months later. In one pithy phrase, Gove manage to articulate the fury of millions of underachievers. He also single-handedly destroyed the last scrap of trust the public had in the system. Suddenly, the opinion of the man in the street was just as valid as that of someone who had spent years mastering her subject.

Gove’s (or, rather, Cummings’) ludicrous argument, that, because some trusted individuals once made an incorrect prediction, all their predictions are worthless, was the final justification for a Brexit vote among a decisive group of waverers. This arrant nonsense, combined with parallel whispering campaigns – Boris Johnson merrily defaming the EU in his columns for the Telegraph and Spectator, and the relentless demonisation of immigrants and Muslims in the Express, Mail, and far-right “news” sites – was what dragged Leave over the line.

For years, liberals ignored all this mudslinging, I guess because they assumed no one was gullible enough to believe it. Hopefully, they’re waking up to the fact that they can’t afford to ignore it any more.

(I’ve got more to say on this subject, but I’ve already wittered on for too long. Part two will follow shortly.)