UK polls get it right

The reputation of the
UK polling industry has been restored following its performance in the December
2019 election.

The snap election held
in the UK on 12 December 2019 was an important test for the country’s opinion
pollsters. They had widely been regarded as having got it ‘wrong’ in the two
previous elections in 2015 and 2017, while the record of the polls in the 2016
referendum on Britain’s membership of the European Union had been widely
criticised. The industry could not afford its reputation to suffer another
battering – but it appears to have avoided that fate.

Ten companies
conducted polls whose fieldwork continued through to at least 10 December, that
is, just two days before polling day, and which thus can be regarded as
‘forecasts’ of the likely outcome. The results of these polls are shown in the
table.

Table: Results of final polls of vote intention
in the 2019 UK Election

The polls by
Ipsos MORI and Survation were conducted by telephone. All other polls undertaken
online. Sample sizes ranged between 1,009 and 3,174, except for YouGov whose
poll of 105,612 respondents was analysed using multi-level regression and
post-stratification. All
figures are for Great Britain (excluding Northern Ireland).

Conservative lead

All of the final polls
put the Conservatives well ahead of their principal rivals, the Labour Party,
and so, by implication at least, suggested that – given the country’s use of a
single member plurality electoral system that tends to advantage whichever
party comes first – the Conservatives would win a parliamentary majority. As a
result, given that what most voters and commentators wish to ascertain from the
polls is who will ‘win’, the polls are widely regarded as having made the right
‘call’. It was certainly a much better performance than in 2015, when on
average the polls put the Conservatives and Labour neck and neck but the
Conservatives enjoyed a seven-point lead, and, indeed, in 2017 when the final
estimate of an eight-point Conservative lead compared with what proved to be a
three-point one. Indeed, one company, Opinium, was not only spot on in its
estimate of the share of the vote won by the Conservatives and Labour, but also
in its figures for most of the smaller parties too.

That said, none of the
polls overestimated the Conservative share of the vote and, at nine points, the
average Conservative lead over Labour was three points below what actually
transpired. The ‘errors’ in the polls were not simply randomly distributed on
either side of the true estimate of the Conservatives’ performance, but rather
demonstrated a slight tendency to overestimate the Conservatives’ strength
relative to that of Labour. This replicates a pattern that has been in evidence
at most recent elections (with 2017 being a notable exception).

Room for improvement?

So, what lessons did
the polls learn from 2015 and 2017 – and why might there still be room for improvement?

The substantial overestimate
of the Conservatives’ strength in 2015 occasioned an independent inquiry
(sponsored by the British Polling Council and the Market Research Society) into
what went wrong. Its principal conclusion was that the polls had too many
Labour voters in their samples. There has long been evidence that polls tend to
be more successful at contacting Labour supporters than their Conservative
counterparts within the limited fieldwork periods they employ, a tendency they
attempt to counteract through stratification and/or weighting. However, in 2015
these efforts came to nought – the underestimate of the Conservative lead in
the published estimates simply replicated an underestimate that was also in
evidence in the unweighted data.

A key reason for the
oversampling of Labour voters appeared to be a failure to estimate correctly
the extent to which younger voters were less likely than their older
counterparts to vote. This mattered because in 2015 support for Labour
increased (and that for the Conservatives fell) among younger voters while the
opposite happened among older citizens. Those younger voters whom the polls
interviewed did support Labour in large numbers – but they proved atypical in
their willingness to vote in the first place.

The industry’s
response in 2017 was two-fold. One was to try to improve the quality of their
samples, including securing the participation of more people who do not vote.
The other was to weight their data so that their estimates of the age gap in
turnout matched the evidence from high quality academic surveys of what the
size of that gap had been in 2015. Trouble is, in taking the second step many
companies appear to have overcompensated for whatever over-representation of
Labour voters existed in their samples. Indeed, at one point, the average Conservative
lead in the unweighted data in the final polls was only slightly less than the
actual lead of 2.5 points and was much closer than the eight-point lead the
polls reported after weighting their data.

This time, most companies
dropped their attempts to weight their data to match pre-determined turnout
targets, simply relying on respondents’ self-reported probability of voting. That
step ensured that they did not repeat the error that occurred in 2017. However,
younger voters continue to be much more likely to vote Labour, still leaving
the polls potentially vulnerable to the problems that beset them in 2015. That
in the event they have come much closer to the mark suggests significant
progress has indeed been made in overcoming it. At the same time, however, the
fact that there was once again some tendency to underestimate the Conservative
lead over Labour suggests there is still room for improvement.