Sam Bright speaks to Amelia Showalter – Barack Obama’s analytics chief – about big data, the shock of Hillary Clinton’s defeat, and how to fundraise $1bn
Across the western world, progressive politics is being forced into retreat by the raging forces of populism and isolationism. In the midst of this pandemonium, Barack Obama’s campaigns possess an almost mythical aura.
To many people, the Obama campaigns provide much-needed evidence that – not too long ago – an articulate, intelligent candidate managed to persuade the United States to elect hope over fear.
But the Obama campaigns are not only ideologically inspirational. They also provide a model of how to run an innovative campaign. As the Economist wrote in January 2012: ‘Eagerness to master new technologies fast has been a feature of both Obama’s campaigns – with an emphasis on social media in the first and on big data and analytics in the second.’
Amelia Showalter knows more about these innovations than pretty much anyone else, having led the efforts of the 2012 Obama campaign to master big data and analytics.
‘I was the director of digital analytics … I was focused on our digital communications, which largely involved communicating to our supporters’, Showalter tells me over the phone from the office of her digital consultancy firm in Washington DC.
‘A lot of our success was in fundraising. The Obama campaign raised a billion dollars, which is kind of ridiculous. I think it’s silly that we spend so much money on politics in our country, but it’s what you need to do to win.’
Showalter was one of many young prospects hired by the Obama campaign – poached from assorted Ivy League universities. Within four years of graduating from Harvard, Showalter was appointed to lead Obama’s digital analytics team.
In theory, her job was quite simple: to run tests, mainly via email, to see which messages and tactics raised the most money, and then implement the most successful variations. But, with the ability to test unlimited strategies and slogans, Showalter had to be discerning about which experiments to prioritise.
One of her most successful involved assessing how many fundraising emails to send to Obama supporters.
‘I really pushed for this experiment. I thought we needed to test this over the course of a long period of time to see what happens when we send more emails versus fewer emails. What we found was this it was significantly better to send more emails.’
As a consequence, Obama supporters received a barrage of emails over the last few months of the campaign – helping to raise ‘somewhere between $20m and $30m in additional donations for the campaign’.
So, if you are wondering why politicians are constantly bugging you with emails, Showalter is probably to blame. ‘I realise that it has made everyone annoyed’, she says, chuckling. ‘Now, politicians know that it’s better to send more emails – but we had an election to win.’
Showalter insists that forensic data testing can benefit a campaign. But, is it possible to assess the impact of digital campaigning on an election result? After all, commentators in the UK have been eager to attribute Labour’s underwhelming-but-not-disastrous general election performance to Jeremy Corbyn’s social media success.
‘It’s all about having a marginally better operation than you would have otherwise’, Showalter contends.
‘It gets you maybe an extra percentage or two of the vote. These models are not going to dramatically change the nature of the election … it’s not going to change the minds of massive portions of the electorate. But, in a close race, a few percentage points matter.’
Digital innovation could certainly have tipped the presidential election in 2012. In the pursuit of a second term, Obama won 51.1 per cent of the popular vote – with Republican candidate Mitt Romney less than four per cent behind. And Romney’s digital operation was widely viewed as rudimentary, compared to Obama’s. In the wake of the election, Breitbart editor Mike Flynn singled out Romney’s digital director Zac Moffatt for criticism, tweeting: ‘Zac Moffatt made millions while he screwed the country because of his incompetence. Can we please agree that he is done in politics?’
Yet Moffatt is not the only digital expert to have been chastened since Obama’s re-election. The big data industry – in particular political polling – has been publically crucified, both in Britain and the United States. Indeed, there is a widespread belief that pollsters have got the results of elections decisively wrong in recent years, notably the 2015 British general election, the 2016 referendum on the United Kingdom’s membership of the European Union, and the 2016 United States presidential election.
‘I definitely thought Hillary [Clinton] would win’, says Showalter – who has been forced to question her own faith in the industry.
‘I believed all the data. 2016 was a moment of humility for everyone who believed in data.
‘I actually spent election night at a party at a major political data firm – hanging out with other data nerd friends there. At the beginning of the night, all these very smart people who know everything about political data were hearing from their friends on the Hillary campaign that the numbers were looking good.
‘But, as the night progressed, we heard rumours and realised that it wasn’t going to turn out well. I don’t actually want to think about that night very much anymore.’
In the aftermath of this political shock, Showalter says her pre-existing ideas about political polling and data analysis were thrown into question.
‘I wish I could say I knew all along, but I didn’t know. None of us knew. We had too much confidence in our data process’, she says.
‘As a data person I used to be extremely sceptical of focus groups. I used to think focus groups were really silly because it’s not a large sample size and it’s not a controlled environment. It’s just a few people, spouting off their grievances or whatever.
‘Now I think that focus groups are actually really important for helping to contextualise the data.’
However, in Showalter’s view, her peers working in political data are not the only ones responsible for the political surprises of the past two years. Journalists and pundits also need to reflect on how they report political polling. After all, she says, a poll is a prediction, not a prophecy.
‘People don’t understand that there’s a margin of error. Sometimes, no matter what you do, there’s always going to be variability that you cannot account for.
‘We didn’t have a good way of measuring that there were Trump supporters who would vote, even though they hadn’t voted before … When something really unusual is happening out there in the world, by definition it’s hard to predict what is going to happen.’
This analysis is difficult to critique. In the past, pollsters have predicted election results with devastating accuracy. In 2012, while Showalter was harvesting a billion dollars for the Obama campaign, Nate Silver was building a formula that would correctly predict the winning candidate in all 50 states (after correctly predicting 49 out of 50 in 2008).
Data gurus undoubtedly need to adapt to changing political circumstances. The rules of politics are not static, and pollsters will suffer major setbacks if they simply copy and paste their blueprint from one election to the next. There are a number of strategies that can help data experts to read the mood in a nation, but face-to-face conversations with voters cannot be neglected.
Few people in journalism and politics understand big data (ie the process of analysing large data sets – such as how people react to different fundraising emails). Preferring to find an excuse to dismiss big data rather than understand it, many people have used recent polling errors to malign the idea. This is surely not the right approach. To get the most out of big data, we need a better understanding of both its possibilities and its limitations.
Journalists should not solely rely on polling to tell them whether an election will swing one way or another, just as politicians should not solely rely on data analysis to win the same election. But polling is undoubtedly the most accurate indicator of an election result, and data testing can tip an election – if only by shifting public opinion by one or two percentage points.
As we enter a new political era, it is crucial for progressives to adapt. We cannot recycle the strategies of the past and expect to win in the future. But equally, we should not dismiss the lessons of previous victories. Obama’s digital success proves that we must innovate with technology to stand the best chance of winning.
Fortunately, Showalter is confident that data will not be ditched in political campaigning.
‘I don’t think that data is dead in American politics. There is just a little bit more scepticism.’
Sam Bright is digital editor at Progress. He tweets @SamBright_Ltd.
Progressive centre-ground Labour politics does not come for free.
Our work depends on you.