Our General Election coverage in 2017 was our best yet, with live results dashboards working in conjunction with timely analysis that brought out of our best news lines of the election. We built live-updating results graphics that could be embedded anywhere on our website to tell the immediate story of the night, which meant that data journalists could be free to focus on what we do best: analysis. One of the best examples of this was our live seat-by-seat machine learning predictor model. The data used to power the model was a mixture of demographic data – obtained through various governmental departments and the 2011 Census – and historic polling figures compiled from various polling companies. We used this information to predict the votes for each party in each constituency. These predictions were then communicated through fun dials with illustrated images of Theresa May and Jeremy Corbyn. It allowed readers to pull out predictions for their specific seat – something that wasn’t offered elsewhere – and also consistently forecast a hung parliament when others were indicating the Conservatives may sneak a majority.We also produced news stories out of data that led the website and were featured heavily in the newspaper. An example of this was a story showing how, due to the UK’s First Past the Post electoral system, Theresa May was just 400 votes short of a majority. If she had convinced 400 people to vote Conservative across eight constituencies, she would have won. This focused on the key story of the night: How was it possible that Theresa May lost her majority, when almost every pollster, commentator and expert was expecting her to dramatically increase the slender Conservative majority? Such a piece of analysis was not the most complex piece of data journalism, but it did what a good piece of data journalism should. It pulled out the key news story of the night, and found an original line of analysis that highlighted the crux of it. Alongside our live-updating results page (http://www.telegraph.co.uk/news/general-election-results-2017-maps-breakdown/), it worked effectively in allowing our readers to understand just how close the election really was.
What makes this project innovative?
This was the first time the Telegraph had been this digitally ambitious in covering a general election. The main results page was innovative because it was composed of a series of individually parcelled elements - all driven by live data - that were also available to be embedded in any of our other stories. This innovation was a huge step forward for us, as it meant that we could put our focus and resource entirely into analysis instead. The story on how Theresa May was just 400 votes short of a majority was a result of this, because our data journalists had the time to combine live data visualisations and traditional reporting methods to pull out a key news line of the General Election. None of our competitors had communicated this story yet, so it was important for us to break this data-led story and tell it with succinct and effective graphics. Our live predictor was innovative due to the fact that we created our own machine learning model based on historic polling trends and demographic data on a constituency level. It was very much an experiment, but one that paid off on the night and gave us an edge over our competitors. Despite being an experiment, it was still incredibly accurate. It predicted a hung parliament immediately in the model’s first iteration - and it stuck with this prediction, even when local results were coming in that meant that others thought that the Conservative Party might sneak a majority.
What was the impact of your project? How did you measure it?
All of our election content did very well in terms of traffic with the main results file picking up the bulk of this. The 400-votes story was featured on page five of the newspaper in the aftermath of the election, and splashed the website. Its main results graphic was used across social media as its clarity in making the point was immediate and clear for our readership. Our predictor was successful in its accuracy, consistently forecasting a hung parliament when others were indicating the Conservatives may sneak a majority. It performed particularly well in Scotland, forecasting the exact number of seats that the Conservatives would gain from the SNP within the first iteration of our forecast.
Source and methodology
When it came to the General Election results we set up a live-updating spreadsheet and a corresponding set of live results graphics in order to give our readers immediate analysis. The Press Association provided the latest data via FTP server. We created a serverless application from which we programmatically translated the data and stored it in a Google spreadsheet, allow us to add any manual edits we might want. Throughout the results night, we would scour over this dataset and point out key findings for our live blog and social media teams. As well as powering our live analysis through the spreadsheet, the feed also powered our live graphics, which were created in HTML/CSS, in order to visualise our live analysis.Using R and Microsoft Excel, we analysed data to try and find the best possible lines for the story - whether that was found in correlations, outliers, patterns or any other surprising or noteworthy data points. These would then be communicated to the relevant subject desks and we would collaborate with specialists to figure out the best way to tell data-led stories to our readers.The data used to power the model was a mixture of demographic data - obtained through various governmental departments and the 2011 Census - and historic polling figures compiled from various polling companies. We used this information to power a model which predicted the votes for each party in each constituency.
R and Microsoft Excel were used for sourcing and analysing our data, while R, QGIS, HTML/CSS and the Adobe Suite were some of the tools used for visualisation. While many day-to-day tasks can be performed adequately with Microsoft Excel, we also use templated R scripts to repeat analysis and visualisation tasks that we will regularly repeat. All of these visuals are then uploaded to The Telegraph’s Particle CMS, making them visible, accessible and reusable for all the newsroom’s journalists.