RADAR (Reporters and Data and Robots) is a news agency that can bring data journalism to any local newsroom every day. We combine human editorial skills and AI tools to write stories from open data on a previously unimagined scale.
We are an eight-person start-up that began operating in September 2017. In 18 months, we have invented a new method to produce data-driven news and devised a unique technical workflow from data to distribution. We produce over 8,000 stories each month for our subscribers.
Our work is based on systematically mining open data – a source of local news stories that we felt had been neglected. Much open data in the UK is localised but newsrooms lack the time and skills to exploit it.
We mine the data to find the local stories, then we write our stories in template form using Natural Language Generation software. This allows one journalist to amplify their work, producing hundreds of localised versions of the story.
We write stories across all main news beats – health, crime, education, transport, lifestyle etc.
Our stories give news organisations a strong supply of fact-based public interest journalism and reveal information about local communities that would otherwise be inaccessible to citizens.
What makes this project innovative?
The biggest innovation is perhaps the scale of what we are doing at RADAR – ten data journalism projects each week resulting in thousands of published stories across the UK. Other organisations have used NLG-based bots for match reports in sport or financial results. RADAR is the only organisation to take the next step in daily, hard news production. To enable this, we have built a bespoke data-to-distribution production system. Working across so many data sources requires a flexible approach that is editorially led. Instead of NLG as a developer-led project our approach is to put the tech solely in the hands of the journalists, who treat it as a writing tool. This new way of working for data journalism means we can harness open data and automation to deliver compelling and important data stories every day. Such large-scale production requires targeted distribution. We work with multiple data granularities, a potential barrier to easy story discovery. We have geo-mapped all the major data granularities into clearly identified “channels” that mirror the local government structure of the UK and fit with publisher footprints. It results in a smart, easy to use web interface for our customers or delivery of their “channel” streams via API.
What was the impact of your project? How did you measure it?
RADAR offers benefits for large and small news publishers. Our pricing model is based on the number of geographical “channels” bought, so we serve large organisations which own many titles, small independents and one-person community hyperlocals. During our beta period (Jan-Dec 2018) we distributed to the whole UK local media market. In January 2019 we moved to a paid subscription model. Already, we are providing a rich spine of public interest journalism to 329 digital, print and broadcast outlets across the UK. We see the impact in the number of published stories. In broad terms, the great majority of our distributed stories are quickly published online by subscribers. In most cases, these are ‘as is’ using our copy with perhaps a change to the headline. As an agency we do not have access to our subscribers’ audience metrics. Overall, subscribers report that impact is as good as, or better than, their local newsroom originated stories. Our largest subscriber helped us with an impact study last autumn which showed that, for one of their units, 16 of their 30 best performing stories in a month were produced by RADAR. The value placed on our stories is evident in print usage. In the impact study we found that RADAR stories appeared as page leads in the main news section, and that the most common position for RADAR content was front page splash. Our work has had a relatively high profile with media news coverage (FT, BBC), and recognition in industry reports (GEN, WAN/IFRA) and scholarly articles (CJR, RISJ). The government-commissioned Cairncross Review on the future of sustainable journalism in UK said that RADAR “holds particular promise” for public interest journalism
Source and methodology
The RADAR process begins with a systematic surveillance of open data releases. Over the past year we have monitored over 4,000 data releases to select those with strong news potential and localised data breakdowns. Each working week has a fresh workflow. We start by assembling a weekly data release calendar, and supplement that with story ideas where we feel data research can offer fresh local insight on a topical issue. We focus on official open data sources such as UK government departments, the Office for National Statistics and public bodies such as the NHS and police forces. We have begun to add other sources - NGOs and charities, and some private providers. The data we use is, broadly, robust and well-structured. Individual reporters are allocated a data release, and have full ownership of its progress. They undertake data analysis and story discovery. They conduct research to fully understand the context of the data and undertake supporting interviews with experts, politicians etc for inclusion in the copy. The reporter will then write their project in the form of a Natural Language Generation template. We have trained all our reporters to use NLG software. Each template, on average, will generate 200 versions of the project for local distribution.
We have built an end-to-end production system to support our unique workflow. This starts with a bespoke data management system to operate the RADAR service. This system is used for storage of source data material and is integrated with third party NLG software from Arria to power our production system. The production system sends a dataset to the completed NLG template at a URL, accessed with an API key. The resulting story versions are returned and sorted into geographical “channels” for distribution by our geo-mapping system. We built our geo-mapping system using ONS area coding, plus some bespoke number coding for institutions without an ONS reference number. Stories are made available to clients on the RADAR website or can be distributed directly to their CMS via an API.
Gary Rogers Alan Renwick Ralph Blackburn Joseph Hook Miguel Rodriguez Harriet Clugston Isabelle Kirk