Project description

The Trace\’s NRA Campaign Spending Tracker and its related Twitter bot gave the public real-time access to the group’s federal campaign spending. Starting 100 days before the 2018 midterm elections, our code pulled and sorted the NRA’s latest filings to the Federal Election Commission, which allowed us to give media, analysts, researchers and the voting public a running tally of every dollar the group had spent on each congressional candidate it backed, and how much it spent to oppose those it wanted to see defeated.

Together, these tools helped us serve our audiences with distinctive reporting on a consequential election while establishing one of the major narratives of the 2018 midterms.

What makes this project innovative?

The Trace’s resident reporter-slash-coder, Daniel Nass, built this automated NRA Campaign Spending Tracker to make it easy for journalists and the public to follow the group’s spending in the 2018 midterms. We initially conceived of this as a tool that would live on The Trace's website, but we extended its use to Twitter, where many journalists, analysts and voters gathered on election night to watch the returns. Our Twitter bot recorded whether NRA-backed candidates won or lost their races, compared those results to previous election cycles, and shared those results and comparisons with our Twitter audience using a reader-friendly format. The tracker and bot are the first tools to watch the NRA's spending this closely in real time.

What was the impact of your project? How did you measure it?

The NRA owes its fearsome political reputation to its muscular election spending. In previous years, the gun group had poured tens of millions of dollars into dozens of races, winning big in many of them. But our tool uncovered a significant shift for the group: the NRA was spending far less than it had in recent electoral cycles. When the tool flagged the NRA’s sharply reduced spending, it identified a key storyline that informed reporting by major national outlets (including CNN, the Washington Post, Bloomberg News, and McClatchy) on the group’s diminished political sway. More than 10 local and national media outlets cited our tracker in their coverage. Over the election season, nearly 30,000 users viewed our tracker, spending an average of 3:30 minutes with the tool.

Source and methodology

All of the data used in this news app came from publicly available sources. Some was obtained through APIs, some was scraped, and some was compiled by hand. The campaign finance data was sourced from the Federal Election Commission’s API and ProPublica’s campaign finance API, both of which draw on raw filings submitted to the FEC. For independent expenditure data, ProPublica’s API was used because it parses the FEC filings faster than the FEC itself — ProPublica’s API generally makes new data available within a few minutes, rather than the following day. Direct contribution data, which is filed on a monthly basis, came from the FEC. Candidates’ states and party affiliations also came from the FEC; this information was manually checked against other sources such as Ballotpedia and corrected if necessary (e.g. there were a few cases where the FEC listed candidates currently running for Senate seats as running for House seats they had previously held). The campaign finance data was stored as two arrays: one of individual line-item expenditures, and one of candidates. These two datasets joined on a candidate ID field. The analysis of the campaign finance data mainly involved grouping the spending data on various fields, such as state, race, candidate, and party, and then filtering or summarizing. This allowed the user to access a wide variety of views of the data, such as a summary of spending totals in every state, or itemized spending in a single race over time. Because independent expenditures disclosed in initial 24- or 48-hour reports are sometimes amended in monthly reports, the script powering the project automatically cross-referenced new filings with their older counterparts and replaced the data when needed. As Daniel developed the project, he consulted with campaign finance specialists at the Center for Responsive Politics and ProPublica in order to make sure he was using and interpreting the data correctly. The project incorporated a few other data sources. Because the FEC’s API does not differentiate between general elections and special elections, a list of the special elections in the 2018 cycle was obtained from Ballotpedia. The Twitter bot used a hand-built list of candidates’ Twitter handles to tweet at the candidates who were receiving the NRA’s support. That list was compiled by manually searching Twitter and Google to find the username of every candidate whose name appeared in the NRA’s filings. After the NRA published its candidate grades, those were incorporated into the tracker and bot as well. The grades were concealed behind a search function that only showed results for candidates on the ballot at a specific street address. To obtain all of the House candidate grades, Census shapefiles of the congressional districts were analyzed to identify a point within each one. Those points were fed into Google’s reverse geocoder API to map each point to an address, which were then plugged into the search form to scrape the results. Both the list of candidate Twitter handles and the list of grades were manually edited to standardize the candidate names so that they would join cleanly with the names present in the FEC data.

Technologies Used

The real-time campaign finance data at the core of the project came from ProPublica’s Campaign Finance API and the Federal Election Commission’s API. A serverless AWS Lambda function written in Node.js periodically hit both APIs and wrote the latest data to a JSON file stored in an S3 bucket. That data was consumed by a single-page news app written in Svelte. Since Svelte is a young framework, all of the components were written from scratch, such as the typeahead and the sticky header and footer. The campaign finance dataset was fairly small (less than 2,000 line items by the time of the midterms) and updated frequently, so the grouping, summarizing, and filtering were calculated on the frontend in JavaScript. The line chart visualizations were created with D3. For the previous years of NRA spending shown in the last chart, filing data was downloaded in CSV format from the FEC, analyzed in R using the tidyverse suite of packages, and exported as JSON which was included in the frontend bundle. Another Node.js Lambda function powered the Twitter bot, which used the Twitter API to post a tweet every time a new filing appeared in the data. To scrape the NRA grades, the shapefiles were analyzed using the SF package in R. Node.js scripts were used to access the Google reverse geocoding API and to scrape the results from the NRA’s website.

Project members

Daniel Nass

Link

Additional links

Followers

Click Follow to keep up with the evolution of this project:
you will receive a notification anytime the project leader updates the project page.