Instant sellouts. Soaring ticket prices. Gouging on scalper websites.
Anyone who’s tried to buy seats for big concerts or sports events knows there’s something rotten going on within the online ticketing market.
CBC News and the Toronto Star decided to investigate.
We began with a simple idea: track online ticket sales for big events – to understand the supply and demand – and the profits being squeezed from fans through the ‘secondary market’ on scalping websites like StubHub, SeatGeek and Vivid Seats.
In early 2018, we launched a big data project, building Python-powered web scrapers to repeatedly scour TicketMaster.com, the official box office platform, and five popular resale sites to track how tickets are bought and sold – and at what prices.
Target # 1: Toronto Blue Jays home-opener. We watched in near real-time as the first baseball game of the season went on sale and tickets quickly moved from the primary market to the secondary market. We exposed how the Major League Baseball team is colluding with StubHub, profiting from the scalping of its own tickets. Fans were outraged.
Target # 2: We expanded our focus, and began tracking tickets sales for the Toronto Maple Leafs’ first home playoff hockey game, and 10 high-profile summer concerts in multiple cities featuring artists like Taylor Swift, Elton John and the Foo Fighters. We exposed how fans are losing out to scalpers, corporations and season ticket holders, for the first time presenting hard data and proof of why it is so tough to get affordable sports and concert tickets from the box office.
Target # 3: We zeroed in on Ticketmaster, the world’s biggest box office, on its way to become the biggest scalper, too. Our scraping and analysis of tickets for Bruno Mars’ 2018 world tour exposed a bag of dirty tricks now being used by the box office to make ever-higher profits off fans. With these findings in hand, we went undercover to Las Vegas where we caught Ticketmaster on hidden camera recruiting industrial scale scalpers and offering online tools to help share in the profits.
What makes this project innovative?
Scale. This is the largest ever investigation into online ticket sales, which made global headlines. CBC News senior developer William Wolfe-Wylie built web scrapers that regularly tracked ticket listings offered on six different platforms, and tied to more than a dozen sporting events and music concerts across Canada for the better part of a year. When all was said and done, our team collected over two million rows of data for 14 events in 6 cities, several of which were monitored closely from the time the box office opened to the moment the artist set foot on stage. Real-time analysis became vital to the project as CBC News senior data journalist Valerie Ouellet wrote and ran dozens of SQL scripts to track price changes and fluctuations in supply and demand as they happened. We were able to determine that only 96 regular tickets were initially put on sale to the public for a Toronto Maple Leafs playoff game, yet at the last minute, Ticketmaster released hundreds of tickets that had been held back, manipulating both the supply and the price. The data analysis allowed CBC and Toronto Star to expose some of the most absurd distortions in the ticketing market, including finding Ontario’s most overpriced concert ticket of the summer: a regularly-priced $325 ticket to a Taylor Swift concert posted at $66,000 on a resale platform. Our model on numerous stories published in 2018 relied on SVG venue maps of stadiums and concert halls to allow our audience to see exactly which tickets in the stands were scalped for the highest prices. We were able to track individual seats, rows and sections, sharing our findings widely through social media using innovative graphics and GIFs, to help tell the story of how the ticketing industry really works. The data was the cornerstone to a string of stories and investigative reports that gave fans around the globe hard proof of how ticket sales are manipulated for profit by scalpers and the world’s biggest box office - Ticketmaster.
What was the impact of your project? How did you measure it?
The CBC News / Toronto Star stories reached huge audiences around the globe. Our project sparked investigations by authorities in Canada and the U.S., including the U.S. Senate committee on consumer protection. Band managers for major, well-known acts, including Mumford and Sons, Radiohead and the Pixies, were outraged and demanded changes to their contracts with Ticketmaster. Fans launched four class action lawsuits accusing Ticketmaster of deception. On the CBC News website, these stories each garnered hundreds of thousands of page views. One of the stories in this series was among the most viewed stories of the year on our cbc.ca website and had hundreds of thousands of engaged minutes. Our ticket scalping series also generated significant engagement on social media. Several tweets and Facebook posts relayed by both media outlets were shared hundreds of times and generated hundreds of comments - including several tips - as well. One Facebook Live with Dave Seglins, CBC’s lead investigative reporter, and music journalist Alan Cross was viewed more than 48,000 times. For additional context, on the same Facebook page, a live video featuring Canadian Prime Minister Justin Trudeau has 124,000 views, while another live ‘Q&A’ addressing a fatal bus crash that became a national tragedy has 157,000 views.
Source and methodology
We collected three databases from Ticketmaster: venue maps, offer lists, and listings. With that data, we created a master map of the stadium that included the original face value, section, row number and seat number of each ticket. We compared this master map of the venue against the tickets on five resale platforms (StubHub, VividSeats, SeatGeek, TicketsNow and Ticketmaster Resale) against the tickets originally listed for sale by the primary vendor (TicketMaster) and see how prices and the total stock of tickets varied. For clarity’s sake, we collected the same variables for all six vendors and structured our tables exactly the same way, which helped with cleaning, queries and analysis. By comparing the exact same listings over time, we could see how many new tickets were released on the market every day, as well as compare prices and markups for tickets posted on multiple websites. Our project came with several important caveats, just like any web scraping operation. Some tickets were simultaneously posted for resale on multiple platforms. Some tickets were posted for resale with no specific seat, row or section, for example, VIP packages or general admission sections, and as such could not be counted accurately. For that reason, we restricted our methodology to sections and events with assigned seating.
A Python app drove a Selenium-based scraper to gather the data directly from the API endpoints offered by each major vendor. Data collected was pushed into a SQL database, which was then cleaned and analyzed using MySQL Workbench, with a bit of Excel to tidy up and present pivot tables to reporters. Our analysis workflow duplicated effort to double-verify conclusions. William and Valerie performed the analysis independently before comparing process and conclusions to verify each other’s results. Once our findings were confirmed, we would share them with our whole team via email.
Valérie Ouellet, William Wolfe-Wylie, Dave Seglins, Rachel Houlihan, Marco Chown Oved, Robert Cribb, Richard Grasley, Andrew Bailey. (Senior editors: Lynn McAuley, Raj Ahluwalia)