Project description

What is the probability that a defendant in Latin America will be convicted of a crime if he or she is represented by a public defender or a private attorney? With this question in mind, the data unit at Miami-based Univision News formed a team of journalists, attorneys and students who spent four years building a database in Costa Rica. This database allowed us to credibly and authoritatively answer this question.We chose Costa Rica because we are familiar with how the justice system works there. Compared to other Latin American countries, Costa Rica’s judiciary branch makes public information abundantly available to its citizens, although that information is not systematized and is hard to access. Nevertheless, a project of this nature is viable. The public defense system in Costa Rica is perceived as a model for other judicial systems in Latin America. We wanted to know if that perception reflects reality. We spent countless hours of our personal time to read and extract 25 variables from more than 8,000 judicial rulings from the digital archives of the Second Circuit Criminal Court in San José. We processed a total of 11,200 offenses. To clean the database, we teamed up with Costa Rica’s most prestigious research center, the State of the Nation. We then created a statistical model called logistic regression. Our research reveals that in Costa Rica, for certain crimes, the probability that a public defender’s client will be convicted at trial increases by 12% to 26% when compared to defendants who have private attorneys. This is especially true when the crime is complex and difficult to investigate.In an ideal world, whether the defense attorney is public or private should not affect the likelihood of conviction. In Costa Rica, it appears this is not the case. Univision’s findings cast doubt on the belief that a criminal defendant’s economic status does not affect decisions made in imparting justice. In five of the nine types of crimes analyzed – where most of the studied cases are concentrated (75%) – judges were more likely to convict defendants with public counsel.Defendants charged with fraud or forgery who have a government-appointed attorney have a greater risk of being convicted. In these cases – classified as crimes against public faith, which often require an expensive and complex technical defense – the probability of conviction increases by 26% for those with public defenders.By qualitative research, we also found out the Costa Rican Public Defender’s Office doesn’t keep official statistics on the number of cases assigned to each of its attorneys.This gap in information is important, as several academics, judges and former public defenders believe the number of cases handled by each public defender could be linked to poor results when compared to those of private attorneys.The lack of data also shows that the assignment of cases by the Public Defender’s Office is not based on technical criteria.This reality is even more relevant given the analysis by Univision Data, which shows that in Costa Rica, defendants represented by public attorneys in criminal cases, for certain offenses, are more likely to be convicted than if they had private lawyers.

What makes this project innovative?

1. This is and unprecedented investigation in Latin America:When it comes to statistics on the performance of public defense attorneys and their quality of service, Latin America lives in an age of darkness. Data and studies on public defenders – attorneys appointed to people who can’t afford private ones in a criminal case – are practically nonexistent. While in the United States one can easily find 13 quality investigations comparing the work of public and private defense attorneys, in Latin America there are none. Univision Data did exactly this, publishing a series of articles based on a study carried out in Costa Rica. It is likely the first investigation of its kind in Latin America.2. We built from scratch a database that no one else had ever done:Confronted by systemic disinterest by Costa Rica’s Public Defender’s Office, a group of journalists and attorneys decided to voluntarily build a database to measure the performance of public and private defense attorneys. The database took four years to complete and was based on thousands of judicial rulings. Our goal was to fill the gap in information left by the Public Defender’s Office, which despite its annual budget of $56 million, appears uninterested in filling it. This database is the first of its kind. It is the largest effort to date in Costa Rica to compare the performances of public and private attorneys. After several rounds of filtration, analysis and other applied testing (detailed below), the primary tendencies that emerged did not vary, making the database a valid source of information.3. We used a research tool barely used by journalists:For the analysis, Univision Data used a statistical method common in social sciences but practically unexplored in newsrooms, called logistic regression. This tool estimates the probability of conviction, acquittal and dismissal of defendants when they have a public defender or a private defense attorney by excluding several factors that could influence the outcome of a criminal trial. Among them are type of crime; the number of years of experience of the defense attorneys, judges and prosecutors who participate in a judicial process; the victim’s gender; the defendant’s gender; whether the legal process is normal or expedited (a variable representing cases in which a defendant has overwhelming evidence either for or against them); and whether the defendant is a national or a foreigner.4. We created an interactive tool to estimate the chances to be convicted conditional on the type of defender and other variables:This is the first time in Latin America that the readers can estimate from their smartphone the influence of several factors in the result of a trial. Based on a version of our statistical model, the interactive tool estimates the probability of conviction of defendants when they have a public defender or a private defense attorney by considering several factors that could influence the outcome of a criminal trial. Among them are type of crime; the number of years of experience of the defense attorneys, judges and prosecutors who participate in a judicial process; the victim’s gender; the defendant’s gender; whether the legal process is normal or expedited (a variable representing cases in which a defendant has overwhelming evidence either for or against them); and whether the defendant is a national or a foreigner.4. The results we obtained in Costa Rica were just the start of a 5 month collaborative investigation developed in seven Latin American nations:After reviewing the results, our investigative team knew that we had discovered important and unprecedented news. Following further research, we were troubled to find that no one else in Latin America had conducted this type of investigation before. We decided that the Costa Rican investigation was incomplete. We needed reinforcements to construct a solid argument that would help explain to the world, from Univision, the injustices that Latin American judicial systems cause for the region’s poor. We decided to form alliances with Animal Político (Mexico), Plaza Pública (Guatemala), Semanario Universidad (Costa Rica), El Tiempo (Colombia), CIPER (Chile), Ojo Público (Peru) and Agência Lupa (Brazil).For five months, Univision Data worked with these seven news organizations posing the following question: Would you be interested in collaborating on a project that shows the world what type of public defense systems poor people have access to in your countries? All of them accepted. We called the final product, which was pioneering in Latin America, “Attorneys for the poor” (www.abogadosdelospobres.com). The results were dramatic. Below, we describe those results according to country and the corresponding media organization: 1) In Peru (Ojo-publico.com), soldiers accused of human rights violations have more access to government legal aid than their victims do. We published a list of more than 40 members of the military accused of crimes ranging from torture to assassination who have private defense attorneys paid for by the Peruvian government. Yet, in October 2016, the administration of Pedro Pablo Kuczynski fired 200 public defenders and victims’ advocates who provided legal aid to citizens, including victims of human rights violations.2) In Colombia (ElTiempo.com), a small group of 153 public defenders and victims’ advocates must represent 234,964 cases filed by victims of the civil war. That comes to about 1,535 cases per attorney. The Colombian Attorneys Association warns that this saturation of cases public defenders must handle could worsen during the transition, before the Special Jurisdiction for Peace begins operating.3) In Mexico (AnimalPolítico.com), 8,000 mostly non-Spanish-speaking indigenous defendants are imprisoned while awaiting rulings on their cases. This process often takes several years, because public defenders are swamped with cases, attorneys who speak an indigenous language are lacking, and there isn’t enough time to properly work each case. The problem with the Mexican justice system is exemplified by the case of Adán Cruz Gallegos. Cruz, an indigenous Zapotec man from Oaxaca, has been in prison for seven years while waiting for courts to rule on his case. Cruz’s attorneys say he was forced to sign a confession in Spanish, a language he doesn’t understand. He also did not have a defense attorney who spoke his language.4) In Guatemala (PlazaPublica.com.gt), 15 offices that have an intercultural focus, each of them with an attorney on staff, are tasked with providing the fundamental right to defense for the 40% of Guatemalans who describe themselves as indigenous. That’s about 6.5 million people, most of them living in poverty. Having a public defense attorney who knows the local language is important to avoid what happened to Petrona Xol, a 63-year-old Q’eqchi’ woman who has been imprisoned for the past 12 years for a crime she says she didn’t commit. Xol never understood what was happening in the courtroom – not the charges the prosecutor’s office brought against her, the witness testimony, or the court’s verdict. The entire trial was conducted in Spanish, a language Xol doesn’t speak or understand. Xol communicates only in Q’eqchi’, her native tongue, which not even her defense attorney speaks.5) In Chile (CiperChile.cl), the criminal defense of thousands of people, most with limited incomes, was in the hands of a bankrupt company. On Sept. 20, 2016, the public agency signed yet another three-year contract for $1.36 million with a new company, Asesorías Jurídicas Integrales SpA (Integral Legal Consultants, or AJI), founded by DJP – the same company that had declared itself insolvent and stopped paying its attorneys three months earlier.6) Finally, Brazil (lupa.news) needs 10,578 more public defenders to guarantee public access to justice. Brazil has two prosecutors and two judges for every public attorney in the country. Thirty years after the country’s public defense system was created, the right to free legal assistance is nearly inaccessible for society’s most vulnerable citizens.

What was the impact of your project? How did you measure it?

1. After our publication, for the first time the Costa Rican Public Defender’s Office started to collect data to compare the performance of public and private defenders.2. Our research was published as a full chapter in the State of the Justice Report, edited by the most prestigious think tank in Costa Rica, called State of the Nation Program, a partially government funded organization. Because of that publication, our findings were discussed and accepted by the Judiciary System of Costa Rica. This means the highest authorities of the judicial system decided to consider our results and create a solution.

Source and methodology

---Methodology--- https://www.univision.com/noticias/criminalidad-y-justicia/how-did-we-reach-these-conclusionsThe investigation of Costa Rica’s public defense system was born from a single question: Is there a link between the type of lawyer that defends a criminal case and the probability that a defendant will be convicted?Answering that question in Latin America is practically impossible given the systemic lack of official data organized for that purpose. The task was more viable in a small Central American country like Costa Rica.Costa Rica’s judicial branch has an abundance of unorganized public information that allows the first test to be conducted in the region.For four years, 10 Costa Rican journalists and lawyers built from scratch a database that is sufficiently solid to begin to explore this question. We took all rulings from the Second Circuit Criminal Court in San José from 2004-13 and extracted multiple variables.The database has partial or complete information on 11,183 crimes, distributed in approximately 8,000 rulings.For the analysis, Univision Data used a statistical method common in social sciences but practically unexplored in newsrooms, called logistic regression.This tool estimates the probability of conviction, acquittal and dismissal of defendants when they have a public defender or a private defense attorney by excluding several factors that could influence the outcome of a criminal trial. Among them are type of crime; the number of years of experience of the defense attorneys, judges and prosecutors who participate in a judicial process; the victim’s gender; the defendant’s gender; whether the legal process is normal or expedited (a variable representing cases in which a defendant has overwhelming evidence either for or against them); and whether the defendant is a national or a foreigner.Following these parameters, the study aims to define the degree of probability of being convicted in a criminal process that is attributable to whether a defendant cannot afford an attorney and must be appointed one by the state, compared with the option of having a private defense attorney.Using this methodology, the investigation also examined if the type of crime associated with Public Defender’s Office clients explains the possible disadvantage of using public attorneys.For example, a possible argument to explain the poorer performance of government-appointed public defenders compared to private ones is that people with low incomes who commit crimes against property in the public space, such as robberies and theft, could more easily be convicted. The concentration of these crimes among public defenders is undeniable. According to the database, of all processed crimes against property, 70% of defendants had a public defender.Our analysis demonstrated that the greater probability of conviction with government-paid attorneys is invariable, regardless of the concentration of certain crimes in this group.To be clear, while the majority of crimes against property are handled by public defenders, in general, crime categories by type of attorney are very similar.We also tested whether the poorer results of public attorneys is explained by the fact that they defend many cases where the defendant has an abundance of evidence against them. On average, 73% of expedited processes that we analyzed corresponded to public defense.In those processes, the defendant has enough damning evidence that they plead guilty in exchange for a lesser sentence if it is an ordinary trial and if the judge approves. However, when we looked at the probability of conviction excluding the influence of this variable, the disadvantage associated with a public defender was maintained for various crimes.--Limitations of the study--Our results should be interpreted with caution. This investigation does not conclude that public defenders are the direct cause of clients being convicted more frequently than those with private attorneys. Other factors not contemplated by the model could explain, at least in part, this disadvantage.One example is the fact that public defenders can’t choose which cases they represent, while private defense attorneys can.The results also can’t be generalized for the entire country, but are confined to the metropolitan area where the study was conducted. In this case, that area is the jurisdiction of the Second Judicial Circuit of San José.The results do suggest, for the first time, the possibility that the service of public defense in Costa Rica – considered a model in Latin America – does not counterbalance the weight of criminal defendants’ economic status in the administration of justice.---Our database--- https://www.univision.com/noticias/criminalidad-y-justicia/about-our-databaseDownload our database used in the statistical model: https://github.com/UnivisionBeta/abogado-de-los-pobres-code/raw/master/base-datos-modelo-publicada.xlsxDownload the code used to build the statistical model: https://github.com/UnivisionBeta/abogado-de-los-pobres-codeConfronted by systemic disinterest by Costa Rica’s Public Defender’s Office, a group of journalists and attorneys decided to voluntarily build a database to measure the performance of public and private defense attorneys. The database took four years to complete and was based on thousands of judicial rulings.Our goal was to fill the gap in information left by the Public Defender’s Office, which despite its annual budget of $56 million, appears uninterested in filling it.This database is the first of its kind. It is the largest effort to date in Costa Rica to compare the performances of public and private attorneys. After several rounds of filtration, analysis and other applied testing (detailed below), the primary tendencies that emerged did not vary, making the database a valid source of information.Of course, inconsistencies can be found because the original source of all of the information (judicial rulings) isn’t always precise. Second, despite putting controls in place, no database this large is error-free, due to human error (8,000 rulings and 11,000 offenses were analyzed). Third, sufficient official data is lacking that would have allowed some of the variables to be cross-checked.We are aware that our work is perfectible, so we are offering complete access to the database that we built. This allows readers to review it and, if necessary, alert us to inaccuracies or errors.We will evaluate all corrections and determine if they change the results of our analysis. As with all proper use of scientific method, we believe that new information will allow us to reach updated and better conclusions.Transparency and opennessWe decided to publish our study knowing that the database was perfectible for three reasons:1. There is no indication that the potential errors are extensive and would cause the model’s primary results to drastically change.2. The Costa Rican Public Defender’s Office has taken a systemic disinterest in generating statistics that would help to evaluate the performance of its attorneys. This would allow them to be compared with private defense attorneys, whose clients tend to be wealthier. Additionally, the country’s judicial branch currently cannot issue rulings based on complete and precise information. Beyond these two factors, there is a need for citizens to understand and discuss relevant issues. Apathy in the management of public institutions is dangerous. Not only does it hinder self-evaluation and criticism by the seats of governmental power, particularly given the lack of basic statistics, but it also allows these powers to benefit. As such, private, voluntary efforts to inspect in good faith the labors of the state, such as what Univision Noticias seeks to do with this investigation, are blocked.3. Journalism should aspire to a high degree of certitude for stories that are published. This report is no exception. But our study uses social research methods, and in the social sciences it’s usually impossible to obtain absolute certainty; more so when encountering complex research problems. Often, the generation of knowledge and public debate in a certain area leads to conclusions that are based on the best information available at a specific moment.Details regarding the filtering processBuilding this database was a four-year effort. In the last phase, with the support of the State of the Nation, we managed to finish the filtration of the database by using digital case file data possessed by the judicial branch.In this phase of filtering, several risks were identified regarding the database’s consistency:The data included in rulings often lacked standardization, which produced many empty cells. One of the missing variables in some rulings was the type of defense attorney. As important as it is, the judicial branch is still unaccustomed to registering this detail.Electronic rulings that are a primary source of data have proven to contain incorrect information on occasion.Building such an extensive database over a long period of time, and with a group of people working simultaneously, generates human typing errors. To minimize risks, the team conducted a series of cross-checks last year with both internal judicial branch sources and external ones. This filtration concentrated on finalizing rows that had the variables “type of ruling” and “type of process.”Some of these applied controls are:1. The classification was cross-checked with a list of public defenders.2. The status of the attorney was checked at the Attorneys Association.3. A database (report) generated by the judicial branch’s official statistical system (SIGMA) was requested, from which some rulings were verified to fill out the information. The report’s usefulness, however, is limited because the system has gaps and is only available for recent years, not the entire decade analyzed.4. It would have been possible to recover even more information by reviewing physical case files stored at the court, but access to these files was denied. The denial was based on a Superior Council resolution (Session 76-16, on Aug. 11, 2016, using the argument of the coordinating judge).5. In cases of doubt, where a defense attorney was classified as both public and private (listed as serving as both during the 10 years analyzed), this variable was cross-checked. It was done by calling many of them on the telephone; reviewing rulings from other courts (especially the Penal Branch of the Supreme Court) to corroborate their status around the time the Criminal Court in Goicochea was intervened; and through Internet searches. As a result, errors were corrected in which the ruling did not coincide with the database, or the ruling contained incorrect data (220 cases were corrected, which represents a little more than 3% of the database).6. In cases where doubt persisted, and the attorney couldn’t be located in other sources, the decision was made to eliminate the offense. After this filtration, the base used for the model included 6,584 offenses.7. Analysis models were calculated before and after filtration, maintaining general research tendencies in both cases. The statistical team from the State of the Nation Program also carried out a study of 100 routines in which the elimination of approximately 680 cases was simulated (by not being taken into consideration). The model’s central tendencies remained intact.

Technologies Used

To build the database: Google forms, Open Refine and Excel.Data modeling: R and R Studio.Building the interactive tool: D3 and Java Script.

Project members

DATA RESEARCH: Alejandro Fernández S., Andrés Fernández A., Daniel Salazar M.REPORTING AND TEXT: Alejandro Fernández S. and Hulda MirandaEDITING: Ronny RojasVISUALIZATION: Luis MelgarINTERACTIVE: Antonio Cucho and Juan Jesús GómezDATABASE BUILDING AND FILTERING: Alejandro Fernández S., Samantha Fonseca U., Guillermo Fernández R., Daniel Salazar M., Joanna Nelson, Mariana Álvarez, Ernesto Nuñez, Alejandro Robles L., Marilyn Carvajal P., María Jesús Hernández C, Alina Rodríguez, David Bolaños, Carolina Mendoza, Rolando Leiva (Estado de la Nación) and Mario Herrera (Estado de la Nación).

Link

Additional links

Project owner administration

Contributor username

Followers

Click Follow to keep up with the evolution of this project:
you will receive a notification anytime the project leader updates the project page.