Source and methodology
---Methodology--- https://www.univision.com/noticias/criminalidad-y-justicia/how-did-we-reach-these-conclusionsThe investigation of Costa Rica’s public defense system was born from a single question: Is there a link between the type of lawyer that defends a criminal case and the probability that a defendant will be convicted?Answering that question in Latin America is practically impossible given the systemic lack of official data organized for that purpose. The task was more viable in a small Central American country like Costa Rica.Costa Rica’s judicial branch has an abundance of unorganized public information that allows the first test to be conducted in the region.For four years, 10 Costa Rican journalists and lawyers built from scratch a database that is sufficiently solid to begin to explore this question. We took all rulings from the Second Circuit Criminal Court in San José from 2004-13 and extracted multiple variables.The database has partial or complete information on 11,183 crimes, distributed in approximately 8,000 rulings.For the analysis, Univision Data used a statistical method common in social sciences but practically unexplored in newsrooms, called logistic regression.This tool estimates the probability of conviction, acquittal and dismissal of defendants when they have a public defender or a private defense attorney by excluding several factors that could influence the outcome of a criminal trial. Among them are type of crime; the number of years of experience of the defense attorneys, judges and prosecutors who participate in a judicial process; the victim’s gender; the defendant’s gender; whether the legal process is normal or expedited (a variable representing cases in which a defendant has overwhelming evidence either for or against them); and whether the defendant is a national or a foreigner.Following these parameters, the study aims to define the degree of probability of being convicted in a criminal process that is attributable to whether a defendant cannot afford an attorney and must be appointed one by the state, compared with the option of having a private defense attorney.Using this methodology, the investigation also examined if the type of crime associated with Public Defender’s Office clients explains the possible disadvantage of using public attorneys.For example, a possible argument to explain the poorer performance of government-appointed public defenders compared to private ones is that people with low incomes who commit crimes against property in the public space, such as robberies and theft, could more easily be convicted. The concentration of these crimes among public defenders is undeniable. According to the database, of all processed crimes against property, 70% of defendants had a public defender.Our analysis demonstrated that the greater probability of conviction with government-paid attorneys is invariable, regardless of the concentration of certain crimes in this group.To be clear, while the majority of crimes against property are handled by public defenders, in general, crime categories by type of attorney are very similar.We also tested whether the poorer results of public attorneys is explained by the fact that they defend many cases where the defendant has an abundance of evidence against them. On average, 73% of expedited processes that we analyzed corresponded to public defense.In those processes, the defendant has enough damning evidence that they plead guilty in exchange for a lesser sentence if it is an ordinary trial and if the judge approves. However, when we looked at the probability of conviction excluding the influence of this variable, the disadvantage associated with a public defender was maintained for various crimes.--Limitations of the study--Our results should be interpreted with caution. This investigation does not conclude that public defenders are the direct cause of clients being convicted more frequently than those with private attorneys. Other factors not contemplated by the model could explain, at least in part, this disadvantage.One example is the fact that public defenders can’t choose which cases they represent, while private defense attorneys can.The results also can’t be generalized for the entire country, but are confined to the metropolitan area where the study was conducted. In this case, that area is the jurisdiction of the Second Judicial Circuit of San José.The results do suggest, for the first time, the possibility that the service of public defense in Costa Rica – considered a model in Latin America – does not counterbalance the weight of criminal defendants’ economic status in the administration of justice.---Our database--- https://www.univision.com/noticias/criminalidad-y-justicia/about-our-databaseDownload our database used in the statistical model: https://github.com/UnivisionBeta/abogado-de-los-pobres-code/raw/master/base-datos-modelo-publicada.xlsxDownload the code used to build the statistical model: https://github.com/UnivisionBeta/abogado-de-los-pobres-codeConfronted by systemic disinterest by Costa Rica’s Public Defender’s Office, a group of journalists and attorneys decided to voluntarily build a database to measure the performance of public and private defense attorneys. The database took four years to complete and was based on thousands of judicial rulings.Our goal was to fill the gap in information left by the Public Defender’s Office, which despite its annual budget of $56 million, appears uninterested in filling it.This database is the first of its kind. It is the largest effort to date in Costa Rica to compare the performances of public and private attorneys. After several rounds of filtration, analysis and other applied testing (detailed below), the primary tendencies that emerged did not vary, making the database a valid source of information.Of course, inconsistencies can be found because the original source of all of the information (judicial rulings) isn’t always precise. Second, despite putting controls in place, no database this large is error-free, due to human error (8,000 rulings and 11,000 offenses were analyzed). Third, sufficient official data is lacking that would have allowed some of the variables to be cross-checked.We are aware that our work is perfectible, so we are offering complete access to the database that we built. This allows readers to review it and, if necessary, alert us to inaccuracies or errors.We will evaluate all corrections and determine if they change the results of our analysis. As with all proper use of scientific method, we believe that new information will allow us to reach updated and better conclusions.Transparency and opennessWe decided to publish our study knowing that the database was perfectible for three reasons:1. There is no indication that the potential errors are extensive and would cause the model’s primary results to drastically change.2. The Costa Rican Public Defender’s Office has taken a systemic disinterest in generating statistics that would help to evaluate the performance of its attorneys. This would allow them to be compared with private defense attorneys, whose clients tend to be wealthier. Additionally, the country’s judicial branch currently cannot issue rulings based on complete and precise information. Beyond these two factors, there is a need for citizens to understand and discuss relevant issues. Apathy in the management of public institutions is dangerous. Not only does it hinder self-evaluation and criticism by the seats of governmental power, particularly given the lack of basic statistics, but it also allows these powers to benefit. As such, private, voluntary efforts to inspect in good faith the labors of the state, such as what Univision Noticias seeks to do with this investigation, are blocked.3. Journalism should aspire to a high degree of certitude for stories that are published. This report is no exception. But our study uses social research methods, and in the social sciences it’s usually impossible to obtain absolute certainty; more so when encountering complex research problems. Often, the generation of knowledge and public debate in a certain area leads to conclusions that are based on the best information available at a specific moment.Details regarding the filtering processBuilding this database was a four-year effort. In the last phase, with the support of the State of the Nation, we managed to finish the filtration of the database by using digital case file data possessed by the judicial branch.In this phase of filtering, several risks were identified regarding the database’s consistency:The data included in rulings often lacked standardization, which produced many empty cells. One of the missing variables in some rulings was the type of defense attorney. As important as it is, the judicial branch is still unaccustomed to registering this detail.Electronic rulings that are a primary source of data have proven to contain incorrect information on occasion.Building such an extensive database over a long period of time, and with a group of people working simultaneously, generates human typing errors. To minimize risks, the team conducted a series of cross-checks last year with both internal judicial branch sources and external ones. This filtration concentrated on finalizing rows that had the variables “type of ruling” and “type of process.”Some of these applied controls are:1. The classification was cross-checked with a list of public defenders.2. The status of the attorney was checked at the Attorneys Association.3. A database (report) generated by the judicial branch’s official statistical system (SIGMA) was requested, from which some rulings were verified to fill out the information. The report’s usefulness, however, is limited because the system has gaps and is only available for recent years, not the entire decade analyzed.4. It would have been possible to recover even more information by reviewing physical case files stored at the court, but access to these files was denied. The denial was based on a Superior Council resolution (Session 76-16, on Aug. 11, 2016, using the argument of the coordinating judge).5. In cases of doubt, where a defense attorney was classified as both public and private (listed as serving as both during the 10 years analyzed), this variable was cross-checked. It was done by calling many of them on the telephone; reviewing rulings from other courts (especially the Penal Branch of the Supreme Court) to corroborate their status around the time the Criminal Court in Goicochea was intervened; and through Internet searches. As a result, errors were corrected in which the ruling did not coincide with the database, or the ruling contained incorrect data (220 cases were corrected, which represents a little more than 3% of the database).6. In cases where doubt persisted, and the attorney couldn’t be located in other sources, the decision was made to eliminate the offense. After this filtration, the base used for the model included 6,584 offenses.7. Analysis models were calculated before and after filtration, maintaining general research tendencies in both cases. The statistical team from the State of the Nation Program also carried out a study of 100 routines in which the elimination of approximately 680 cases was simulated (by not being taken into consideration). The model’s central tendencies remained intact.