SDSU Researchers Watered Down a Police Racial-Profiling Study
When a long-awaited study on whether the San Diego Police Department engages in racial profiling finally dropped in late November, the results were unsurprising: It found that black and Hispanic drivers were more likely to be searched, though they were less likely to actually have contraband items, and that minority drivers were more likely to be subjected to field interviews.
When it came to the overarching question of whether officers and SDPD as a whole showed racial bias, however, San Diego State researchers were restrained: Though they found significant differences in the way minorities and white drivers were treated, researchers were careful to point out that such differences 鈥渁re by no means unique to the SDPD鈥 and that findings only 鈥渟uggest鈥 that implicit, or unconscious, bias 鈥渕ay exist鈥 among officers.
But a draft copy of the study obtained by Voice of San Diego through public records requests was far more aggressive. City officials fought to avoid releasing the drafts publicly and said their disclosure 鈥渨ould likely increase community tension and discontent,鈥 but copies were provided by San Diego State University, which conducted the study.
During the process of revising the study from a draft to the final version that was presented to the City Council, harsh language was softened and some findings were taken out entirely.
Among the changes:
鈥聽In more than two-dozen instances, the word 鈥渂ias鈥 was replaced with 鈥渄isparities.鈥
鈥聽An early draft recommended the department stop making traffic stops for minor violations unrelated to public safety, and instead simply issue citations by mail鈥攕omething, researchers noted, other departments are exploring.
鈥聽Also cut from the final draft was the finding, via police survey, that the majority of officers felt they wouldn鈥檛 benefit from additional training in fair and impartial policing.
鈥聽The final version of the study found that black drivers were more likely than white drivers to be stopped in only one of the San Diego Police Department鈥檚 nine divisions, northeastern, which includes the largely white neighborhoods of Mira Mesa, Rancho Bernardo, Rancho Pe帽asquitos and Scripps Ranch. A draft version reached a different conclusion, finding evidence of racial disparities in three divisions, not just one, and, in aggregate, all police divisions located north of Interstate 8, often considered to be San Diego鈥檚 racial and economic dividing line.
On that last point, Joshua Chanin, professor of public affairs at聽San Diego聽State University and the study鈥檚 lead researcher, said the final change was made because the researchers decided to use a different threshold to determine whether a finding was statistically significant.
Put simply, researchers initially considered a finding statistically significant鈥攎eaning the data showed evidence of racial profiling鈥攊f there was at least a 90 percent chance the finding was true. This was later changed to 95 percent.
Findings for southeastern and northern divisions were just a few points away from 95 percent: In the Northern division, the likelihood that racial disparities in traffic stops weren鈥檛 due to chance was 93.4 percent. In the southeastern division, it was 92.3 percent, and for all divisions located above the I-8 divide, 94.2 percent. This was noted in a draft of the study provided to the city on Oct. 27:
鈥淸O]ur analysis of combined stops from all five divisions above Interstate 8 shows that when compared to Whites, Black drivers are 15 percent more likely to be stopped during daylight hours, when driver race is visible, than after dark, when driver race is obscured,鈥 it read.
But by the final draft the finding was the opposite: 鈥淎nalysis of the aggregated data from [divisions north I-8] shows no statistically significant difference in the daylight-darkness stop patterns of Black and White drivers.鈥
The 95 percent threshold is considered a standard in scientific research, and Chanin said the goal was to produce a study whose findings were 鈥渦nassailable鈥 by the city and police department.
City Councilman David Alvarez said he wishes researchers had found a way to explain that some findings were on the cusp of statistical significance.
鈥淚t鈥檚 even more important when you鈥檙e dealing with the public who aren鈥檛 statisticians and who need to get a narrative and a description of what the findings were because that鈥檚 how you鈥檙e transparent about information,鈥 he said.
Chanin said his team believed the revisions were necessary in order to persuade SDPD to take the study seriously.
鈥淭he argument that won out,鈥 he said, 鈥渋s if we want this to make any sort of meaningful impact, it needs to be something that is, to the extent possible, unassailable in terms of its orientation.鈥
After a tense period that included San Diego Police聽, sustained complaints from minority communities of聽,听,听聽and the officer-involved聽聽of at least two聽, city officials in 2015 decided to act: Then-Councilwoman Marti Emerald tapped Chanin and his team to research whether people of color were really being pulled over by SDPD at a disproportionate rate.
The study employed a method called the 鈥淰eil of Darkness.鈥 Here鈥檚 how VOSD聽:
The Veil of Darkness assumes two things. First, that it鈥檚 more or less the same people driving on a given street between 5:30 and 9 p.m. They鈥檙e coming home from a 9-to-5聽or heading out for the night shift. Second, it assumes than an officer can better observe a driver鈥檚 skin color when the sun is up. So, researchers can compare traffic stop data from 5:30 to 9 p.m. in July, when it鈥檚 light out, to the same timeframe in January, when it鈥檚 dark. If more people of color are pulled over in that area during the summer, one can assume race is at play.
In addition to examining traffic stops, the researchers also looked at what happened after a car was stopped. They found that citywide, black and Hispanic drivers were more likely to be searched following a traffic stop and less likely to be found with contraband.
The study got聽聽at a meeting of the City Council鈥檚 Public Safety and Livable Neighborhoods Committee on Dec. 7. There, members of the public raised concerns that city officials had pressured Chanin and his team to tone down their findings. On Dec. 1 we asked the city for copies of all drafts of the study. State law requires public agencies to turn over drafts that are 鈥渞etained in the ordinary course of business鈥濃攎eaning, if a draft exists in electronic form, like an email attachment, it must be disclosed.
The city denied the request, and said no electronic copies existed. Lea Bernard-Fields, the city鈥檚 public records administrator, said there was only one hard copy of a draft, but it had been 鈥渆xtensively annotated by an individual staff member鈥 and therefore was not subject to disclosure.
Nearly three months later, after VOSD put in a similar request with San Diego State University, Bernard-Fields followed up to say an unmarked hardcopy of a draft had been located, but releasing it 鈥渨ould likely increase community tension and discontent in an environment already fraught with friction over the issue of vehicle stops.鈥
San Diego State University ultimately provided emails between Chanin and city staff that included two drafts of the report.
The earliest draft of the study provided by SDSU鈥攖he version emailed to the city on Oct. 27鈥攊s far different in tone than the final draft. Between that draft and the final draft, there are small changes, like 鈥渟ignificantly more likely鈥 being changed to 鈥渕ore likely.鈥 The line about how findings suggest that implicit bias 鈥渕ay exist鈥 initially said findings 鈥渟uggest that implicit bias exists鈥 among SDPD officers 鈥渢o an extent that 鈥 is on par with other departments nationwide that have come under federal supervision for such bias.鈥 And, in more than two-dozen instances, the word 鈥渂ias鈥 was changed to 鈥渄isparities.鈥 For example, the recommendation that the police department 鈥淸a]cknowledge the existence of racial/ethnic bias鈥 became 鈥淸a]cknowledge the existence of racial/ethnic disparities.鈥
Chanin said other academics who study policing strongly recommended changing 鈥渂ias鈥 to 鈥渄isparities.鈥 Experts I spoke to agreed with the change.
鈥淪ometimes words have a generally accepted meaning in society, but a far more specific definition in academic disciplines,鈥 said Seth Stoughton, a law professor at the University of South Carolina and former police officer.
Lorie Fridell, a criminology professor at the University of South Florida and a leading expert on racial bias in policing, said she would have made the same recommendation. 鈥淚t鈥檚 not difficult for social scientists to measure disparities [in traffic stops],鈥 she said. 鈥淲hat鈥檚 very difficult is to identify the causes of that disparity.鈥
In an earlier draft of the study, researchers urged the police department to cut back on its use of traffic stops鈥攕pecifically for equipment violations like broken brake lights or tinted windows. Doing so could improve community relations and officer safety, the draft said.
鈥淲e note that other police departments are currently in the process of reconsidering their approach to traffic stops in this way by directing officers to focus on the violations most related to safety, such as speeding and the running of red lights,鈥 the draft said.
Researchers recommended officers record the car鈥檚 license plate number and that the department implement a system to issue the driver a warning or a 鈥渇ix-it鈥 ticket by mail.
But the department pushed back against the recommendation, Chanin said. He and his team were left with a choice: Keep a recommendation that would never be implemented, or cut it to 鈥渇oster goodwill and enable some of the other [changes] that we felt were possibilities.鈥
That was the only significant change due to pressure from the city, Chanin said.
But even the toned-down report had little impact. At the Dec. 7 committee meeting, Police Chief Shelly Zimmerman acknowledged the study鈥檚 findings, but when asked repeatedly by Emerald whether people of color were sometimes treated differently by her police officers, Zimmerman responded only that 鈥渆very human being has bias.鈥
Zimmerman did say that her department was committed to taking a 鈥減roactive approach鈥 to combating bias.
In February, the City Council voted to accept the report, but declined to implement any of its recommendations. Alvarez and Councilwoman Georgette Gomez, who replaced Emerald, were the only 鈥渘o鈥 votes. They both argued that the study鈥檚 recommendations, particularly that the department improve its data-collection efforts, warranted further discussion. The study had recommended that the department, at the least, collect basic information on the officer making the stop. 鈥淲ithout these data, it is impossible to know, for example, whether black drivers were treated differently by white cops,鈥 Chanin said.
In an exchange with Alvarez at the meeting, Chanin said the department could easily collect such data under its existing system. Alvarez said he was disappointed recommendations went ignored.
鈥淲e need to have this information,鈥 Alvarez said at the meeting. 鈥淚t does start with the data. It鈥檚 a shame that we wasted so many people鈥檚 time 鈥 hoping to actually get real action and instead what we get is saying this report is before us鈥攖here it is鈥攁nd we鈥檙e not doing anything about it.鈥
Chanin said that regardless of any changes made between the first and final drafts, 鈥渢here鈥檚 clear evidence in this report that there is a difference in the way that black and brown people are treated than whites.鈥
Like Alvarez, he questions why the city asked for the study, if only to shelve it.
鈥淗ere we are, two years removed from the initiation of this process and I don鈥檛 know if things have changed,鈥 he said.
罢丑颈蝉听聽originally appeared in the聽.