Princeton made errors in reporting 2017 We Speak Survey

The recently released We Speak report used potentially misleading techniques to report some data and contained errors with basic arithmetic, a review by the University Press Club has found.Undergraduate students reported 116 rapes in the 12 months preceding the 2017 survey, among the roughly half of the student body that responded. More than 500 undergraduates reported experiencing sexual misconduct in the same period, including sexual assault, stalking, dating violence and sexual harassment.These results were similar to those found in past years, but a change in reporting techniques altered how some statistics associated with rape and sexual assault were reported, including whether the perpetrator was another Princeton student and whether victims were drinking before their assault.Respondents were given the option on this year’s survey to decline to answer further questions after reporting that they had been raped, which had not been an option in previous years, and people who did not respond to further questions were included in the results.For example, the share of undergraduates who reported that their rape was committed by another Princeton undergraduate is 20 percent lower in the 2017 report than in the 2016 report. The University used the same language when reporting the statistic in each report.“Was the person a student at Princeton?,” the survey asked referring to a respondent’s rapist. “Yes, and undergraduate student,” one of the answers, is reported as the chosen answer 82 percent of the time on the 2016 survey, and only 61 percent of the time in the 2017 survey.A decline in the share of rapes that were committed by other students could be interpreted as a good thing. A slightly closer look at the data, though, reveals that the drop might not have happened. Of the students who responded to the question, three in four reported being raped by another Princeton student, which would be not be significant change.Princeton added the option to decline to answer follow-up questions because recounting details of sexual assault was overly traumatic for many people, Jed Marsh, the University’s vice provost for institutional research, told the University Press Club.Marsh said that the change in reporting would not mislead anyone who was reading the report.The We Speak survey was originally pitched as a centerpiece of the University’s efforts to seriously combat sexual assault and harassment on campus soon after the Department of Education found that Princeton was in violation of Title IX for neglecting issues of sexual misconduct on its campus.The reported proportion of rapes that occurred in University-owned dorms also dropped from last year. According to the 2017 report, 32 percent of respondents said they were raped in an University-owned dorm room; however, that number rises to 58 percent if one removes respondents who did not answer the question.(Either number would be a decrease from what was reported in the 2016 survey, but it would only be a small decrease if one excludes students who did not answer from calculations.)A pie chart in this year’s report representing the “involvement of alcohol” in rape said that 54 percent of victims had been drinking before their assault. It lumped victims who had not been drinking and who had not answered the question into the same category: “Neither or unknown.”

There are two problems with this chart that make the data misleading. The first is that the survey said that 59 percent, not 54 percent, of rape victims reported either drinking and using drugs or just drinking before being attacked. This chart leaves out the five percent of rape victims who reported both drinking and using drugs before being assaulted.

Marsh said that the chart was only supposed to show the data for rapes where the victim indicated only drinking alcohol (as opposed to drinking and doing drugs), but that is not marked on the report. The chart’s header spoke only of “involvement of alcohol.”Another issue is with the reporting of “Neither or unknown.” A reasonable person, reading the graph without seeing any of the detailed information in the report, would not know that respondents who did not answer the question were lumped in with people who were not drinking before being assaulted.One is led to believe that only a slim majority of rape cases involved alcohol, which is probably  not accurate. About three in four respondents who answered the question indicated that they had been drinking before being raped, which is a nearly identical statistic to the 2016 report.There is no evidence that everyone who declined to answer the question had not been drinking, which the pie chart suggests.Part of the idea behind the We Speak surveys was that numbers would be comparable across all three surveys, to track any trends. The 2017 report echoed that sentiment, saying that the survey and its results were “largely comparable” with past versionsMarsh said in his interview with the Press Club that questions about the events surrounding rape and sexual assault should not, in fact, be viewed as comparable between the different surveys.Because there was no data on whether the group that declined to answer further questions was statistically representative of the respondents as a whole, Marsh did not want to exclude them from the report, he said.“The best way to read the follow-up details section is to say that we know at least this many people experienced this behavior,” Marsh said. The phrase “at least” never appeared in the section of the We Speak with the follow-up details.In addition to potentially misleading reporting techniques, some of the data that Princeton reported was simply incorrect.A one-page infographic released on the Title IX Office’s website said that “1 in 5” undergraduate women experienced sexual misconduct. This is a mistake. It should have said “1 in 4” or, more accurately, “1 in 3.7,” since 27 percent of undergraduate women who responded to the survey reported experiencing sexual misconduct.(Marsh said that the infographic was fixed after the University Press Club requested a comment on the mistake, but the infographic on the Title IX Office’s site still has the incorrect “1 in 5” statistic, as of press time.)Many members of the University community relied upon the University’s infographic for the most important findings of the 322-page report. The Daily Princetonian copied the University’s infographic onto its front page, and used the incorrect “1 in 5” statistic in its article on the report.The Undergraduate Student Government included an announcement of the 2017 report’s release in one of its emails to the undergraduate student body, but included the infographic from last year, not this year.“Thank you for alerting me of the error in my last email,” Myesha Jemison, USG President, wrote in an email. USG later sent out an email with the infographics from both this year and last year.In the past two years, Princeton’s support for the We Speak survey has become less vocal, and the University has not addressed the link between alcohol and sexual assault on campus, even after the link was highlighted in the year of the first survey.University President Christopher Eisgruber was alarmed when the first edition of the We Speak report, released in 2015, showed a high correlation between alcohol and the 143 cases of rape against Princeton undergraduates reported on the survey.“We need to find ways to reduce the harms and risks that result from the misuse of alcohol on this campus,” Eisgruber wrote in a letter to the university community after the report was released.Princeton’s Faculty-Student Committee on Sexual Misconduct, which Eisgruber had convened after the 2014 settlement with the Department of Education, made a series of recommendations for improving the We Speak survey and lowering the rates of sexual misconduct on campus.Not a single recommendation explicitly mentioned alcohol, although greater cooperation with the eating clubs was proposed, and the committee said that they wanted to wait for the second iteration of the survey to make more substantive recommendations.Last November, Princeton released the results of the second round of the We Speak survey, which again showed a close connection between drinking and rape committed against Princeton students by Princeton students. That year, 98 rapes were reported on the survey.Eisgruber did not release a statement after the 2016 report was released.The committee on sexual misconduct that was created after the first We Speak survey released a new report, which included nine specific recommendations for Princeton to reduce the incidence of sexual harassment and assault. Not a single one of the nine specifically mentioned alcohol, even though almost eight in ten undergraduate rape victims reported that they had been drinking before their assault.One recommendation mentioned closer cooperation with the eating clubs, but the We Speak report showed that alcohol was not limited to Prospect Avenue. A majority of respondents who were drinking in the hours before their rape had done so in University-owned dorms, and most were drinking hard liquor, which is not widely available in the eating clubs.The 2017 report had its problems with reporting data, but adjusting the data to match the reporting techniques of previous years revealed a similar correlation between drinking and rape on campus, and most of the people who reported drinking before being raped did so in University-owned dorms.Eisgruber did not release a statement to accompany this year’s report. The committee on sexual misconduct is expected to release another set of recommendations in the spring.“The President is as committed to this as he was the first time the survey happened,” Marsh said. He said that Eisgruber would have had less an impact if he had written a letter after each report was released.More than 100 Princeton undergraduates are victims of traumatic violent crimes every year, and these are overwhelmingly committed by other Princeton undergraduates, usually in Princeton-owned dorms. Princeton has a responsibility to its students to take reasonable steps to stop these rapes.The We Speak surveys were created with the goal of increasing transparency and demonstrating that the University could hold itself accountable on something that virtually everybody agreed was a crisis Princeton needed to address.More than two years later, it seems like Princeton has faltered in fulfilling that promise.