Anonymity and Integrity in Researching Judicial Performance

Recently, Chicago Appleseed was asked about unsolicited anonymous comments about judicial retention candidates and whether they play a role in the data collection for the Judicial Performance Commission of Cook County’s (the JPC) evaluations. The integrity of our data is paramount and we strive to use only valid sources of feedback. Nonetheless, the most valuable commentary we can collect from survey respondents is candid, honest commentary. In our data collection effort, therefore, we balance the privacy needs of data sources against our need to verify their responses.

Generally, our research into the performance of judicial retention candidates focuses on court-watching, anonymous electronic surveys and confidential phone interviews of attorneys with experience in the relevant courtrooms in the last three years. Chicago Appleseed acts as independent researchers for the JPC. Chicago Appleseed sends court-watchers into courtrooms to record their impressions of how the courtroom functions and to observe the atmosphere in the courtroom. Although volunteers identify themselves in their court-watching reports, the reports are stripped of identifying information when presented to the JPC for use in evaluations. Court-watching data, therefore, carries some integrity because we have trained the observers ourselves. Our court-watching data is further controlled for individual biases by having more than one volunteer watch each courtroom and by completing several observations over a year. Although our court-watchers do not identify themselves while they are observing courtrooms, judicial retention candidates are informed about that court-watching project.

Chicago Appleseed uses electronic and telephone surveys to collect the experiences of attorneys who have been in the judicial retention candidates’ courtrooms in the last three years. The Chief Judge of the Circuit Court of Cook County has authorized the Clerk of the Court to give us appearance data which identifies those attorneys. Because we use the Clerk’s appearance data to determine which attorneys to survey, we are capturing a broad and neutral base for survey respondents. The electronic survey is sent only to attorneys in the Clerk’s data, so we know persons responding to the survey have had recent professional contact with the candidate. Because the survey is administered by a third party software company, responses are entirely anonymous, ensuring to a reasonable degree that respondents will be both candid and honest.  Researchers also conduct telephone interviews of attorneys appearing in the Clerk’s data. Names of attorneys who participate in the telephone interviews are kept in strict confidence. The Commissioners receive reports of the interviews, but no identifying information about the attorneys who spoke to researchers.

Chicago Appleseed promises confidentiality  to attorneys agreeing to participate in interviews for the process; however, Appleseed does not accept unsolicited anonymous comments from unknown sources for use in the JPC’s evaluation process. In the event a member of the community contacted us with comments or complaints about their experiences before a judicial retention candidate, we would need to verify the account as much as possible and therefore we would expect the commentator to identify himself or herself. However, just as we preserve the confidentiality of attorneys we interview, we would keep those sources confidential as well.

Finally, each judicial candidate is given the opportunity to review the JPC’s recommendation prior to its finalization and release to the public. Judges are invited to respond to draft recommendations and provide rebuttals to the findings.

The Judicial Performance Commission model for evaluating judges does not commonly limit data sources to the experiences of attorneys. Many Commissions survey lay people with experience before the judges—such as witnesses, jurors or litigants—and Commissions in Arizona and Tennessee hold public hearings where members of the community may voice their opinions. Inviting the public to participate in the process, and considering the experiences of non-attorneys in court, strengthens the work of any judicial performance commission. It acknowledges the public as stakeholders in the judicial system and recognizes the value of their input. However, the integrity of the data which a performance commission uses in its evaluations cannot be compromised, if the evaluations are to have credibility.

At Chicago Appleseed, we are quite concerned with the credibility of the research we present to the JPC for use in evaluating the judicial retention candidates and have not yet devised a protocol for accepting anonymous input from the citizenry at large.  For the 2012 judicial retention election, most data presented to the JPC for evaluating the candidates will come from attorneys identified by the Clerk of the Court as having appeared before the candidate in the last three years. Additional data will come from court-watching performed by Chicago Appleseed’s volunteers and from news databases. In the future, we hope to expand our research to capture the experiences of litigants, witnesses and/or jurors. However, at this time we do not have a credible means of collecting these experiences.