Technology is not the answer to everything. I have recently been reminded of this when talking with more than one potential client who was curious about using internet based “jury research.” Specifically, we’ve recently been asked to bid on mock jury research only to learn that the competitor’s bid was for an online “mock jury.” In another case, the client wanted an online “mock jury” for reasons that did not really make sense; I’ll get back to that. The reason I’m putting parentheses around mock juries relative to the online variety is that these are not really mock juries. There are several variations on the theme of internet based “mock juries” but most include using people as mock jurors to participate via an internet based connection. Some of these amount to little more than an online survey. Some include a group “chat” (verbal or typed). But, they all rely on people who, in the comfort of their homes or workplaces (which raises another set of issues, including confidentiality), participate on research on a case, often, for very little pay. Some of these use avatars so that the presenting attorneys are looking at “faces” though not necessarily of the real people. This type of research is sometimes sold as more cost effective than traditional mock jury research. But, based on my investigations, it appears that properly conducted internet based research does not present a major cost savings over traditional, live, in-person groups. Instead, it appears that most of the applications of this type of research are gimicky ways to entice clients to sign up for something, without revealing the limitations of online “mock jury” research. In the interest of creating informed consumers, I’m going to list some of the limitations of such research. There are certainly limitations to any mock jury, or social science research, but here are 6 which immediately come to mind regarding online research:
1) The source for obtaining the online participants is suspect. Though this can be overcome, to some degree, by using proper recruiting techniques, in many instances, the participants are convenience samples who may have signed up online. And, in that most online research is conducted without using a camera, it is also questionable as to whether the people are who they say they are. Again, this can be overcome by verifying identity; otherwise, one does not know whether the person on the other end of the internet is who they say they are. Is that a 45 year old man posing as a 25 year old woman? Do they even live in the trial venue?
2) In an online “mock jury” project, the sample of participants is almost always biased to include only those participants who have access to a computer, preferably at home, as opposed at their workplace, or a public location like a library. Many people do not have a home computer, instead, they rely on their smart phones or tablets. While a tablet may be a substitute, it is still a bias in that not everyone has internet access via any type of acceptable device. Further, more than in live mock juries, it appears, again, based on our study of the methodologies being used for online research, that online research is more likely than a live mock jury to include “repeat” participants from a list maintained by the consultant or, perhaps the recruiter, if one is used.
3) There are numerous variations in the way the presentation is made to the mock jurors. In some instances, it is a pre-recorded video or a live presentation. In others, it is simply a written narrative for the participants to read. The latter amounts to little more than an online survey in many ways, though, as above, with limitations. And, for the attorneys, the loss of face validity should seem important. In a live mock jury, the attorneys can see and hear the participants in real time. Presenting to a camera or an array of avatars might seem kind of cool, but it is unnatural.
4) The “research” may or may not involve all participants being connected simultaneously in any way. Again, this creates more of a survey based methodology. But, when they are participating without being linked together – it is mock juror research, not a mock jury, (see #6).
5) There is often no video feed of the participants except in the more sophisticated projects which utilize cameras. I have noted over the years that our clients are often more interested in the degree of engagement and “body language” of mock jurors than we are. Having this relegated to being a remote function takes it one step more distant. (No, not everything is like you see it on TV with “Dr. Bull.”)
6) An online “mock jury” is not a jury for one reason, it does not involve a group! At best, it is like a conference call, perhaps one with visuals. But, the participants are not grouped like a jury. We’ve all been on enough bad conference calls to know that they are poor substitutes for in-person meetings, especially on critical topics. Even though, during in-person meetings, some participants have little to say, communicating using conferencing technology seems to reduce participation. Though one can prompt less engaged participants (live or electronically) – doing this online also eliminates the ability to view them (except in the most sophisticated online research which uses cameras) in order to evaluate the individual and his/her behaviors. The fact is, online research can never replicate the group dynamics of people who are actually in a group. This is, to me, the number 1 problem with online research. Stay tuned for Part 2 of this Topic.
Social psychology is the study of group behavior, among other things. By definition, a group is comprised of two or more people who assemble together. A jury, is of course, a unique type of group that is tasked with making important decisions that have a substantial impact on other people’s lives. When an individual or a corporation professes to be in the “jury” consulting business, every effort must be made to conduct focus groups, mock trials, attitude surveys, etc. in ways that can be generalized to the actual trial (or mediation, or arbitration). Absent taking every step to ensure that the research findings can be generalized to the actual case, there is no point in conducting the research in the first place. Therefore, the fact that online “mock jury” research does not involve a group of people who work together in the exact ways a jury works together to reach a verdict means there is no generalizability of the research findings. It is as simple as that. If one accepts the premises that: (1) a group is comprised of two or more individuals who come together to achieve a common goal; (2) a jury is a specific type of group, with the purpose of a jury to reach one decision regarding a trial; and (3) a jury verdict is a single decision, as opposed to individual decisions made by each juror, separate from the others; then it is logical to conclude that online research is something other than “mock jury” research. Based on the absence of credentials I have observed among people willing to conduct online “mock jury” research, I am not sure what these people are doing is actually research, but that is another point for another post. Suffice it to say, Magnus has always, and will continue to, conduct social psychological research of the highest quality, using all appropriate scientific methods, with jury eligible citizens in the trial venue. That’s just what we do, and we do it well.