I have recently been working with a client to explain some concepts that are part of our work. These concepts are some of the fundamentals of social science research, concepts which are so well understood by those educated in this area that they are second nature. This particular concept is easily explained by the adage, “the more the merrier.” As it turns out, this phrase originated in a poem in the 14th century and usually relates to one of two things. More people = more fun or a greater amount of something is better. Now, I know this is not always true and sometimes less is more, but, in our trial consulting work, a big variable in the design of research is the number of participants – usually mock jurors (but it can be arbitrators or other decision makers). The other variable is the number of deliberations groups. These variables drive the costs of mock jury research which we know is not inexpensive. What is difficult, and sometimes frustrating, is when clients try to cut corners, sometimes, with the assistance of competing trial consultants who are apparently not skilled or trained in research design, and ask us to cut corners suggested by the other consultants. For example, we have been asked, “Can you recruit 8 instead of 10 mock jurors for each group?” (I need to step back first: In many venues, including Florida, most trial juries consist of 6 jurors, and some number of alternates. We know that not everyone who is recruited will show up – our national historic show rate is 80% so we are most likely to get 8 out of 10. But this is not always true, so we recruit extras to ensure a full panel and to give us some room in case a mock juror who shows up needs to be sent home for some reason.) Recruiting 8 instead of 10 with an 80% show rate gives us no “wiggle” room. And, it would only save a few hundred dollars to begin with, but that is not the end of the story. The rest of the story is that, while we “need” only 6 jurors, we want as many as possible to show. From a research perspective, the more data points we have, the more confidence we have in the results. Small group research, like a mock jury, has limitations and the primary one is the “small” group aspect. In fact, for any basic statistical analysis to be conducted in small group research, a minimum of 30 participants is required. If our research can be 30+, as it is often with a 4 group methodology we employee when feasible, then we have the ability to develop stronger profiles of favorable or unfavorable trial jurors, as well as to provide other insights based on statistical analyses – quantitative analyses in addition to qualitative analyses. So more jurors, rather than fewer jurors, is preferred. The more the merrier. Sounds like a simple concept, but it is one we find ourselves explaining when we wish clients would trust our expertise and knowledge of how to help them most effectively. And, we struggle to ensure that we explain such things without alienating the client. More on that in another post.
One of the most challenging, not to mention frustrating, aspects of my career has been explaining scientific research methods, statistical principles, and other complex aspects of the work I perform to people who lack the education, knowledge, and experience to understand what I am talking about. There is only so much “dumbing down” that is possible, to the point that it is almost as if I am speaking a different language than the person who has no basis required to understand what I am talking about. The first time I became painfully aware that the average person could not grasp complex statistical analyses, despite my repeated attempts to explain them in words with few syllables and in a patient tone, was when I worked as Director of Marketing Research at a large hospital. My boss, who was not very bright, to put it nicely, asked me to explain a very basic statistical technique known as “analysis of variance” in as few words as possible, without using any statistical or psychological jargon (or, as he actually said, “mumbo jumbo”) in my description. Because he was my boss, I soon learned to not say what I was thinking and instead, began to grasp that working as a social psychologist in an applied setting, that is, outside academia, would involve many challenges I, as a recent Ph.D., had never considered. So, here I am, all these years later, faced with constantly having to explain sample size, generalizability, reliability, validity, random sampling, and many other concepts involved in my work to attorneys, insurance adjusters, corporate executives, and other people who, like my former boss, have the audacity to second guess my expertise even though they are completely lacking in knowledge about what I am trying to do on their behalf. I am very fortunate it is David, not I, who most often has to explain to a client or prospective client why Magnus does things “the right way,” with scientific rigor and precision, even though the majority of our competitors cut every possible corner to under bid us on a proposal. One of the most fundamental aspects of good research, as opposed to bad research, is to have a large enough sample size to yield meaningful results. Sure, it’s possible to get results with a small sample size (sample size refers to the number of research participants), but the results won’t be meaningful and in fact, they could be wrong. Why would a client hire someone who provides the wrong results for any amount of money? Why?
Comments are closed.