In thinking about my prior post on confirmation bias, I thought about one aspect of being hired as a professional trial consultant. It happens that I recently saw an announcement of a bar association seminar on do it yourself (DIY) mock trials. I know that mock trials are often expensive when conducted by a qualified trial consultant (what constitutes the distinction of “qualified” has been the topic of other posts). I get it, sometimes a case does not warrant the cost of a professional. But just as other DIY projects, DIY mock juries often suffer from the lack of expertise of the person conducting it. Melissa and I have written before about some of the dangers of DIY mock trials but, what struck me in writing on confirmation bias is the realization that it is highly likely this is a dangerous problem with DIY mock trials (despite the books and seminars purporting to teach lawyers to DIY). Suppose the mock jury verdict seems outrageous. Suppose the mock jurors award significantly more than one expected. What if it was much less? The next question should be “why”? What was done to cause the unexpected result? Were biases involved that skewed the results? Should one then rely on results that are unexpected? Is it fair to the end client to be presented with potentially faulty information, or was the information correct? I know of a trial consultant who believed the results of a particular mock trial were unrealistic, thus, she changed the results in her report to the client so as not to report an “outlier” result. The result did not confirm her expectations to the point she falsified the results rather than add a disclaimer! On a case Magnus handled, a plaintiff’s attorney was astonished that the mock jurors awarded considerably more damages than he anticipated. An insurance adjuster was once so upset at being wrong upon hearing mock jury verdicts that he literally lost his appetite. Both of these individuals reacted to contrary information in a positive fashion; they had us there to tell them what seemed to be driving the unexpected results. The plaintiff’s attorney got a larger settlement than he ever expected. The insurance adjuster got the case settled rather than risk a huge loss. But, in a DIY mock jury, there is no devil’s advocate watching the outcome (not to mention ensuring that as much bias is eliminated from the process as possible). It seems entirely possible that a failure to understand confirmation bias could lead to disastrous results for the ultimate client. Whether that is that malpractice for the DIY attorney remains to be seen. But, clearly the risk of not knowing one’s limitations and abilities in terms of conducting DIY research is dangerous.
I’ll begin my part of this topic by saying that, if social psychologists, who study confirmatory bias and are, therefore, presumed to be experts on it, are subject to confirmation bias in their decision making, then almost anyone can engage in this type of biased information processing. Attorneys may be more educated and more intelligent than the average person, however, when they choose to perform “Do It Yourself” mock trials, they are acting more like a DIY home owner who, instead of hiring an expert to re-tile her kitchen floor, decides to do the best she can by watching a “how to tile a kitchen floor” YouTube video before making a mess of the project. Abraham Lincoln famously said, “A lawyer who represents himself has a fool for a client.” If one accepts the premise of this oft repeated quote and further, accepts the premise that a lawyer possesses expertise on legal matters, but not social science research principles, then a lawyer who conducts DIY research is worse than a mere fool. The definition of “fool” is a person lacking in judgment or a person lacking in common powers of understanding. Conducting highly complex research on humans requires an advanced education in social scientific research methods, as well as considerable experience, traits which most attorneys lack. Yes, one can save a lot of money on DIY mock trials, just as one can save a lot of money by DIY re-tiling a kitchen floor, but, unlike the rather insignificant negative consequences of DIY home improvement projects, DIY mock trials sacrifice the attorney’s client’s case at the expense of saving money. Confirmation bias is a powerful predictor of human behavior. It is, in my opinion, tantamount to legal malpractice to conduct DIY mock trials, then potentially interpret the data according to what one wants or expects to hear, due to the fact that one’s clients place their implicit trust in attorneys to make decisions based on actual, not wishful, information. The benefits in hiring a professional litigation research consultant are numerous, with the greatest benefit being providing the best possible assistance to one’s clients. Confirmation bias? DIY? Think about it before you make a regrettable error.