Craig Ball of Craig D. Ball, P.C. – eDiscovery Trends, Part 2

March 7, 2013

This is the tenth (and final) of the 2013 LegalTech New York (LTNY) Thought Leader Interview series. eDiscoveryDaily interviewed several thought leaders at LTNY this year and generally asked each of them the following questions:

What are your general observations about LTNY this year and how it fits into emerging trends?
If last year’s “next big thing” was the emergence of predictive coding, what do you feel is this year’s “next big thing”?
What are you working on that you’d like our readers to know about?

Today’s thought leader is Craig Ball. A frequent court appointed special master in electronic evidence, Craig is a prolific contributor to continuing legal and professional education programs throughout the United States, having delivered over 1,000 presentations and papers. Craig’s articles on forensic technology and electronic discovery frequently appear in the national media, and he writes a monthly column on computer forensics and eDiscovery for Law Technology News called Ball in your Court, as well as blogs on those topics at ballinyourcourt.com.

Craig was very generous with his time again this year and our interview with Craig had so much good information in it, we couldn’t fit it all into a single post. Yesterday was part 1. Today is part 2 and part 3 will be published in the blog on Friday. A three-parter!

Note: I asked Craig the questions in a different order and, since the show had not started yet when I interviewed him, instead asked about the sessions in which he was speaking.

I noticed that you are speaking at a couple of sessions here. What would you like to tell me about those sessions?

{Interviewed the evening before the show} I am on a Technology Assisted Review panel with Maura Grossman and Ralph Losey that should be as close to a barrel of laughs as one can have talking about technology assisted review. It is based on a poker theme – which was actually Matt Nelson’s (of Symantec) idea. I think it is a nice analogy, because a good poker player is a master or mistress of probabilities, whether intuitively or overtly performing mental arithmetic that are essentially statistical and probability calculations. Such calculations are key to quality assurance and quality control in modern review.

We have to be cautious not to require the standards for electronic assessments to be dramatically higher than the standards applied to human assessments. It is one thing with a new technology to demand more of it to build trust. That’s a pragmatic imperative. It is another thing to demand so exalted a level of scrutiny that you essentially void all advantages of the new technology, including the cost savings and efficiencies it brings. You know the old story about the two hikers that encounter the angry grizzly bear? They freeze, and then one guy pulls out running shoes and starts changing into them. His friend says “What are you doing? You can’t outrun a grizzly bear!” The other guy says “I know. I only have to outrun you”. That is how I look at technology assisted review. It does not have to be vastly superior to human review; it only has to outrun human review. It just has to be as good or better while being faster and cheaper.

We cannot let the vague uneasiness about the technology cause it to implode. If we have to essentially examine everything in the discard pile, so that we not only pay for the new technology but also back it up with the old. That’s not going to work. It will take a few pioneers who get the “arrows in the back” early on—people who spend more to build trust around the technology that is missing at this juncture. Eventually, people are going to say “I’ve looked at the discard pile for the last three cases and this stuff works. I don’t need to look at all of that any more.

Even the best predictive coding systems are not going to be anywhere near 100% accurate. They start from human judgment where we’re not even sure what “100% accurate” is, in the context of responsiveness and relevance. There’s no “gold standard”. Two different qualified people can look at the same document and give a different assessment and approximately 40% of the time, they do. And, the way we decide who’s right is that we bring in a third person. We indulge the idea that the third person is the “topic authority” and what they say goes. We define their judgment as right; but, even their judgments are human. To err being human, they’re going to make misjudgments based on assumptions, fatigue, inattention, whatever.

So, getting back to the topic at hand, I do think that the focus on quality assurance is going to prompt a larger and long overdue discussion about the efficacy of human review. We’ve kept human review in this mystical world of work product for a very long time. Honestly, the rationale for work product doesn’t naturally extend over to decisions about responsiveness and relevance. Even though, most of my colleagues would disagree with me out of hand. They don’t want anybody messing with privilege or work product. It’s like religion or gun control—you can’t even start a rational debate.

Things are still so partisan and bitter. The notions of cooperation, collaboration, transparency, translucency, communication – they’re not embedded yet. People come to these processes with animosity so deeply seated that you’re not really starting on a level playing field with an assessment of what’s best for our system of justice. Justice is someone else’s problem. The players just want to win. That will be tough to change.

We “dinosaurs” will die off, and we won’t have to wait for the glaciers to advance. I think we will have some meteoric events that will change the speed at which the dinosaurs die. Technology assisted review is one. We’ve seen a meteoric rise in the discussion of the topic, the interest in the topic, and I think it will have a meteoric effect in terms of more rapidly extinguishing very bad and very expensive practices that don’t carry with them any more superior assurance of quality.

eDiscovery Daily Blog