eDiscovery Daily Blog
Mike Q Says the Weakest Link in TAR is Humans: eDiscovery Best Practices
We started the week with a post from Tom O’Connor (his final post in his eDiscovery Project Management from Both Sides series). And, we’re ending the week covering an article from Mike Quartararo on Technology Assisted Review (TAR). You would think we were inadvertently promoting our webcast next week or something. :o)
Remember The Weakest Link? That was the early 2000’s game show with the sharp-tongued British hostess (Anne Robinson) telling contestants that were eliminated “You are the weakest link. Goodbye!” Anyway, in Above the Law (Are Humans The Weak Link In Technology-Assisted Review?), Mike takes a look at the debate as to which tool is the superior tool for conducting TAR and notes the lack of scientific studies that point to any particular TAR software or algorithm being dramatically better or, more importantly, significantly more accurate, than any other. So, if it’s not the tool that determines the success or failure of a TAR project, what is it? Mike says when TAR has problems, it’s because of the people.
Of course, Mike knows quite a bit about TAR. He’s managed his “share of” of projects, has used “various flavors of TAR” and notes that “none of them are perfect and not all of them exceed all expectations in all circumstances”. Mike has also been associated with the EDRM TAR project (which we covered earlier this year here) for two years as a team leader, working with others to draft proposed standards.
When it comes to observations about TAR that everyone should be able to agree on, Mike identifies three: 1) that TAR is not Artificial Intelligence, just “machine learning – nothing more, nothing less”, 2) that TAR technology works and “TAR applications effectively analyze, categorize, and rank text-based documents”, and 3) “using a TAR application — any TAR application — saves time and money and results in a reasonable and proportional outcome.” Seems logical to me.
So, when TAR doesn’t work, “the blame may fairly be placed at the feet (and in the minds) of humans.” We train the software by categorizing the training documents, we operate the software, we analyze the outcome. So, it’s our fault.
Last month, we covered this case where the plaintiffs successfully requested additional time for discovery when defendant United Airlines, using TAR to manage its review process, produced 3.5 million documents. However, sampling by the plaintiffs (and later confirmed by United) found that the production contained only 600,000 documents that were responsive to their requests (about 17% of the total production). That seems like a far less than ideal TAR result to me. Was that because of human failure? Perhaps, when it comes down to it, the success of TAR being dependent on humans points us back to the long-used phrase regarding humans and computers: Garbage In, Garbage Out.
So, what do you think? Should TAR be considered Artificial Intelligence? As always, please share any comments you might have or if you’d like to know more about a particular topic.
Image Copyright © British Broadcasting Corporation
Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.