Analysis

Last month, we discussed whether to perform keyword search culling before performing Predictive Coding/Technology Assisted Review (TAR) and, like many have concluded before (even a judge in FCA US, LLC v. Cummins, Inc.), we agree that you shouldn’t perform keyword search culling before TAR.  But, should TAR be performed instead of keyword search – in all cases?  Is TAR always preferable to keyword search?

I was asked that question earlier this week by a colleague, so I thought I would relay what I essentially told him.

Many attorneys that I have observed over the years have typically tried to approach keyword search this way: 1) Identify a bunch of potentially responsive terms, 2) string them together with OR operators in between (i.e., {term 1} OR {term 2}, etc.), 3) run the search, 4) add family members (emails and attachments linked to the files with hits) to the results, and 5) begin review.

If that’s the keyword search methodology you plan to use, then, yes, a sound TAR approach is preferable to that approach pretty much every time.  Sure, proportionality concerns can affect the decision, but I would recommend a sound approach over an unsound approach every time.  Unfortunately, that’s the approach a lot of attorneys still use when it comes to keyword search.

However, it’s important to remember that the “A” in TAR stands for “Assisted” and that TAR is not just about the technology, it’s as much about the process that accompanies the technology.  A bad approach to using TAR will generally lead to bad results with the technology, or at least inefficient results.  “Good TAR” includes a sound process for identifying training candidates for the software, reviewing those candidates and repeating the process iteratively until the collection has been classified at a level that’s appropriate to meet the needs of the case.

What about keyword search?  “Good keyword search” also includes a sound process for identifying potentially responsive terms, using various mechanisms to refine those terms (which can include variations, at an appropriate level, that can also be responsive), performing a search for each term, testing the result set (to determine if the term is precise enough and not overbroad) and testing what was not retrieved (to determine what, if anything, might have been missed).  We covered some useful resources for testing and sampling earlier this week here.

Speaking of this week, apparently, this is my week for the “wayback machine” on this blog.  In early 2011, I described a defensible search approach for keyword search for which I created an acronym – “STARR”.  Not Ringo or Bart, but Search, Test, Analyze, Revise (if necessary), Repeat (the first four steps until precision and recall is properly balanced).  While you might think that “STARR” sounds a lot like “TAR”, I coined my acronym for the keyword search approach well before the TAR acronym became popular (just sayin’).

Regardless whether you use STARR or TAR, the key is a sound approach.  Keyword search, if you’re using a sound approach in performing it, can still be an appropriate choice for many cases and document collections.

So, what do you think?  Do you think that keyword search still has a place in eDiscovery?  If not, why not?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

We’re seeing a lot of discussion about whether to perform keyword searching before predictive coding.  We’ve even seen a recent case where a judge weighed in as to whether TAR with or without keyword searching is preferable.  Now, we have a new article published in the Richmond Journal of Law and Technology that weighs in as well.

In Calling an End to Culling: Predictive Coding and the New Federal Rules of Civil Procedure (PDF version here), Stephanie Serhan, a law student, looks at the 2015 Federal Rules amendments (particularly Rules 1 and 26(b)(1)) as justification for applying predictive coding “at the outset on the entire universe of documents in a case.”  Serhan concludes that doing so is “far more accurate, and is not more costly or time-consuming, especially when the parties collaborate at the outset.”

Serhan discusses the importance of timing to predictive coding and explains the technical difference between predictive coding at the outset of a case vs. predictive coding after performing keyword searches.  One issue of keyword culling that Serhan notes is that it “is not as accurate because the party may lose many relevant documents if the documents do not contain the specified search terms, have typographical errors, or use alternative phraseologies”.  Serhan assumes that those “relevant documents removed by keyword culling would likely have been identified using predictive coding at the outset instead.”

Serhan also takes a look at the impact on efficiency and cost between the two methods and concludes that the “actual cost of predictive coding will likely be substantially equal in both methods since the majority of the costs will be incurred in both methods.”  She also looks at TAR related cases, both before and after the 2015 Rules changes.

More and more people have concluded that predictive coding should be done without keyword culling and with good reason.  Applying predictive coding to a set unaltered by keywords would not only likely be more accurate, but also be more efficient as keyword searching requires its own methodology that includes testing of results (and documents not retrieved) before moving on.  Unless there’s a need to limit the volume of collected data because of cost considerations, there is no need to apply keyword culling before predictive coding.

Culling that does make sense is Hash based deduplication, elimination of clearly non-responsive domains and other activities where clearly redundant or non-responsive ESI can be removed from the collection.  That’s a different type of culling that does make sense.

So, what do you think?  To keyword cull or not to keyword cull?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.