eDiscovery Daily Blog

Court Agrees with Plaintiffs, Orders Provision for Qualitative Sampling of Disputed Search Terms: eDiscovery Case Law

In the case In Re: Lithium Ion Batteries Antitrust Litigation, No. 13-MD-02420 YGR (DMR) (N.D. Cal., Feb. 24, 2015), California Magistrate Judge Donna M. Ryu ordered the defendants to comply with the plaintiffs’ proposed qualitative sampling process for keyword search terms, citing DaSilva Moore that keywords “often are overinclusive”.

Case Background

In this multi-district litigation (MDL), the court ordered the parties to meet and confer to negotiate a protocol for the use of search terms in December 2014. The parties agreed upon an iterative process for the development and testing of search terms, summarized as follows:

  1. The producing/responding party will develop an initial list of proposed search terms and provide those terms to the requesting party;
  2. Within 30 days, the requesting party may propose modifications to the list of terms or provide additional terms (up to 125 additional terms or modifications); and
  3. Upon receipt of any additional terms or modifications, the producing/responding party will evaluate the terms, and
  4. Run all additional/modified terms upon which the parties can agree and review the results of those searches for responsiveness, privilege, and necessary redactions, or
  5. For those additional/modified terms to which the producing/responding party objects on the basis of overbreadth or identification of a disproportionate number of irrelevant documents, that party will provide the requesting party with certain quantitative metrics and meet and confer to determine whether the parties can agree on modifications to such terms. Among other things, the quantitative metrics include the number of documents returned by a search term and the nature and type of irrelevant documents that the search term returns. In the event the parties are unable to reach agreement regarding additional/modified search terms, the parties may file a joint letter regarding the dispute.

The parties requested the court’s guidance on a single remaining issue regarding their search term protocol: the steps the parties needed to take if they could not resolve a disagreement over a particular term. The plaintiff wanted the defendant to conduct a randomized qualitative sampling of documents retrieved by searching for any disputed terms, and to then allow the plaintiff to review the resulting documents following a privilege review.

The defendants objected to the proposed sampling provision “solely on the grounds that it will provide Plaintiffs with access to non-responsive, irrelevant documents that will be generated through the procedure.” They argued that the provision was unnecessary due to the detailed quantitative information that they agreed to produce regarding disputed search terms and because “there has been no showing that any Defendant’s production is incomplete.” The plaintiffs countered “that the proposed provision incorporates ESI best practices, including those embodied in materials developed by this Court” and contended that “the best way to refine searches and eliminate unhelpful search terms is to analyze a random sample of documents, including irrelevant ones, to modify the search in an effort to improve precision.”

Judge’s Opinion

With regard to the plaintiffs’ argument, Judge Ryu stated simply, “The court agrees. The point of random sampling is to eliminate irrelevant documents from the group identified by a computerized search and focus the parties’ search on relevant documents only. As the court noted in Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012), a problem with keywords ‘is that they often are overinclusive, that is, they find responsive documents but also large numbers of irrelevant documents.’”

Noting, however, that the defendants “raise a valid concern that the sampling protocol will result in the production of irrelevant information”, Judge Ryu ordered the following parameters to alleviate that concern:

  • At the hearing, the plaintiffs agreed that the defendants “may review the random qualitative sample and remove any irrelevant document(s) from the sample for any reason, provided that they replace the document(s) with an equal number of randomly generated document(s)”;
  • The parties also agreed that the defendants would conduct the qualitative sampling only after they had exhausted an agreed-upon quantitative evaluation process;
  • Judge Ryu ordered that irrelevant documents in the sample “shall be used only for the purpose of resolving disputes regarding search terms in this action, and for no other purpose in this litigation or in any other litigation” and that those irrelevant documents, as well as any attorney notes regarding the sample, “shall be destroyed within fourteen days of resolution of the search term dispute”;
  • Only one attorney from each law firm designated co-lead class counsel for Direct Purchaser Plaintiffs and Indirect Purchaser Plaintiffs (total of six attorneys) would be allowed to review the random sample;
  • The plaintiffs could invoke the random sampling process with respect to no more than five search terms per defendant group.

So, what do you think? Was the court right to order random sampling? Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.