eDiscovery Daily Blog

• August 28, 2018

In City of Rockford v. Mallinckrodt ARD Inc., No. 17 CV 50107, No. 18 CV 379 (N.D. Ill. Aug. 7, 2018), Illinois Magistrate Judge Iain D. Johnston adopted the parties’ proposed order establishing the production protocol for ESI with the inclusion of the plaintiffs’ proposal that a random sample of the null set will occur after the production and that any responsive documents found as a result of that process will be produced.

Case Background

In this case involving alleged breach of contract, racketeering and antitrust violations related to the defendant’s prescription medication, the parties agreed on several aspects of discovery, including a plan to use keyword searching and a protocol for agreeing on search terms, date restrictions, and custodian restrictions.  The protocol also addressed the steps to be taken if a party were to dispute a specific term as being overly broad, with the producing party to review a statistically valid sample of documents to determine if the term is returning mostly responsive documents, followed by negotiation as to any modifications to the term, with a plan to submit to the Court if they could not agree.

However, the parties could not agree on what to do after the production.  The defendants’ proposed that if “the requesting party reasonably believes that certain categories of requested documents exist that were not included in the production, the parties will meet and confer to discuss whether additional terms are necessary.”  On the other hand, the plaintiffs proposed a random sample of the null set (the documents not returned via search), with the following specific provision:

“The producing party agrees to quality check the data that does not hit on any terms (the Null Set) by selecting a statistically random sample of documents from the Null Set. The size of the statistically random sample shall be calculated using a confidence level of 95% and a margin of error of 2%. If responsive documents are found during the Null Set review, the producing party agrees to produce the responsive documents separate and apart from the regular production. The parties will then meet and confer to determine if any additional terms, or modifications to existing terms, are needed to ensure substantive, responsive documents are not missed.”

Judge’s Ruling

While noting that “the parties have agreed to use key word searching”, Judge Johnston evaluated the “pros and cons” of keyword searching as compared to technology assisted review (TAR), but ultimately decided that he “will not micromanage the litigation and force TAR onto the parties.”

As for the proposal in dispute, Judge Johnston ruled that sampling the null set is reasonable under Rule 26(g), stating that “Defendants provide no reason establishing that a random sampling of the null set cannot be done when using key word searching. Indeed, sampling the null set when using key word searching provides for validation to defend the search and production process, and was commonly used before the movement towards TAR.”

Judge Johnston also ruled that sampling the null set is proportionate under Rule 26(b)(1), stating: “The Court’s experience and understanding is that a random sample of the null set will not be unreasonably expensive or burdensome. Moreover and critically, Defendants have failed to provide any evidence to support their contention…Indeed, the Court’s experience and understanding is that the random sample will not be voluminous in the context of a case of this magnitude.”  Judge Johnston also cited the issues at stake, the potential amount in controversy, asymmetrical discovery (with the defendants having access to the vast majority of the relevant information), the “substantial resources” of the defendant and that “the burden and expense of a random sampling of the null set does not outweigh its likely benefit of ensuring proper and reasonable – not perfect – document disclosure” all as reasons as to why sampling was proportionate in this case.

As a result, Judge Johnston ordered a random sample of the null set, determining that “Plaintiffs’ proposed 95% confidence level with +/-margin of 2% is acceptable.”

Editor’s Note: It’s worth noting that if you plug the proposed confidence level and margin of error into the Raosoft sample size calculator, you get no more than 2,401 documents that need to be sampled — even if the size of the null set is as large as 10 million documents.  Conducting a random sample is one of the most proportionate activities associated with eDiscovery review.

So, what do you think?  Should random sampling of the null set always be required in cases like this to help confirm a comprehensive search result?  Please let us know if any comments you might have or if you’d like to know more about a particular topic.

Case opinion link courtesy of eDiscovery Assistant.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.