Close
Enter your
text here

Searching

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Court Orders Defendant to Produce Additional ESI Responsive to 78 “Unopposed” Search Terms: eDiscovery Case Law

In Venturdyne, Ltd. d/b/a Scientific Dust Collectors v. Carbonyx, Inc., d/b/a Carbonyx Carbon Technologies, et. al., No. 14-00351 (N.D. Ind., Nov. 15, 2016), Indiana District Judge John E. Martin ordered the defendant to produce additional ESI to the plaintiff to be retrieved via 78 “unopposed” search terms that neither party objected to during negotiation over the plaintiff’s original list of 126 search terms.

Case Background

In this breach of contract case, the parties agreed to use keywords to search the defendant’s electronically stored information (ESI) and the defendant turned over sample emails related to the case to help determine what keywords would be appropriate.  On February 25, 2016, the plaintiff’s counsel sent the defendant’s counsel a list of 126 keywords to use for retrieving ESI.  On March 10, the defendant’s counsel sent an email to the plaintiff striking 20 search terms from the list and, on April 28, the plaintiff’s counsel responded by voluntarily removing 28 additional terms, leaving 78 terms and also objecting to 7 of the 20 terms deleted by the defendant.  In all, neither party objected to 78 of the original 126 search terms during this negotiation.

The plaintiff’s counsel sent the defendant’s counsel follow-up emails on May 6, May 24, June 3, June 13, and June 30 and the defendant’s counsel did not respond to any of those emails, causing the plaintiff to file a Motion to Compel on July 12.

Judge’s Ruling

While noting that “Carbonyx contends that it has produced ‘more than 12,000 pages of documents, consisting of emails and their attachments, as well as paper documents’”, Judge Martin also noted that “these documents were generated using the search terms ‘Scientific Dust Collectors’ – the name Venturdyne does business under – and ‘Scientific.’” (those terms happen to be redundant, by the way, as the second term would would include the results of the first term – just sayin’)  But, he also noted that “Carbonyx’s assertion that the documents it has already produced…are completely responsive is not convincing.”

As a result, Judge Martin stated:

“Accordingly, Carbonyx must produce documents flagged by the 78 unopposed search terms contained in DE 48-7. Those terms appear ‘reasonably calculated to lead to discovery of admissible evidence’ in that the terms were based on sample emails related to this case and the parties already eliminated many generic words like ‘money.’”

Judge Martin also noted that “the court must, after giving an opportunity to be heard, require the party . . . whose conduct necessitated the motion, the party or attorney advising that conduct, or both to pay the movant’s reasonable expenses incurred in making the motion, including attorney’s fees.”  So, he ordered the plaintiff to file an itemization of its costs and fees, including attorney’s fees, incurred in making the Motion to Compel by November 21, with the defendant to file a response by December 5 and the plaintiff to file a reply to that by December 12.

So, what do you think?  Would the ruling have turned out differently if the defendant had continued to work with the plaintiff?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Defendant Ordered to Produce Source Code Responsive to Agreed Upon Search Protocol: eDiscovery Case Law

In ACI Worldwide Corp. v. MasterCard Technologies, LLC et. al., No. 14-31 (D. Nebraska, Oct. 27, 2016), Nebraska Magistrate Judge F.A. Gossett granted the plaintiff’s Motion to Compel Production of the defendant’s Full Source Code in part, “to the extent there are any files that MasterCard has not produced responsive to the parties’ previously established joint Search Protocol.”

Case Background

In this action where the plaintiff alleged the defendants violated a licensing agreement and disclosed confidential information regarding the plaintiff’s software, the disclosure of the defendant’s source code had been the subject of ongoing discovery disputes for more than a year.  In an order dated July 13, 2015 (and covered by us here), the court found the plaintiff had shown the relevance and a particular need for electronically stored information (“ESI”) constituting or containing the plaintiff’s proprietary information, but, because the court did not have the expertise necessary to determine the best way to retrieve the requested information, the parties were directed to devise a joint search protocol or methodology to retrieve the information requested by the plaintiff; otherwise, the court would appoint a special master.

The parties then generally agreed to a search protocol, but were unable to reach a full agreement on the search terms to be used.  In August 2015, the court ordered the defendant to run the plaintiff’s “Counterproposal Search Protocol” and produce the search results no later than September 16, 2015; after the defendant’s objections were overruled, the parties subsequently conferred and agreed that the defendant would run the Search Protocol and produce the requested materials by October 9, 2015. The defendant continued to hesitate to produce actual source code without an additional protective order, and after another motion to compel by the plaintiff, the court again ordered the defendant to run the Search Protocol and produce the requested information, which included source code. In February 2016, the court denied the defendant’s motion for partial reconsideration, but stated it would entertain inclusions of additional provisions to the protective order in place, if the parties agreed. In March, the court adopted the parties’ joint protective order for source code.

Ultimately, the plaintiff filed the current motion to compel disclosure of the entire source code, claiming that the defendant had not produced all items identified by the Search Protocol, stating production of the entire source code was warranted due to “MasterCard’s history of refusing to produce clearly relevant source code” and alleging that the source code produced to date supported its allegations of misappropriation. The defendant counters that the remedy is the production of missing files pursuant to the Search Protocol, and not the production of the entire source code.

Judge’s Ruling

Noting its previous emphasis that “The Federal Rules of Civil Procedure . . . emphasize that electronic discovery should be a party-driven process”, Judge Gossett ruled that:

“ACI’s request now for the entire source code to the MDS is not proportional to the needs of the case, would include information irrelevant to ACI’s claims, and would defeat the purpose of the protracted efforts by the parties to reach a compromise regarding production of MDS source code…To the extent MasterCard’s production has been deficient, the remedy is not to compel the production of its entire source code for the MDS. Rather, the court will compel MasterCard to produce only the source code retrieved using the parties’ joint search protocol that MasterCard has not yet produced.”

So, what do you think?  Given the defendant’s numerous delays in producing source code, should the court have granted the plaintiff’s motion in total?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Defendant Not Required to Use Predictive Coding by Court: eDiscovery Case Law

Regardless how the election turned out last night, eDiscovery case law marches on…

In the case In re Viagra Products Liability Litigation, 16-02691 (N.D. Cali., Oct. 14, 2016), California Magistrate Judge Sallie Kim, noting that other courts had declined to force a party to use predictive coding, denied the plaintiff’s motion to force the defendant to use predictive coding instead of its preferred approach using search terms.

Case Background

In this multi-district litigation (MDL) against drug company Pfizer regarding its highly popular drug Viagra and alleged correlations to incidences of melanoma, the plaintiffs urged the Court to order the defendant to use predictive coding with the plaintiffs’ input to identify the locations of relevant information and the responsive ESI from those locations. The plaintiffs argued that TAR and/or predictive coding is a more sophisticated tool than the traditional search term or search query approach, that using that suggested approach would save time and money for both sides and indicated that they wanted representatives from both parties to participate in process of creating and working with the search process in this iterative process.

The defendant offered stiff opposition to the plaintiff’s request (sorry, I couldn’t resist) proposing instead that it use search terms to identify potentially relevant documents. The defendant described its preferred methodology as an iterative process – though not the same iterative process as TAR and/or predictive coding – where the defendant tests search terms and validates them using rigorous sampling of potentially responsive documents, verifying that the search terms yield high rates of response. In the defendant’s proposed process, the parties would exchange lists of proposed search terms and the defendant would agree to run any search terms that appeared on both parties’ lists.

The defendant pointed out that the plaintiffs did not cite to any case law in support of their proposal to require the defendant, over its objection, to use TAR and/or predictive coding. At the hearing on the matter, the plaintiffs conceded that no court has ordered a party to engage in TAR and/or predictive coding over the objection of the party.

Judge’s Ruling

Adding to the plaintiff’s concession, Judge Kim noted that “[t]he few courts that have considered this issue have all declined to compel predictive coding”.  Judge Kim cited Hyles v. New York City, stating:

“As the court reasoned in Hyles, the responding party is the one best situated to decide how to search for and produce ESI responsive to discovery requests.  The responding party ‘can use the search method of its choice. If [the propounding party] later demonstrates deficiencies in the . . . production, the [responding party] may have to re-do its search. But that is not a basis for Court intervention at this stage of the case…[I]t is not up to the Court, or the requesting party . . ., to force the . . . responding party to use TAR when it prefers to use keyword searching. While [the propounding party] may well be correct that production using keywords may not be as complete as it would be if TAR were used . . ., the standard is not perfection, or using the ‘best’ tool . . ., but whether the search results are reasonable and proportional.”

In denying the plaintiffs’ motion, Judge Kim concluded: “The Court finds Hyles well-reasoned. Even if predictive coding were a more efficient and better method, which Pfizer disputes, it is not clear on what basis the Court could compel Pfizer to use a particular form of ESI, especially in the absence of any evidence that Pfizer’s preferred method would produce, or has produced, insufficient discovery responses.”

So, what do you think?  Should a court ever require a party to use a particular method to search for and produce ESI?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Plaintiff Must Cooperate on Search Terms, Says Court: eDiscovery Case Law

In Pyle v. Selective Insurance Company of America, No. 2:16-cv-335 (W.D. Pa. Sept. 30, 2016), Pennsylvania Senior District Judge Terrence F. McVerry granted the defendant’s Motion to Compel Plaintiff to Provide ESI Search Terms “insofar as it seeks to compel Plaintiff to confer and come to an agreement on the search terms Defendant will use to cull through the additional email archives that Defendant has identified as having been retrieved.”

Case Background

In this case, plaintiff’s counsel served the defendant with a Request for Production of Documents in July 2016, which sought “all emails, correspondence, memorandum, and/or other documents” from various defendant employees relating to the plaintiff’s employment, particularly her receipt of disability benefits and the termination of her employment. The defendant responded by providing some relevant documents and also advised Plaintiff that “[i]n addition to the documents produced, additional email archives for various Selective employees have been retrieved”,  requesting that “Plaintiff agree with Defendant on appropriate electronic search terms to be used to located [sic] any relevant documents therein.”  After plaintiff’s counsel refused, defendant’s counsel e-mailed plaintiff’s counsel again in early September asking for a “suggested list of search terms . . . so that we can agree on a common set of terms so that the electronic records [Selective] has gathered can be searched for relevance.”  Plaintiff’s counsel responded by saying that the requests “are not requests for access to the email accounts of the individuals referenced in these requests that would necessitate our providing search terms and/or a protocol for the review of these accounts.”

Not satisfied with that response, the defendant filed a motion seeking an order compelling the plaintiff to provide ESI search terms or else relieving Defendant of the obligation to produce any ESI.  The plaintiff objected, arguing that the defendant cited no authority to support its request, nor identified any burden that it faces in locating and producing the requested emails.

Judge’s Ruling

In response to the plaintiff’s argument, Judge McVerry’s stated:

“Plaintiff’s argument totally misses the mark; in fact, it borders on being incomprehensible. Far from being baseless, Defendant’s request is entirely consistent with both the letter and spirit of the Federal Rules of Civil Procedure regarding the discovery of electronically stored information and this Court’s Local Rules. It is well settled by now that ‘electronic discovery should be a party-driven process.’…The Federal Rules expressly require counsel to meet and confer on ‘any issues about disclosure, discovery, or preservation of electronically stored information, including the form or forms in which it should be produced[.]’ Fed. R. Civ. P. 26(f)(3)(C). ‘Among the items about which the court expects counsel to `reach practical agreement’ without the court having to micro-manage e-discovery are `search terms, date ranges, key players and the like.’’…Indeed, this principle is incorporated into this Court’s Local Rules, which direct counsel to ‘meet and confer, and attempt to agree, on the discovery of ESI, including . . . an ESI search protocol, including methods to filter data, such as application of search terms or date ranges.’”

As a result, Judge McVerry granted the defendant’s motion “insofar as it seeks to compel Plaintiff to confer and come to an agreement on the search terms Defendant will use to cull through the additional email archives that Defendant has identified as having been retrieved.”  Judge McVerry denied the defendant’s Motion to Compel Production of Medical Records and Executed HIPAA Authorizations as “premature” since the 30 day time period for the plaintiff to respond had not yet elapsed.

So, what do you think?  Should receiving parties be required to negotiate search terms with producing parties?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Searching Only File Names is Not the “Safe Way” to Avoid Sanctions: eDiscovery Case Law

See what I did there?… :o)

In Rodman v. Safeway, No. 11-cv-03003-JST (N.D. Cal., Oct. 6, 2016), California District Judge Jon S. Tigar ordered the defendant to pay plaintiff’s Class Counsel $688,646 as a discovery sanction under Rule 26(g), ruling that “failure to search within the contents of the legacy drive constituted an unreasonable inquiry”, but denied without prejudice the plaintiff’s request for a negative jury instruction.

Case Background

In this certified class action for breach of contract, the Court previously granted the plaintiff’s motion for partial summary judgment that the defendant breached its contract with class members who registered to shop online after 2006 “by charging higher prices for groceries on its online Safeway.com delivery service than it charged in the stores where the groceries were selected.” After the Court denied the plaintiff’s motion for partial summary judgment regarding pre-2006 liability, only one issue remained to be tried: whether class members who registered for the delivery service prior to 2006 had agreed to the same contract as class members who registered after 2006.

Trial was scheduled to begin on October 6, 2015. Approximately seven days before that date, the defendant produced ten highly relevant documents related to its pre-2006 terms and conditions. The documents were found on a “legacy” computer drive by the defendant’s Director of Marketing, Steve Guthrie, who was reviewing that drive in anticipation for trial. Apparently, Guthrie’s initial search of the legacy drive did not involve a search within the contents of the files, but rather merely searched for file names within the legacy drive. Guthrie did not find the ten documents-in-question until approximately ten days before trial – over five months after fact discovery closed on April 24, 2015. Ultimately, the parties stipulated that the Court’s prior summary judgment orders were equally applicable to Class members who registered before January 1, 2006 and in November 2015, the Court issued a Judgment in favor of the certified class in the amount of $30,979,262 in damages and $10,905,505 in prejudgment interest, for a total of $41,884,767. The defendant appealed.

On April 6, 2016, Plaintiff filed a motion for discovery sanctions under Rule 26(g) based primarily on the defendant’s “false and inaccurate statements” in response to interrogatories and document requests “concerning the non-existences of documents reflecting historic copies of” the defendant’s pre-2006 terms and conditions. The Court held a hearing on the motion on August 25, 2016.

Judge’s Ruling

Judge Tigar disagreed with the defendant that its inquiry was unreasonable, stating “Although there is no indication of bad faith in Safeway’s inability to find the ten responsive documents during fact discovery, the Court concludes that Safeway’s initial search of the legacy drive was unreasonable”, giving three reasons that the inquiry was unreasonable:

  1. There was “no indication that Safeway’s counsel guided or monitored Mr. Guthrie’s search of the legacy drive in any significant way”;
  2. There was no evidence that Guthrie had any experience searching large repositories; and
  3. The evidence indicated that the search was objectively unreasonable.

Ultimately, Judge Tigar concluded that the defendant’s failure to search within the contents of the legacy drive constituted an unreasonable inquiry and that the defendant offered no substantial justification for its violation of Rule 26(g). The plaintiff requested attorneys’ fees totaling over $1 million, but Judge Tigar reduced that request by one-third, resulting in a fee award of $688,646. Judge Tigar also denied without prejudice the plaintiff’s request for a negative jury instruction, but indicated that the plaintiff may renew the request for a negative jury instruction should the Ninth Circuit remand this case for trial.

So, what do you think? Should the lack of finding of bad faith have resulted in a different outcome? Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

To Be or Not To Be? Not to Be, if Your Search Contains Noise Words: eDiscovery Best Practices

This is an issue that comes up frequently when my clients ask me to review their proposed search terms, or need help in understanding why a particular term doesn’t retrieve the intended result.

When providing searching assistance to my clients and reviewing their proposed list of search terms, one of the considerations I use for evaluating those terms is whether they contain any potential “noise” words that might affect their search results.  Noise words (also known as stop words) are words – such as “to”, “or”, “not”, etc. – which are so common that they are not considered useful in searches.

Search engines rely on indices to find information quickly – these indices are built and updated each time documents are loaded into the database.  To save time, both in creating the indexing and (especially) in performing searches, noise words are not indexed and are ignored in indexed searches. This enables even the most complex searches on the largest databases to be performed quickly and effectively; for example, a search for 30 to 40 terms within a 15 million document database in CloudNine usually takes a matter of seconds to retrieve the results.  I know, because I recently performed several of those searches for a client on their 15 million document database.

The advantage of excluding these words is smaller indexes and improved indexing, and searching, time.  If every “the”, “do”, “can” and “up” was indexed, searching eDiscovery databases would be way slower – painfully slow, in fact.

However, there can be drawbacks to not indexing these noise words.  One disadvantage is that if your searches are typically for common phrases, you may not be able to search with precision and you may either get additional non-responsive results or (even worse) miss some responsive results.

Years ago, I attended a presentation by Craig Ball, where he identified the perfect phrase that illustrates the problems with noise words:

“To Be or Not to Be”

This famous phrase in Shakespeare’s Hamlet would typically not be indexed at all in most search engines – every word in the phrase is a typical noise word.

If a quoted phrase in a search query includes a noise word, the search results may contain results with any word in place of the noise word. For example, a search query for “deed of trust”, might contain documents with the phrases “deed and trust” or “deed under trust” in the search results.

Most search tools can provide a list of the noise words used, so that you can adjust accordingly when constructing your searches.  So, when preparing a list of search terms, it’s important to remember that noise words exist and they could affect your search results.  If you have to search for phrases that contain noise words, you may retrieve some non-responsive hits in those results, so you want to be prepared to review to determine how effective the search was able to retrieve the desired results.  Don’t let noise words drown out your ability to effectively search your collection!

So, what do you think?  Have you encountered issues with noise words in your searches?  How have you addressed those issues?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Denies Defendant’s Motion to Overrule Plaintiff’s Objections to Discovery Requests

Defendant Not Required to Produce All Documents Responsive to Search Terms: eDiscovery Case Law

Remember earlier this week, when we once again discussed the perils of not checking your wildcard terms before agreeing to them?  This case law summary is a prime example of that.  Perhaps more narrowed search terms would have reduced the review required.  Frankly, I’m surprised more documents weren’t retrieved.

In Bancpass, Inc. v. Highway Toll Administration, LLC, No. A-14-CV-1062-SS (W.D. Tex. July 26, 2016), Texas Magistrate Judge Andrew W. Austin (no relation) denied the plaintiff’s Motion to Enforce Discovery Agreements with regard to the plaintiff’s request for the defendant to produce all non-privileged documents responsive to search terms agreed to over email.

Case Background

In this defamation case between two competing highway toll mechanism companies, the parties, negotiating over email, agreed to produce ESI responsive to the following search terms:

  • Smartphone /50 toll!
  • Smartphone /50 threat
  • Smartphone /10 app!
  • Phone! /10 app!
  • Double /10 bill
  • Geotoll

Slash followed by a number means within that number of words (e.g., /50 = within 50 words).  The ! after a term indicates a wildcard term.

*My own commentary: See terms 3 and 4 above?  They call for the phone related terms within 10 words of any word that begins with “app” (because they’re looking for phone applications or apps, got it?).  As you can see here, there are 306 words in the English language that begin with “app”.  Apparently, it appears that the parties (one of whom may someday be an appellant) appreciated the thought that appending a wildcard after “app” would be an appropriate application of the wildcard in that approach.  OK, a bit extreme, but you get the point – don’t get wild with wildcards!  A simple “(application! OR app OR apps)” would have saved a lot of false hits.

Anyway, the plaintiff stated that it believed the parties agreed that “all non-privileged documents responsive to the additional search terms will be produced,” which it understood to mean that every non-privileged document the search turned up would be produced, regardless of whether it had anything to do with the parties’ dispute and the plaintiff did so in its production.  Conversely, the defendant produced only those documents that it determined were relevant to the case and withheld the rest, which the plaintiff argued was a violation of the agreement.  In response, the defendant stated that it ran the searches as requested, but that the search terms turned up over 20,000 non-privileged but also non-responsive documents. After de-duplicating the search results against previous production and identifying clearly irrelevant documents, the defendant independently reviewed the remaining 3,489 documents for responsiveness and privilege, ultimately producing 34 documents captured by the agreed search terms that were both responsive to the plaintiff’s discovery requests and were non-privileged.

Judge’s Ruling

Ruling on this issue, Judge Austin stated:

“If the Court were to construe the parties’ email correspondence on this topic to constitute a contract, it is likely that HTA’s actions would amount to a breach. The Court’s reading of the e-mail chain is that the parties agreed to produce all of the results of the searches save privileged documents. But the parties’ e-mail exchange is not a contract. Rather, it was a means to simplify and limit the scope of production responsive to BancPass’s requests for production 6, 11, 29, 38, 39, and 45. Having reviewed the attached affidavits detailing HTA’s search and review process there is no reason to believe that HTA has withheld documents it was obligated to produce. Nor is it clear that additional searches with the identified search terms would produce more documents responsive to BancPass’s requests for production. Accordingly, as to the search terms, BancPass’s motion is denied.”

So, what do you think?  Should the defendant have been required to produce all of the responsive, non-privileged documents per the email discussion?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Sometimes, Your Wildcard May Not Be “Wild” Enough: eDiscovery Best Practices

On the very first day we launched this blog nearly six years ago (next Tuesday is our six year anniversary), one of our first blog posts was called “Don’t Get ‘Wild’ with Wildcards” where we showed how a poorly constructed wildcard of “min*” to retrieve variations like “mine”, “mines” and “mining” actually retrieved over 300,000 files with hits because there are 269 words in the English language that begin with the letters “min” (such as words like “mink”, “mind”, “mint” and “minion”).  Sometimes, though, you have the opposite problem – your wildcard isn’t “wild” enough.

Last week a client of mine provided some search terms to me for review.  One of the searches he proposed included a wildcard term for depreciate* to reflect assets that depreciate.  See any problem with that term?

That wildcard would have picked up variations such as depreciates and depreciated, but would have missed other obvious variations like depreciating and, of course, depreciation.  Oops.

So, how do you find the actual variations of the word you want?  One way, as we noted back in September 2010, is to list all of the words that begin with your search string.  Morewords.com is one site that shows a list of words that begin with your search string.  So, to get all 269 words beginning with “min”, go here – simply substitute any characters for “min” to see the words that start with those characters.  You can choose the variations you want and incorporate them into the search instead of the wildcard – i.e., use “(mine or “mines or mining)” instead of “min*” to retrieve a more relevant result set.

However, if you don’t want to search through 269 words to get the ones you want, or if you picked a place to insert your wildcard character so that all desired terms don’t even display, there’s another way.  As we discussed a couple of years ago, you can use a dictionary.

Dictionary.com, that is.  Type in the word that you want at the top of the form and find all of the uses of it (e.g., the yellow sweater is mine, which tells you not all of the hits may be relevant to mining terms) and also variations of a term (e.g., depreciated, depreciating, depreciation).  You can even find synonyms of the word (e.g., reserve, excavate) on the left hand side of the form (via Thesaurus.com) that might lead to additional terms you may want to include in your search.

Believe it or not, a poorly placed wildcard may sometimes not be “wild” enough.  If you want to make sure you cover all of the variations you need (and only those variations), use a dictionary.

So, what do you think? Do you use wildcards in your eDiscovery searches? If so, how do you check them to ensure that they are neither over-inclusive nor under-inclusive?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Considering All of the Considered TAR Responses: eDiscovery Trends

Our webinar panel discussion conducted by ACEDS a couple of weeks ago has generated some interesting discussion and debate over the past week or so about the content of the webinar, what it should have covered and what it was intended to cover.

The webinar panel discussion, titled How Automation is Revolutionizing eDiscovery was sponsored by CloudNine and presented on August 10 (here is a link to last week’s blog post with an embedded video of the webinar).  Our panel discussion provided an overview of eDiscovery automation technologies and we took a hard look at the technology and definition of TAR and potential limitations associated with both.  Mary Mack, Executive Director of ACEDS moderated the discussion and I was one of the panelists, along with Bill Dimm, CEO of Hot Neuron and Bill Speros, Evidence Consulting Attorney with Speros & Associates, LLC.

On the next day, ACEDS published A Considered Response from Gordon Cormack, which was a letter from Gordon Cormack, Professor with the School of Computer Science at the University of Waterloo in Ontario, Canada who is an expert in the area of technology-assisted review in litigation and has co-authored several influential works with Maura Grossman, a fellow researcher at the University of Waterloo (and, before that, an attorney with Wachtell, Lipton, Rosen & Katz).  Cormack and Grossman authored the 2011 Richmond Journal of Law and Technology (JOLT) study (that I inaccurately referred to in the webinar as a “white paper” – sorry about that) titled Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, which was discussed to some extent during the webinar that both Cormack and Grossman attended.

In his letter, Dr. Cormack expressed several concerns about the content of the webinar, including their belief “that the webinar presented the false impression that we, and the courts, are resting on our laurels and that no legitimate empirical work has been done with respect to TAR.”  He provided links to several other works that have been authored by Cormack and Grossman that were not discussed during the webinar and also noted his opinion that the webinar lacked “any constructive suggestion as to how to proceed” (among other concerns he noted).

Then, last week, ACEDS also published responses from two of the presenters, Bill Speros (Reconsidering Dr. Cormack’s Considered Response) and Bill Dimm (ACEDS Commentary: Bill Dimm Responds to Gordon Cormack) where both “Bills” (Speros and Dimm) provided responses to the concerns that Dr. Cormack raised in his letter the previous week.

I feel that both Speros and Dimm made several good points in both clarifying the intended scope of the webinar and also in what we feel the webinar accomplished.  Dimm noted that “our goal was to deliver a large amount of information that is useful to a broad e-discovery audience within the confines of a 60-minute webinar” (of which about 40 minutes were devoted to TAR) and that we covered the JOLT report as extensively as we did (instead of other Cormack/Grossman works) “because it is the study that judicial opinions rely upon, and we’re not aware of any subsequent study comparing the quality (not merely the cost) of TAR results to those of human review.”

Speros identified several constructive suggestions that we felt the webinar provided, including “Clarifying the (general lack of) judicial acceptance of TAR”, “Differentiating alternative TAR techniques and technologies” and “Developing independent and valid TAR assessments”.  Speros also noted that “rather than attacking Dr. Cormack and his work, the webinar’s content spoke to the quality of the court’s interpretations [of the JOLT report] in a manner entirely consistent with thoughtful and professional analysis” – a position with which I agree wholeheartedly.

So, what is my response to Dr. Cormack’s letter?

As the other presenter in the webinar, I don’t have much to add to the responses provided by Speros and Dimm, except that they essentially reflect my own thoughts about the intent and accomplishments of the webinar.  Our goal was to challenge several industry-accepted assumptions about TAR and to take a look at the current state of acceptance of TAR, both judicially and within organizations contemplating the use of TAR.  And, I feel we accomplished that.

Nonetheless, I have tremendous respect, not just for my co-presenters, but also for Gordon Cormack and Maura Grossman and the numerous contributions that they have made to the industry through their research and various works (including the 2011 JOLT report).  I consider this to be a healthy discussion and debate among industry thought leaders and look forward to hopefully seeing that healthy discussion and debate continue.  I encourage you to view the webinar and read the commentaries by Cormack, Speros and Dimm and draw your own conclusions.

So, what do you think?  Do you have an opinion on the webinar or on the topic of TAR in general?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

There’s a New TAR Case Law Primer Published by The Sedona Conference: eDiscovery Best Practices

One of the major topics discussed during last month’s panel discussion at The Masters Conference and also last week during our ACEDS webinar was the debate over the current state of judiciary acceptance of technology assisted review (TAR).  Whether intentional or not, to help with that debate The Sedona Conference® has just published the Public Comment Version of a new TAR Case Law Primer.

This was a project of The Sedona Conference Working Group on Electronic Document Retention and Production (WG1).  As noted in the Preface of the Primer, it is “the product of more than a year of development and dialogue within WG1. It was originally conceived as a chapter of a larger Commentary on the use of TAR in civil litigation, but the rapid development of the case law, the volume of court decisions, and the importance of those decisions in shaping legal practice in real time required that an exposition of the case law be made available on a faster timetable than WG1’s usual dialogue and consensus-building process allowed. For that reason, the Primer strives to present the case law in as neutral a fashion as possible. It avoids making any recommendations regarding particular TAR methodologies, nor does it propose any principles, guidelines, or best practices for TAR application, independent of those suggested by the courts themselves.”

The 33 page PDF file covers several topics related to Technology Assisted Review, including:

  • Da Silva Moore: Obviously as the first case to authorize the use of TAR, Judge Andrew Peck’s Da Silva Moore ruling merits its own section as a logical starting point, with the advantages of TAR and the emphasis on process (including advice to “follow the Sedona Cooperation Proclamation model”);
  • Other Courts’ Encouragement of the Use of TAR and Additional Cases Reflecting the Parties’ Use of TAR: The Primer touches on other cases where the use of TAR was either encouraged or used;
  • Disputed Issues Regarding TAR: This is the significant portion of the Primer, touching on issues ranging from efforts by requesting parties (and by courts) to compel the use of TAR, two contradictory decisions about whether it’s OK to switch to TAR in the middle of discovery, cases that address the appropriateness of using search terms to cull the document population before applying TAR, issues associated with disclosure of seed/training/validation sets and advance court approval of the use of TAR (such as the Dynamo Holdings case, which recently had another significant opinion about the results of the TAR process, despite cooperation between the parties).

The Primer also provides a review of evolving views of TAR and the state of international adoption, with discussion of both the Pyrrho Investments and Brown v BCA Trading cases.  The final section is an Index of Cases (at least those covered in the Primer).

The Conclusion section (just before the Index of Cases) offers this conclusion:

“While the case law reflects a broad consensus that TAR is an acceptable search and review methodology, certain issues regarding the details of its use remain unresolved. The general principles set forth in the cases discussed in this Primer should provide useful guidance to courts and parties seeking to use TAR to achieve the goals of Federal Rule 1 (the just, speedy, and inexpensive resolution of legal proceedings) and Rule 26(b)(1) (proportionality). The Bench and Bar should continue to actively monitor research and case law developments in this area.”

You can download the Primer here.  The Sedona Conference encourages Working Group Series members and others to spread the word and share this link so they can get comments in before the public comment period closes on October 15, 2016 (consider the word spread!). Questions and comments regarding the Primer may be sent to comments@sedonaconference.org.

So, what do you think? What do you consider to be the state of judicial acceptance of TAR?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.