Close
Enter your
text here

Searching

Law Firm Partner Says Hourly Billing Model “Makes No Sense” with AI: eDiscovery Trends

Artificial intelligence (AI) is transforming the practice of law and we’ve covered the topic numerous times (with posts here, here and here, among others).  And, I’m not even including all of the posts about technology assisted review (TAR).  According to one law firm partner at a recent panel discussion, it could even (finally) spell the end of the billable hour.

In Bloomberg Law’s Big Law Business blog (Billable Hour ‘Makes No Sense’ in an AI World, written by Helen Gunnarsson), the author covered a panel discussion at a recent American Bar Association conference, which included Dennis Garcia, an assistant general counsel for Microsoft in Chicago, Kyle Doviken, a lawyer who works for Lex Machina in Austin and Anthony E. Davis, a partner with Hinshaw & Culbertson LLP in New York.  The panel was moderated by Bob Ambrogi, a Massachusetts lawyer and blogger (including the LawSites blog, which we’ve frequently referenced on this blog).

Davis showed the audience a slide quoting Andrew Ng, a computer scientist and professor at Stanford University: “If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.” AI can “automate expertise,” Davis said. Because software marketed by information and technology companies is increasingly making it unnecessary to ask a lawyer for information regarding statutes, regulations, and requirements, “clients are not going to pay for time,” he said. Instead, he predicted, they will pay for a lawyer’s “judgment, empathy, creativity, adaptability, and emotional intelligence.”

Davis said AI will result in dramatic changes in law firms’ hiring and billing, among other things. The hourly billing model, he said, “makes no sense in a universe where what clients want is judgment.” Law firms should begin to concern themselves not with the degrees or law schools attended by candidates for employment but with whether they are “capable of developing judgment, have good emotional intelligence, and have a technology background so they can be useful” for long enough to make hiring them worthwhile, he said.

The panelists provided examples of how the use of artificial intelligence can enhance lawyers’ efficiency in areas such as legal research, document review in eDiscovery, drafting and evaluating contracts, evaluating lateral hires and even assessing propensities of federal judges.  Doviken indicated that a partner at a large firm had a “hunch” that a certain judge’s rulings favored alumni of the judge’s law school. After reviewing three years’ worth of data, the firm concluded the hunch was valid, assigned a graduate of that law school to a matter pending before that judge, and started winning its motions.

“The next generation of lawyers is going to have to understand how AI works” as part of the duty of competence, said Davis.  Want one example of how AI works that you are probably already using?  Click here.

So, what do you think?  Do you think that AI could spell the end of the billable hour?  Please let us know if any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Rejects Search Terms by Both Sides as Overly Inclusive: eDiscovery Case Law

Monday, I asked for a call for key eDiscovery case law cases in 2018 to cover.  While this one wasn’t overtly suggested, it was covered by Ralph Losey in his excellent e-Discovery Team® blog the same day, so that works too… :o)

In Am. Municipal Power, Inc. v. Voith Hydro, Inc., No. 2:17-cv-708 (S.D. Ohio June 4, 2018), Ohio Magistrate Judge Elizabeth A. Preston Deavers ruling on the parties’ arguments from a May discovery conference, concluded that search terms proposed by both parties in the case were overly inclusive.

Case Background

The parties provided extensive letter briefing for a discovery conference on May 24, 2018 regarding discovery disputes relating to the production of ESI and other documents, with the parties’ dispute centered around two ESI-related issues: (1) the propriety of a single-word search by Project name proposed by the defendant which it sought to have applied to the plaintiff’s ESI and (2) the propriety of the plaintiff’s request that the defendant run crafted search terms which the plaintiff proposed that were not limited to the Project’s name.

Judge’s Ruling

After careful consideration of the parties’ letter briefing and their arguments during the discovery conference, Judge Deavers concluded as follows with regard to the defendant’s proposed search terms:

“Voith’s single-word Project name search terms are over-inclusive. AMP’s position as the owner of the power-plant Projects puts it in a different situation than Voith in terms of how many ESI “hits” searching by Project name would return. As owner, AMP has stored millions of documents for more than a decade that contain the name of the Projects which refer to all kinds of matters unrelated to this case. Searching by Project name, therefore, would yield a significant amount of discovery that has no bearing on the construction of the power plants or Voith’s involvement in it, including but not limited to documents related to real property acquisitions, licensing, employee benefits, facility tours, parking lot signage, etc. While searching by the individual Project’s name would yield extensive information related to the name of the Project, it would not necessarily bear on or be relevant to the construction of the four hydroelectric power plants, which are the subject of this litigation. AMP has demonstrated that using a single-word search by Project name would significantly increase the cost of discovery in this case, including a privilege review that would add $100,000 — $125,000 to its cost of production. The burden and expense of applying the search terms of each Project’s name without additional qualifiers outweighs the benefits of this discovery for Voith and is disproportionate to the needs of even this extremely complicated case.”

Judge Deavers also concluded this with regard to the plaintiff’s proposed search terms:

“AMP’s request that Voith search its ESI collection without reference to the Project names by using as search terms including various employee and contractor names together with a list of common construction terms and the names of hydroelectric parts is overly inclusive and would yield confidential communications about other projects Voith performed for other customers. Voith employees work on and communicate regarding many customers at any one time. AMPs proposal to search terms limited to certain date ranges does not remedy the issue because those employees still would have sent and received communications about other projects during the times in which they were engaged in work related to AMP’s Projects. Similarly, AMP’s proposal to exclude the names of other customers’ project names with “AND NOT” phrases is unworkable because Voith cannot reasonably identify all the projects from around the world with which its employees were involved during the decade they were engaged in work for AMP on the Projects. Voith has demonstrated that using the terms proposed by AMP without connecting them to the names of the Projects would return thousands of documents that are not related to this litigation. The burden on Voith of running AMP’s proposed search terms connected to the names of individual employees and general construction terms outweighs the possibility that the searches would generate hits that are relevant to this case. Moreover, running the searches AMP proposes would impose on Voith the substantial and expensive burden of manually reviewing the ESI page by page to ensure that it does not disclose confidential and sensitive information of other customers. The request is therefore overly burdensome and not proportional to the needs of the case.”

So, what do you think?  Are these parties overreaching, do they need a course in search best practices or do they need a TAR approach?  Please share any comments you might have or if you’d like to know more about a particular topic.

Case opinion link courtesy of eDiscovery Assistant.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

EDRM Needs Your Input on its TAR Guidelines: eDiscovery Best Practices

I’m here in Durham, NC at the annual EDRM Spring Workshop at Duke Law School and, as usual, the Workshop is a terrific opportunity to discuss the creation of standards and guidelines for the legal community, as well as network with like minded people on eDiscovery topics.  I’ll have more to report about this year’s Workshop next week.  But, one part of the Workshop that I will touch on now is the release of the public comment version of EDRM’s Technology Assisted Review (TAR) Guidelines.

Last Friday, EDRM released the preliminary draft of its TAR Guidelines for public comment (you can download it here).  EDRM and the Bolch Judicial Institute at Duke Law are seeking comments from the bench, bar, and public on a preliminary draft of Technology Assisted Review (TAR) Guidelines. Nearly 50 volunteer lawyers, e-discovery experts, software developers, scholars and judges worked on this draft under the auspices of EDRM. A version of the document was presented at the Duke Distinguished Lawyers’ conference on Technology Assisted Review, held Sept. 7-8, 2017. At that event, 15 judges and nearly 100 lawyers and practitioners provided feedback and comments on the draft. The document was further revised based on discussions at that conference, additional review by judges and additional review by EDRM members over the past couple of months (which involved significant changes and a much tighter and briefer guideline document). With the assistance of four law student fellows of the Bolch Judicial Institute, this draft was finalized in May 2018 for public comment.

So, calling this a preliminary draft is a bit of a misnomer as it has already been through several iterations of review and edit.  Now, it’s the public’s turn.

EDRM states that “Comments on this preliminary draft will be carefully considered by the drafting team and an advisory group of judges as they finalize the document for publication. Please send comments on this draft, whether favorable, adverse, or otherwise, as soon as possible, but no later than Monday, July 16, 2018. Comments must be submitted in tracked edits (note: the guidelines are in a Word document for easy ability to track changes) and submitted via email to edrm@law.duke.edu. All comments will be made available to the public.”

That’s all well and good and EDRM will hopefully get a lot of useful feedback on the guideline document.  However, one thing I have observed about public comment periods is that the people who tend to provide comments (i.e., geeks like us who attend EDRM workshops) are people who already understand TAR (and think they know how best to explain it to others).  If the goal of the EDRM TAR guidelines is to help the general bench and bar better understand TAR, then it’s important for the average attorney to review the document and provide comments as to how useful it is.

So, if you’re an attorney or legal technology practitioner who doesn’t understand TAR, I encourage (even challenge) you to review these guidelines and provide feedback.  Point out what you learned from the document and what was confusing and whether or not you feel that you have a better understanding of TAR and the considerations for when to use it and where it can be used.  Ask yourself afterward if you have a better idea of how to get started using TAR and if you understand the difference between TAR approaches.  If these guidelines can help a lot of members of the legal profession better understand TAR, that will be the true measure of its effectiveness.

Oh, and by the way, Europe’s General Data Protection Regulation is now in effect!  Are you ready?  If not, you might want to check out this webcast.

So, what do you think?  Will these guidelines help the average attorney or judge better understand TAR?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

A Fresh Comparison of TAR and Keyword Search: eDiscovery Best Practices

Bill Dimm of Hot Neuron (the company that provides the product Clustify that provides document clustering and predictive coding technologies, among others) is one of the smartest men I know about technology assisted review (TAR).  So, I’m always interested to hear what he has to say about TAR, how it can be used and how effective it is when compared to other methods (such as keyword searching).  His latest blog post on the Clustify site talk about an interesting exercise that did exactly that: compared TAR to keyword search in a real classroom scenario.

In TAR vs. Keyword Search Challenge on the Clustify blog, Bill challenged the audience during the NorCal eDiscovery & IG Retreat to create keyword searches that would work better than technology-assisted review (predictive coding) for two topics.  Half of the room was tasked with finding articles about biology (science-oriented articles, excluding medical treatment) and the other half searched for articles about current law (excluding proposed laws or politics).  Bill then ran one of the searches against TAR in Clustify live during the presentation (the others he couldn’t do during the session due to time constraints, but did afterward and covered those on his blog, providing the specific searches to which he compared TAR).

To evaluate the results, Bill measured the recall from the top 3,000 and top 6,000 hits on the search query (3% and 6% of the population respectively) and also included the recall achieved by looking at all docs that matched the search query, just to see what recall the search queries could achieve if you didn’t worry about pulling in a ton of non-relevant docs.  For the TAR results he used TAR 3.0 (which is like Continuous Active Learning, but applied to cluster centers only) trained with (a whopping) two seed documents (one relevant from a keyword search and one random non-relevant document) followed by 20 iterations of 10 top-scoring cluster centers, for a total of 202 training documents.  To compare to the top 3,000 search query matches, the 202 training documents plus 2,798 top-scoring documents were used for TAR, so the total document review (including training) would be the same for TAR and the search query.

The result: TAR beat keyword search across the board for both tasks.  The top 3,000 documents returned by TAR achieved higher recall than the top 6,000 documents for any keyword search.  Based on this exercise, TAR achieved better results (higher recall) with half as much document review compared to any of the keyword searches.  The top 6,000 documents returned by TAR achieved higher recall than all of the documents matching any individual keyword search, even when the keyword search returned 27,000 documents.

Bill acknowledges that the audience had limited time to construct queries, they weren’t familiar with the data set, and they couldn’t do sampling to tune their queries, so the keyword searching wasn’t optimal.  Then again, for many of the attorneys I’ve worked with, that sounds pretty normal.  :o)

One reader commented about email headers and footers cluttering up results and Bill pointed out that “Clustify has the ability to ignore email header data (even if embedded in the middle of the email due to replies) and footers” – which I’ve seen and is actually pretty cool.  Irrespective of the specifics of the technology, Bill’s example is a terrific fresh example of how TAR can outperform keyword search – as Bill notes in his response to the commenter “humans could probably do better if they could test their queries, but they would probably still lose”.  Very interesting.  You’ll want to check out the details of his test via the link here.

So, what do you think?  Do you think this is a valid comparison of TAR and keyword searching?  Why or why not?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Court Compels Discovery in Response to Party That Was Using Outdated Rule 26 Standard: eDiscovery Case Law

In Cen Com, Inc. v. Numerex Corp., No. C17-0560 RSM, (W.D. Wash., April 11, 2018), Washington Chief District Judge Ricardo S. Martinez ruled that the Plaintiff’s refusal to comply with the Defendant’s request for discovery using specific search terms was not justified, and that the Plaintiff must “fully comply with the subpoenas that Defendants served upon them and shall produce all responsive documents in a format that is accessible/readable by Defendants.”

Case Background

A request for discovery was issued by the Defendant for the founder and owner of the Plaintiff, along with two current employees of the Plaintiff, all of whom were former employees of the Defendant. The plaintiffs objected to the subpoenas “on the basis that it was an improper attempt to obtain discovery from a party employee,” and “that the subpoena is overbroad, unduly burdensome, and that the costs outweigh the potential for acquiring relevant information.”

The Defendant also filed a motion to compel the Plaintiff to use specific electronic search terms (“attorney w/2 general” and “consent w/2 decree”) related to a 2012 consent decree that Plaintiff entered into with Washington State’s Attorney General. The Plaintiff objected to the search terms regarding the consent decree as irrelevant.

As part of a counterclaim, the Plaintiff requested sanctions against the Defendant, claiming they withheld certain documents because of a pending motion for protective order, which was later denied by the Court. However, the Plaintiff continued to seek sanctions for the time period that it alleges Defendants were not in compliance with the stipulated ESI Order.

Judge’s Ruling

In Judge Martinez’s ruling, all of the Defendants’ motions were granted. Regarding the scope and relevance of the discovery request, it was noted that the Plaintiff was basing their refusal to comply on the former FRCP Rule 26 standard and not in line with the current version of Rule 26, which states discovery must be relevant to the claim and proportional to the needs of the case, while taking into account the parties’ access to relevant information and available resources, the importance of the discovery in resolving the matter, and whether the burden or expense of discovery outweighs its likely benefit.

Additionally, under Rule 37, “The party who resists discovery has the burden to show that discovery should not be allowed, and has the burden of clarifying, explaining, and supporting its objections.” Here the Plaintiff failed to explain specifically why the documents are not relevant, or that a search of the documents would be unduly burdensome, and instead only made the blanket statement that the documents sought “do not concern this matter and could not lead to relevant information.”

In regard to the Plaintiff’s counterclaim, Judge Martinez denied the motion for sanctions, citing Rule 37(d)(2): “A failure described in Rule 37(d)(1)(A) is not excused on the ground that the discovery sought was objectionable, unless the party failing to act has a pending motion for a protective order under Rule 26(c).”

So, what do you think?  Was the ruling correct or were the Defendant’s requests “overly burdensome”?  Please share any comments you might have or if you’d like to know more about a particular topic.

Case opinion link courtesy of eDiscovery Assistant.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Don’t Miss Our Webcast Today on Technology Assisted Review!: eDiscovery Webcasts

What is Technology Assisted Review (TAR)? Why don’t more lawyers use it? Find out in our webcast today!

Today at noon CST (1:00pm EST, 10:00am PST), CloudNine will conduct the webcast Getting Off the Sidelines and into the Game using Technology Assisted Review. In this one-hour webcast that’s CLE-approved in selected states, will discuss what TAR really is, when it may be appropriate to consider for your case, what challenges can impact the use of TAR and how to get started. Topics include:

  • Understanding the Goals for Retrieving Responsive ESI
  • Defining the Terminology of TAR
  • Different Forms of TAR and How They Are Used
  • Acceptance of Predictive Coding by the Courts
  • How Big Does Your Case Need to Be to use Predictive Coding?
  • Considerations for Using Predictive Coding
  • Challenges to an Effective Predictive Coding Process
  • Confirming a Successful Result with Predictive Coding
  • How to Get Started with Your First Case using Predictive Coding
  • Resources for More Information

Once again, I’ll be presenting the webcast, along with Tom O’Connor, who recently wrote an article about TAR that we covered on this blog.  To register for it, click here.  Even if you can’t make it, go ahead and register to get a link to the slides and to the recording of the webcast (if you want to check it out later).  If you want to learn about TAR, what it is and how to get started, this is the webcast for you!

So, what do you think?  Do you use TAR to assist in review in your cases?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Why Is TAR Like a Bag of M&M’s?, Part Four: eDiscovery Best Practices

Editor’s Note: Tom O’Connor is a nationally known consultant, speaker, and writer in the field of computerized litigation support systems.  He has also been a great addition to our webinar program, participating with me on several recent webinars.  Tom has also written several terrific informational overview series for CloudNine, including eDiscovery and the GDPR: Ready or Not, Here it Comes (which we covered as a webcast), Understanding eDiscovery in Criminal Cases (which we also covered as a webcast) and ALSP – Not Just Your Daddy’s LPO.  Now, Tom has written another terrific overview regarding Technology Assisted Review titled Why Is TAR Like a Bag of M&M’s? that we’re happy to share on the eDiscovery Daily blog.  Enjoy! – Doug

Tom’s overview is split into four parts, so we’ll cover each part separately.  The first part was covered last Tuesday, the second part was covered last Thursday and the third part was covered this past Tuesday.  Here’s the final part, part four.

Justification for Using TAR

So where does this leave us? The idea behind TAR – that technology can help improve the eDiscovery process – is a valuable goal. But figuring out what pieces of technology to apply at what point in the workflow is not so easy, especially when the experts disagree as to the best methodology.

Is there a standard, either statutory or in case law to help us with this determination?  Unfortunately, no. As Judge Peck noted on page 5 of the Hyles case mentioned above, “…the standard is not perfection, or using the “best” tool, but whether the search results are reasonable and proportional.”

FRCP 1 is even more specific.

These rules govern the procedure in all civil actions and proceedings in the United States district courts, except as stated in Rule 81. They should be construed, administered, and employed by the court and the parties to secure the just, speedy, and inexpensive determination of every action and proceeding.  (emphasis added)

The Court in any given matter decides if the process being used is just.  And although we have seen ample evidence that computers are faster than humans, speed may not always equate to accuracy. I’ll leave aside the issue of accuracy for another day since two of the most interesting case studies, the EDI/Oracle study and the most recent Lex Geek “study” in which a human SME scored exactly the same number of accurate retrievals as the computer system.

I am most interested in pointing out that few if any studies or case law opinions address the issue of inexpensive.  To his credit, Judge Peck did note in footnote 2 on page 3 of the Hyles opinion that “…some vendor pricing models charge more for TAR than for keywords.” but went on to note that typically those costs are offset by review time savings.  With all due respect to Judge Peck, to whose opinion I give great credence, I am not sure that is necessarily the case.

Most case studies I have seen emphasize speed or accuracy and don’t even mention cost. Yet the increased emphasis on proportionality in eDiscovery matters makes this third requirement more important than ever. Maura Grossman does provide for this concern in her Broiler Chicken protocol but only to the extent that a concerned party should bring any issues to the Special Master.

The proportionality issue is an important one. Principle 4 of the Sedona Conference Commentary on Proportionality in Electronic Discovery states that “The application of proportionality should be based on information rather than speculation.” Absent specific statistics regarding TAR costs, it seems we are all too often engaging in speculation about the true cost a specific technology.

I am mindful of the decision in the case of In Re State Farm Lloyds in March of 2017 (covered by eDiscovery Daily here), in which the Texas Supreme Court, deciding a matter involving the form of production and noting it’s parity with the Federal Rules, remarked that one party made an assertion of an “… extraordinary and burdensome undertaking … without quantifying the time or expense involved.”   Meaningful case studies and their statistics about the actual costs of various technologies would go a long way towards resolving these sort of disputes and fulfilling the requirement of FRCP 1.

Conclusions

Although the use of TAR has been accepted in the courts for several years, there is still a great deal of confusion as to what TAR actually is. As a result, many lawyers don’t use TAR at all.

In addition, the lack of definitions makes pricing problematic. This means that the several of the Federal Rules of Civil Procedure are difficult if not impossible to implement including FRCP 1 and FRCP 26(b)(1).

It is essential for the proper use of technology to define what TAR means and to determine not only the different forms of TAR but the costs of using each of them.  Court approval of technology such as predictive coding, clustering and even AI all depend on clear concise information and cost analysis.  Only then will technology usage be effective as well as just, speedy and inexpensive.

So, what do you think?  How would you define TAR?  As always, please share any comments you might have or if you’d like to know more about a particular topic.

Image Copyright © Mars, Incorporated and its Affiliates.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Why Is TAR Like a Bag of M&M’s?, Part Three: eDiscovery Best Practices

Editor’s Note: Tom O’Connor is a nationally known consultant, speaker, and writer in the field of computerized litigation support systems.  He has also been a great addition to our webinar program, participating with me on several recent webinars.  Tom has also written several terrific informational overview series for CloudNine, including eDiscovery and the GDPR: Ready or Not, Here it Comes (which we covered as a webcast), Understanding eDiscovery in Criminal Cases (which we also covered as a webcast) and ALSP – Not Just Your Daddy’s LPO.  Now, Tom has written another terrific overview regarding Technology Assisted Review titled Why Is TAR Like a Bag of M&M’s? that we’re happy to share on the eDiscovery Daily blog.  Enjoy! – Doug

Tom’s overview is split into four parts, so we’ll cover each part separately.  The first part was covered last Tuesday and the second part was covered last Thursday.  Here’s part three.

Uses for TAR and When to Use or Not Use It

Before you think about using more advanced technology, start with the basic tools early on: dedupe, de-nist, cull by dates and sample by custodians. Perhaps even keyword searches if your case expert fully understands case issues and is consistent in his or her application of that understanding.

When you have all (or at least most) of your data at the outset, some examples are:

  • Review-for-production with very large data sets
  • First pass review for Responsive/Not Responsive
  • First pass review for Privileged/Not Privileged
  • Deposition preparation
  • Working with an expert witness

Then when you are ready to move on to more advanced analytics, get an expert to assist you who has legal experience and can explain the procedure to you, your opponent and the Court in simple English.

Advanced tools may also be helpful when all of the data is not yet collected, but you need to:

  • Identify and organize relevant data in large datasets
  • When the objective is more than just identifying relevance or responsiveness
  • If you need to locate a range of issues
  • If you have a very short deadline for a motion or hearing

There are several operational cautions to keep in mind however.

  1. TAR isn’t new: it’s actually the product of incremental improvements over the last 15 years
  2. TAR isn’t one tool: just as there is no one definition of the tools, there is likewise no single approach to how they’re employed
  3. TAR tools do not “understand” or “read” documents. They work off of numbers, not words

And when do you NOT want to use TAR? Here is a good example.

This is a slide that Craig Ball uses in his presentation on TAR and eDiscovery:

Image Copyright © Craig D. Ball, P.C.

The point is clear. With large data sets that require little or no human assessment, TAR … and here we are specifically talking about predictive coding …. is your best choice. But for the close calls, you need a human expert.

How does this work with actual data? The graphic below from the Open Source Connections blog shows a search result using a TAR tool in a price fixing case involving wholesale grocery sales.  The query was to find and cluster all red fruits.

Image Copyright © Open Source Connections blog

What do see from this graphic?  The immediate point is that the bell pepper is red, but it is a vegetable not a fruit. What I pointed out to the client however was there were no grapes in the results.  A multi modal approach with human intervention could have avoided both these errors.

We’ll publish Part 4 – Justification for Using TAR and Conclusions – on Thursday.

So, what do you think?  How would you define TAR?  As always, please share any comments you might have or if you’d like to know more about a particular topic.

Image Copyright © Mars, Incorporated and its Affiliates.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Getting Off the Sidelines and into the Game using Technology Assisted Review: eDiscovery Webcasts

The use of Technology Assisted Review (TAR) has been accepted in the courts for several years, but most lawyers still don’t use it and many still don’t know what it is or how it works. Why not?  We will discuss this and other questions in a webcast next week.

On Wednesday, April 25 at noon CST (1:00pm EST, 10:00am PST), CloudNine will conduct the webcast Getting Off the Sidelines and into the Game using Technology Assisted Review. In this one-hour webcast that’s CLE-approved in selected states, will discuss what TAR really is, when it may be appropriate to consider for your case, what challenges can impact the use of TAR and how to get started. Topics include:

  • Understanding the Goals for Retrieving Responsive ESI
  • Defining the Terminology of TAR
  • Different Forms of TAR and How They Are Used
  • Acceptance of Predictive Coding by the Courts
  • How Big Does Your Case Need to Be to use Predictive Coding?
  • Considerations for Using Predictive Coding
  • Challenges to an Effective Predictive Coding Process
  • Confirming a Successful Result with Predictive Coding
  • How to Get Started with Your First Case using Predictive Coding
  • Resources for More Information

Once again, I’ll be presenting the webcast, along with Tom O’Connor, who recently wrote an article about TAR that we are currently covering on this blog (parts one and two were published last week, the remaining two parts will be published this week).  To register for it, click here.  Even if you can’t make it, go ahead and register to get a link to the slides and to the recording of the webcast (if you want to check it out later).  If you want to learn about TAR, what it is and how to get started, this is the webcast for you!

So, what do you think?  Do you use TAR to assist in review in your cases?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

You May Be a User of Predictive Coding Technology and Not Realize It: eDiscovery Trends

At the Houston ACEDS luncheon/TAR panel last week, we asked a few questions of the audience to gauge their understanding and experience with Technology Assisted Review (TAR).  Some of the questions (like “have you used TAR on a case?”) were obvious questions to ask.  Others might have not been so obvious.

Like, “do you watch movies and TV shows on Netflix or Amazon Prime?”  Or, “do you listen to music on Pandora or Spotify”?

So, why would we ask a question like that on a TAR panel?

Because those sites are examples of uses of artificial intelligence and supervised machine learning.

But first, this week’s eDiscovery Tech Tip of the Week is about Boolean Searching.  When performing searches, the ability to combine multiple criteria into a single search to be performed is key to help achieve a proper balance of recall and precision in that search.  Using OR operators between search terms helps expand recall by retrieving documents that meet ANY of the criteria; while using AND or AND NOT operators between search terms help improve precision by only retrieving documents that are responsive if they include all terms (AND) or exclude certain terms (AND NOT).

Grouping of those parameters properly is important as well.  My first name is Dozier, so a search for my name could be represented as Doug or Douglas or Dozier and Austin or it could be represented as (Doug or Douglas or Dozier) and Austin.  One of them is right.  Guess which one!  Regardless, boolean searching is an important part of efficient search and retrieval of documents to meet discovery requirements.

To see an example of how Boolean Searching is conducted using our CloudNine platform, click here (requires BrightTalk account, which is free).

Anyway, back to the topic of the day.  Let’s take Pandora, for example.  I was born in the 60’s – yes, I look GREAT for my age, :o) – and so I’m a fan of classic rock.  Pandora is a site where you can set up “stations” of your favorite artists.  If you’re a fan of classic rock and you’re born in the 60’s, you probably love an artist like Jimi Hendrix.  Right?

Well, I do and I have a Pandora account, so I set up a Jimi Hendrix “station”.  But, Pandora doesn’t just play Jimi Hendrix on that station, it plays other artists and songs it thinks I might like that are in a similar genre.  Artists like Stevie Ray Vaughan (The Sky is Crying), Led Zeppelin (Kashmir), The Doors (Peace Frog) and Ten Years After (I’d Love to Change the World), which is the example you see above.  For each song, you can listen to it, skip it, or give it a “thumbs up” or “thumbs down” (for the record, I wouldn’t give any of the above songs a “thumbs down”).  If you give a song a “thumbs up”, you’re more likely to hear the song again and if you give the song a “thumbs down”, you’re less likely to hear it again (at least in theory).

Does something sound familiar about that?

You’re training the system.  Pandora is using the feedback you give it to (hopefully) deliver more songs that you like and less of the songs you don’t like to improve your listening experience.  One nice thing about it is that you get to listen to songs or artists you may not have heard before and learn to enjoy them as well (that’s how I got to be a fan of The Black Keys, for example).

If you watch a show or movie on Netflix and you log in sometime afterward, Netflix will suggest shows for you to watch, based on what you’ve viewed previously (especially if you rate what you watched highly).

That’s what supervised machine learning is and what a predictive coding algorithm does.  “Thumbs up” is the same as marking a document responsive, “thumbs down” is the same as marking a document non-responsive.  The more documents (or songs or movies) you classify, the more likely you’re going to receive relevant and useful documents (or songs or movies) going forward.

When it comes to teaching the legal community about predictive coding, “I’d love to save the world, but I don’t know what to do”.  Maybe, I can start by teaching people about Pandora!  So, you say you’ve never used a predictive coding algorithm before?  Maybe you have, after all.  :o)

Speaking of predictive coding, is that the same as TAR or not?  If you want to learn more about what TAR is and what it could also be, check out our webcast Getting Off the Sidelines and into the Game using Technology Assisted Review on Wednesday, April 25.  Tom O’Connor and I will discuss a lot of topics related to the use of TAR, including what TAR is (or what people think it is), considerations and challenges to using TAR and how to get started using it.  To register, click here!

So, what do you think?  Have you used a predictive coding algorithm before?  Has your answer changed after reading this post?  :o)  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.