Review

On his Complex Discovery site, Rob Robinson does a great job of analyzing trends in the eDiscovery industry and often uses surveys to gauge sentiment within the industry for things like industry business confidence.  Now, Rob is proving and overview and conducting a survey regarding predictive coding technologies and protocols for representatives of leading eDiscovery providers that should prove interesting.

On his site at Predictive Coding Technologies and Protocols: Overview and Survey, Rob notes that “it is increasingly more important for electronic discovery professionals to have a general understanding of the technologies that may be implemented in electronic discovery platforms to facilitate predictive coding of electronically stored information.”  To help in that, Rob provides working lists of predictive coding technologies and TAR protocols that is worth a review.

You probably know what Active Learning is.  Do you know what Latent Semantic Analysis is? What about Logistic Regression?  Or a Naïve Bayesian Classifier?  If you don’t, Rob discusses definitions for these different types of predictive coding technologies and others.

Then, Rob also provides a list of general TAR protocols that includes Simple Passive Learning (SPL), Simple Active Learning (SAL), Continuous Active Learning (CAL) and Scalable Continuous Active Learning (S-CAL), as well as the Hybrid Multmodal Method used by Ralph Losey.

Rob concludes with a link to a simple three-question survey designed to help electronic discovery professionals identify the specific machine learning technologies and protocols used by eDiscovery providers in delivering the technology-assisted review feature of predictive coding.  It literally take 30 seconds to complete.  To find out the questions, you’ll have to check out the survey.  ;o)

So far, Rob has received 19 responses (mine was one of those).  It will be interesting to see the results when he closes the survey and publishes the results.

So, what do you think?  Are you an expert in predictive coding technologies and protocols?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Artificial intelligence (AI) is transforming the practice of law and we’ve covered the topic numerous times (with posts here, here and here, among others).  And, I’m not even including all of the posts about technology assisted review (TAR).  According to one law firm partner at a recent panel discussion, it could even (finally) spell the end of the billable hour.

In Bloomberg Law’s Big Law Business blog (Billable Hour ‘Makes No Sense’ in an AI World, written by Helen Gunnarsson), the author covered a panel discussion at a recent American Bar Association conference, which included Dennis Garcia, an assistant general counsel for Microsoft in Chicago, Kyle Doviken, a lawyer who works for Lex Machina in Austin and Anthony E. Davis, a partner with Hinshaw & Culbertson LLP in New York.  The panel was moderated by Bob Ambrogi, a Massachusetts lawyer and blogger (including the LawSites blog, which we’ve frequently referenced on this blog).

Davis showed the audience a slide quoting Andrew Ng, a computer scientist and professor at Stanford University: “If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.” AI can “automate expertise,” Davis said. Because software marketed by information and technology companies is increasingly making it unnecessary to ask a lawyer for information regarding statutes, regulations, and requirements, “clients are not going to pay for time,” he said. Instead, he predicted, they will pay for a lawyer’s “judgment, empathy, creativity, adaptability, and emotional intelligence.”

Davis said AI will result in dramatic changes in law firms’ hiring and billing, among other things. The hourly billing model, he said, “makes no sense in a universe where what clients want is judgment.” Law firms should begin to concern themselves not with the degrees or law schools attended by candidates for employment but with whether they are “capable of developing judgment, have good emotional intelligence, and have a technology background so they can be useful” for long enough to make hiring them worthwhile, he said.

The panelists provided examples of how the use of artificial intelligence can enhance lawyers’ efficiency in areas such as legal research, document review in eDiscovery, drafting and evaluating contracts, evaluating lateral hires and even assessing propensities of federal judges.  Doviken indicated that a partner at a large firm had a “hunch” that a certain judge’s rulings favored alumni of the judge’s law school. After reviewing three years’ worth of data, the firm concluded the hunch was valid, assigned a graduate of that law school to a matter pending before that judge, and started winning its motions.

“The next generation of lawyers is going to have to understand how AI works” as part of the duty of competence, said Davis.  Want one example of how AI works that you are probably already using?  Click here.

So, what do you think?  Do you think that AI could spell the end of the billable hour?  Please let us know if any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Bill Dimm of Hot Neuron (the company that provides the product Clustify that provides document clustering and predictive coding technologies, among others) is one of the smartest men I know about technology assisted review (TAR).  So, I’m always interested to hear what he has to say about TAR, how it can be used and how effective it is when compared to other methods (such as keyword searching).  His latest blog post on the Clustify site talk about an interesting exercise that did exactly that: compared TAR to keyword search in a real classroom scenario.

In TAR vs. Keyword Search Challenge on the Clustify blog, Bill challenged the audience during the NorCal eDiscovery & IG Retreat to create keyword searches that would work better than technology-assisted review (predictive coding) for two topics.  Half of the room was tasked with finding articles about biology (science-oriented articles, excluding medical treatment) and the other half searched for articles about current law (excluding proposed laws or politics).  Bill then ran one of the searches against TAR in Clustify live during the presentation (the others he couldn’t do during the session due to time constraints, but did afterward and covered those on his blog, providing the specific searches to which he compared TAR).

To evaluate the results, Bill measured the recall from the top 3,000 and top 6,000 hits on the search query (3% and 6% of the population respectively) and also included the recall achieved by looking at all docs that matched the search query, just to see what recall the search queries could achieve if you didn’t worry about pulling in a ton of non-relevant docs.  For the TAR results he used TAR 3.0 (which is like Continuous Active Learning, but applied to cluster centers only) trained with (a whopping) two seed documents (one relevant from a keyword search and one random non-relevant document) followed by 20 iterations of 10 top-scoring cluster centers, for a total of 202 training documents.  To compare to the top 3,000 search query matches, the 202 training documents plus 2,798 top-scoring documents were used for TAR, so the total document review (including training) would be the same for TAR and the search query.

The result: TAR beat keyword search across the board for both tasks.  The top 3,000 documents returned by TAR achieved higher recall than the top 6,000 documents for any keyword search.  Based on this exercise, TAR achieved better results (higher recall) with half as much document review compared to any of the keyword searches.  The top 6,000 documents returned by TAR achieved higher recall than all of the documents matching any individual keyword search, even when the keyword search returned 27,000 documents.

Bill acknowledges that the audience had limited time to construct queries, they weren’t familiar with the data set, and they couldn’t do sampling to tune their queries, so the keyword searching wasn’t optimal.  Then again, for many of the attorneys I’ve worked with, that sounds pretty normal.  :o)

One reader commented about email headers and footers cluttering up results and Bill pointed out that “Clustify has the ability to ignore email header data (even if embedded in the middle of the email due to replies) and footers” – which I’ve seen and is actually pretty cool.  Irrespective of the specifics of the technology, Bill’s example is a terrific fresh example of how TAR can outperform keyword search – as Bill notes in his response to the commenter “humans could probably do better if they could test their queries, but they would probably still lose”.  Very interesting.  You’ll want to check out the details of his test via the link here.

So, what do you think?  Do you think this is a valid comparison of TAR and keyword searching?  Why or why not?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

What is Technology Assisted Review (TAR)? Why don’t more lawyers use it? Find out in our webcast today!

Today at noon CST (1:00pm EST, 10:00am PST), CloudNine will conduct the webcast Getting Off the Sidelines and into the Game using Technology Assisted Review. In this one-hour webcast that’s CLE-approved in selected states, will discuss what TAR really is, when it may be appropriate to consider for your case, what challenges can impact the use of TAR and how to get started. Topics include:

• Understanding the Goals for Retrieving Responsive ESI
• Defining the Terminology of TAR
• Different Forms of TAR and How They Are Used
• Acceptance of Predictive Coding by the Courts
• How Big Does Your Case Need to Be to use Predictive Coding?
• Considerations for Using Predictive Coding
• Challenges to an Effective Predictive Coding Process
• Confirming a Successful Result with Predictive Coding
• How to Get Started with Your First Case using Predictive Coding
• Resources for More Information

Once again, I’ll be presenting the webcast, along with Tom O’Connor, who recently wrote an article about TAR that we covered on this blog.  To register for it, click here.  Even if you can’t make it, go ahead and register to get a link to the slides and to the recording of the webcast (if you want to check it out later).  If you want to learn about TAR, what it is and how to get started, this is the webcast for you!

So, what do you think?  Do you use TAR to assist in review in your cases?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.