Industry Trends

eDiscovery Trends: Review Attorneys, Are You Smarter than a High Schooler?

 

Review attorneys are taking a beating these days.  There’s so much attention being focused on technology assisted review, with the latest study noting the cost-effectiveness of technology assisted review (when compared to manual review) having just been released this month.  There is also the very detailed and well known white paper study written by Maura Grossman and Gordon Cormack (Technology-Assisted Review in E-Discovery can be More Effective and More Efficient that Exhaustive Manual Review) which notes not only the cost-effectiveness of technology assisted review but also that it was actually more accurate.

The latest study, from information scientist William Webber (and discussed in this Law Technology News article by Ralph Losey) seems to indicate that trained reviewers don’t provide any better review accuracy than a pair of high schoolers that he selected with “no legal training, and no prior e-discovery experience, aside from assessing a few dozen documents for a different TREC topic as part of a trial experiment”.  In fact, the two high schoolers did better!  He also notes that “[t]hey worked independently and without supervision or correction, though one would be correct to describe them as careful and motivated.”  His conclusion?

“The conclusion that can be reached, though, is that our assessors were able to achieve reliability (with or without detailed assessment guidelines) that is competitive with that of the professional reviewers — and also competitive with that of a commercial e-discovery vendor.”

Webber also cites two other studies with similar results and notes “All of this raises the question that is posed in the subject of this post: if (some) high school students are as reliable as (some) legally-trained, professional e-discovery reviewers, then is legal training a practical (as opposed to legal) requirement for reliable first-pass review for responsiveness? Or are care and general reading skills the more important factors?”

I have a couple of observations about the study.  Keep in mind, I’m not an attorney (and don’t play one on TV), but I have worked with review teams on several projects and have observed the review process and how it has been conducted in a real world setting, so I do have some real-world basis for my thoughts:

  • Two high schoolers is not a significant sample size: I’ve worked on several projects where some reviewers are really productive and others are highly unproductive to the point of being useless.  It’s difficult to determine a valid conclusion on the basis of two non-legal reviewers in his study and four non-legal reviewers in one of the studies that Webber cites.
  • Review is typically an iterative process: In my experience, most legal reviews that I’ve seen start with detailed instructions and training provided to the reviewers, followed up with regular (daily, if not more frequent) changes to instructions to reflect information gathered during the review process.  Instructions are refined as the review commences and more information is learned about the document collection.  Since Webber noted that “[t]hey worked independently and without supervision or correction”, it doesn’t appear that his review test was conducted in this manner.  This makes it less of a real world scenario, in my opinion.

I also think some reviews especially benefit from a first pass review with legal trained reviewers (for example, a reviewer who understands intellectual property laws is going to understand potential IP issues better than someone who hasn’t had the training in IP law).  Nonetheless, these studies are bound to “fan the flames” of debate regarding the effectiveness of manual attorney review (even more than they already are).

So, what do you think?  Do you think his study is valid?  Or do you have other concerns about the conclusions he has drawn?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: Need to Catch Up on Trends Over the Last Six Weeks? Take a Time Capsule.

 

I try to set aside some time over the weekend to catch up on my reading and keep abreast of developments in the industry and although that’s sometimes that’s easier said than done, I stumbled across an interesting compilation of legal technology information from my friend Christy Burke and her team at Burke & Company.  On Friday, Burke & Company released The Legal Technology Observer (LTO) Time Capsule on Legal IT Professionals. LTO was a 6 week concentrated collection of essays, articles, surveys and blog posts providing expert practical knowledge about legal technology, eDiscovery, and social media for legal professionals.

The content has been formatted into a PDF version and is available for free download here.  As noted in their press release, Burke & Company's bloggers, including Christy, Melissa DiMercurio, Ada Spahija and Taylor Gould, as well as many distinguished guest contributors, set out to examine the trends, topics and perspectives that are driving today's legal technology world for 6 weeks from June 6 to July 12. They did so with help of many of the industry's most respected experts and LTO acquired more than 21,000 readers in just 6 weeks.  Nice job!

The LTO Time Capsule covers a wide range of topics related to legal technology.  There were several topics that have impact to eDiscovery, some of which included thought leaders previously interviewed on this blog (links to their our previous interviews with them below), including:

  • The EDRM Speaks My Language: Written by – Ada Spahija, Communications Specialist at Burke and Company LLC; Featuring – Experts George Socha and Tom Gelbmann.
  • Learning to Speak EDRM: Written by – Ada Spahija, Communications Specialist at Burke and Company LLC; Featuring – Experts George Socha and Tom Gelbmann.
  • Predictive Coding: Dozens of Names, No Definition, Lots of Controversy: Written by – Sharon D. Nelson, Esq. and John W. Simek.
  • Social Media 101 for Law Firms – Don’t Get Left Behind: Written by – Ada Spahija, Communications Specialist at Burke and Company LLC; Featuring – Kerry Scott Boll of JustEngage.
  • Results of Social Media 101 Snap-Poll: Written by – Ada Spahija, Communications Specialist at Burke and Company LLC.
  • Getting up to Speed with eDiscovery: Written by – Taylor Gould, Communications Intern at Burke and Company LLC; Featuring – Browning Marean, Senior Counsel at DLA Piper, San Diego.
  • LTO Interviews Craig Ball to Examine the Power of Computer Forensics: Written by – Melissa DiMercurio, Account Executive at Burke and Company LLC; Featuring – Expert Craig Ball, Trial Lawyer and Certified Computer Forensic Examiner.
  • LTO Asks Bob Ambrogi How a Lawyer Can Become a Legal Technology Expert: Written by – Melissa DiMercurio, Account Exectuive at Burke and Company LLC; Featuring – Bob Ambrogi, Practicing Lawyer, Writer and Media Consultant.
  • LTO Interviews Jeff Brandt about the Mysterious Cloud Computing Craze: Written by – Taylor Gould, Communications Intern at Burke and Company LLC; Featuring – Jeff Brandt, Editor of PinHawk Law Technology Daily Digest.
  • Legal Technology Observer eDiscovery in America – A Legend in the Making: Written by – Christy Burke, President of Burke and Company LLC; Featuring – Barry Murphy, Analyst with the eDJ Group and Contributor to eDiscoveryJournal.com.
  • IT-Lex and the Sedona Conference® Provide Real Help to Learn eDiscovery and Technology Law: Written by – Christy Burke, President of Burke and Company LLC.

These are just some of the topics, particularly those that have an impact on eDiscovery.  To check out the entire list of articles, click here to download the report.

So, what do you think?  Do you need a quick resource to catch up on your reading?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Case Law: Judge Scheindlin Says “No” to Self-Collection, “Yes” to Predictive Coding

 

When most people think of the horrors of Friday the 13th, they think of Jason Voorhees.  When US Immigration and Customs thinks of Friday the 13th horrors, do they think of Judge Shira Scheindlin?

As noted in Law Technology News (Judge Scheindlin Issues Strong Opinion on Custodian Self-Collection, written by Ralph Losey, a previous thought leader interviewee on this blog), New York District Judge Scheindlin issued a decision last Friday (July 13) addressing the adequacy of searching and self-collection by government entity custodians in response to Freedom of Information Act (FOIA) requests.  As Losey notes, this is her fifth decision in National Day Laborer Organizing Network et al. v. United States Immigration and Customs Enforcement Agency, et al., including one that was later withdrawn.

Regarding the defendant’s question as to “why custodians could not be trusted to run effective searches of their own files, a skill that most office workers employ on a daily basis” (i.e., self-collect), Judge Scheindlin responded as follows:

“There are two answers to defendants' question. First, custodians cannot 'be trusted to run effective searches,' without providing a detailed description of those searches, because FOIA places a burden on defendants to establish that they have conducted adequate searches; FOIA permits agencies to do so by submitting affidavits that 'contain reasonable specificity of detail rather than merely conclusory statements.' Defendants' counsel recognize that, for over twenty years, courts have required that these affidavits 'set [ ] forth the search terms and the type of search performed.' But, somehow, DHS, ICE, and the FBI have not gotten the message. So it bears repetition: the government will not be able to establish the adequacy of its FOIA searches if it does not record and report the search terms that it used, how it combined them, and whether it searched the full text of documents.”

“The second answer to defendants' question has emerged from scholarship and caselaw only in recent years: most custodians cannot be 'trusted' to run effective searches because designing legally sufficient electronic searches in the discovery or FOIA contexts is not part of their daily responsibilities. Searching for an answer on Google (or Westlaw or Lexis) is very different from searching for all responsive documents in the FOIA or e-discovery context.”

“Simple keyword searching is often not enough: 'Even in the simplest case requiring a search of on-line e-mail, there is no guarantee that using keywords will always prove sufficient.' There is increasingly strong evidence that '[k]eyword search[ing] is not nearly as effective at identifying relevant information as many lawyers would like to believe.' As Judge Andrew Peck — one of this Court's experts in e-discovery — recently put it: 'In too many cases, however, the way lawyers choose keywords is the equivalent of the child's game of 'Go Fish' … keyword searches usually are not very effective.'”

Regarding search best practices and predictive coding, Judge Scheindlin noted:

“There are emerging best practices for dealing with these shortcomings and they are explained in detail elsewhere. There is a 'need for careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or keywords to be used to produce emails or other electronically stored information.' And beyond the use of keyword search, parties can (and frequently should) rely on latent semantic indexing, statistical probability models, and machine learning tools to find responsive documents.”

“Through iterative learning, these methods (known as 'computer-assisted' or 'predictive' coding) allow humans to teach computers what documents are and are not responsive to a particular FOIA or discovery request and they can significantly increase the effectiveness and efficiency of searches. In short, a review of the literature makes it abundantly clear that a court cannot simply trust the defendant agencies' unsupported assertions that their lay custodians have designed and conducted a reasonable search.”

Losey notes that “A classic analogy is that self-collection is equivalent to the fox guarding the hen house. With her latest opinion, Schiendlin [sic] includes the FBI and other agencies as foxes not to be trusted when it comes to searching their own email.”

So, what do you think?  Will this become another landmark decision by Judge Scheindlin?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: TREC Study Finds that Technology Assisted Review is More Cost Effective

 

As reported in Law Technology News (Technology-Assisted Review Boosted in TREC 2011 Results by Evan Koblentz), the Text Retrieval Conference (TREC) Legal Track, a government sponsored project designed to assess the ability of information retrieval techniques to meet the needs of the legal profession, has released its 2011 study results (after several delays).  The overview of the 2011 TREC Legal Track can be found here.

The report concludes the following: “From 2008 through 2011, the results show that the technology-assisted review efforts of several participants achieve recall scores that are about as high as might reasonably be measured using current evaluation methodologies. These efforts require human review of only a fraction of the entire collection, with the consequence that they are far more cost-effective than manual review.” 

However, the report also notes that “There is still plenty of room for improvement in the efficiency and effectiveness of technology-assisted review efforts, and, in particular, the accuracy of intra-review recall estimation tools, so as to support a reasonable decision that 'enough is enough' and to declare the review complete. Commensurate with improvements in review efficiency and effectiveness is the need for improved external evaluation methodologies that address the limitations of those used in the TREC Legal Track and similar efforts.”

Other notable tidbits from the study and article:

  • Ten organizations participated in the 2011 study, including universities from diverse locations such as Beijing and Melbourne and vendors including OpenText and Recommind;
  • Participants were required to rank the entire corpus of 685,592 documents by their estimate of the probability of responsiveness to each of three topics, and also to provide a quantitative estimate of that probability;
  • The document collection used was derived from the EDRM Enron Data Set;
  • The learning task had three distinct topics, each representing a distinct request for production.  A total of 16,999 documents was selected – about 5,600 per topic – to form the “gold standard” for comparing the document collection;
  • OpenText had the top number of documents reviewed compared to recall percentage in the first topic, the University of Waterloo led the second, and Recommind placed best in the third;
  • One of the participants has been barred from future participation in TREC – “It is inappropriate –- and forbidden by the TREC participation agreement –- to claim that the results presented here show that one participant’s system or approach is generally better than another’s. It is also inappropriate to compare the results of TREC 2011 with the results of past TREC Legal Track exercises, as the test conditions as well as the particular techniques and tools employed by the participating teams are not directly comparable. One TREC 2011 Legal Track participant was barred from future participation in TREC for advertising such invalid comparisons”.  According to the LTN article, the barred participant was Recommind.

For more information, check out the links to the article and the study above.  TREC previously announced that there would be no 2012 study and is targeting obtaining a new data set for 2013.

So, what do you think?  Are you surprised by the results or are they expected?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: The Da Silva Moore Case Has Class (Certification, That Is)

 

As noted in an article written by Mark Hamblett in Law Technology News, Judge Andrew Carter of the U.S. District Court for the Southern District of New York has granted conditional class certification in the Da Silva Moore v. Publicis Groupe & MSL Group case.

In this case, women employees of the advertising conglomerate Publicis Groupe and its U.S. subsidiary, MSL, have accused their employer of company-wide discrimination, pregnancy discrimination, and a practice of keeping women at entry-level positions with few opportunities for promotion.

Judge Carter concluded that “Plaintiffs have met their burden by making a modest factual showing to demonstrate that they and potential plaintiffs together were victims of a common policy or plan that violated the law. They submit sufficient information that because of a common pay scale, they were paid wages lower than the wages paid to men for the performance of substantially equal work. The information also reveals that Plaintiffs had similar responsibilities as other professionals with the same title. Defendants may disagree with Plaintiffs' contentions, but the Court cannot hold Plaintiffs to a higher standard simply because it is an EPA action rather an action brought under the FLSA.”

“Courts have conditionally certified classes where the plaintiffs have different job functions,” Judge Carter noted, indicating that “[p]laintiffs have to make a mere showing that they are similarly situated to themselves and the potential opt-in members and Plaintiffs here have accomplished their goal.”

This is just the latest development in this test case for the use of computer-assisted coding to search electronic documents for responsive discovery. On February 24, Magistrate Judge Andrew J. Peck of the U.S. District Court for the Southern District of New York issued an opinion making it likely the first case to accept the use of computer-assisted review of electronically stored information (“ESI”) for this case.  However, on March 13, District Court Judge Andrew L. Carter, Jr. granted plaintiffs’ request to submit additional briefing on their February 22 objections to the ruling.  In that briefing (filed on March 26), the plaintiffs claimed that the protocol approved for predictive coding “risks failing to capture a staggering 65% of the relevant documents in this case” and questioned Judge Peck’s relationship with defense counsel and with the selected vendor for the case, Recommind.

Then, on April 5, Judge Peck issued an order in response to Plaintiffs’ letter requesting his recusal, directing plaintiffs to indicate whether they would file a formal motion for recusal or ask the Court to consider the letter as the motion.  On April 13, (Friday the 13th, that is), the plaintiffs did just that, by formally requesting the recusal of Judge Peck (the defendants issued a response in opposition on April 30).  But, on April 25, Judge Carter issued an opinion and order in the case, upholding Judge Peck’s opinion approving computer-assisted review.

Not done, the plaintiffs filed an objection on May 9 to Judge Peck's rejection of their request to stay discovery pending the resolution of outstanding motions and objections (including the recusal motion, which has yet to be ruled on.  Then, on May 14, Judge Peck issued a stay, stopping defendant MSLGroup's production of electronically stored information.  Finally, on June 15, Judge Peck, in a 56 page opinion and order, denied the plaintiffs’ motion for recusal

So, what do you think?  What will happen in this case next?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Best Practices: Judge Facciola Discusses Competency and Ethics

 

The American Bar Association (ABA) Model Rules of Professional Conduct (Model Rules) require that an attorney possess and demonstrate a certain requisite level of knowledge in order to be considered competent to handle a given matter.  Specifically, Model Rule 1.1 states that, "[a] lawyer shall provide competent representation to a client. Competent representation requires the legal knowledge, skill, thoroughness, and preparation reasonably necessary for the representation."

As noted in Law Technology News, eDiscovery vendor iConect hosted a free webinar last week entitled "Duty of Competency and E-Discovery", in which Joshua Gilliland, author of Bow Tie Law's Blog and founder of legal iPad app developer Majority Opinion, discussed ethics and eDiscovery with Magistrate Judge John M. Facciola of the United States District Court for the District of Columbia.  The “sheriff” speaks!

The LTN article notes that, according to Judge Facciola, the requirement for competency now requires "a fundamental understanding of the way information is produced." This entails: 1) some understanding of the information systems you and your client are relying upon; 2) knowing your own limitations; and 3) if you don't understand, have someone at your side, i.e. an expert, who does, he declared.

With regard to ethics and eDiscovery, Judge Facciola gave an example of what might occur in a 26(f) meet and confer, which he called "the linchpin" of the 2006 amendments to the Federal Rules of Civil Procedure. Two lawyers are meeting in discovery in a case involving pharmaceuticals. One lawyer knows that the drug Pharmadine is spelled with an "e" not an "a" but doesn't correct the opposing party even though he knows it will disrupt his opponent's search, prolonging the discovery period by six months. Labeling this the difference between a material fact and not speaking to correct a mistake, Facciola says there's "no ethical rule" for this, but, ultimately that lawyer is going to have to go before a judge and account for those extra six months. Model rules regulate a profession's ethics, they don't influence a judge's decision. "Ethics rules are not a safe harbor," Facciola warned.

For more observations from Judge Facciola topics such as cooperation, preservation and search methodology, click on this link to access the article from Law Technology News.  And, for more on the subject of competency and ethics as it relates to eDiscovery, check out this post and this post from our archives.

So, what do you think?  Are you addressing ethics and competency requirements in your firm as it relates to eDiscovery?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Case Law: Judge Peck Denies Recusal Motion in Da Silva Moore

 

It’s been a few weeks since we heard anything from the Da Silva Moore case.  If you’ve been living under a rock the past few months, Magistrate Judge Andrew J. Peck of the U.S. District Court for the Southern District of New York issued an opinion in this case in February making it one of the first cases to accept the use of computer-assisted review of electronically stored information (“ESI”).  However, the plaintiffs objected to the ruling and questioned Judge Peck’s relationship with defense counsel and with the selected vendor for the case, Recommind and ultimately formally requested the recusal of Judge Peck.  For links to all of the recent events in the case that we’ve covered, click here.

Last Friday, in a 56 page opinion and order, Judge Peck denied the plaintiffs’ motion for recusal.  The opinion and order reviewed the past several contentious months and rejected the plaintiffs’ arguments for recusal in the following areas:

Participation in conferences discussing the use of predictive coding:

“I only spoke generally about computer-assisted review in comparison to other search techniques…The fact that my interest in and knowledge about predictive coding in general overlaps with issues in this case is not a basis for recusal.”

“To the extent plaintiffs are complaining about my general discussion at these CLE presentations about the use of predictive coding in general, those comments would not cause a reasonable objective observer to believe I was biased in this case. I did not say anything about predictive coding at these LegalTech and other CLE panels that I had not already said in in my Search,Forward article, i.e., that lawyers should consider using predictive coding in appropriate cases. My position was the same as plaintiffs’ consultant . . . . Both plaintiffs and defendants were proposing using predictive coding in this case.  I did not determine which party’s predictive coding protocol was appropriate in this case until the February 8, 2012 conference, after the panels about which plaintiffs complain.”

“There are probably fewer than a dozen federal judges nationally who regularly speak at ediscovery conferences. Plaintiffs' argument that a judge's public support for computer-assisted review is a recusable offense would preclude judges who know the most about ediscovery in general (and computer-assisted review in particular) from presiding over any case where the use of predictive coding was an option, or would preclude those judges from speaking at CLE programs. Plaintiffs' position also would discourage lawyers from participating in CLE programs with judges about ediscovery issues, for fear of subsequent motions to recuse the judge (or disqualify counsel).”

Relationship with defense counsel Ralph Losey:

“While I participated on two panels with defense counsel Losey, we never had any ex parte communication regarding this lawsuit. My preparation for and participation in ediscovery panels involved only ediscovery generally and the general subject of computer-assisted review. Losey's affidavit makes clear that we have never spoken about this case, and I confirm that. During the panel discussions (and preparation sessions), there was absolutely no discussion of the details of the predictive coding protocol involved in this case or with regard to what a predicative coding protocol should look like in any case. Plaintiffs' assertion that speaking on an educational panel with counsel creates an appearance of impropriety is undermined by Canon 4 of the Judicial Code of Conduct, which encourages judges to participate in such activities.”

Relationship with Recommind, the selected vendor in the case:

“The panels in which I participated are distinguishable. First, I was a speaker at educational conferences, not an audience member. Second, the conferences were not one-sided, but concerned ediscovery issues including search methods in general. Third, while Recommind was one of thirty-nine sponsors and one of 186 exhibitors contributing to LegalTech's revenue, I had no part in approving the sponsors or exhibitors (i.e., funding for LegalTech) and received no expense reimbursement or teaching fees from Recommind or LegalTech, as opposed to those companies that sponsored the panels on which I spoke. Fourth, there was no "pre-screening" of MSL's case or ediscovery protocol; the panel discussions only covered the subject of computer-assisted review in general.”

Perhaps it is no surprise that Judge Peck denied the recusal motion.  Now, the question is: will District Court Judge Andrew L. Carter, Jr. weigh in?

So, what do you think?  Should Judge Peck recuse himself in this case or does he provide an effective argument that recusal is unwarranted?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: Where Does the Money Go? RAND Provides Some Answers

 

The RAND Corporation, a nonprofit research and analysis institution recently published a new 159 page report related to understanding eDiscovery costs entitled Where the Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery by Nicholas M. Pace and Laura Zakaras that has some interesting findings and recommendations.  To obtain either a paperback copy or download a free eBook of the report, click here.

For the study, the authors requested case-study data from eight Fortune 200 companies and obtained data for 57 large-volume eDiscovery productions (from both traditional lawsuits and regulatory investigations) as well as information from extensive interviews with key legal personnel from the participating companies.  Here are some of the key findings from the research:

  • Review Makes Up the Largest Percentage of eDiscovery Production Costs: By a whopping amount, the major cost component in their cases was the review of documents for relevance, responsiveness, and privilege (typically about 73 percent). Collection, on the other hand, only constituted about 8 percent of expenditures for the cases in the study, while processing costs constituted about 19 percent in the cases.  It costs about $14,000 to review each gigabyte and $20,000 in total production costs for each gigabyte (click here for a previous study on per gigabyte costs).  Review costs would have to be reduced by about 75% in order to make those costs comparable to processing, the next highest component.
  • Outside Counsel Makes Up the Largest Percentage of eDiscovery Expenditures: Again, by a whopping amount, the major cost component was expenditures for outside counsel services, which constituted about 70 percent of total eDiscovery production costs.  Vendor expenditures were around 26 percent.  Internal expenditures, even with adjustments made for underreporting, were generally around 4 percent of the total.  So, almost all eDiscovery expenditures are outsourced in one way or another.
  • If Conducted in the Traditional Manner, Review Costs Are Difficult to Reduce Significantly: Rates currently paid to “project attorneys during large-scale reviews in the US may well have bottomed out” and foreign review teams are often not a viable option due to “issues related to information security, oversight, maintaining attorney-client privilege, and logistics”.  Increasing the rate of review is also limited as, “[g]iven the trade-off between reading speed and comprehension…it is unrealistic to expect much room for improvement in the rates of unassisted human review”.  The study also notes that techniques for grouping documents, such as near-duplicate detection and clustering, while helpful, are “not the answer”.
  • Computer-Categorized Document Review Techniques May Be a Solution: Techniques such as predictive coding have the potential of reducing the review hours by about 75% with about the same level of consistency, resulting in review costs of less than $2,000 and total production costs of less than $7,000.  However, “lack of clear signals from the bench” that the techniques are defensible and lack of confidence by litigants that the techniques are reliable enough to reliably identify the majority of responsive documents and privileged documents are barriers to wide-scale adoption.

Not surprisingly, the recommendations included taking “the bold step of using, publicly and transparently, computer-categorized document review techniques” for large-scale eDiscovery efforts.

So, what do you think?  Are you surprised by the cost numbers?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: Wednesday LTWC 2012 Sessions

 

As noted yesterday, LegalTech West Coast 2012 (LTWC) is happening this week and eDiscoveryDaily is here to report about the latest eDiscovery trends being discussed at the show.  There’s still time to check out the show if you’re in the Los Angeles area with a number of sessions (both paid and free) available and 69 exhibitors providing information on their products and services, including (shameless plug warning!) my company, CloudNine Discovery, which just announced yesterday release of Version 11 of our linear review application, OnDemand®, and will be exhibiting at booth #216 along with our partners, First Digital Solutions.  Come by and say hi!

Perform a “find” on today’s LTNY conference schedule for “discovery” and you’ll get 21 hits.  More eDiscovery sessions happening!  Here are some of the sessions in the main conference tracks:

10:30 – 12:00 AM:

Information Governance and Information Management

With the volume of electronically stored information (ESI) growing exponentially and the challenges surrounding managing it, protecting it, and developing effective policies are essential. Social media, email, IMs, web pages, mobile devices and the cloud have made a big job even bigger. How much or how little should you collect? How aggressive should you be? How can you be certain your approach and results are defensible?

Speakers are: Richard E. Davis, JD, e-Discovery Solutions Architect & Founder, Litigation Logistics, LLC; Jack Halprin, Head of eDiscovery, Google; Dawson Horn, III, Senior Litigation Counsel, Tyco International and David Yerich, Director, eDiscovery, UHG Legal Department, United Health Group.

The GARP® Principles and eDiscovery

Attendees will hear from experts on the GARP Principles and eDiscovery as well as:

  • Understand the importance of proactive records management through the eight GARP® Principles
  • Revisit the GARP® Principles and learn how their role is magnified by recent case law
  • Learn what to do before eDiscovery: how GARP® precedes and complements the EDRM

Speakers are: Gordon J. Calhoun, Esq., Lewis Brisbois Bisgaard &, Smith LLP; Lorrie DeCoursey, Former Law Firm Administrator and John J. Isaza, Esq., Partner, Rimon P.C.  Moderator: David Baskin, Vice President of Product Management, Recommind.

1:30 – 3:00 PM:

Practical Handbook for Conducting International eDiscovery – Tips and Tricks

This session will present a truly international view on how to conduct global eDiscovery from a practical perspective, including developing proactive global document retention policies and assuring multi-jurisdictional compliance, best practices of global data preservation and collection, successful data migration across jurisdictions, navigating unique cultural and procedural challenges in various global regions, handling multi-lingual data sets as well as strategic positioning of hosting data centers.

Speakers are: Monique Altheim, Esq., CIPP, The Law Office of Monique Altheim; George I. Rudoy, Founder & CEO, Integrated Legal Technology, LLC and David Yerich, Director, eDiscovery, UHG Legal Department, United Health Group.

Litigation Preparedness Through Effective Data Governance

Be prepared. This panel will go through the benefits of data governance in your litigation preparedness and discuss benefits such as:

  • Auto-classification of legacy and newly created content
  • What is email management and is it ready for prime-time?
  • Review the court's findings on the complexities of ESI, including metadata, native formats, back-up tapes, mobile devices, and legacy technology
  • Key questions to ask before outsourcing ESI to the cloud

Speakers are: Lorrie DeCoursey, Former Law Firm Administrator; John J. Isaza, Esq., Partner, Rimon P.C. and Ayelette Robinson, Director – Knowledge Technology, Littler Mendelson.  Moderator: Derek Schueren, GM, Information Access and Governance, Recommind.

3:30 – 5:00 PM:

Managed and Accelerated Review

As costs for review soar and volumes of data multiply at an almost exponential rate, traditional linear review seems to be giving way to new technologies that will enable faster, better, more defensible eDiscovery results. How can you be assured that this new approach will catch everything that needs to be captured? Will human review become obsolete? What do you need to ask when considering this new technology? How should it be incorporated into your overall litigation strategy?

Speakers are: Matthew Miller, Manager, Fraud Investigation & Dispute Services, Ernst & Young; Robert Miller, Founder, Rise Advisory Group, LLC; Former Discovery Counsel, BP; David Sun, Discovery Project Manager, Google.

eDiscovery Circa 2015: Will Aggressive Preservation/Collection and Predictive Coding be Commonplace?

Who's holding back on Predictive Coding, clients or outside counsel? This session will discuss if aggressive preservation/collection of predictive coding will become commonplace as well as:

  • How aggressive should clients be with preservation/collection?
  • How to use effective searching, sampling, and targeting tools and techniques to not over-collect

Speakers are: Gordon J. Calhoun, Esq., Lewis Brisbois Bisgaard &, Smith LLP; Lorrie DeCoursey, Former Law Firm Administrator and Greg Chan, Senior Regional Litigation Technology Manager, Bingham McCutchen LLP.  Moderator: David Baskin, Vice President of Product Management, Recommind.

In addition to these, there are other eDiscovery-related sessions today.  For a complete description for all sessions today, click here.

So, what do you think?  Are you planning to attend LTWC this year?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

eDiscovery Trends: Welcome to LegalTech West Coast 2012!

 

Today is the start of LegalTech® West Coast 2012 (LTWC) and eDiscoveryDaily is here to report about the latest eDiscovery trends being discussed at the show.  Over the next two days, we will provide a description each day of some of the sessions related to eDiscovery to give you a sense of the topics being covered.  If you’re in the Los Angeles area, come check out the show – there are a number of sessions (both paid and free) available and 69 exhibitors providing information on their products and services, including (shameless plug warning!) my company, CloudNine Discovery, which just announced today release of Version 11 of our linear review application, OnDemand®, and will be exhibiting at booth #216 along with our partners, First Digital Solutions.  Come by and say hi!

Perform a “find” on today’s LTNY conference schedule for “discovery” and you’ll get 19 hits.  So, there is plenty to talk about!  Sessions in the main conference tracks include:

10:30 AM – 12:00 PM:

A "Stormy" Subject…Exploring Cloud-Based eDiscovery

Can your organization better manage costs and increase control over discovery by bringing eDiscovery tools in-house or in-firm? What are the advantages and drawbacks of eDiscovery in the cloud? In this session, the panel will:

  • Explore insourcing v. outsourcing market trends
  • Discuss the pros and cons inherent in cloud/SaaS v. on premises e-discovery software solutions
  • Examine challenges when collecting and preserving discoverable data stored in the cloud

Speakers are: Scott Sachs, eDiscovery Attorney, Atkinson Andelson and Adam Sand, Associate General Counsel, Ancestry.com.  Moderator: Wayne Wong, Managing Consultant, Kroll Ontrack.

1:30 – 3:00 PM:

Under Fire: Defending and Challenging Technology-Assisted Review

Intelligent Review? Predictive Coding? Smart review? Whatever you call it, amidst growing data volumes and dwindling resources, traditional linear document review is quickly going the way of the dinosaur. In this session, the panel will:

  • Explore the "what", "why", and "how" behind  technology-assisted review
  • Discuss cutting-edge opinions from the bench
  • Provide you with tips to help overcome your organization's objections to using intelligent review technology

Speakers are: Tom Werner, Associate, Irell & Manella, LLP; Jeffrey Fowler, Partner, O'Melveny & Myers, LLP and Pallab Chakraborty, Director of eDiscovery, Oracle.  Moderator: Andrea Gibson, Product Director, Kroll Ontrack.

3:30 – 5:00 PM:

Exploring Hot eDiscovery Trends: FRCP Amendments, Social Media, and Emerging Case Law

eDiscovery evolves at the speed of light. If your organization is standing still, you are losing ground. In this session, the panel will:

  • Explore how eDiscovery evolved in 2011, with a look into how it will continue to change in the remainder of 2012
  • Analyze whether potential amendments to the Federal Rules of Civil Procedure are even possible, and what the amendments might entail discuss "hot" trends impacting eDiscovery such as social media.

Panelists are: Ron S. Best, EDD Staff Attorney & Director, Litigation Systems, Munger, Tolles & Olson, LLP and Eric Chan, Associate, O'Melveny & Myers, LLP.  Moderator: Joel Vogel, Vice President, Discovery Products and Services, Kroll Ontrack.

In addition to these, there are other eDiscovery-related sessions today.  For a complete description for all sessions today, click here.

So, what do you think?  Are you planning to attend LTWC this year?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.