Close
Enter your
text here

eDiscoveryDaily

Managing an eDiscovery Contract Review Team: Starting the Project
 

Throughout the life of the project, you will implement some standard project management techniques for monitoring quality and ensuring that questions are resolved efficiently.  We’re going to cover those techniques in the next blog posts in this series.  There are, however, a couple of special steps you should take at the beginning of a project to ensure that it gets off to a smooth start.

First, you want to make sure very quickly that everyone on the team understands the criteria and is applying it correctly to the documents.  The best way to do this is to check everybody’s initial work right away and to provide feedback as quickly as you can.   Make arrangements with the supervisory staff to work extra hours the first few days of the project.  During the first couple of days of the project, have supervisors check the work after the review staff leaves.   Have the supervisors give feedback – one-on-one – to each team member within the first couple of days of the project.  In addition, make sure that the supervisors are communicating with one another and with the project manager about what they are finding.  Wide-spread misunderstandings will uncover holes in the training and can easily be cleared up with the group in short re-training sessions. 

When we talked about who should be on the review team, we talked about “decision makers” and subject matter experts.  Make sure these team members are onsite full-time the first few days of the project.  There will be a lot of questions the first few days, and you’ll want to resolve those questions quickly.  Once the project gets underway, the level of questions will subside, and the supervisors and project manager will be better equipped to answer the questions that do arise.  At that point, you probably don’t need the decision makers and subject matter experts on hand full time.  But make sure they are present at the start of the project. 

How do you approach starting a document review project?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Trends: Tom Gelbmann of Gelbmann & Associates, LLC
 

This is the first of the LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and asked each of them the same three questions:

  1. What do you consider to be the current significant trends in eDiscovery that people in the industry are, or should be, focused on?
  2. Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?
  3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Tom Gelbmann. Tom is Principal of Gelbmann & Associates, LLC, co-author of the Socha-Gelbmann Electronic Discovery Survey and co-founder of the Electronic Discovery Reference Model (EDRM).  Since 1993, Gelbmann & Associates, LLC has helped law firms and Corporate Law Departments realize the full benefit of their investments in Information Technology.  As today is Valentine’s Day, consider this interview with Tom as eDiscoveryDaily’s Valentine’s Day present to you!

What do you consider to be the current significant trends in eDiscovery that people in the industry are, or should be, focused on?

The first thing that comes to mind is the whole social media thing, which is something you’re probably getting quite a bit of (in your interviews), but with the explosion of the use of social media, personally and within organizations, we’re seeing a huge explosion (in eDiscovery).  One of the issues is that there is very little in terms of policy and management around that, and I look at it in a very similar vein to the late ’80s and early ‘90s when electronic mail came about and there were no real defining guidelines.  It wasn’t until we got to a precipitating event where “all of a sudden, organizations get religion” and say “oh my god, we better have a policy for this”.  So, I think the whole social media thing is one issue.

On top of that, another area that is somewhat of an umbrella to all this is information management and EDRM with the Information Management Reference Model (IMRM) is certainly part of that. What is important in this context is that corporations are beginning to realize the more they get their “electronic house in order”, the better off they’re going to be in many ways.  Less cost, less embarrassment and so forth.

The third thing is that, and this is something that I’ve been tracking for awhile, the growth in tools and solutions available for small organizations and small cases.  For a long time, everything was about millions of documents and gigabytes of data – that’s what got the headlines and that what the service bureaus and providers were focusing on.  The real “gold” in my mind is the small cases, the hundreds of thousands of small cases that are out there.  The providers that can effectively reach that market in a cost-effective way will be positioned very well and I think we’re starting to see that happen.  And, I think the whole “cloud” concept of technology is helping that.

Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?

{Interviewed on the first afternoon of the show} Well, so far it’s been a blur [laughs].  But, I think we’re definitely seeing social media as a big issue at this LegalTech and I also think we’re seeing more solutions toward the smaller cases and smaller organizations here at this year’s show.

What are you working on that you’d like our readers to know about?

From an EDRM standpoint, I just came from a meeting for the EDRM Testing pilot project.  Last fall, at the mid-year meeting, there was a groundswell to address testing, and the basic issue is applying some principles of testing to software products associated with electronic discovery to answer the question of “how do you know?” when the court asks if the results are true and what sort of testing process did you go through.  There is very little as far as a testing regimen or even guidelines on a testing regimen for electronic discovery software and so the EDRM testing group is looking to establish some guidelines, starting very basically looking at bands of rigor associated with bands of risk.  So, you will see that at this year’s EDRM annual meeting in May that EDRM Testing will become a full-fledged project.

And the other thing that I’m happy to announce is that George Socha and I have launched a web site called Apersee, which is the next step in the evolution of the (Socha-Gelbmann) rankings.  We killed the rankings two years ago because they were being misused.  Consumers wanted to know who do I send the RFP to, who do I engage and they would almost mindlessly send to the Socha-Gelbmann Top Ten.  But, now the consumers can specify what they’re looking for, starting with areas of the model, whether it’s Collection, Preservation, Review, etc., and provide other information such as geography and types of ESI and what will be returned on those searches is a list of providers with those services or products.  We have right now about 800 providers in the database and many of those have very basic listings at this point.  As this is currently in beta, we have detailed information that we pre-populated for about 200 providers and are expanding rapidly.  Over the next couple of months, we’re working hard with providers to populate their sites with whatever content is appropriate to describe their products and services in terms of what they do, where they do it, etc., that can feed the search engine.  And, we have been getting very good feedback from both the consumer side and the provider side as being a very valuable service.

Thanks, Tom, for participating in the interview!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

eDiscovery Case Law: Responses to FOIA Requests Must Be Searchable

Southern District of New York Judge Shira A. Scheindlin is at it again!  Her latest ruling is that the federal government must provide documents “in a usable format” when it responds to Freedom of Information Act (FOIA) requests.

Noting that “Once again, this Court is required to rule on an eDiscovery issue that could have been avoided had the parties had the good sense to ‘meet and confer,’ ‘cooperate’ and generally make every effort to ‘communicate’ as to the form in which ESI would be produced.”, Judge Scheindlin ruled that federal agencies must turn over documents that include “metadata,” which allows them to be searched and indexed.  Indicating that “common sense dictates” that the handling of FOIA requests should be informed by “the spirit if not the letter” of the Federal Rules of Civil Procedure, Judge Scheindlin indicated the government offered “a lame excuse” for delivering non-searchable documents.

In National Day Laborer Organizing Network v. U.S. Immigration and Customs Enforcement Agency, 10 Civ. 3488, the National Day Laborer Organizing Network, Center for Constitutional Rights and the Immigration Justice Clinic at the Benjamin N. Cardozo School of Law sued to require production of a wide range of documents under the Freedom of Information Act in August 2010.  In response, the government agency defendants produced documents grouped together in large files that were not searchable, for which individual documents could not be easily identified, with emails separated from their attachments.

Consistent with the decisions of several state courts regarding their own FOIA statutes, Judge Scheindlin ruled that the federal law requires that metadata, which allows for electronic files to be organized and searched, must be retained in the records agencies produce.  While the federal act doesn’t specifically specify the form in which documents must be delivered, it does require that documents be provided in any “format” that is “readily reproducible” by the agency in that format.  Metadata, in the FOIA context, is “readily reproducible,” Judge Scheindlin noted.

Judge Scheindlin also observed that “whether or not metadata has been specifically requested,” the production of non-searchable documents is “an inappropriate downgrading” of electronically stored information and provision of files “stripped of all metadata and lumped together without any indication of where a record begins and ends” is not an “acceptable form of production,” she said.

A copy of the opinion and order can be found here.

So, what do you think?  Have you been the recipient of a “lumped together” non-searchable production recently?  Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Trends: EDRM Metrics Privilege Survey
 

As a member of the EDRM Metrics Project for the past four years, I have seen several accomplishments by the group to provide an effective means of measuring the time, money and volumes associated with eDiscovery activities, including:

  • Code Set: An extensive code set of activities to be tracked from Identification through Presentation, as well as Project Management.
  • Case Study: A hypothetical case study that illustrates at each phase why metrics should be collected, what needs to be measured, how metrics are acquired and where they’re recorded, and how the metrics can be used.
  • Cube: A simple graphical model which illustrates the EDRM phases, aspects to be tracked (e.g., custodians, systems, media, QA, activities, etc.) and the metrics to be applied (i.e., items, cost, volume, time).

The EDRM Metrics project has also been heavily involved in proposing a standard set of eDiscovery activity codes for the ABA’s Uniform Task Based Management System (UTBMS) series of codes used to classify the legal services performed by a law firm in an electronic invoice submission.

Now, we need your help for an information gathering exercise.

We are currently conducting a Metrics Privilege survey to get a sense throughout the industry as to typical volumes and percentages of privileged documents within a collection.  It’s a simple, 7 question survey that strives to gather information regarding your experiences with privileged documents (whether you work for a law firm, corporation, provider or some other organization).

If you have a minute (which is literally all the time it will take), please take the survey and pass along to your colleagues to do so as well.  The more respondents who participate, the more representative the survey will be as to the current eDiscovery community.  To take the survey, go to edrm.net or click here.  EDRM will publish the results in the near future.

So, what do you think?  What are your typical metrics with regard to privileged documents?  Please share any comments you might have or if you’d like to know more about a particular topic.

Managing an eDiscovery Contract Review Team: Training a Review Team
 

Yesterday, we discussed the assembling the project team for document review.  It’s also important that the review team gets good training.  As a starting point, prepare a training manual for each team member that includes this information:

  • The document review criteria
  • A list of the custodians.  For each, provide the custodian’s title, a job description, a description of his/her role in the events that are at issue in the case, and a description of the types of documents you expect will be found in his/her files
  • Lists of keywords, key characters, key events, and key dates
  • Samples of responsive documents that you collected when you reviewed the collection
  • The review procedures
  • The review schedule
  • Instructions for use of the review tool

Cover these topics in training:

  • Case background information
    • A description of the parties
    • A description of the events that led to the case
    • A description of the allegations and defenses
    • An overview of the expected case schedule
  • Project overview information
    • A description of the goals of the review project
    • A description of the process
    • An overview of the expected project schedule
  • Responsive criteria
    • Go through the criteria – point-by-point – to ensure that the group understands what is responsive
    • Provide samples of responsive documents
  • Mechanics
    • Describe the roles of individuals on the team
    • Review the procedures for reviewing documents
    • Train the reviewers in use of the online review tool

Give the team training exercises – that is, give them sample documents to review.  Collect the work, review it, and give feedback to the group.

And let me give you two more suggestions that will help make your training effective:

  1. Train the team together, rather than one-on-one or in sub-groups.  Under this team-training approach, you ensure that everyone hears the same thing, and that responses to questions asked by individuals will benefit the entire group.
  2. Involve a senior attorney in the training.  You might, for example, ask a senior attorney to give the case background information.  Attention from a senior litigation team member is good for morale.  It tells the team that the work they are doing is important to the case.

How do you approach training a document review team?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

Managing an eDiscovery Contract Review Team: Assembling the Project Team
 

Before assembling the review team, think through how the project will be structured.  This will drive decisions on the type of people that you’ll need.  Your goal is to get the work done as cost effectively as possible – using less expensive personnel where possible — without sacrificing work quality or the utility of the work product.

The “base” of the project will be comprised of contract reviewers and qc staff.  In the project plan, you determined the number of people that you need.  At this point, don’t worry about who will be a reviewer and who will do qc work.  Everybody can start as a reviewer.  After a few days, you can identify who will do qc work.  You’ve got options for assembling this staff, but you should consider working with a litigation support vendor who offers staffing services.  A good vendor already has access to a pool of people with document review experience.  This can save you lots of time and work.

In addition to the contract review staff, you’ll need project management staff.  We’ve already talked about a project manager.  For a large project, you’ll want project supervisors — each responsible for a team of reviewers/qc personnel.  Each supervisor is responsible for overseeing the flow of work to the team, the quality of the work done by the team, the productivity of team members, and answering questions raised by reviewers (or ensuring that questions are resolved).  I usually create teams of 10 to 12 and assign one supervisor to a team.   The supervisors might be law firm litigation support professionals, or supervisory staff provided by the vendor with whom you are working.

You’ll also need “decision makers” and experts in the subject matter to round out the team.  At a minimum, you’ll want an attorney from the litigation team.  Depending on the complexity of the documents, you might need a client employee who is familiar with the company’s operations and documents.  These people should be on-site, full-time for the first few days of a project.  Eventually there will be fewer questions and it’s probably sufficient to have phone access to these team members.

Later in this blog series we’ll talk about how these staff levels interact so that decisions are made by attorneys but effectively implemented by review staff.

How do you structure a document review team?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Best Practices: Judges’ Guide to Cost-Effective eDiscovery
 

Last week at LegalTech, I met Joe Howie at the blogger’s breakfast on Tuesday morning.  Joe is the founder of Howie Consulting and is the Director of Metrics Development and Communications for the eDiscovery Institute, which is a 501(c)(3) nonprofit research organization for eDiscovery.

eDiscovery Institute has just released a new publication that is a vendor-neutral guide for approaches to considerably reduce discovery costs for ESI.  The Judges’ Guide to Cost-Effective E-Discovery, co-written by Anne Kershaw (co-Founder and President of the eDiscovery Institute) and Joe Howie, also contains a foreword by the Hon. James C. Francis IV, Magistrate Judge for the Southern District of New York.  Joe gave me a copy of the guide, which I read during my flight back to Houston and found to be a terrific publication that details various mechanisms that can reduce the volume of ESI to review by up to 90 percent or more.  You can download the publication here (for personal review, not re-publication), and also read a summary article about it from Joe in InsideCounsel here.

Mechanisms for reducing costs covered in the Guide include:

  • DeNISTing: Excluding files known to be associated with commercial software, such as help files, templates, etc., as compiled by the National Institute of Standards and Technology, can eliminate a high number of files that will clearly not be responsive;
  • Duplicate Consolidation (aka “deduping”): Deduping across custodians as opposed to just within custodians reduces costs 38% for across-custodian as opposed to 21% for within custodian;
  • Email Threading: The ability to review the entire email thread at once reduces costs 36% over having to review each email in the thread;
  • Domain Name Analysis (aka Domain Categorization): As noted previously in eDiscoveryDaily, the ability to classify items based on the domain of the sender of the email can significantly reduce the collection to be reviewed by identifying emails from parties that are clearly not responsive to the case.  It can also be a great way to quickly identify some of the privileged emails;
  • Predictive Coding: As noted previously in eDiscoveryDaily, predictive coding is the use of machine learning technologies to categorize an entire collection of documents as responsive or non-responsive, based on human review of only a subset of the document collection. According to this report, “A recent survey showed that, on average, predictive coding reduced review costs by 45 percent, with several respondents reporting much higher savings in individual cases”.

The publication also addresses concepts such as focused sampling, foreign language translation costs and searching audio records and tape backups.  It even addresses some of the most inefficient (and therefore, costly) practices of ESI processing and review, such as wholesale printing of ESI to paper for review (either in paper form or ultimately converted to TIFF or PDF), which is still more common than you might think.  Finally, it references some key rules of the ABA Model Rules of Professional Conduct to address the ethical duty of attorneys in effective management of ESI.  It’s a comprehensive publication that does a terrific job of explaining best practices for efficient discovery of ESI.

So, what do you think?  How many of these practices have been implemented by your organization?  Please share any comments you might have or if you’d like to know more about a particular topic.

Deadline Extended to Vote for the Most Significant eDiscovery Case of 2010
 

Our ‘little experiment’ to see what the readers of eDiscoveryDaily think about case law developments in 2010 needs more time as we have not yet received enough votes yet to have a statistically significant result.  So, we’ve extended the deadline to select the case with the most significant impact on eDiscovery practices in 2010 to February 28.  Evidently, calling out the vote on the last business day before LegalTech is not the best timing.  Live and learn!

As noted previously, we have “nominated” five cases, which we feel were the most significant in different issues of case law, including duty to preserve and sanctions, clawback agreements under Federal Rule of Evidence 502, not reasonably accessible arguments and discoverability of social media content.  If you feel that some other case was the most significant case of 2010, you can select that case instead.  Again, it’s very important to note that you can vote anonymously, so we’re not using this as a “hook” to get your information.  You can select your case without providing any personal information.  However, we would welcome your comments as to why you selected the case you did and you can – optionally – identify yourself as well.

To get more information about the nominated cases (as well as other significant cases), click here.  To cast your vote, click here.

And, as always, please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Trends: Announcing LTNY Thought Leader Series!

I lied.

OK, I didn’t really lie.  I told you on Monday that eDiscoveryDaily would be tweeting throughout LegalTech New York (LTNY) this week.  Which is what I intended to do; but, I forgot how crazy it can be at the show with so many meetings, sessions and exhibit hall time.  So, I only managed one tweet per day.  I’ll have a better plan for next year, I promise.  😉

One of the reasons that I only managed the one tweet per day is that, during that time, eDiscoveryDaily was conducting interviews with several eDiscovery industry thought leaders and we’re pleased to introduce the schedule for the series, which will begin on Monday, February 14.  Happy Valentine’s Day!

Here are the interviews that we will be publishing over the next few weeks:

Monday, February 14: Tom Gelbmann, Principal Analyst of Gelbmann & Associates and co-founder of the Electronic Discovery Reference Model (EDRM).  Since 1993, Tom has helped law firms and Corporate Law Departments realize the full benefit of their investments in Information Technology.

Wednesday, February 16: Alon Israely, Senior Advisor, BIA.  Alon has over fifteen years of experience in a variety of advanced computing-related technologies and currently oversees BIA’s product development for its core technology products.

Friday, February 18: Jim McGann, Vice President of Index Engines.  Jim has extensive experience with eDiscovery and Information Management in the Fortune 2000 sector and has worked for leading software firms that provided financial services and information management solutions.

Monday, February 21: Christine Musil, Director of Marketing of Informative Graphics Corporation (IGC).  Christine has applied her in-depth knowledge of IGC’s products and benefits to marketing initiatives, including branding, overall messaging, and public relations. She has also been a contributing author to a number of publications on archiving formats, redaction, and viewing technology in the enterprise.

Wednesday, February 23: Jack Halprin, Vice President eDiscovery and Compliance, Autonomy.  Jack serves as a subject matter expert and assists clients with building best practices and defensible processes around eDiscovery and compliance related issues and manages the product line strategy for Autonomy’s Legal Hold and Early Case Assessment solutions.

Friday, February 25: Deidre Paknad, President & CEO of PSS Systems.  A well-known thought leader in the legal and information governance domain, Deidre founded the Compliance, Governance and Oversight Council (CGOC), a professional community on retention and preservation that analyst firm IDC labeled a “think tank.” She has been a member of several Sedona working groups since 2005 and leads the EDRM IMRM working group.

Monday, February 28: George Socha, President of Socha Consulting LLC and co-founder of the Electronic Discovery Reference Model (EDRM).  As President of Socha Consulting LLC, George offers services as an eDiscovery expert witness, special master and advisor to corporations, law firms and their clients, and legal vertical market software and service providers in the areas of electronic discovery and automated litigation support.

Wednesday, March 2: Tom O’Connor, Director, Gulf Coast Legal Tech Center. Tom is a nationally known consultant, speaker and writer in the area of computerized litigation support systems.  A frequent lecturer on the subject of legal technology, Tom has been on the faculty of numerous national CLE providers and has taught college level courses on legal technology.

Friday, March 4: Craig Ball, Law Offices of Craig D. Ball, P.C.  Craig has delivered over 600 presentations and papers to continuing legal and professional education programs throughout the United States.  Craig’s articles on forensic technology and electronic discovery frequently appear in the national media and he also writes a monthly column on computer forensics and e-discovery for Law Technology News called “Ball in your Court”.

Thanks to everyone for their time in participating in these interviews, especially during a busy LegalTech week!

So, what do you think?  Please share any comments you might have or if you’d like to know more about a particular topic.

Managing an eDiscovery Contract Review Team: Prepare a Review Plan
 

Before starting a review project, prepare a plan that will be your map for moving forward and that you’ll use to monitor progress throughout the project.  The components of the plan should include:

  • A schedule:  The schedule will be dictated by the date on which you’re required to produce documents.  Determine when you can start  (you’ll need time in advance to assemble the team, draft criteria, train the team, and get documents loaded into the review tool).  Plan to complete the project well in advance of the production deadline, so you’ve got some pad for schedule slippage.  Once you have a start date and an end date, look at the size of the collection and calculate how many documents you need to process in a day. 
  • Identify required staff resources:  Determine the number of units a reviewer can do in a day (use statistics from prior projects or do test runs on a sampling of the collection).  From there, do the math to determine how many reviewers you’ll need to complete the project within your schedule.  Add in people to do quality control (a good ratio is 1 qc reviewer for every 4 or 5 reviewers).
  • A budget:  Include costs for review/qc staff, project management, equipment, and any other incidentals.
  • Prepare a monitoring plan:  Every day you should look at the progress that the review team is making so you’ll know where you stand with schedule and budget.  You’ll need reports that provide statistics on what has been completed – reports that can probably be generated by the review tool.  Look at those reports to determine if they will meet your needs or if you’ll need to establish additional tracking mechanisms.
  • Prepare a quality control plan:  Determine how reviewed documents will be funneled to the quality control staff, the mechanism by which quality control reviewers will report findings to project management, and the mechanism by which feedback will be provided to reviewers.

A good plan isn’t hard to prepare and it can make a world of difference in how smoothly the project will run.

How do you plan a document review?  Have you run into glitches?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.