Close
Enter your
text here

Review

Managing an eDiscovery Contract Review Team: Ensuring High-Quality, Consistent Work

The most common problem with work product done by a group is inconsistent work – which means that some of the work has been done wrong.  This is a problem, however, that can be avoided.  Of course there will always be some inconsistencies, but there are steps that you can take and procedures that you can follow that will minimize inconsistencies and result in high-quality work.  Let’s go over those.

  • Continuously sample the documents that are coming through the pipeline:  Regardless of how much time you spent looking at the documents before the project started, there will be surprises in the collection.  And often, similar documents appear together in a collection.  Your project will run much smoother if you know what’s coming before the documents are in the hands of the reviewer.  Have supervisors spend time each day looking at what’s coming next.  When you see documents that will be new to the group, alert the staff.  Have a short meeting with the group.  Show them what they’ll be seeing and tell them how to categorize the documents.
  • Disseminate updated rules and new examples:  As you move through the review, you’ll be continuously seeing new types of documents and refining the criteria to cover what you are seeing.  Make sure this information gets disseminated to the staff.  Update the criteria frequently (in the beginning, this may be daily) and distribute the revised rules and examples to the staff.  Have meetings with the staff to go over new rules.  While meetings will disrupt the work, it will save you time in the long run – time in having to do re-work and fix errors.
  • Quality Control:  As soon as you can, identify members of the review team to do quality control work and make it their job to review the work of the review staff.  At first, they’ll probably need to look at everything.  As the project progresses, they should be able to sample the work.
  • Attorney sampling:  Attorneys should sample work product throughout the life of the project.  They should spot-check the documents in each category (responsive, not responsive, and privileged).

What steps do you take to ensure that a review team is doing high-quality work?  Have you run into glitches?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

Managing an eDiscovery Contract Review Team: Keeping Decisions in the Hands of the Attorneys
 

The main objective in most document reviews is to categorize each document into one of three categories:

  • Responsive, to be produced
  • Not Responsive
  • Privileged

Attorneys on the litigation team will make the decisions regarding how to categorize the documents, and the contract review staff will implement those decisions.  Before the review even starts, you’ve done a lot of work to ensure that this process will work well:

  • You sampled the collection, looked at a lot of documents, and made decisions regarding how to categorize them
  • You drafted detailed, objective criteria that encapsulates that work product
  • You thoroughly trained the contract staff

And, you’ve staffed the project in a hierarchy that works well for keeping decisions in the hands of the attorneys.  Let’s review how this works in practice.

The base of your project is made up of contract reviewers who have been trained and armed with solid, objective criteria.  They won’t make decisions regarding what constitutes responsiveness or privilege.  They are simply applying rules that have already been developed by attorneys responsible for the case.  When a reviewer comes across a document that isn’t covered by the rules, they bring it to a supervisor.  The supervisor won’t make substantive decisions either, but the supervisor has had more access to the attorneys and broader exposure to the document collection than an individual reviewer, so the supervisor will in many cases know how to categorize a document in question.  When the supervisor can’t do that, it gets kicked up to the next level (most likely the project manager) who has yet broader exposure to the collection.  Some documents will get funneled up to an attorney for a decision.  In fact, that’s likely to happen frequently at the start of the project.  That’s why it’s important that attorney decision-makers are on-site and available full-time in the beginning.  As the project moves forward, you may be able to get by with attorneys being available remotely.

There are other steps you’ll take and mechanisms that you’ll employ to ensure that this model works well.  We’ll cover those in the next post in this series.

How do you structure and manage a document review project?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Trends: Christine Musil of Informative Graphics Corporation (IGC)
 

This is the fourth of the LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and asked each of them the same three questions:

  1. What do you consider to be the current significant trends in eDiscovery on which people in the industry are, or should be, focused?
  2. Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?
  3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Christine Musil.  Christine has a diverse career in engineering and marketing spanning 15 years. Christine has been with IGC since March 1996, when she started as a technical writer and a quality assurance engineer. After moving to marketing in 2001, she has applied her in-depth knowledge of IGC's products and benefits to marketing initiatives, including branding, overall messaging, and public relations. She has also been a contributing author to a number of publications on archiving formats, redaction, and viewing technology in the enterprise.

What do you consider to be the current significant trends in eDiscovery on which people in the industry are, or should be, focused?

For us, the biggest trend is elevation of the importance of eDiscovery, from what happens the minute you find out you have a lawsuit until the end of the case.  There’s a lot more discussion about how you can prevent it, how you can be better prepared, and I think that’s where the new buzzword, information governance, comes in.  We partner with OpenText and we partner with EMC on their content management side and we definitely see them pushing into the eDiscovery market to provide an end-to-end solution and stop trying to treat eDiscovery as an isolated issue. I think that the elevation of eDiscovery and inclusion of eDiscovery more into the regular business workflow of an organization is a pretty significant trend to watch.

Another trend that I see is an elevation of the use of search and how people can get more out of their searches to save time and cost.  This may be somewhat skewed based on our perspective in the market, but we’ve had a lot of requests for our redaction software to pick up the search that has already been done. Providers work so hard to come up with amazingly complicated algorithms to find data.  Why reinvent the wheel?  The companies all ask why all the other vendors can’t just take those search results and use it. 

Since you’ve written a white paper about native review and redaction, where do you see that heading?  Well, I hope that people will stop printing things out, scanning it back in to TIFF, then OCRing it and handing everybody back a disk of flat images and a separate disk with OCR text.  I sort of understand why they do it, but I think a less paranoid or adversarial approach through more effective “meet and confer” agreements on how you are going to present things are going to make it so much easier for everybody.  I hope in three to five years people say “I’m not afraid to hand you my native files because I know how to check them and know what metadata they contain and whether there are any tracked changes or other potential issues”.  So, the paranoia and fear that people have about the unknown that they can’t see in their documents and whether there is a smoking gun in there should die down.  I think people are getting smarter – now that they’re not producing paper – as to what  electronic files contain.  Hopefully, they will understand that native format is OK and when they need to redact, it’s OK to use PDF format to do so.  You tell the other side what you’re doing and what they’re going to get and it becomes a more open and well understood process.

I’m also on the EDRM XML committee and hope a standard load file format that transmits data seamlessly from one side to the other and contains all the information about what has been redacted, among other things, will make things easier on everybody, getting information through the process more seamlessly.  We’re writing white papers about the data set to educate the vendors on how to use it and I have high hopes for what we will be able to accomplish there.

Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?

{Interviewed on the first morning of LTNY}  Well, that’s hard since LegalTech just started [smiles].  I can tell you that in discussions with some of our partners, we’re seeing more support for mobile devices, support for the iPad, etc., to help lawyers work wherever they are and be more efficient wherever they are.  And, I think that literally goes all the way to the courtroom.  So, you’re seeing support for more devices and smaller screens, wherever attorneys get information.

What are you working on that you’d like our readers to know about?

I’m moderating a panel discussion {at LegalTech} titled, The Debate on Native Format Production and Redaction, which includes Craig Ball, George Socha, Tom O’Connor and Browning Marean.  I wrote a white paper last year entitled The Reality of Native Format Production and Redaction, which has inspired this panel discussion here at LegalTech.  So, that should be informative and interesting.  We’ve noticed that there’s just an awful lot of confusion in terms of what’s really required and what’s acceptable and the white paper and panel discussion really speaks to that.  We’re trying to educate our customers and help our partners educate their clients.

The other thing we’re announcing here is the release of integration to OpenText eDOCS.  We’ve been partners with OpenText for content management since 2002 and are very excited to extend our partnership to include this new area. eDOCS has a great presence in the legal space and we look forward to working with them.

Thanks, Christine, for participating in the interview!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

eDiscovery Trends: Alon Israely, Esq., CISSP of BIA
 

This is the second of the LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and asked each of them the same three questions:

  1. What do you consider to be the current significant trends in eDiscovery on which people in the industry are, or should be, focused?
  2. Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?
  3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Alon Israely.  Alon is a Senior Advisor in BIA’s Advisory Services group and when he’s not advising clients on e-discovery issues he works closely with BIA’s product development group for its core technology products.  Alon has over fifteen years of experience in a variety of advanced computing-related technologies and has consulted with law firms and their clients on a variety of technology issues, including expert witness services related to computer forensics, digital evidence management and data security.

What do you consider to be the current significant trends in eDiscovery on which people in the industry are, or should be, focused?

I think one of the important trends for corporate clients and law firms is cost control, whether it’s trying to minimize the amount of project management hours that are being billed or the manner in which the engagement is facilitated.  I’m not suggesting going full-bore necessarily, but taking baby steps to help control costs is a good approach.  I don’t think it’s only about bringing prices down, because I think that the industry in general has been able to do that naturally well.  But, I definitely see a new focus on the manner in which costs are managed and outsourced.  So, very specifically, scoping correctly is key, making sure you’re using the right tool for the right job, keeping efficiencies (whether that’s on the vendor side or the client side) by doing things such as not having five phone calls for a meeting to figure out what the key words are for field searching or just going out and imaging every drive before deciding what’s really needed. Bringing simple efficiencies to the mechanics of doing e-discovery saves tons of money in unnecessary legal, vendor and project management fees.  You can do things that are about creating efficiencies, but are not necessarily changing the process or changing the pricing.

I also see trends in technology, using more focused tools and different tools to facilitate a single project.  Historically, parties would hire three or four different vendors for a single project, but today it may be just one or two vendors or maybe even no vendors, (just the law firm) but, it’s the use of the right technologies for the right situations – maybe not just one piece of software, but leveraging several for different parts of the process.  Overall, I foresee fewer vendors per project, but more vendors increasing their stable of tools.  So, whereas a vendor may have had a review tool and one way of doing collection, now they may have two or three review tools, including an ECA tool, and one or two ways of doing collections. They have a toolkit from which they can choose the best set of tools to bring to the engagement.  Because they have more tools to market, vendors can have the right tool in-their-back-pocket whereas before the tool belonged to just one service provider so you bought from them, or you just didn’t have it.

Which of those trends are evident here at LTNY, which are not being talked about enough, and/or what are your general observations about LTNY this year?

{Interviewed on the first morning of LTNY} I think you have either a little or a lot of – depending on how aggressive I want to be with my opinion – that there seems to be a disconnect between what they’re speaking about in the panels and what we’re seeing on the floor.  But, I think that’s OK in that the conference itself, is usually a little bit ahead of the curve with respect to topics, and the technology will catch up.  You have topics such as predictive coding and social networking related issues – those are two big ones that you’ll see.  I think, for example, there are very few companies that have a solution for social networking, though we happen to have one.  And, predictive coding is the same scenario.  You have a lot of providers that talk about it, but you have a handful that actually do it, and you have probably even fewer than that who do it right.  I think that next year you’ll see many predictive coding solutions and technologies and many more tools that have that capability built into them.  So, on the conference side, there is one level of information and on the floor side, a different level.

What are you working on that you’d like our readers to know about?

BIA has a new product called TotalDiscovery.com, the industry’s first SaaS (software-as-a-service), on-demand collection technology that provides defensible collections.  We just rolled it out, we’re introducing it here at LegalTech and we’re starting a technology preview and signing up people who want to use the application or try it.  It’s specifically for attorneys, corporations, service providers – anyone who’s in the business and needs a tool for defensible data collection performed with agility (always hard to balance) – so without having to buy software or have expert training, users simply login or register and can start immediately.  You don’t have to worry about the traditional business processes to get things set up and started.  Which, if you think about it on the collections side of e-discovery it means that  the client’s CEO or VP of Marketing can call you up and say “I’m leaving, I have my PST here, can you just come get it?” and you can facilitate that process through the web, download an application, walk through a wizard, collect it defensibly, encrypt it and then deliver a filtered set, as needed, for review..

The tool is designed to collect defensibly and to move the collected data – or some subset of that data –to delivery, from there you would select your review tool of choice and we hand it off to the selected review tool.  So, we’re not trying to be everything, we’re focused on automating the left side of the EDRM.  We have loads to certain tools, having been a service provider for ten years, and we’re connecting with partners so that we can do the handoff, so when the client says “I’m ready to deliver my data”, they can choose OnDemand or Concordance or another review tool, and then either directly send it or the client can download and ship it.  We’re not trying to be a review tool and not trying to be an ECA tool that helps you find the needle in the haystack; instead, we’re focused on collecting the data, normalizing it, cataloguing it and handing if off for the attorneys to do their work.

Thanks, Alon, for participating in the interview!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

Managing an eDiscovery Contract Review Team: Starting the Project
 

Throughout the life of the project, you will implement some standard project management techniques for monitoring quality and ensuring that questions are resolved efficiently.  We’re going to cover those techniques in the next blog posts in this series.  There are, however, a couple of special steps you should take at the beginning of a project to ensure that it gets off to a smooth start.

First, you want to make sure very quickly that everyone on the team understands the criteria and is applying it correctly to the documents.  The best way to do this is to check everybody’s initial work right away and to provide feedback as quickly as you can.   Make arrangements with the supervisory staff to work extra hours the first few days of the project.  During the first couple of days of the project, have supervisors check the work after the review staff leaves.   Have the supervisors give feedback – one-on-one – to each team member within the first couple of days of the project.  In addition, make sure that the supervisors are communicating with one another and with the project manager about what they are finding.  Wide-spread misunderstandings will uncover holes in the training and can easily be cleared up with the group in short re-training sessions. 

When we talked about who should be on the review team, we talked about “decision makers” and subject matter experts.  Make sure these team members are onsite full-time the first few days of the project.  There will be a lot of questions the first few days, and you’ll want to resolve those questions quickly.  Once the project gets underway, the level of questions will subside, and the supervisors and project manager will be better equipped to answer the questions that do arise.  At that point, you probably don’t need the decision makers and subject matter experts on hand full time.  But make sure they are present at the start of the project. 

How do you approach starting a document review project?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

Managing an eDiscovery Contract Review Team: Training a Review Team
 

Yesterday, we discussed the assembling the project team for document review.  It’s also important that the review team gets good training.  As a starting point, prepare a training manual for each team member that includes this information:

  • The document review criteria
  • A list of the custodians.  For each, provide the custodian’s title, a job description, a description of his/her role in the events that are at issue in the case, and a description of the types of documents you expect will be found in his/her files
  • Lists of keywords, key characters, key events, and key dates
  • Samples of responsive documents that you collected when you reviewed the collection
  • The review procedures
  • The review schedule
  • Instructions for use of the review tool

Cover these topics in training:

  • Case background information
    • A description of the parties
    • A description of the events that led to the case
    • A description of the allegations and defenses
    • An overview of the expected case schedule
  • Project overview information
    • A description of the goals of the review project
    • A description of the process
    • An overview of the expected project schedule
  • Responsive criteria
    • Go through the criteria – point-by-point – to ensure that the group understands what is responsive
    • Provide samples of responsive documents
  • Mechanics
    • Describe the roles of individuals on the team
    • Review the procedures for reviewing documents
    • Train the reviewers in use of the online review tool

Give the team training exercises – that is, give them sample documents to review.  Collect the work, review it, and give feedback to the group.

And let me give you two more suggestions that will help make your training effective:

  1. Train the team together, rather than one-on-one or in sub-groups.  Under this team-training approach, you ensure that everyone hears the same thing, and that responses to questions asked by individuals will benefit the entire group.
  2. Involve a senior attorney in the training.  You might, for example, ask a senior attorney to give the case background information.  Attention from a senior litigation team member is good for morale.  It tells the team that the work they are doing is important to the case.

How do you approach training a document review team?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

Managing an eDiscovery Contract Review Team: Assembling the Project Team
 

Before assembling the review team, think through how the project will be structured.  This will drive decisions on the type of people that you’ll need.  Your goal is to get the work done as cost effectively as possible – using less expensive personnel where possible — without sacrificing work quality or the utility of the work product.

The “base” of the project will be comprised of contract reviewers and qc staff.  In the project plan, you determined the number of people that you need.  At this point, don’t worry about who will be a reviewer and who will do qc work.  Everybody can start as a reviewer.  After a few days, you can identify who will do qc work.  You’ve got options for assembling this staff, but you should consider working with a litigation support vendor who offers staffing services.  A good vendor already has access to a pool of people with document review experience.  This can save you lots of time and work.

In addition to the contract review staff, you’ll need project management staff.  We’ve already talked about a project manager.  For a large project, you’ll want project supervisors — each responsible for a team of reviewers/qc personnel.  Each supervisor is responsible for overseeing the flow of work to the team, the quality of the work done by the team, the productivity of team members, and answering questions raised by reviewers (or ensuring that questions are resolved).  I usually create teams of 10 to 12 and assign one supervisor to a team.   The supervisors might be law firm litigation support professionals, or supervisory staff provided by the vendor with whom you are working.

You’ll also need “decision makers” and experts in the subject matter to round out the team.  At a minimum, you’ll want an attorney from the litigation team.  Depending on the complexity of the documents, you might need a client employee who is familiar with the company’s operations and documents.  These people should be on-site, full-time for the first few days of a project.  Eventually there will be fewer questions and it’s probably sufficient to have phone access to these team members.

Later in this blog series we’ll talk about how these staff levels interact so that decisions are made by attorneys but effectively implemented by review staff.

How do you structure a document review team?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Best Practices: Judges’ Guide to Cost-Effective eDiscovery
 

Last week at LegalTech, I met Joe Howie at the blogger’s breakfast on Tuesday morning.  Joe is the founder of Howie Consulting and is the Director of Metrics Development and Communications for the eDiscovery Institute, which is a 501(c)(3) nonprofit research organization for eDiscovery.

eDiscovery Institute has just released a new publication that is a vendor-neutral guide for approaches to considerably reduce discovery costs for ESI.  The Judges’ Guide to Cost-Effective E-Discovery, co-written by Anne Kershaw (co-Founder and President of the eDiscovery Institute) and Joe Howie, also contains a foreword by the Hon. James C. Francis IV, Magistrate Judge for the Southern District of New York.  Joe gave me a copy of the guide, which I read during my flight back to Houston and found to be a terrific publication that details various mechanisms that can reduce the volume of ESI to review by up to 90 percent or more.  You can download the publication here (for personal review, not re-publication), and also read a summary article about it from Joe in InsideCounsel here.

Mechanisms for reducing costs covered in the Guide include:

  • DeNISTing: Excluding files known to be associated with commercial software, such as help files, templates, etc., as compiled by the National Institute of Standards and Technology, can eliminate a high number of files that will clearly not be responsive;
  • Duplicate Consolidation (aka “deduping”): Deduping across custodians as opposed to just within custodians reduces costs 38% for across-custodian as opposed to 21% for within custodian;
  • Email Threading: The ability to review the entire email thread at once reduces costs 36% over having to review each email in the thread;
  • Domain Name Analysis (aka Domain Categorization): As noted previously in eDiscoveryDaily, the ability to classify items based on the domain of the sender of the email can significantly reduce the collection to be reviewed by identifying emails from parties that are clearly not responsive to the case.  It can also be a great way to quickly identify some of the privileged emails;
  • Predictive Coding: As noted previously in eDiscoveryDaily, predictive coding is the use of machine learning technologies to categorize an entire collection of documents as responsive or non-responsive, based on human review of only a subset of the document collection. According to this report, “A recent survey showed that, on average, predictive coding reduced review costs by 45 percent, with several respondents reporting much higher savings in individual cases”.

The publication also addresses concepts such as focused sampling, foreign language translation costs and searching audio records and tape backups.  It even addresses some of the most inefficient (and therefore, costly) practices of ESI processing and review, such as wholesale printing of ESI to paper for review (either in paper form or ultimately converted to TIFF or PDF), which is still more common than you might think.  Finally, it references some key rules of the ABA Model Rules of Professional Conduct to address the ethical duty of attorneys in effective management of ESI.  It’s a comprehensive publication that does a terrific job of explaining best practices for efficient discovery of ESI.

So, what do you think?  How many of these practices have been implemented by your organization?  Please share any comments you might have or if you’d like to know more about a particular topic.

Managing an eDiscovery Contract Review Team: Prepare a Review Plan
 

Before starting a review project, prepare a plan that will be your map for moving forward and that you’ll use to monitor progress throughout the project.  The components of the plan should include:

  • A schedule:  The schedule will be dictated by the date on which you’re required to produce documents.  Determine when you can start  (you’ll need time in advance to assemble the team, draft criteria, train the team, and get documents loaded into the review tool).  Plan to complete the project well in advance of the production deadline, so you’ve got some pad for schedule slippage.  Once you have a start date and an end date, look at the size of the collection and calculate how many documents you need to process in a day. 
  • Identify required staff resources:  Determine the number of units a reviewer can do in a day (use statistics from prior projects or do test runs on a sampling of the collection).  From there, do the math to determine how many reviewers you’ll need to complete the project within your schedule.  Add in people to do quality control (a good ratio is 1 qc reviewer for every 4 or 5 reviewers).
  • A budget:  Include costs for review/qc staff, project management, equipment, and any other incidentals.
  • Prepare a monitoring plan:  Every day you should look at the progress that the review team is making so you’ll know where you stand with schedule and budget.  You’ll need reports that provide statistics on what has been completed – reports that can probably be generated by the review tool.  Look at those reports to determine if they will meet your needs or if you’ll need to establish additional tracking mechanisms.
  • Prepare a quality control plan:  Determine how reviewed documents will be funneled to the quality control staff, the mechanism by which quality control reviewers will report findings to project management, and the mechanism by which feedback will be provided to reviewers.

A good plan isn’t hard to prepare and it can make a world of difference in how smoothly the project will run.

How do you plan a document review?  Have you run into glitches?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.

Managing an eDiscovery Contract Review Team: Identify a Project Manager
 

Yesterday, we talked about applying topic codes to the documents to identify helpful or harmful documents.  Today, we will talk about identifying a project manager for the review.

A good, experienced project manager is critical to the success of your review project.  In fact, the project manager is the most important part of the equation.  The project manager will be responsible for:

  • Creating a schedule and a budget
  • Determining the right staff size
  • Lining up all the resources that you’ll need like computer equipment, software, and supplies
  • Preparing training materials.
  • Coordinating training of the review team
  • Serving as a liaison with the service providers who are processing the data, loading data into the review tool, and making the review tool available to the review team
  • Monitoring status of the project and reporting to the litigation team
  • Identifying potential problems with schedule and budget and developing resolutions
  • Ensuring that questions are resolved quickly and that lines of communication between the review team and decision makers are open
  • Supervising workflow and quality control work

Choose someone who has project management experience and is experienced in litigation, technology, electronic discovery, working with vendors, and working with attorneys.  Identify the project manager early on and get him or her involved in the project planning steps. 

What do you look for in a project manager?  Please share any comments you have and let us know if you’d like to know more about an eDiscovery topic.