Outsourcing

Working Successfully with eDiscovery and Litigation Support Service Providers: Preventing Quality Problems

 

In the past few posts in this series, we’ve discussed ways to minimize the possibility of problems with costs and meeting deadlines.  There are also steps you can take to increase the likelihood of high quality work that will meet your expectations. 

It starts when you are evaluating and selecting a vendor.  Based on your review of a vendor’s qualifications, procedures and technology, along with feedback you’ve received from references, make sure you select a vendor that is likely to do high quality work and that has a good track record. 

Once a project is underway, look at the vendor’s work to ensure it meets your expectations.  This is especially important with work that is labor-intensive.  In fact, there are three levels of quality control that you should be doing:

  1. A review of initial work done by each staff member:  As staff members finish initial batches of work, look at it right away and verify that it is correct.  This way, mistakes made because someone misunderstood instructions can be cleared up very quickly, before a lot of work is done that will need to be re-done.
  2. Ongoing QC of work throughout the project:  Even if everyone on the staff knows how to do the work, that doesn’t mean they will do it right all the time.  Mistakes are inevitable and work should be spot-checked throughout a project to ensure that errors are kept to a minimum.  It’s a good idea to have project staff dedicated to doing on-going quality control reviews.
  3. Random sampling done by senior litigation team members:  In addition to routine quality control done by project staff members, its always a good idea to ask senior members of the litigation team to periodically look at a sampling of the work, to ensure that stakeholder expectations are in sync with the work of the project staff.

What steps do you take to ensure high quality work from a vendor with which you are working?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

Working Successfully with eDiscovery and Litigation Support Service Providers: Preventing Unexpected Schedule Problems

 

Of all the problems you might have to report to an attorney, the worst may be that you are going to miss a production deadline.  Deadlines get missed because document or data collections are larger than expected, the task is more difficult than expected, sufficient technical resources aren’t allocated to the project, or the project simply isn’t staffed properly.  Good communication up front with a vendor, and good monitoring throughout a project can minimize the chance of missing deadlines. 

Here are some steps you can take to prevent deadline problems and keep a vendor on track with your schedule requirements:

  • During the vendor evaluation and selection process, get commitments from a vendor on scaling up to accommodate changes in volume.  Ask about their ability to add staff, technical resources, or second shifts to meet deadlines if things get off track.
  • Before signing a contract with a vendor, ask for a schedule that includes weekly estimates of completed work, and that includes expected staffing levels for labor-intensive work.
  • In addition, require that the vendor provide you with weekly status reports that include estimated weekly numbers, actual weekly numbers, estimated to-date numbers, actual to-date numbers, and variances.  This will enable you to see – each week – how the project is going and whether the schedule is slipping.  Carefully review status reports as they are received and speak to the vendor right away if things are getting off-track.  Be proactive in requesting additional resources if the project is getting off-schedule.
  • Require that the vendor notify you in writing – in advance – if a deadline will be missed.
  • When you communicate schedule information to a vendor, build in pad!  For example, give the vendor a deadline well in advance of a production date.
  • Stay in constant communication with the litigation team about discovery issues.  The more notice that you have about additional documents or data, the more notice you can give to the vendor.

What steps do you take to prevent schedule problems with vendor work?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

Working Successfully with eDiscovery and Litigation Support Service Providers: Preventing Unexpected Costs

 

Unfortunately, it’s just not possible to know up-front precisely what a litigation support or eDiscovery project will cost.  There are too many unknowns, and, therefore, too many opportunities for surprises regarding costs.  You can, however, avoid some of those surprises with good upfront communication with the vendor.

Here are some steps you can take to prevent surprises and control costs:

  • Make sure that you have a clear understanding of every unit price the vendor provides to you in an estimate.  Some tasks may be charged per-page, others per-document, others per gigabyte, and others on an hourly basis.  Make sure that you understand how the work is being charged and that you agree that the price model makes sense.
  • Require that the service provider prepare an estimate of total project costs in addition to providing unit pricing.  That estimate should be based on a reasonable estimate of the size of the collection and on assumptions about the project that you provide to the vendor.  Ask the vendor to include all incidental cost components in the estimate.  For example, it the vendor charges for project management, supplies and shipping, require that the estimate include these costs based on their experience with similar projects.
  • Before signing a contract, come to agreement on how any changes in the assumptions will affect the price.
  • Review a sample invoice and make sure that you understand its components.  You may need to verify the vendor’s invoices, so make sure that they provide an invoice you understand.  Changing the format of an invoice may not be easy for a vendor, so it’s probably not reasonable to ask them to do so.  You could, however, ask for a cover letter with each invoice explaining charges that may not be clear or evident .  It’s always better to ask for these types of “extras” before signing a contract with a vendor.
  • Find out what the vendor’s policy is on price changes or notification of such.

These are easy steps that don’t take much time.  This extra effort up-front, however, can avoid surprises and disagreements down the road.

What steps do you take to prevent cost problems with vendor work?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

Working Successfully with eDiscovery and Litigation Support Service Providers: Monitoring Work and Preventing Problems

 

So many law firm litigation support folks that I know have an unpleasant war story or two about a project they did with a vendor.  I usually find these frustrating, because more often than not, the problems could have been avoided.  Taking a few preventative steps early on, and doing a good job of monitoring project activity can make all the difference.   In the next few posts in this series, we’re going to look at ways to make the projects you do with vendors run smoother.

First, let’s look at the potential problem areas.  There are three:

  • Unexpected project costs
  • Missed deadlines
  • Poor work quality

Very often, the problems in these areas result from poor communication between the litigation team and the service provider.  Sometimes, the problems stem from poor up-front communication, which results in a misunderstanding of requirements, priorities and expectations.  Other times, the problems arise because of poor communication during the project itself.

There are several steps you can take up-front to minimize the potential for problems.  These steps will foster a good understanding of expectations and requirements and build lines of communication into the process.  Constant communication with the service provider throughout the life of the project is likewise important.  If you are in frequent communication you are likely to catch problems before they get out of hand and before they turn into major headaches.

The next several blog posts will cover steps you can take to minimize the potential for problems with unexpected project costs, problems with meeting deadlines, and problems with sub-standard work quality.

What steps do you take to prevent problems with vendor work?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Best Practices: 4 Steps to Effective eDiscovery With Software Analytics

 

I read an interesting article from Texas Lawyer via Law.com entitled “4 Steps to Effective E-Discovery With Software Analytics” that has some interesting takes on project management principles related to eDiscovery and I’ve interjected some of my thoughts into the analysis below.  A copy of the full article is located here.  The steps are as follows:

1. With the vendor, negotiate clear terms that serve the project's key objectives.  The article notes the important of tying each collection and review milestone (e.g., collecting and imaging data; filtering data by file type; removing duplicates; processing data for review in a specific review platform; processing data to allow for optical character recognition (OCR) searching; and converting data into a tag image file format (TIFF) for final production to opposing counsel) to contract terms with the vendor. 

The specific milestones will vary – for example, conversion to TIFF may not be necessary if the parties agree to a native production – so it’s important to know the size and complexity of the project, and choose only an experienced eDiscovery vendor who can handle the variations.

2. Collect and process data.  Forensically sound data collection and culling of obviously unresponsive files (such as system files) to drastically decrease the overall review costs are key services that a vendor provides in this area.  As we’ve noted many times on this blog, effective culling can save considerable review costs – each gigabyte (GB) culled can save $16-$18K in attorney review costs.

The article notes that a hidden cost is the OCR process of translating extracted text into a searchable form and that it’s an optimal negotiation point with the vendor.  This may have been true when most collections were paper based, but as most collections today are electronic based, the percentage of documents requiring OCR is considerably less than it used to be.  However, it is important to be prepared that there are some native files which will be “image only”, such as TIFFs and scanned PDFs – those will require OCR to be effectively searched.

3. Select a data and document review platform.  Factors such as ease of use, robustness, and reliability of analytic tools, support staff accessibility to fix software bugs quickly, monthly user and hosting fees, and software training and support fees should be considered when selecting a document review platform.

The article notes that a hidden cost is selecting a platform with which the firm’s litigation support staff has no experience as follow-up consultation with the vendor could be costly.  This can be true, though a good vendor training program and an intuitive interface can minimize or even eliminate this component.

The article also notes that to take advantage of the vendor’s more modern technology “[a] viable option is to use a vendor's review platform that fits the needs of the current data set and then transfer the data to the in-house system”.  I’m not sure why the need exists to transfer the data back – there are a number of vendors that provide a cost-effective solution appropriate for the duration of the case.

4. Designate clear areas of responsibility.  By doing so, you minimize or eliminate inefficiencies in the project and the article mentions the RACI matrix to determine who is responsible (individuals responsible for performing each task, such as review or litigation support), accountable (the attorney in charge of discovery), consulted (the lead attorney on the case), and informed (the client).

Managing these areas of responsibility effectively is probably the biggest key to project success and the article does a nice job of providing a handy reference model (the RACI matrix) for defining responsibility within the project.

So, what do you think?  Do you have any specific thoughts about this article?   Please share any comments you might have or if you’d like to know more about a particular topic.

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References, Part 3

 

Last week, we started and continued a list of questions to ask when you are doing a reference check.  So far, the list includes general project questions, questions about communicating with the vendor, questions about costs, and questions aimed at evaluating the quality of a vendor’s work. Today we’ll complete the list with questions about the vendor’s ability to meet schedule requirements, and questions aimed at assessing the vendor’s overall performance. 

Questions regarding schedule requirements, meeting deadlines, and flexibility:

  1. Did you establish a schedule with milestones and deadlines for the vendor?
  2. Did they meet your milestones and deadlines?  If not, why was there a problem?
  3. Did the scope of the project – in terms of volume – change after the project started?  How did the vendor handle that? 
  4. Did the vendor need to add additional resources to your project to meet deadlines?  If not, should they have?
  5. If deadlines were missed, how much advance notice were you given?
  6. Did you change priorities that required adjustments to work flow during the project?  If so, how did the vendor handle that?  Were they able to accommodate changing needs?

Questions aimed at assessing overall performance

  1. Will you use this vendor again?
  2. Are there changes you would make to the way in which you work with this vendor?  Or changes you would request regarding their services?
  3. Are there project thresholds under which you wouldn’t use this vendor (for example, not for a collection above a certain size, or not for a rush project)?
  4. What do you believe is this vendor’s greatest strength?  Weakness?
  5. What is the one thing you would change about this vendor if you could?
  6. How would you rank this vendor against other vendors you have used?  Are they on your list of preferred vendors for future projects?  Under what circumstances might you choose another vendor over this vendor?

The list of questions we’ve compiled in the last several posts should be a good starting point for you!  Remember, though, that a good reference check session will be interactive.  Some of the answers you get will prompt additional questions.  Don’t be afraid to do some digging to uncover what you need to know.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References, Part 2

 

Yesterday, we started a list of questions to ask when you are doing a reference check.  That list included general project questions, questions about communicating with the vendor, and questions about costs. Here are some questions aimed at uncovering information on the quality of the vendor’s work:

  1. Were you satisfied with the quality of the vendor’s work and did it meet your expectations?
  2. Were you involved in initial project training of the vendor’s staff?  Do you recommend that with this vendor?
  3. Did you do quality control checks of the vendor’s work?  How did you check the work?  How much of the work did you check?
  4. Did you spend time checking work at the vendor’s facility?  Do you recommend that with this vendor?
  5. Did you provide feedback to the vendor on quality?  How was that received?  Did the vendor incorporate your feedback and did the work quality improve?
  6. If there were quality problems, were they dealt with quickly?  Was it necessary to escalate problems above the project manager to get them resolved?
  7. Do you know what quality control procedures the vendor used?  How much work did they check?  What quality control was done manually?  What automated checks were done?
  8. Do you know if the vendor needed to do rework?  Was rework precipitated by issues you brought to their attention?  Did you have to ask the vendor to do any work over?
  9. If you had the technology tools and resources to do this work yourself, do you think you could have done a better job?  If so, in what way?
  10. Do you feel that the vendor had a good grasp on your requirements and the experience to do the work efficiently?
  11. Did the project manager have the right background and experience to handle your project?

In the next issue, we’ll continue this list with questions aimed at getting a handle on a vendor’s ability to meet schedule requirements and some closing questions aimed at assessing overall performance.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

In observance of Good Friday, eDiscovery Daily will not publish a new post tomorrow.  See you Monday!

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References

 

Getting good information about a vendor in a reference check is a skill worth developing.  As a starting point, you should be armed with a good set of questions.  Don’t limit yourself to the list, though.  A good reference-check call will be interactive and answers to some questions will probably prompt additional questions.

Here’s a good starting point list that might apply to any type of vendor or service you are inquiring about:

First, get a handle on the scope of the project the vendor handled for the client:

  1. What services were provided?
  2. What volume of data / documents were handled?
  3. What deliverables did the vendor provide to you?
  4. What was the schedule?

Ask questions about the communication frequency and effectiveness between the vendor and the client:

  1. Were you the primary point of contact?
  2. If not, can you point me to the person who was?
  3. Did you have regular contact with the project manager?  With anyone else?
  4. Did you get regular status reports from the vendor?  How often did you get them?  What information was included in the status reports?  Were the reports useful?  Could you tell if the project was on schedule?
  5. How quickly did the vendor respond to phone calls?  Email?
  6. Did you ever have trouble reaching the vendor, or reaching the right person at the vendor organization?

Ask questions about the project costs:

  1. Were the costs what you expected?
  2. Were costs within the vendor’s estimates?
  3. Were the invoices clear and sufficiently detailed?  What information was included on the invoices?
  4. Were there any cost surprises?  Any cost line items on the invoices that you weren’t expecting?

Tomorrow we’ll continue this list with questions aimed at getting a handle on the quality of a vendor’s work.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

What eDiscovery Professionals Can Learn from the Internet Gambling Crack Down

 

Many of you may have heard about the FBI cracking down on the three largest online gambling sites in the past few days, as the owners of those sites in the United States have been indicted and charged with bank fraud, money laundering and illegal gambling offenses and the sites have been essentially shut down in the US.  Restraining orders have been issued against more than 75 bank accounts in 14 countries used by the poker companies PokerStars, Full Tilt Poker and Absolute Poker.  Many US customers of these sites are now scrambling to try to get their funds out of the sites and finding it difficult to do so.

So what?  This is an eDiscovery blog, right?  What does an Internet gambling crack down have to do with eDiscovery?

PokerStars, Full Tilt Poker, Absolute Poker and other gambling sites are cloud-based, Software-as-a-Service (SaaS) applications.  Just like Amazon, Facebook, Twitter, eBay, YouTube and SalesForce.com, these sites provide an application via the web that enables its clients to receive a service.  In the case of Amazon, it’s the ability to purchase any number of products.  For Facebook, it’s the ability to share information with friends and family.  For these gambling sites, it’s the ability to play poker for money with anyone else in the world who has the same gambling itch that you do and a broadband connection.

The problem is: in the US, it’s illegal.  The Unlawful Internet Gambling Enforcement Act of 2006 prohibits gambling businesses from knowingly accepting payments related to a bet or wager that involves the use of the Internet and that is unlawful under any federal or state law.  So, these sites are hosted in other countries to attempt to skirt the law.

What many US customers of these sites are finding out is the same thing that eDiscovery professionals discover when they need to retrieve cloud-based data in response to a discovery request: it’s imperative to know where your data is stored.  It’s likely that many customers of these gambling sites knew that their funds were kept off-shore, while others may not have known this was the case.  Regardless, they’re now scrambling to get their data (i.e., funds) back — if they can.

Many organizations are “in the same boat” when it comes to their SaaS providers – it may be unclear where that data is being stored and it may be difficult to retrieve if it’s stored in a foreign country with a different set of laws.  It’s important to establish (in writing if possible) with the provider up front where the data will be stored and agree on procedures such as records management/destruction schedules so that you know where your data is stored and can get access to it when you need it.  Don’t gamble with your data.

So, what do you think?  Do you have organizational data in a SaaS-based solution?  Do you have a plan for getting that data when you need it?  Please share any comments you might have or if you’d like to know more about a particular topic.

Full disclosure: I work for Trial Solutions, which provides SaaS-based eDiscovery review applications FirstPass® (for first pass review) and OnDemand® (for linear review and production).  Our clients’ data is hosted in a secured Tier 4 Data Center in Houston, Texas, where Trial Solutions is headquartered.

Working Successfully with eDiscovery and Litigation Support Service Providers: Checking References – General Suggestions

 

Yesterday, we talked about the importance of checking references when considering eDiscovery and Litigation Support Service Providers.  In the next blogs in this series, I’m going to suggest some questions you may consider asking when doing a reference check.  First, however, let me make some general suggestions:

  • When you ask a vendor for references, ask for clients that had a project similar in size and scope to yours.  You want to speak with people who had similar requirements regarding services and schedules, and that had document or data collections similar in size and format to yours.
  • When you ask a vendor for references, let them know that you’d like to speak with two different types of people:
    • End-users of the vendor’s work product
    • The vendor’s main, day-to-day point of contact
  • Don’t call a reference out of the blue.  Make an initial contact by email to introduce yourself and to schedule a call.  If the call is scheduled, you are more likely to get better attention and more time. 
  • Be prepared with a list of specific questions.  Don’t call and ask only general question like “Were you satisfied with the quality?” and “Did they meet your deadlines?”
  • Try to engage the person you are speaking with in conversation.  Don’t settle for yes and no answers.  When someone responds “yes” or “no” to a question, ask them to provide details.  This may uncover information that is important to you or it may trigger additional questions that you should ask.
  • Always send follow-up emails to the people with whom you speak thanking them for their time and information. 

Next we’ll talk about specific questions you can consider asking when checking references, so stay tuned.

How do you approach checking references?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.