Close
Enter your
text here

eDiscoveryDaily

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References, Part 3
 

Last week, we started and continued a list of questions to ask when you are doing a reference check.  So far, the list includes general project questions, questions about communicating with the vendor, questions about costs, and questions aimed at evaluating the quality of a vendor’s work. Today we’ll complete the list with questions about the vendor’s ability to meet schedule requirements, and questions aimed at assessing the vendor’s overall performance. 

Questions regarding schedule requirements, meeting deadlines, and flexibility:

  1. Did you establish a schedule with milestones and deadlines for the vendor?
  2. Did they meet your milestones and deadlines?  If not, why was there a problem?
  3. Did the scope of the project – in terms of volume – change after the project started?  How did the vendor handle that? 
  4. Did the vendor need to add additional resources to your project to meet deadlines?  If not, should they have?
  5. If deadlines were missed, how much advance notice were you given?
  6. Did you change priorities that required adjustments to work flow during the project?  If so, how did the vendor handle that?  Were they able to accommodate changing needs?

Questions aimed at assessing overall performance

  1. Will you use this vendor again?
  2. Are there changes you would make to the way in which you work with this vendor?  Or changes you would request regarding their services?
  3. Are there project thresholds under which you wouldn’t use this vendor (for example, not for a collection above a certain size, or not for a rush project)?
  4. What do you believe is this vendor’s greatest strength?  Weakness?
  5. What is the one thing you would change about this vendor if you could?
  6. How would you rank this vendor against other vendors you have used?  Are they on your list of preferred vendors for future projects?  Under what circumstances might you choose another vendor over this vendor?

The list of questions we’ve compiled in the last several posts should be a good starting point for you!  Remember, though, that a good reference check session will be interactive.  Some of the answers you get will prompt additional questions.  Don’t be afraid to do some digging to uncover what you need to know.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

eDiscovery Trends: Thought Leader Interview with Jeffrey Brandt, Editor of Pinhawk Law Technology Daily Digest
 

As eDiscovery Daily has done in the past, we have periodically interviewed various thought leaders in eDiscovery and legal technology to provide insight as to trends in the industry for our readers to consider.  Recently, I was able to interview Jeffrey Brandt, Editor of the Pinhawk Law Technology Daily Digest and columnist for Legal IT Professionals.

With an educational background in computer science and mathematics from the University of Pittsburgh, Jeff has over twenty four years of experience in the field of legal automation working with various organizations in the United States, Canada, and the United Kingdom.  As a technology and management consultant to hundreds of law firms and corporate law departments he has worked on information management projects including: long range strategic planning, workflow management and reengineering, knowledge management, IT structure and personnel requirements and budgeting. Working as a CIO at several large law firms, Jeff has helped bring oversight, coordination and change management to initiatives including: knowledge management, library & research services, eDiscovery, records management, technology and more. Most recently, he served as the Chief Information and Knowledge Officer with an AMLaw 100 law firm based out of Washington, DC.

Jeff has also been asked to serve on numerous advisory councils and CIO advisory boards for key vendors in the legal space, advising them on issues of client service and future product direction.  He is a long time member (and former board member) of the International Legal Technology Association (ILTA) and has taught CLE classes on topics ranging from litigation support to ethics and technology.

What do you consider to be the current significant trends in eDiscovery in 2011 and beyond on which people in the industry are, or should be, focused?

I would say that the biggest two are the project management component and, for lack of a better term, automated or artificial intelligence.

The whole concept and the complexities of what it takes to manage a case today are more challenging than ever, including issues like the number of sources, the amount of data in the sources, the format in which you’re producing, where can the data go and who can see it.  I remember the days when people used to take a couple of bankers boxes, put them in their car and go home and work on the documents.  You simply cannot do that today – the amount of information today is just insane.

As for artificial intelligence, as was discussed in the (Pinhawk) digest recently, you’re seeing the emergence of predictive coding and using computers to cull through the massive amounts of information so that a human can take the final pass.  I think more and more we’re going to see people relying on those types of technologies – some because they embrace it, others because there is no other way to humanly do it.

I think if there’s any third trend it would probably be where do we go next to get the data?  In terms of social media, mining Facebook and Twitter and all the other various sources for additional data as part of the discovery process has become a challenge.

You recently became editor of the Pinhawk Law Technology Daily Digest.  Tell me about that and about your plans for the digest.

Well, I think there are several things going forward.  My role is to keep up the good work that Curt Meltzer, the founding editor, started and fill the “big shoes” that Curt left behind.  My goal is to expand the sources of information from which Pinhawk draws.  There are about 400 sources today and I think by the time my sources (and possibly a few others) are added in, there will be over 500.  We’ve also talked about going to our readership and asking them “what are your go-to and must read sources?” to include those sources as well.  We’ll also be looking to incorporate social media tools to hopefully make the experience much more comprehensive and easier to participate in for the Pinhawk digest reader.

And, what should we be looking for in your column in Legal IT Professionals?

Well, I like to dabble in multiple areas – in the small consulting practice that I have, I do a little bit of everything.  I’ve recently done some very interesting work in communities of practice, using social media tools, focusing them inward in law firms to provide the forum for lawyers to open up, share and mentor to others.  I like KM (Knowledge Management) and related topics and we had a recent post in Pinhawk talking about the future of the law firm.  To me, those types of discussions are fascinating.

You take the extremes and you’ve got the “law factory”, you take the high-end and you’ve got the “bet the farm” law firm.  How technology plays a role in whatever culture, whatever focus a law firm puts itself on is interesting.  And then you watch and see some of the rumblings and inklings of what can be done in places like Australia, where you have third-party investment of law firms and the United Kingdom, where they are about to get third-party investment.  There was a recent article about third-party ownership of law firms in North Carolina.  You look at examples like that and you say “is the model of partnership alive?”  When you get into “big law”, are they really partnerships?  Where are they in the spectrum of a thousand sole practitioners operating under one letterhead to a firm of a thousand lawyers?  That’s where I think that communities of practice and social media tools are going to help lawyers know more about their own partners and own firms. 

It’s sad that in some firms the lawyers on the north side of the building don’t even know the lawyers on the south side of the building, let alone the people on the eighth floor vs. the tenth floor.  It’s a changing landscape.  When I got into legal and was first a CIO at Porter, Wright, Morris & Arthur, 250 lawyers in Columbus, Ohio was the 83rd largest law firm in the US – an AMLAW 100 firm.  Today, does that size a firm even make it into the AMLAW 250?

In my column at Legal IT Professionals, you’ll see more about KM and change management.  Another part of my practice is mentoring IT executives in how to deal with business problems related to the business of law and I think that might be my next post – free advice to the aspiring CIO.

This might sound odd coming from a technologist, but…it’s not really about the technology.  From a broad standpoint, you can be successful with most software tools.  A law firm isn’t going to be made or broken whether it chose OpenText or iManage as a document management tool or chose a specific litigation support tool.  It is more about the people, the education and the process than it is the actual tool.  Yes, there are some horrible tools that you should avoid, but, all things being equal, it’s really more the other pieces of the equation that determine your success.

Thanks, Jeff, for participating in the interview!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References, Part 2
 

Yesterday, we started a list of questions to ask when you are doing a reference check.  That list included general project questions, questions about communicating with the vendor, and questions about costs. Here are some questions aimed at uncovering information on the quality of the vendor’s work:

  1. Were you satisfied with the quality of the vendor’s work and did it meet your expectations?
  2. Were you involved in initial project training of the vendor’s staff?  Do you recommend that with this vendor?
  3. Did you do quality control checks of the vendor’s work?  How did you check the work?  How much of the work did you check?
  4. Did you spend time checking work at the vendor’s facility?  Do you recommend that with this vendor?
  5. Did you provide feedback to the vendor on quality?  How was that received?  Did the vendor incorporate your feedback and did the work quality improve?
  6. If there were quality problems, were they dealt with quickly?  Was it necessary to escalate problems above the project manager to get them resolved?
  7. Do you know what quality control procedures the vendor used?  How much work did they check?  What quality control was done manually?  What automated checks were done?
  8. Do you know if the vendor needed to do rework?  Was rework precipitated by issues you brought to their attention?  Did you have to ask the vendor to do any work over?
  9. If you had the technology tools and resources to do this work yourself, do you think you could have done a better job?  If so, in what way?
  10. Do you feel that the vendor had a good grasp on your requirements and the experience to do the work efficiently?
  11. Did the project manager have the right background and experience to handle your project?

In the next issue, we’ll continue this list with questions aimed at getting a handle on a vendor’s ability to meet schedule requirements and some closing questions aimed at assessing overall performance.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

In observance of Good Friday, eDiscovery Daily will not publish a new post tomorrow.  See you Monday!

Working Successfully with eDiscovery and Litigation Support Service Providers: Questions to Ask when Checking References
 

Getting good information about a vendor in a reference check is a skill worth developing.  As a starting point, you should be armed with a good set of questions.  Don’t limit yourself to the list, though.  A good reference-check call will be interactive and answers to some questions will probably prompt additional questions.

Here’s a good starting point list that might apply to any type of vendor or service you are inquiring about:

First, get a handle on the scope of the project the vendor handled for the client:

  1. What services were provided?
  2. What volume of data / documents were handled?
  3. What deliverables did the vendor provide to you?
  4. What was the schedule?

Ask questions about the communication frequency and effectiveness between the vendor and the client:

  1. Were you the primary point of contact?
  2. If not, can you point me to the person who was?
  3. Did you have regular contact with the project manager?  With anyone else?
  4. Did you get regular status reports from the vendor?  How often did you get them?  What information was included in the status reports?  Were the reports useful?  Could you tell if the project was on schedule?
  5. How quickly did the vendor respond to phone calls?  Email?
  6. Did you ever have trouble reaching the vendor, or reaching the right person at the vendor organization?

Ask questions about the project costs:

  1. Were the costs what you expected?
  2. Were costs within the vendor’s estimates?
  3. Were the invoices clear and sufficiently detailed?  What information was included on the invoices?
  4. Were there any cost surprises?  Any cost line items on the invoices that you weren’t expecting?

Tomorrow we’ll continue this list with questions aimed at getting a handle on the quality of a vendor’s work.

Do you get good information when you check vendor references?  What questions do you ask?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.

CORRECTION: Announcing Thought Leader Interview with Jeffrey Brandt, Editor of Pinhawk Law Technology Daily Digest
 

As eDiscovery Daily has done in the past, we have periodically interviewed various thought leaders in eDiscovery and legal technology to provide insight as to trends in the industry for our readers to consider.  Recently, I was able to interview Jeffrey Brandt, Editor of the Pinhawk Law Technology Daily Digest and columnist for Legal IT Professionals.

With an educational background in computer science and mathematics from the University of Pittsburgh, Jeff has over twenty four years of experience in the field of legal automation working with various organizations in the United States, Canada, and the United Kingdom.  I believe you will find he has some unique insights to share with the readers of eDiscovery Daily regarding legal technology trends.

I had stated that Jeff's interview will be published this Friday, April 22, but forgot that this Friday is Good Friday (no blog post that day, sorry!).  So, Jeff's interview will be published next Monday, April 25 instead.  Mark your calendars!

What eDiscovery Professionals Can Learn from the Internet Gambling Crack Down
 

Many of you may have heard about the FBI cracking down on the three largest online gambling sites in the past few days, as the owners of those sites in the United States have been indicted and charged with bank fraud, money laundering and illegal gambling offenses and the sites have been essentially shut down in the US.  Restraining orders have been issued against more than 75 bank accounts in 14 countries used by the poker companies PokerStars, Full Tilt Poker and Absolute Poker.  Many US customers of these sites are now scrambling to try to get their funds out of the sites and finding it difficult to do so.

So what?  This is an eDiscovery blog, right?  What does an Internet gambling crack down have to do with eDiscovery?

PokerStars, Full Tilt Poker, Absolute Poker and other gambling sites are cloud-based, Software-as-a-Service (SaaS) applications.  Just like Amazon, Facebook, Twitter, eBay, YouTube and SalesForce.com, these sites provide an application via the web that enables its clients to receive a service.  In the case of Amazon, it’s the ability to purchase any number of products.  For Facebook, it’s the ability to share information with friends and family.  For these gambling sites, it’s the ability to play poker for money with anyone else in the world who has the same gambling itch that you do and a broadband connection.

The problem is: in the US, it’s illegal.  The Unlawful Internet Gambling Enforcement Act of 2006 prohibits gambling businesses from knowingly accepting payments related to a bet or wager that involves the use of the Internet and that is unlawful under any federal or state law.  So, these sites are hosted in other countries to attempt to skirt the law.

What many US customers of these sites are finding out is the same thing that eDiscovery professionals discover when they need to retrieve cloud-based data in response to a discovery request: it’s imperative to know where your data is stored.  It’s likely that many customers of these gambling sites knew that their funds were kept off-shore, while others may not have known this was the case.  Regardless, they’re now scrambling to get their data (i.e., funds) back — if they can.

Many organizations are “in the same boat” when it comes to their SaaS providers – it may be unclear where that data is being stored and it may be difficult to retrieve if it’s stored in a foreign country with a different set of laws.  It’s important to establish (in writing if possible) with the provider up front where the data will be stored and agree on procedures such as records management/destruction schedules so that you know where your data is stored and can get access to it when you need it.  Don’t gamble with your data.

So, what do you think?  Do you have organizational data in a SaaS-based solution?  Do you have a plan for getting that data when you need it?  Please share any comments you might have or if you’d like to know more about a particular topic.

Full disclosure: I work for Trial Solutions, which provides SaaS-based eDiscovery review applications FirstPass® (for first pass review) and OnDemand® (for linear review and production).  Our clients’ data is hosted in a secured Tier 4 Data Center in Houston, Texas, where Trial Solutions is headquartered.

eDiscovery Trends: 2011 eDiscovery Errors Survey
 

As noted in Legal IT Professionals on Friday, LDM Global on Friday announced the results of its 2011 eDiscovery Errors survey. The company asked a selection of industry professionals their views on which errors they experienced most often during the discovery process. Results were collected from across the USA, Europe and Australia.

According to Scott Merrick, LDM Global Marketing Director and survey author, “Our goal was to find out what the real, day to day issues and problems are around the discovery process.”  He also noted that “Of particular interest was the ongoing challenge of good communication. Technology has not solved that challenge and it remains at the forefront of where mistakes are made.”

The respondents of the survey were broken down into the following groups: Litigation Support Professionals 47%, Lawyers 30%, Paralegals 11%, IT Professionals 9% and Others 3%.  Geographically, the United States and Europe had 46% of the respondents each, with the remaining 8% of respondents coming from Australia.  LDM Global did not identify the total number of respondents to the survey.

For each question about errors, respondents were asked to classify the error as “frequently occurs”, “occasionally occurs”, “not very common” or “never occurs”.  Based on responses, the most common errors are:

  • Failure to Effectively Communicate across Teams: 50% of the respondents identified this error as one that frequently occurs
  • An Inadequate Data Retention Policy: 47% of the respondents identified this error as one that frequently occurs
  • Not Collecting all Pertinent Data: 41% of the respondents identified this error as one that frequently occurs
  • Failure to Perform Critical Quality Control (i.e., sampling): 40% of the respondents identified this error as one that frequently occurs
  • Badly Thought Out, or Badly Implemented, Policy: 40% of the respondents identified this error as one that frequently occurs

Perhaps one of the most surprising results is that only 14% of respondents identified Spoliation of evidence, or the inability to preserve relevant emails as an error that frequently occurs.  So, why are there so many cases in which sanctions have been issued for that very issue?  Interesting…

For complete survey results, go to LDMGlobal.com.

So, what do you think?  What are the most common eDiscovery errors that your organization has encountered?   Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Trends: Sedona Conference Database Principles
 

A few months ago, eDiscovery Daily posted about discovery of databases and how few legal teams understand database discovery and know how to handle it.  We provided a little pop quiz to test your knowledge of databases, with the answers here.

Last month, The Sedona Conference® Working Group on Electronic Document Retention & Production (WG1) published the Public Comment Version of The Sedona Conference® Database Principles – Addressing the Preservation & Production of Databases &Database Information in Civil Litigation to provide guidance and recommendations to both requesting and producing parties to simplify discovery of databases and information derived from databases.  You can download the publication here.

As noted in the Executive Overview of the publication, some of the issues that make database discovery so challenging include:

  • More enterprise-level information is being stored in searchable data repositories, rather than in discrete electronic files,
  • The diverse and complicated ways in which database information can be stored has made it difficult to develop universal “best-practice” approaches to requesting and producing information stored in databases,
  • Retention guidelines that make sense for archival databases (databases that add new information without deleting past records) rapidly break down when applied to transactional databases where much of the system’s data may be retained for a limited time – as short as thirty days or even thirty seconds.

The commentary is broken into three primary sections:

  • Section I: Introduction to databases and database theory,
  • Section II: Application of The Sedona Principles, designed for all forms of ESI, to discovery of databases,
  • Section III: Proposal of six new Principles that pertain specifically to databases with commentary to support the Working Group’s recommendations.  The principles are stated as follows:
    • Absent a specific showing of need or relevance, a requesting party is entitled only to database fields that contain relevant information, not the entire database in which the information resides or the underlying database application or database engine.
    • Due to differences in the way that information is stored or programmed into a database, not all information in a database may be equally accessible, and a party’s request for such information must be analyzed for relevance and proportionality.
    • Requesting and responding parties should use empirical information, such as that generated from test queries and pilot projects, to ascertain the burden to produce information stored in databases and to reach consensus on the scope of discovery.
    • A responding party must use reasonable measures to validate ESI collected from database systems to ensure completeness and accuracy of the data acquisition.
    • Verifying information that has been correctly exported from a larger database or repository is a separate analysis from establishing the accuracy, authenticity, or admissibility of the substantive information contained within the data.
    • The way in which a requesting party intends to use database information is an important factor in determining an appropriate format of production.

To submit a public comment, you can download a public comment form here, complete it and fax (yes, fax) it to The Sedona Conference® at 928-284-4240.  You can also email a general comment to them at tsc@sedona.net.

eDiscovery Daily will be delving into this document in more detail in future posts.  Stay tuned!

So, what do you think?  Do you have a need for guidelines for database discovery?   Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Case Law: Conclusion of Case Does Not Preclude Later Sanctions

In Green v. Blitz U.S.A., Inc., (E.D. Tex. Mar. 1, 2011), the defendant in a product liability action that had been settled over a year earlier was sanctioned for “blatant discovery abuses” prior to the settlement. Defendant was ordered to add $250,000 to its settlement with plaintiff, to provide a copy of the court’s order to every plaintiff in every lawsuit against defendant for the past two years or else forfeit an additional $500,000 “purging” sanction, and to include the order in its first responsive pleading in every lawsuit for the next five years in which defendant became involved.

Defendant, a manufacturer of gasoline containers, was named in several product liability lawsuits, including this case in which plaintiff alleged that her husband’s death was caused in part by the lack of a flame arrestor on defendant’s gas cans. The jury in plaintiff’s case returned a verdict for defendant after counsel for defendant argued that “science shows” that flame arrestors did not work. The case was settled after the jury verdict for an undisclosed amount, but two years later, counsel for plaintiff sought sanctions and to have the case reopened after learning in another case against defendant that while the gas can lawsuits were underway, defendant had been instructing its employees to destroy email.

The court described defendant’s failure to implement a litigation hold as gas can cases were filed. A single employee met with other employees to ask them to look for documents, but he did not have any electronic searches made for documents and he did not consult with defendant’s information technology department on how to retrieve electronic documents.

The court held that defendant willfully violated the discovery order in the case by not producing key documents such as a handwritten note indicating a desire to install flame arrestors on gas cans and an email noting that the technology for flame arrestors existed given the common use of flame arrestors in the marine industry. “Any competent electronic discovery effort would have located this email,” according to the court, through a key word search. Defendant’s employee in charge of discovery did not conduct a key word search and, despite acknowledging that he was as computer “illiterate as they get,” did not seek help from defendant’s information technology department, which was routinely sending out instructions to employees to delete email and rotating backup tapes every two weeks while the litigation was underway.

The court declined to reopen the case since it had been closed for a year. However, based on its knowledge of the confidential settlement of the parties, the court ordered defendant to pay plaintiff an additional $250,000 as a civil contempt sanction to match the minimum amount that the settlement would have been if plaintiff had been provided documents withheld by defendant. The court also ordered a “civil purging sanction” of $500,000 which defendant could avoid upon showing proof that a copy of the court’s decision had been provided to every plaintiff in a lawsuit against defendant for the past two years. The court added a requirement that defendant include a copy of the court’s opinion in its first pleading in any lawsuit for the next five years in which defendant became a party.

As Yogi Berra would say, “It ain’t over ‘til it’s over”.

So, what do you think?  Should cases be re-opened after they’re concluded for discovery violations?  Please share any comments you might have or if you’d like to know more about a particular topic.

Case Summary Source: Applied Discovery (free subscription required).  For eDiscovery news and best practices, check out the Applied Discovery Blog here.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

Working Successfully with eDiscovery and Litigation Support Service Providers: Checking References – General Suggestions
 

Yesterday, we talked about the importance of checking references when considering eDiscovery and Litigation Support Service Providers.  In the next blogs in this series, I’m going to suggest some questions you may consider asking when doing a reference check.  First, however, let me make some general suggestions:

  • When you ask a vendor for references, ask for clients that had a project similar in size and scope to yours.  You want to speak with people who had similar requirements regarding services and schedules, and that had document or data collections similar in size and format to yours.
  • When you ask a vendor for references, let them know that you’d like to speak with two different types of people:
    • End-users of the vendor’s work product
    • The vendor’s main, day-to-day point of contact
  • Don’t call a reference out of the blue.  Make an initial contact by email to introduce yourself and to schedule a call.  If the call is scheduled, you are more likely to get better attention and more time. 
  • Be prepared with a list of specific questions.  Don’t call and ask only general question like “Were you satisfied with the quality?” and “Did they meet your deadlines?”
  • Try to engage the person you are speaking with in conversation.  Don’t settle for yes and no answers.  When someone responds “yes” or “no” to a question, ask them to provide details.  This may uncover information that is important to you or it may trigger additional questions that you should ask.
  • Always send follow-up emails to the people with whom you speak thanking them for their time and information. 

Next we’ll talk about specific questions you can consider asking when checking references, so stay tuned.

How do you approach checking references?  Please share any comments you might have and let us know if you’d like to know more about an eDiscovery topic.