Close
Enter your
text here

Review

eDiscovery Trends: Myth of SaaS Insecurity Finally Busted

Eleven years ago, when I first began talking to attorneys about hosting document collections online to manage the review and production process for discovery, the typical response that I got was “I would never consider putting my client’s documents online – it’s just not secure”.  Let’s face it – lawyers are not exactly early adopters of technology… 😉

These days, few folks seem to have that concern any more when it comes to putting sensitive data and documents online.  Many people bank online, buy items from Amazon and other “etailers”, share pictures and other personal information on Facebook, etc.  As for business data, SalesForce.com has become the top customer relationship management (CRM) application and many business users are using Google Docs to share documents with colleagues, as just two examples.

What do all of these applications have in common?  They are Software as a Service (SaaS) applications, delivering data and functionality via an online application.  As noted previously on this blog, a new IDC study forecasts the SaaS market to reach $40.5 billion by 2014, an annual growth rate of 25.3%.  Also by 2014, about 34% of all new business software purchases will be via SaaS applications, according to IDC.

SaaS review applications have also become increasingly popular in eDiscovery with several eDiscovery SaaS applications available that provide benefits including: no software to install, intuitive browser-based interfaces and ability to share the collection with your client, experts, and co-counsel without distributing anything more than a login.

As for security concerns, most litigators have come to accept that these systems are secure.  But, do they realize just how secure they are?

As an example, at Trial Solutions, the servers hosting data for our OnDemand® and FirstPass™ (powered by Venio FPR™) platforms are housed in a Tier 4 data center in Houston (which is where our headquarters is).  The security at this data center is military grade: 24 x 7 x 365 onsite security guards (I feel sorry for the folks who have to work this Saturday!), video surveillance, biometric and card key security required just to get into the building.  Not to mention a building that features concrete bollards, steel lined walls, bulletproof glass, and barbed wire fencing.  And, if you’re even able to get into the building, you then have to find the right server (in the right locked room) and break into the server security.  It’s like the movie Mission Impossible where Tom Cruise has to break into the CIA, except for the laser beams over the air vent (anyone who watches movies knows those can be easily thwarted by putting mirrors over them).  To replicate that level of security infrastructure would be cost prohibitive for even most large companies.

From the outside, SaaS applications secure data with login authentication and Secured Sockets Layer (SSL) encryption.  SSL encryption is like taking a piece of paper with text on it, scrambling the letters on that piece of paper and then tearing it up into many pieces and throwing the scraps into the wind.  To intercept a communication (one request to the server), you have to intercept all of the packets of a communication, then unscramble each packet individually and then reassemble them in the correct order.

Conversely, desktop review application data could be one stolen laptop away from being compromised.  No wonder why nobody talks about security concerns anymore with SaaS applications.

So, what do you think?  How secure is your document collection?  Please share any comments you might have or if you’d like to know more about a particular topic.

Happy Holidays from all of us at Trial Solutions and eDiscovery Daily!

eDiscovery Trends: Predictive Coding Strategy and Survey Results

Yesterday, we introduced the Virtual LegalTech online educational session Frontiers of E-Discovery: What Lawyers Need to Know About “Predictive Coding” and defined predictive coding while also noting the two “learning” methods that most predictive coding mechanisms use to predict document classifications.  To get background information regarding the session, including information about the speakers (Jason Baron, Maura Grossman and Bennett Borden), click here.

The session also focused on strategies for using predictive coding and results of the TREC 2010 Legal Track Learning Task on the effectiveness of “Predictive Coding” technologies.  Strategies discussed by Bennett Borden include:

  • Understanding the technology used by a particular provider:  Not only will supervised and active learning mechanisms often yield different results, but there are differing technologies within each of these learning mechanisms.
  • Understand the state of the law regarding predictive coding technology: So far, there is no case law available regarding use of this technology and, while it may eventually be the future of document review, that has yet to be established.
  • Obtain buy-in by the requesting party to use predictive coding technology: It’s much easier when the requesting party has agreed to your proposed approach and that agreement is included in an order of the court which covers the approach and also includes a FRE 502 “clawback” agreement and order.  To have a chance to obtain that buy-in and agreement, you’ll need a diligent approach that includes “tiering” of the collection by probable responsiveness and appropriate sampling of each tier level.

Maura Grossman then described TREC 2010 Legal Track Learning Task on the effectiveness of “Predictive Coding” technologies.  The team took the EDRM Enron Version 2 Dataset of 1.3 million public domain files, deduped it down to 685,000+ unique files and 5.5 GB of uncompressed data.  The team also identified eight different hypothetical eDiscovery requests for the test.

Participating predictive coding technologies were then given a “seed set” of roughly 1,000 documents that had previously been identified by TREC as responsive or non-responsive to each of the requests. Using this information, participants were required to rank the documents in the larger collection from most likely to least likely to be responsive, and estimate the likelihood of responsiveness as a probability for each document.  The study ranked the participants on recall rate accuracy based on 30% of the collection retrieved (200,000 files) and also on the predicted recall to determine a prediction accuracy.

The results?  Actual recall rates for all eight discovery requests ranged widely among the tools from 85.1% actual recall down to 38.2% (on individual requests, the range was even wider – as much as 82% different between the high and the low).  The prediction accuracy rates for the tools also ranged somewhat widely, from a high of 95% to a low of 42%.

Based on this study, it is clear that these technologies can differ significantly on how effective and efficient they are at correctly ranking and categorizing remaining documents in the collection based on the exemplar “seed set” of documents.  So, it’s always important to conduct sampling of both machine coded and human coded documents for quality control in any project, with or without predictive coding (we sometimes forget that human coded documents can just as often be incorrectly coded!).

For more about the TREC 2010 Legal Track study, click here.  As noted yesterday, you can also check out a replay of the session or download the slides for the presentation at the Virtual LegalTech site.

Full Disclosure: Trial Solutions provides predictive coding services using Hot Neuron LLC’s Clustify™, which categorizes documents by looking for similar documents in the exemplar set that satisfy a user-specified criteria, such as a minimum conceptual similarity or near-duplicate percentage.

So, what do you think?  Have you used predictive coding on a case?  Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Trends: What the Heck is “Predictive Coding”?
 

Yesterday, ALM hosted another Virtual LegalTech online "live" day online.  Every quarter, theVirtual LegalTech site has a “live” day with educational sessions from 9 AM to 5 PM ET, most of which provide CLE credit in certain states (New York, California, Florida, and Illinois).

One of yesterday’s sessions was Frontiers of E-Discovery: What Lawyers Need to Know About “Predictive Coding”.  The speakers for this session were:

Jason Baron: Director of Litigation for the National Archives and Records Administration, a founding co-coordinator of the National Institute of Standards and Technology’s Text Retrieval Conference (“TREC”) legal track and co-chair and editor-in-chief for various working groups for The Sedona Conference®;

Maura Grossman: Counsel at Wachtell, Lipton, Rosen & Katz, co-chair of the eDiscovery Working Group advising the New York State Unified Court System and coordinator of the 2010 TREC legal track; and

Bennett Borden: co-chair of the e-Discovery and Information Governance Section at Williams Mullen and member of Working Group I of The Sedona Conference on Electronic Document Retention and Production, as well as the Cloud Computing Drafting Group.

This highly qualified panel discussed a number of topics related to predictive coding, including practical applications of predictive coding technologies and results of the TREC 2010 Legal Track Learning Task on the effectiveness of “Predictive Coding” technologies.

Before discussing the strategies for using predictive coding technologies and the results of the TREC study, it’s important to understand what predictive coding is.  The panel gave the best descriptive definition that I’ve seen yet for predictive coding, as follows:

“The use of machine learning technologies to categorize an entire collection of documents as responsive or non-responsive, based on human review of only a subset of the document collection. These technologies typically rank the documents from most to least likely to be responsive to a specific information request. This ranking can then be used to “cut” or partition the documents into one or more categories, such as potentially responsive or not, in need of further review or not, etc.”

The panel used an analogy for predictive coding by relating it to spam filters that review and classify email and learn based on previous classifications which emails can be considered “spam”.  Just as no spam filter perfectly classifies all emails as spam or legitimate, predictive coding does not perfectly identify all relevant documents.  However, they can “learn” to identify most of the relevant documents based on one of two “learning” methods:

  • Supervised Learning: a human chooses a set of “exemplar” documents that feed the system and enable it to rank the remaining documents in the collection based on their similarity to the exemplars (e.g., “more like this”);
  • Active Learning: the system chooses the exemplars on which human reviewers make relevancy determinations, then the system learns from those classifications to apply to the remaining documents in the collection.

Tomorrow, I “predict” we will get into the strategies and the results of the TREC study.  You can check out a replay of the session at theVirtual LegalTech site. You’ll need to register – it’s free – then login and go to the CLE Center Auditorium upon entering the site (which is up all year, not just on "live days").  Scroll down until you see this session and then click on “Attend Now” to view the replay presentation.  You can also go to the Resource Center at the site and download the slides for the presentation.

So, what do you think?  Do you have experience with predictive coding?  Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Trends: Some SaaS Benefits for eDiscovery

I found an interesting article on Ezine Articles by Sharon Gonzalez, a freelance technical writer with 15 years experience writing on various technical subjects, especially in the areas of cloud computing, Software as a Service (SaaS), and Internet technologies.  The article entitled EDiscovery on SaaS, discusses some of the benefits of SaaS solutions for eDiscovery.

Gonzalez notes that “use of [the] eDiscovery SaaS model which has brought down the costs of many organizations” because the “model is a vendor hosted infrastructure that is highly secured and the customers can run the applications from their own machines”.  Advantages noted by Gonzalez include:

  • Easy Manageable Services: Legal teams are able to process, analyze and review data files using the eDiscovery tools from the SaaS provider via their own browser and control and secure information within those tools.  No software to install.
  • No Problem for Storage Space: The SaaS model “eliminates all requirements of added infrastructure for…increasing storage space”.  While many eDiscovery SaaS models charge a monthly fee based on data stored, that fee is eliminated once the data is no longer needed.
  • Cost-Effective Solutions Provided: Gonzalez notes “Since…the SaaS architecture is maintained by vendors, IT departments are free from the burden of maintaining it. It is also a cost-effective method as it cuts down expenditure on hiring additional IT professionals and other physical components. The companies have to pay a charge to the vendors which work out far cheaper than investing large sums themselves”.
  • Built-In Disaster Recovery: Redundant storage, backup systems, backup power supplies, etc. are expensive to implement, but those mechanisms are a must for SaaS providers to provide their clients with the peace of mind that their data will be secure and accessible.  Because the SaaS provider is able to allocate the cost for those mechanisms across all of its clients, costs for each client are considerably less to provide that secure environment.

There are SaaS applications for eDiscovery throughout the EDRM life cycle from Information Management thru Presentation.

Full disclosure: Trial Solutions is the leader in self service, on demand SaaS litigation document review solutions, offering FirstPass™, powered by Venio FPR™, for early case assessment and first pass review as well as OnDemand™ for linear review and production.

So, what do you think?  Have you used any SaaS hosted solutions for eDiscovery?  Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Project Management: Review the Work

Yesterday, we talked about resolving questions quickly and keeping team members informed about changes to procedures to minimize the chance for significant rework.  However, even with the best staff, mistakes happen — especially on projects that require a team of people.  There are two general types of errors you can expect:

  • Errors that are made because someone doesn’t properly understand the task.  They have misunderstood the procedures or misinterpreted a subjective component of the work.
  • Errors that are made simply because it is inevitable.  People have bad days.  They get tired.  Knowing how to do something doesn’t mean you will do it right every time.

The first type of error is easy to identify and fix.  Check initial work quickly and give immediate feedback.  I always distribute small batches of initial work at the beginning of a task – work that an individual can finish quickly.  Then I have that first initial batch for each person checked thoroughly and right away.  Misinterpretations of the procedures or the criteria are evident and can be dealt with right away, before a lot of work has been done.

Catching the second type of error is a little more difficult unless your schedule and budget permit you to check 100% of the work.  With a good staff, that’s probably overkill.  But, it’s important that work is spot-checked throughout the life of a project, and that an intelligent approach is used to isolate problems.  For example, if you find a few careless errors made by a staff member, see if you can isolate all of the work that person did on that day and check it completely.  Or perhaps you’ll identify a particular type of document or situation that caused problems, and you can take steps to isolate just those documents or situations.  Very often you can apply a systematic approach to finding and fixing errors.

What do you think?  Have you worked on projects where quality control reviews were absent or inadequate and work quality suffered?  Please share your comments or let us know if you would like more information on a topic.

eDiscovery Project Management: Resolve Questions Quickly
 

Even with the best procedures and thorough training, people who are new to a task will likely have questions.   Nuances in a document collection and unexpected situations will surface that don’t fit into your rules.  It is very important that questions are resolved quickly.  Most document work is repetitive.  Many questions, therefore, will apply to more than one document.  If a question is not resolved quickly, there is a good chance that many documents will be affected, and you may face significant rework.   This is especially the case in projects that are being handled by a team of people – for example, a document review project.  So, handle exceptions and questions as they come up and expand and modify the rules to accommodate what you are finding in practice in the document collection.

Depending on the type of project you are managing, you may need to be prepared to answer two types of questions:

  • Questions about the mechanics of the task.  These types of questions are usually best handled by project managers and supervisors.
  • Questions about the substance of the task.  For example, in a document review project there are likely to be questions about the relevance of topics discussed in the documents.  These types of questions are usually best handled by an attorney who is familiar with the case and with the documents.

Make sure that you have the right people on hand to make decisions and answer questions.  If those people can’t be on the project site, make sure they are easily reached and readily available.

And, have a process in place for disseminating updated procedures and criteria to a team doing the work.  If one member of a team has a question, chances are other team members will encounter similar documents and have the same question.   You need to get information quickly into the hands of those doing the work.

What do you think?  Have you worked on projects that required rework because decision makers weren’t available?  Please share your comments or let us know if you’d like more information on a topic.

eDiscovery Searching: Types of Exception Files

Friday, we talked about how to address the handling of exception files through agreement with opposing counsel (typically, via the meet and confer) to manage costs and avoid the potential for spoliation claims.  There are different types of exception files that might be encountered in a typical ESI collection and it’s important to know how those files can be recovered.

Types of Exception Files

It’s important to note that efforts to “fix” these files will often also change the files (and the metadata associated with them), so it’s important to establish with opposing counsel what measures to address the exceptions are acceptable.  Some files may not be recoverable and you need to agree up front how far to go to attempt to recover them.

  • Corrupted Files: Files can become corrupted for a variety of reasons, from application failures to system crashes to computer viruses.  I recently had a case where 40% of the collection was contained in 2 corrupt Outlook PST files – fortunately, we were able to repair those files and recover the messages.  If you have readily accessible backups of the files, try to restore them from backup.  If not, you will need to try using a repair utility.  Outlook comes with a utility called SCANPST.EXE that scans and repairs PST and OST files, and there are utilities (including freeware utilities) available via the web for most file types.  If all else fails, you can hire a data recovery expert, but that can get very expensive.
  • Password Protected Files: Most collections usually contain at least some password protected files.  Files can require a password to enable them to be edited, or even just to view them.  As the most popular publication format, PDF files are often password protected from editing, but they can still be viewed to support review (though some search engines may fail to index them).  If a file is password protected, you can try to obtain the password from the custodian providing the file – if the custodian is unavailable or unable to remember the password, you can try a password cracking application, which will run through a series of character combinations to attempt to find the password.  Be patient, it takes time, and doesn’t always succeed.
  • Unsupported File Types: In most collections, there are some unusual file types that aren’t supported by the review application, such as files for legacy or specialized applications (e.g., AutoCad for engineering drawings).  You may not even initially know what type of files they are; if not, you can find out based on file extension by looking the file extension up in FILExt.  If your review application can’t read the files, it also can’t index the files for searching or display them for review.  If those files may be responsive to discovery requests, review them with the native application to determine their relevancy.
  • No-Text Files: Files with no searchable text aren’t really exceptions – they have to be accounted for, but they won’t be retrieved in searches, so it’s important to make sure they don’t “slip through the cracks”.  It’s common to perform Optical Character Recognition (OCR) on TIFF files and image-only PDF files, because they are common document formats.  Other types of no-text files, such as pictures in JPEG or PNG format, are usually not OCRed, unless there is an expectation that they will have significant text.

It’s important for review applications to be able to identify exception files, so that you know they won’t be retrieved in searches without additional processing.  FirstPass™, powered by Venio FPR™, is one example of an application that will flag those files during processing and enable you to search for those exceptions, so you can determine how to handle them.

So, what do you think?  Have you encountered other types of exceptions?  Please share any comments you might have or if you’d like to know more about a particular topic.

eDiscovery Searching: Exceptions are the Rule
 

Virtually every collection of electronically stored information (ESI) has at least some files that cannot be effectively searched.  Corrupt files, password protected files and other types of exception files are frequent components of your ESI collection and it can become very expensive to make these files searchable or reviewable.  Being without an effective plan for addressing these files could lead to problems – even spoliation claims – in your case.

How to Address Exception Files

The best way to develop a plan for addressing these files that is reasonable and cost-effective is to come to agreement with opposing counsel on how to handle them.  The prime opportunity to obtain this agreement is during the meet and confer with opposing counsel.  The meet and confer gives you the opportunity to agree on how to address the following:

  • Efforts Required to Make Unusable Files Usable: Corrupted and password protected files may be fairly easily addressed in some cases, whereas in others, it takes extreme (i.e., costly) efforts to fix those files (if they can be fixed at all).  Up-front agreement with the opposition helps you determine how far to go in your recovery efforts to keep those recovery costs manageable.
  • Exception Reporting: Because there will usually be some files for which recovery is unsuccessful (or not attempted, if agreed upon with the opposition), you need to agree on how those files will be reported, so that they are accounted for in the production.  The information on exception reports will vary depending on agreed upon format between parties, but should typically include: file name and path, source custodian and reason for the exception (e.g., the file was corrupt).

If your case is in a jurisdiction where a meet and confer is not required (such as state cases where the state has no rules for eDiscovery), it is still best to reach out to opposing counsel to agree on the handling of exception files to control costs for addressing those files and avoid potential spoliation claims.

On Monday, we will talk about the types of exception files and the options for addressing them.  Oh, the suspense!  Hang in there!

So, what do you think?  Have you been involved in any cases where the handling of exception files was disputed?  Please share any comments you might have or if you’d like to know more about a particular topic.

SaaS and eDiscovery: Load Your Own Data

Software as a Service (SaaS) applications hosted “in the cloud” are continuing to become more popular.  A new IDC study forecasts the SaaS market to reach $40.5 billion by 2014, an annual growth rate of 25.3%.  Also by 2014, about 34% of all new business software purchases will be via SaaS applications, according to IDC.

If you haven’t used a SaaS application, you haven’t used the Internet.  Amazon, Facebook, Twitter, eBay and YouTube are all examples of SaaS applications.  Ever shared a document via Google Docs with a colleague or business partner?  Use SalesForce.com for Customer Relationship Management (CRM)?  These are SaaS applications too.

Like any software application, SaaS applications are driven by data.  Many enable you to upload your own data to use and share via the Web.  Facebook and YouTube enable you to upload and share pictures and videos, Google Docs is designed for sharing and maintaining business documents, and even SalesForce.com allows you to upload contacts via a comma-separated values (CSV) file.

eDiscovery SaaS Applications

SaaS applications have also become increasingly popular in eDiscovery (especially for review and production of ESI) with several eDiscovery SaaS applications available that provide benefits including: no software to install, intuitive browser-based interfaces and ability to share the collection with your client, experts, and co-counsel without distributing anything more than a login.

However, most eDiscovery SaaS applications do not enable the user to upload their own data.  Or, if they do, it can be costly.

One exception is OnDemand™, which has now rolled out the new SelfLoader™ module in beta to enable clients to load their own data.  With SelfLoader, clients can load their own images, OCR text files, native files and metadata to an existing OnDemand database using an industry-standard load file (IPRO’s .lfp or Concordance’s .opt) format.

The best part?  You can load your data for free.  With SelfLoader, OnDemand provides full control to load your own data, add your own users and control their access rights.

Is this a start of a trend in eDiscovery?  Will more eDiscovery SaaS providers provide self-loading capabilities?  What do you think?  Please share any comments you might have or if you’d like to know more about a particular topic.

Thought Leader Q&A: Brad Jenkins of Trial Solutions
 

Tell me about your company and the products you represent. Trial Solutions is an electronic discovery software and services company in Houston, Texas that assists corporations and law firms in the collection, processing and review of electronic data. Trial Solutions developed OnDemand™, formerly known as ImageDepot™, an online e-discovery review application which is currently used by over fifty of the top 250 law firms including seven of the top ten.  Trial Solutions also offers FirstPass™, an early case assessment and first-pass review application.  Both applications are offered as a software-as-a-service (SaaS), where Trial Solutions licenses the applications to customers for use and provides access via the Internet. Trial Solutions provides litigation support services in over 90 metropolitan areas throughout the United States and Canada.

What do you see as emerging trends for eDiscovery SaaS solutions?  I believe that one emerging trend that you’ll see is simplified pricing.  Pricing for many eDiscovery SaaS solutions is too complex and difficult for clients to understand.  Many providers base pricing on a combination of collection size and number of users (among other factors) which is confusing and penalizes organizations for adding users into a case,  I believe that organizations will expect simpler pricing models from providers with the ability to add an unlimited number of users to each case.

Another trend I expect to see is provision of more self-service capabilities giving legal teams greater control over managing their own databases and cases.  Organizations need the ability to administer their own databases, add users and maintain their rights without having to rely on the hosting provider to provide these services.  A major self-service capability is the ability to load your own data on your schedule without having to pay load fees to the hosting provider.

Why do you think that more eDiscovery SaaS solutions don’t provide a free self loading capability?  I don’t know.  Many SaaS solutions outside of eDiscovery enable you to upload your own data to use and share via the Web.  Facebook and YouTube enable you to upload and share pictures and videos, Google Docs is designed for sharing and maintaining business documents, and even SalesForce.com allows you to upload contacts via a comma-separated values (CSV) file.  So, loading your own data is not a new concept for SaaS solutions.  OnDemand™ is about to roll out a new SelfLoader™ module to enable clients to load their own data, for free.  With SelfLoader, clients can load their own images, OCR text files, native files and metadata to an existing OnDemand database using an industry-standard load file (IPRO’s .lfp or Concordance’s .opt) format.

Are there any other trends that you see in the industry?  One clear trend is the rising popularity in first pass review/early case assessment (or, early data assessment, as some prefer) solutions like FirstPass as corporate data proliferates at an amazing pace.  According to International Data Corporation (IDC), the amount of digital information created, captured and replicated in the world as of 2006 was 161 exabytes or 161 billion gigabytes and that is expected to rise more than six-fold by 2010 (to 988 exabytes)!  That’s enough data for a stack of books from the sun to Pluto and back again!  With more data than ever to review, attorneys will have to turn to applications to enable them to quickly cull the data to a manageable level for review – it will simply be impossible to review the entire collection in a cost-efficient and timely manner.  It will also be important for there to be a seamless transition from first pass review for culling collections to attorney linear review for final determination of relevancy and privilege and Trial Solutions provides a fully integrated approach with FirstPass and OnDemand.

About Brad Jenkins
Brad Jenkins, President and CEO of Trial Solutions, has over 20 years of experience leading customer focused companies in the litigation support arena. Brad has authored many articles on litigation support issues, and has spoken before national audiences on document management practices and solutions.