Industry Trends

This is the eleventh (and final) of the 2018 Legaltech New York (LTNY) Thought Leader Interview series.  eDiscovery Daily interviewed several thought leaders at LTNY this year (and some afterward) to get their observations regarding trends at the show and generally within the eDiscovery industry.

Today’s thought leader is Chris Dale.  Chris is Editor of the eDisclosure Information Project.  Chris qualified as an English solicitor in 1980. He was a litigation partner in London and then a litigation software developer and litigation support consultant before turning to commentary on electronic disclosure / discovery. He runs the e-Disclosure Information Project which disseminates information about the court rules, the problems, and the technology to lawyers and their clients, to judges, and to suppliers. He was a member of Senior Master Whitaker’s Working Party which drafted Practice Direction 31B and the Electronic Documents Questionnaire. Chris is also a well-known speaker and commentator in the UK, the US and other common law jurisdictions.

{I spoke to Chris during LTNY and this is a rough transcript of our discussion}

Everybody over here in the US is talking about the General Data Protection Regulation (GDPR), what it will mean to American businesses and especially the potential of large fines for lack of compliance.  What are people saying about it in Europe?  Is this as big a deal as everyone is making it out to be?

I’m plotting with somebody to have a conference in London before the implementation date, at which we will not mention the GDPR in the marketing profile.  Whether we’ll get that through the marketing department or the education department I have yet to find out, but there’s no doubt that GDPR is driving a lot of attention, often for the wrong reasons.

You say for the wrong reasons? Why do you say that?

A lot of people are talking about the 4% fines as if that was the only driver which matters. There are lots of people who are talking about “citizens” but have not responded to my challenge find the word ”citizens” anywhere in the GDPR. There’s a lot of pig ignorance about precisely what it says and what its terminology is, let alone what its effect is likely to be. That’s quieting down and the shouters are beginning to shout a bit less about 4% fines. Of course, the fines have to be mentioned, because they are part of the bottom line, and companies like Facebook may well face the very big fines. As a motive for doing anything about GDPR, they should not be the most compelling one for most companies.  It would be good to see people taking a more rounded approach to what the implications are. I interviewed someone this morning who is very good, very knowledgeable about it and yet he (to my surprise) mentioned the 4% fine.  But when we discussed the fine, it was clear that he didn’t mean everybody is at a risk from that. It would be good to see people produce a business case for dealing with GDPR that doesn’t refer either to the 4% fine or to “citizens”, because there’s a lot of nonsense going on about it at the moment. There are an awful lot of people who act as experts on it, but whose first paragraph about it betrays the fact that they haven’t got a clue.  I saw an article promoting a service just recently and the first paragraph had a gross error in it.

So, I guess it’s an understatement to say there’s a lot of misconceptions about the GDPR?

Yes. One of the results of that, to some extent anyway, is that companies just throw their hands up in horror and say, “Well, I can’t comply by the due date, so I’ll just hide and pretend it’s going to go away.” That’s the result of hype and what happens when providers raise the stakes. We saw it in Zubulake and we saw it with the federal changes way back. People are thinking, “Well I can’t comply with it anyway, so why bother?” And that is not exactly the right attitude.

There are people who talk about the 72-hour deadline for breach notification as if that meant you have to do everything, produce every last bit of information to a regulator and possibly to the people affected within the 72 hours.  All this hype tends to make a lot of organizations say, “I can’t, I know I don’t comply with that anyway, I’ll just keep my head down, hope they hit somebody else.” Whereas there needs to be a more moderated approach to what needs to be done and what the implications are of not doing it, and a more positive look at what you gain from compliance.

My favorite quotation came from a chap called Patrick Burke whom you may know. He was in private practice advising on privacy and data protection, and specifically on the GDPR, and I asked him, “What’s your clients’ reactions, are they in fear of fines?” He said, “No, they just want to keep doing business.” Which is a really good line. Very quickly, the clients of the organizations who haven’t complied are probably going to start putting it into their RFP. They’ll be asking not just, “Have you complied?” but ”Can you indicate what you have done to be consistent with compliance with GDPR?” Those who have to say, “Oh, I don’t know what you’re talking about,” which I’m afraid includes quite a lot, will start losing business.  Possibly the companies who are asking that themselves won’t be compliant or know what it means, but it’ll become one of those tick box items like so many other things and the inability to give a satisfactory answer will lose business.

If you look at one of the companies in the UK that’s been fined under the present regime, they were fined 400,000 pounds, which sounds a lot of money until you look at what else they lost – £80 million in direct and indirect costs. It is said that they lost more than 6% of their market share, so you could multiply that £400,000 fine by roughly ten times under the new regime and you’re still not scratching the surface of the losses they’ve suffered overall, because they come across as the sort of company that doesn’t look after its customers’ data properly.

The conventional marketing of, “I know GDPR and I can help you through it,” doesn’t scratch the surface particularly if they start using terminology that doesn’t actually exist in it.  But you’re not offering expert services in guidance through the GDPR, you’re offering the ability to do specific things like to identify personal data and believing it is somebody else’s problem perhaps to decide what personal data is and what the risk profile is. Stick resolutely to the provision of services to meet whatever requirements are offered are sought, such as the ability to identify personal data and the ability to adapt, to show what data you’ve got in case you need to do so.

Perhaps that doesn’t matter as long as one gets business from it.  There’s certainly a lot of work coming out of it. Maybe we are at last finding the ROI for information governance that was missing on the first round through IG. Maybe people will begin to realize that if they get rid of their “crap”, they have less of a problem. That’s valuable.  The end result is less crap, or at least a better handle on their crap through data maps and things like that.  And knowing where it is from the moment of its creation and what in it might be offensive – knowing what ought to be deleted.

Or at least confront the decision. “Yes, I ought to delete that because the EU rules say that once it’s no longer serving the purpose for which it was collected it ought to be deleted.” Weighing that against, “Yes, but then I might be in trouble with a US regulator or court.”  It’s about making the informed decision that you’re keeping it or not keeping it, depending on which of those risks you see as the most important.  We will see gradually US courts acknowledging that there’s an EU requirement to delete data – if it contains personal information that’s no longer required for the purpose for which it was collected –and to acknowledge that that is a reason, an excuse if you like, for its non-availability.

That will take three to five years. For a period you’ll have judges who either don’t give a damn what everybody outside the US says or are too uninformed to understand what all this means with arguments put in front of them by lawyers who neither care nor understand what it all means.  But you’ll reach a point where I think the US courts will acknowledge that there are problems for those who keep EU data at the rate it that has been done most times in the EU. All that will take time and I hope there’ll be some examples. There’ll be some people who get some serious difficulties because of failure to comply. We don’t wish bad things for those people, but until we start seeing that we don’t actually know what target we’re aiming for. Regardless how well we might think we understand the statutes until we start seeing how regulators enforce them, we won’t know what to expect.  Of course, whether we get consistency between EU regulators (as is the hope) or whether in fact they all end up with different shades of interpretation will make a difference. It will take time and very interesting to see. That problem makes it sound even more daunting than it was.

In addition to GDPR, the Supreme Court decision in the Microsoft Ireland case will also have an impact of privacy rights for data subpoenaed by US law enforcement agencies.  What do you think will happen there and what do you think will be the impact?

I think it’s likely that Judge Francis’ original opinion is upheld by the Supreme Court. I think it will be upheld because the politics of it is not Supreme Court’s concern. It will be interesting to see what will happen when the Supreme Court says, “Yes, it’s absolutely fine for US agencies to dip their hands into data stores all over the world even if they don’t know that it’s a US citizen.” That’s a perfectly possible outcome.  What are you going to do then? What’s China going to do? There’s all these sort of political things, which as I say are not strictly the concern of the Supreme Court. What’s the backlash going to be?  Nobody knows.

Regardless of what the decision ultimately is, the CLOUD Act currently before Congress (to amend the Stored Communications Act to allow US federal law enforcement to compel U.S.-based service providers via warrant or subpoena to provide requested data stored on servers regardless of whether they are located within the U.S. or in foreign countries) could make the Supreme Court decision moot.  {Editor’s Note: The CLOUD Act was signed into law as part of the Omnibus Bill in March.}

What would you like our readers to know about things you’re working on?

We have some new civil discovery rules pending in England and Wales, and we have had some cases worthy of comment recently. The main thing is to keep writing – I’m getting 60 more page views per day this year than last year (that is, nearly 22,000 extra page views a year) which suggests a growing interest in this subject.

Part of that, perhaps, is down to the videos which we do and I am keen to make more use of this medium to get messages across, whether about the rules of England and Wales, the GDPR, or the interesting developments in discovery US and worldwide. They are very time-consuming to do properly but are well worth it.

Thanks, Chris, for participating in the interview!

As always, please share any comments you might have or if you’d like to know more about a particular topic!

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation.  Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer:  The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine.  eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance.  eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the tenth of the 2018 Legaltech New York (LTNY) Thought Leader Interview series.  eDiscovery Daily interviewed several thought leaders at LTNY this year (and some afterward) to get their observations regarding trends at the show and generally within the eDiscovery industry.

Today’s thought leader is Brad Jenkins.  Brad is CEO of CloudNine™.  Brad has over 20 years of experience as an entrepreneur, as well as 18 years leading customer focused companies in the litigation technology arena. Brad consults clients on implementing best practices in litigation document management and the impact of technology on managing discovery. Brad is very involved in eDiscovery educational efforts and currently serves as the President of the Houston ACEDS Chapter.  He’s also my boss!  :o)

What were your impressions of LTNY this year?

Personally, I was in meetings with clients, partners and analysts during most of the conference, so I don’t have a lot of observations to share.  It seemed as though a lot of people with whom I was meeting didn’t spend a lot of time at the conference either.  That seems to have become a trend – a lot of people come to New York for LTNY each year (from a lot of corporations, law firms and providers), but it seems that more and more of them are using that opportunity to set up meetings because LTNY provides that opportunity to meet in person.  More attendees seem to actually be spending their time at meetings at hotels near the Hilton instead of actually at the show in the Hilton.

From a CloudNine perspective, we had a booth in the exhibit hall and, from what I understand (and also observed when I stopped by the booth), traffic was generally good.  Though I’ve heard from others overall that traffic in the exhibit hall was down again this year and there were less providers again this year.  CloudNine also co-hosted the “Drinks with Doug and Mary” happy hour at Ruth’s Chris Steak house on Wednesday, along with ACEDS and our partner Compliance Discovery.  That was a huge success, as we had an overflow of people register for the event and a full house during the happy hour.  One of the best things about Legaltech is the opportunity to network with others in the industry and it’s still one of the best conferences for that.

If you were “king of LTNY” for a year, what changes would you make?

I would consider moving the conference to a larger venue – one that provided a large number of meeting rooms for the meetings that currently take place near the show at nearby hotels, so those meetings could actually be held right there at the show.  Assuming they charged a reasonable rate for the meeting rooms, Legaltech could then keep more of the attendees plugged into the show itself and those people could attend sessions and check out the providers in the exhibit hall much more easily.

CloudNine just recently acquired the eDiscovery product lines (Concordance, LAW PreDiscovery and Early Data Analyzer) from LexisNexis.  Why did CloudNine decide to acquire these products and what does CloudNine intend to do with them?

For CloudNine, the opportunity through this acquisition to offer a hybrid of both on-premise and off-premise solutions simply made sense to us as a way to support customer needs regardless of their eDiscovery task, security, and cost requirements.  This purchase enables us to immediately begin serving the on-premise segment with a portfolio of proven and performing products and it provides us the technology that will enable us to deliver fully integrated and automated solutions that can address off-premise requirements, on-premise requirements and combined on/off-premise (hybrid) requirements through a single provider.

CloudNine has been a user of the purchased product line offerings for more than a decade, so we understand what they can do today and the potential of what they can do in the future.  As an example, our production team uses LAW daily to support our clients’ processing and production needs.  Just as the daily use of our CloudNine SaaS platform enables us to uniquely understand how our customers use the product – because we are using it to accomplish the same tasks they are – our regular use of the acquired products provides that same level of unique understanding of how they are used as well.

As for our plans for the products, CloudNine plans to invest significant resources in the support, development, enhancement, and availability of these products.  They will be offered as solutions available as part of the CloudNine portfolio of offerings.  Our acquisition of these products includes the current development and support teams, so the customers using these products will be working with the same people that have been supporting their use of the products today, helping to ensure a smooth transition at the outset.  Our short-term plans include technology integration of our existing cloud offerings and our new on-premise offerings to support client needs by developing connectors using our Outpost technology to automate and accelerate data transfer.  CloudNine partners will also benefit from the purchase as it will give them additional access to proven eDiscovery offerings that will see that significant investment in development and support that I mentioned.

Our ability to acquire and invest in these products is facilitated by our new capital partners, Peak Rock Capital.  Peak Rock’s principals have extensive experience working with businesses in the technology and software industries and they have considerable experience with these types of transactions, so they will be integral to our plans to support and enhance the products.

We know that customers of these products likely have questions (as do others), so we prepared an FAQ document to help address as many of those common questions as possible.  We have already begun to reach out to existing customers for their feedback on the products and encourage them to reach out to us, as well, to enable us to understand what features and capabilities they would like to see added.  If you’re a recent or current customer of these products, we want to hear from you!

What would you like our readers to know about things you’re working on?

Isn’t that enough? {laughs}  Actually, one personal item to note is my involvement in starting a new chapter of the Association of Certified E-Discovery Specialists (ACEDS) in Houston.  I’m the current President of the Houston chapter and our chapter has been in existence for several months now.  The Houston ACEDS chapter just conducted a panel discussion earlier this month on Technology Assisted Review that was very well received and we look forward to future ACEDS events in Houston.

As for CloudNine, we’re obviously very excited about not only our recent acquisition, but about accomplishments at the company as a whole.  Earlier this year, we announced a new preservation and collection capability for CloudNine and, in just the past few days, CloudNine was highlighted in G2 Crowd’s Spring 2018 Grid Report for eDiscovery as one of the best eDiscovery software solutions based on customer satisfaction.  We’re very proud of the highly positive feedback we get from customers for our CloudNine platform and look forward extending that to our new product offerings.  We also continue to emphasize eDiscovery education through this blog and through our monthly webcasts on hot topics related to eDiscovery, information governance and cybersecurity.  All of these initiatives continue to further our mission of simplifying the discovery process for our customers and for others in the legal industry.  The more things change, the more our mission of simplifying discovery remains the same.

Thanks, Brad, for participating in the interview!

As always, please share any comments you might have or if you’d like to know more about a particular topic!

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation.  Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer:  The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine.  eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance.  eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the eighth of the 2018 Legaltech New York (LTNY) Thought Leader Interview series.  eDiscovery Daily interviewed several thought leaders at LTNY this year (and some afterward) to get their observations regarding trends at the show and generally within the eDiscovery industry.

Today’s thought leader is Maura R. Grossman.  Maura is a Research Professor in the David R. Cheriton School of Computer Science at the University of Waterloo and principal of Maura Grossman Law.  Previously, she was Of Counsel at Wachtell, Lipton, Rosen & Katz, where she pioneered the use of technology-assisted review (TAR) for electronic discovery.  Maura’s research with Gordon V. Cormack has been cited in cases of first impression in the United States, Ireland, and (by reference) in the United Kingdom and Australia, approving the use of TAR in civil litigation.  Maura has served as a special master in the Southern District of New York and the Northern District of Illinois to assist with issues involving search methodology.  In 2015 and 2016, Maura served as a coordinator of the Total Recall Track at the National Institute of Standards and Technology’s Text Retrieval Conference (TREC); in 2010 and 2011, she served as a coordinator of the TREC Legal Track.  Maura is also an Adjunct Professor at Osgoode Hall Law School at York University and at the Georgetown University Law Center.  Previously, she taught at Columbia Law School, Rutgers Law School—Newark, and Pace Law School.

I thought I’d start by asking about a couple of cases related to Technology-Assisted Review.  The first one that we talked about that I’d be interested in your thoughts about is FCA US v. Cummins where the judge (Avern Cohn) stated a position that applying TAR without prior keyword culling is the preferred method (although he said he did it rather reluctantly).  What are your thoughts about that decision?  What are your thoughts in general about courts making rulings on preferred TAR approaches?

I don’t believe there is a black or white answer to this question.  I think you have to consider how things might be done in a perfect world, and how they might be done in the real world we live in, which includes time, cost, and burden for ingesting and hosting large volumes of data.  In a perfect world, you wouldn’t perform multiple, sequential culling processes because each one removes potentially relevant information, and that effect is multiplied.  So, if you do keyword culling first—let’s say you do a good job and you get 75% of the relevant data, which would be pretty high for keywords—then, you apply TAR on top of that and, again, you get 75% of the relevant data.  For each of the steps you apply, those steps would sequentially reduce the number of relevant documents you found, resulting in a combined result of about 56% of the relevant data.

In a perfect world, you wouldn’t do that, and you would just put all the data into the TAR system and get 75% recall.  By “all” of the data, I don’t mean the entire enterprise, I mean all of the appropriate data—the appropriate time frame, the appropriate custodians, and so forth.  That’s in a perfect world, if you wanted to maximize your recall, and there were no time and cost considerations.  But, we don’t actually live in that world most of the time.  We live in a world where there generally are time and cost constraints, and per-gigabyte loading and hosting fees. So, often, parties feel the need to reduce the amount of data they are loading into a review platform to reduce processing and hosting fees, and other potential fees as well.  Therefore, parties often want to use keywords first in the real world.  Ultimately, you have to look at the specifics of the matter, including the volume of data, the value and importance of the case, and what’s at stake in the matter.

You also have to look at how effective keywords will be in retrieving information in the particular case at hand.  Keywords may work very well in a case where everything is referred to by its “widget number,” but keywords may not work as well in certain types of fraud cases where parties don’t always know the specific language used, or what the nature of the conspiracy was.  Keywords can be more challenging in those situations.  So, you really have to look at the whole picture.

In FCA v. Cummins, I think what the judge was trying to say was that generally, in a world without any of these other considerations, the best practice would be not to cull first.  I would tend to agree with that from a scientific and technical perspective.  But, that’s not always practical.

Also, I believe it was in Rio Tinto v. Vale where Judge Peck said (in dicta) that in a perfect world, if there were no cost or time or any other considerations, you would take all the data and you would just use the best TAR system you could, and you would be more likely to find the highest number of relevant documents.  But that can drive up costs, and that may not be not proportionate in all cases.  So, it’s really a question of proportionality, and what will work best in each situation, and how much time and resources you have to iterate and test the keywords, and other related factors.

Also, as you know in FCA v. Cummins, the judge didn’t really go into much detail; it was a very short decision.  Maybe it was a relatively small data set and loading it all didn’t make much of a difference.  We just don’t know enough about the facts there.

I got the impression that this case might have involved two equally weighted parties, with equal amounts of data, so the judge may have felt that the parties needed to perform TAR the same way, so he felt he was forced to make a decision.  Do you think that would have an impact as to why a court might decide or not?

I think that where the data volumes (and therefore burdens) are symmetric, there tends to be an understanding that what’s good for the goose is good for the gander.  Parties in those circumstances tend to be more circumspect about what they demand because they know they’ll be subject to the same thing in return.  If I’m representing one party and you’re representing the other, and I ask for everything in native form, I’m probably not going to be able to turn around and argue that I don’t want to produce in native, too, unless I have an awfully good reason for that.

So, I do think that changes the landscape a little bit.  Parties tend not to ask the other side for things that are unduly burdensome if they’re going to be forced to provide those same things themselves.  It can be very different when one side is using TAR and the other side isn’t, and when motivations or incentives are not aligned.  That can affect what parties request.

Another case we talked about was Winfield v. City of New York, and one of the key aspects of the objections by the plaintiffs about the defendants’ TAR process was the process of how they had been designating documents as non-responsive.  What are your thoughts about that?  Do you think arguments that the subjectivity of the subject matter experts will come into play in more and lead to objections in other cases?

Most of the research that I’ve reviewed and most research that I’ve done with Gordon Cormack has suggested that a few documents coded one way or the other are highly unlikely to make a substantial difference in the outcome—at least for CAL algorithms, and even for most robust SAL algorithms.  I know that people like to say, “garbage in, garbage out,” but I’ve never seen any evidence for the proposition that TAR compounds errors made by reviewers, and there is some evidence that TAR can mitigate reviewers’ errors.  The results of most TAR systems appear to be satisfactory unless there are pretty significant numbers of miscoded documents, for example, in the 20 to 25 percent (or higher) range.  Of course, if you’ve coded all of the documents of a particular kind as “not relevant,” you’ve now taught the algorithm not to find anything like that.  Chances are, though, if you have multiple reviewers, whether contract attorneys or even junior associates, not everything will be marked the same way.  There’s going to be a fair amount of noise in the coding decisions, and that doesn’t seem to have a major impact on the results.

With some of the early TAR systems, a lot of commentators said that it had to be a senior partner, or an expert, who trained the system.  But, most of the research that we’ve done, that Jeremy Pickens at Catalyst has done, and that others have done, suggests that a little noise from junior reviewers or contract attorneys, who may be a bit more generous in their definition of relevance, actually yields a better algorithm than a senior partner, who may have a very, narrow view of what’s relevant.  The junior people, who are more generous in their conceptions of relevance, tend to train a better algorithm—meaning a system that will achieve higher recall—in the long run.  So, a little bit of noise actually doesn’t hurt.

I wasn’t particularly surprised that in Winfield there were a few documents that were “arguably relevant,” and about which the two sides disagreed on coding.  That’s going to happen in any matter, and that’s not really going to affect the outcome one way or the other, because those documents are marginal in the first place.  Certainly, if someone is systematically marking all the “hot” documents as non-responsive, that will make a difference, but that wasn’t what was going on there.

In Winfield, the Court said, the documents were marginal, and arguably were relevant.  The Court also said it had reviewed the training process in camera and there was nothing wrong with it.  Most of the case law says that a party shouldn’t get discovery-on-discovery unless there’s a material flaw in the adversary’s process.  If you look at the position taken in the Defense of Process paper that The Sedona Conference published, then that should probably have been the end of the discussion.  But, the judge in Winfield went one step further, and said, “Well, because there’s some evidence that there may have been some disagreements in coding, I’ll permit a sample.”  That’s a little scary to me, because if there was no material deficiency, and we’re talking about a few marginal documents with coding where people disagree, that’s going to occur in every single case.

When you open up the collection to sampling, what happens is that the parties will find more marginal documents where they disagree on the coding.  That often leads to a lot of sparring, as we’ve seen in other cases where the parties disagree about marginal documents and fight about them, and that just drives up cost.  In the long run, those are not the documents that make or break the case.

We’re at a point where more and more people are using TAR, but a lot of people still haven’t really embraced TAR yet.  For those people who have not really gotten started with it, what would be your advice on how they could best get started on learning and applying TAR in their cases?

I would suggest they play with it, and try it out on a set of data that they have already completely manually reviewed and thoroughly QC’d, and where they are confident that everything was well done.  Use that to test some of the different tools out there before you have a live matter you want to use it on, so that you don’t have to decide what tools work and don’t work while you are in a crisis mode, when time is of the essence.

It would be helpful to do that homework, and develop a good understanding of the different work flows, and the different tools, and what kinds of data they work better or worse on.  For example, some are better with spreadsheets than others, some are better with OCR text, and others are better with foreign language or short documents.  Ideally, counsel would do that homework beforehand, and know something about the different tools that are available and their pros and cons.

If they haven’t done that, or feel they can’t, then I would encourage people to use it in ways that don’t impact defensibility considerations as much.  For example, they can use it on an internal investigation, or on incoming data, or simply to prioritize their data for review—even if they plan to review it all—so that they can start reviewing the most-likely responsive documents first and work down from there.

There are also many uses for QC, where the algorithm suggests that there may be errors.  Look at the documents that the TAR system gave a high score for relevance that the reviewers coded as “not relevant,” or vice versa.  There are many uses that don’t implicate defensibility where people can still try TAR, see how it works, and get comfortable with it.  Usually after people see how it works and see that it’s effective—if they’re using a tool that actually is effective—it’s not a hard sell after that.  It’s that first step that’s the hardest, and that’s why I encourage people to do the testing before they’re in a critical situation, before they have to go in front of the court and argue whether they can use it, or not use it.

What would you like our readers to know about things you’re doing, and what you’re working on?

I continue to do research on TAR tools and processes, and on the evaluation of TAR methods.  Gordon Cormack and I are “heads down” doing a lot of work on those things.  One area that we’ve been addressing recently is the notion that some people have been saying that a CAL process can only be used if you’re actually going to put eyes on every document.  Because of that, some people prefer the SAL approach because it can give them a fixed review set.  There is a method we’ve written about, and for which we’ve filed a patent, called S-CAL.  We’ve been doing a lot more work in that area to help parties get the benefits of CAL, but still be able to have the predictability they want of knowing exactly how many documents they’re going to have to review, so they can know how many reviewers they need, how long the review will take, and what it will cost.  Our aim is to be able to do that using a form of CAL, but also to be able to provide an accurate estimate of recall and precision.

That’s one area of research we’re working on.  I’m also becoming increasingly interested in artificial intelligence and the legal, ethical, and policy issues it implicates.  Last semester, I taught the first course (that I’m aware of) that brought together 18 computer science graduate students, and 15 law students, to explore different areas of artificial intelligence and the legal, ethical, and policy issues associated with them.  For example, we looked at autonomous cars, we looked at autonomous weapons, we looked at relationships with robots, and we looked at what to do about job loss.  We looked at data privacy, and the concentration of vast amounts of personal data in the hands of a small number of private companies.  We looked at predictive policing and use of algorithms to predict recidivism in the criminal justice system, and it was a really, really interesting experience, bringing both of those groups together to do that.  I’ve been focused a little more in that area, as well as continuing my information retrieval research and other research in collaboration with Gordon and my other colleagues and students at the University of Waterloo.  And, of course, Gordon and I work on TAR matters.  I still do consulting, expert work, and serve as a special master, and I really love that part of my job.

Thanks, Maura, for participating in the interview!

As always, please share any comments you might have or if you’d like to know more about a particular topic!

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation.  Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer:  The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine.  eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance.  eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

The University of Florida E-Discovery Conference is being held today!  And, for the first time, I’m going to be there!  Regardless of where you are, it’s not too late to attend!

The focus of this year’s conference is effectively managing the everyday case and they will have interesting sessions throughout the day, covering topics ranging from eDiscovery security and data protection to early assessment of the case and the data to keywords, TAR and AI (do I need to spell out those acronyms anymore?).  Want to know about eDiscovery of the JFK files?  It’s here.  Want to get judges’ perspectives on sanctions and other eDiscovery issues?  That’s here too.

The panel of speakers is a regular who’s who in eDiscovery, including Craig Ball, George Socha, Kelly Twigger, David Horrigan, Martin Audet, Mary Mack, Rose Jones, Mike Quartararo and also US Magistrate Judges John Facciola, James Francis, Judges William Matthewman, Mac McCoy, Amanda Arnold Sansone and Gary Jones, and retired Florida Circuit Court Judge Ralph Artigliere.

I’m on a panel discussion at 9am ET in a session titled Getting Critical Information From The Tough Locations – Cloud, IOT, Social Media, And Smartphones! with Craig, Kelly, with Judge Sansone.  We’ll be discussing real solutions for collecting ESI from those difficult locations.  Check it out!

The conference is being conducted in Gainesville, FL on the University of Florida Levin College of Law campus, though I understand it’s a full house.  However, it’s also being livestreamed.  There are CLE-accredited sessions all day from 8am to 5:30pm ET and the conference has been approved for 7.5 Continuing Legal Education (CLE) general credits, 2.0 ethics credits and 3.0 technology credits by the Florida Bar for attorneys attending the conference. The Florida Bar has also approved 7.5 civil trial certification credits.    So, this is a great opportunity to get those needed CLE credits!

Also yesterday, E-Discovery CareerFest was conducted.  And, for the first time, the Law School E-Discovery Core Curriculum Consortium (composed of law professors teaching electronic discovery courses at their respective law schools) will host its first in person workshop focusing on curriculum development on Friday, March 30th from 9am to 12pm ET.

Click here to register for the conference – it’s only $99 for livestream attendance.  And, if you’re a currently enrolled student (in an ABA accredited law school, accredited E-Discovery graduate program or accredited paralegal program), it’s free(!), either in person or livestreamed.  It’s also free if you’re university or college faculty, professional staff, judicial officials, clerks and employees of government bodies and agencies, it’s free(!) for you too.  Come check it out!

So, what do you think?  Are you going to attend the conference next month?  If not, why not?  Please share any comments you might have or if you’d like to know more about a particular topic.

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the fourth of the 2018 Legaltech New York (LTNY) Thought Leader Interview series.  eDiscovery Daily interviewed several thought leaders at LTNY this year (and some afterward) to get their observations regarding trends at the show and generally within the eDiscovery industry.

But first, this week’s eDiscovery Tech Tip of the Week is about The Importance of Metadata.  Metadata is key to the management, tracking and retrieval of documents within the discovery process.  Whether you wish to locate a document sent or received by a particular individual, filter a collection by a relevant date range or locate documents that have been marked as responsive or privileged, the ability to be able to search both the system metadata (that metadata that is extracted from the document when it is processed for review) and user metadata (that metadata used by legal professionals to categorize and track documents during the review and production steps in discovery) is key to being able to manage that document collection efficiently and effectively.  If you don’t think metadata is important to a case, here are 10.8 million reasons why it is.  :o)

To see an example of how Searching for Metadata is conducted using our CloudNine platform, click here (requires BrightTalk account, which is free).

Today’s thought leader is George Socha of BDO.  George is Managing Director at BDO.  Co-founder of EDRM and named an “E-Discovery Trailblazer” by The American Lawyer, George has nearly 30 years’ experience assisting a broad range of organizations with all facets of electronic discovery as well as information governance, domestically and globally. Prior to joining BDO, George spent 16 years as a litigation attorney in private practice and then 13 years running his own consulting firm. He received his law degree from Cornell Law School and his undergraduate degree from the University of Wisconsin-Madison.

What were your general observations about LTNY this year?

Although I have not had the chance to attend much of the show myself, I have been asking others what they think.  The responses have been completely contradictory.  Long-time attendees have said attendance is way down and, for some, the program seems somewhat flat.  Newer attendees find a lot to like, appreciating the programming and the opportunities to mingle.  For me, the conference provides an opportunity to meet in person with colleagues and clients.

With regards to the industry in general, what do you think are some of the hot topics people are talking about?  And, what do you think are some of the hot topics that people should be talking about?

I think that we continue to see a scenario where far too few attorneys have a solid understanding of what they could do and should do with respect to electronic discovery – and even fewer actually are moving forward on those fronts. Even after all these years, it seems the basics don’t get the attention they deserve.

While we spend a lot of time at conferences like this one talking about advanced (and worthy) topics such as predictive coding, those aren’t the commonplace topics of discussion when it comes to actual matters. I continue to encounter folks in law firms and corporate legal departments who are just now are beginning to grapple with electronic discovery; they have never had to deal with it before. I continue to encounter practicing lawyers who have heard the phrase electronic discovery but don’t appreciate what it signifies, aren’t familiar with even the best known players in our space and have never heard of e-discovery-focused organizations such as EDRM and The Sedona Conference.

By now, if you are a litigator, you should know about the EDRM diagram. You should know about the Sedona Principles. You should know about the two sets of FRCP changes in 2006 and 2015. That we encounter folks who aren’t familiar with any of those tells us something about the actual state of the industry.

There has been a lot of increased focus on data privacy this year with GDPR coming into effect.  Where do you think organizations stand with regards to getting compliant with GDPR?  Do you expect to see a lot of organizations scrambling?

From what I’ve seen, there is a lot of uncertainly about what constitutes compliance with GDPR. And, of course, if it’s unclear what constitutes compliance with GDPR, it’s rather hard to figure out how one becomes compliant. I would say that even at this late date, larger corporations are much more focused on GDPR than smaller organizations and law firms.

In larger law firms, there are people who have been spending a fair amount of time looking at GDPR, figuring out what compliance means, and developing and delivering services to help clients be ready for the regulation’s effective date. And there are certainly corporations – mostly larger ones – that also have been spending a fair amount of time and effort and money, taking on the same issues. But, when you look at the larger body of businesses and law firms, GDPR is not exactly front and center for them.

EDRM had a major focus on putting together a guide for technology assisted review last year. Where does that stand and what are you hoping will be the end result of that effort?

Work is underway and progressing well. The guide is divided up into several portions and we’ve got drafts of each portion. Those continue to be developed and reviewed. They are going to be distributed first to the larger EDRM membership for review, just as we’ve done in the past, then released for public comment.  {Editor’s Note: EDRM has since announced that the draft guidelines will be released for EDRM member review in March and a final draft will then be released for public review later this spring.}

I think ultimately we are looking at two sets of materials. The first one will be a nuts and bolts, “how do you do this stuff”, set of guidelines for the people who are in the trenches, mostly people in law firms and corporations. The second set of materials will focus on guidelines for the judiciary to help educate and guide judges through at least the major issues that we see will associated with predictive coding.

When you talk about the nuts and bolts, are we talking a preferred approach or are we just talking a general idea of the things you need to look for in conducting a predictive coding project?

We are talking a framework so that people have a construct to use and a context in which to place all of this. This won’t be so much about specific recommendations as there are too many and those are too tightly tied to specific tools, methodologies, and data seta; just consider the various approaches to predictive coding used today: TAR 1, TAR 2, systems with seed sets, systems with continuous active learning, and so on. We have to be careful to avoid putting out a set of guidelines that box people into the use of only one approach.

In addition, this is a rapidly changing area. We want something that will stand up over time rather than become obsolete as soon as the next big change happens.

And the tools are changing – rapidly. There are things we can now do with predictive coding that just weren’t viable options three years or even two years ago.

If you were “king of LTNY” for a year, what would you do in terms of structuring the curriculum to try to address some of the gaps in eDiscovery knowledge among the legal profession you mentioned?

Not surprisingly, I would look to focus on the practicalities of e-discovery: who can do what, how, and with what tools and techniques, to accomplish common and developing objectives. I would try to provide both basic, medium and advanced level content. To the extent practical and not too difficult, I also would try to do provide sessions that are more hands on.  Of course, this all is a pretty tall order, so I might also want to be a king with limitless time and resources.

I’d try to make the topics narrow enough so that people could get into depth on things and try to offer a wide range. By the way, I would not by any means focus only on electronic discovery, there’s a lot else out there.  Security is a major concern, and so is privacy. There should be a strong focus on that because law firms continue to be areas of risk.  Similarly, a focus on getting your electronic house in order through information governance is very important. And then, let’s see what other areas to cover as other issues emerge.  Again, a pretty tall order.

What would you like our readers to know about what you’re doing and what EDRM’s doing?

As we discussed earlier, EDRM is currently is working on the guidelines of technology assisted review/predictive coding as well as working on a GDPR project. These are the first two undertakings for the “new” EDRM – the EDRM under new management within Duke Law. We should look forward to some new initiatives in the coming year, a broadened scope of activities because EDRM is now part of Duke and, as a result, has a breadth of capabilities that weren’t there before.

Thanks, George, for participating in the interview!

Also, we’re getting ever closer to the University of Florida E-Discovery Conference, which will be held on Thursday, March 29.  As always, the conference will be conducted in Gainesville, FL on the University of Florida Levin College of Law campus (as well as being livestreamed), with CLE-accredited sessions all day from 8am to 5:30pm ET.  I (Doug) am on a panel discussion at 9am ET in a session titled Getting Critical Information From The Tough Locations – Cloud, IOT, Social Media, And Smartphones! with Craig Ball, Kelly Twigger, with Judge Amanda Arnold Sansone.  Click here to register for the conference – it’s only $199 for the entire day in person and only $99 for livestream attendance.  Don’t miss it!

As always, please share any comments you might have or if you’d like to know more about a particular topic!

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the third of the 2018 Legaltech New York (LTNY) Thought Leader Interview series.  eDiscovery Daily interviewed several thought leaders at LTNY this year (and some afterward) to get their observations regarding trends at the show and generally within the eDiscovery industry.

Today’s thought leader is Brett Burney.  Brett is Principal of Burney Consultants LLC, and focuses the bulk of his time on bridging the chasm between the legal and technology frontiers of electronic discovery. Brett is also very active in the Mac-using lawyer community, working with lawyers who want to integrate Macs, iPhones & iPads into their practice.  Prior to establishing Burney Consultants LLC, Brett spent over 5 years at the law firm of Thompson Hine LLP where he worked with litigation teams in building document databases, counseling on electronic discovery issues, and supporting their technology at trial & hearings.  Brett graduated from the University of Dayton School of Law in 2000 and quickly became active in the world of legal technology. Brett speaks around the country on litigation support, eDiscovery, Mac, and iOS-related topics and today authors a variety of blogs and online courses for legal professionals.  Brett also served as the Chair of the ABA TECHSHOW 2015 Conference Planning Board and recently published the book “Macs in Law” for the American Bar Association. Brett is also the co-author the 2018 eDiscovery Buyers Guide.

What were your general observations about LTNY this year?

It seemed that the 2018 Legaltech/Legalweek Conference had a distinct flavor of maturity this year. The Exhibit Hall was a bit thinner and less wacky than previous years, and the pulse was more focused and business-like.

On the “future is coming soon so you better be ready” side, there were multiple warnings about GDPR and AI, with some Blockchain sprinkled throughout.  On the more practical side were continuing discussions on how to collect and monitor social media and IoT sources.

Overall, this year’s conference appeared to deliver what attendees were seeking in education and information.

If you were “king of LTNY” for a year, what kind of changes would you make?

While I fully understand the push last year to make this a “Legalweek” instead of just “Legaltech,” it’s still very confusing with all kinds of sessions and mini-shows happening at the same time. So I would simplify things as much as possible.

Second, I would try to reach out to small and mid-size firms outside of the New York area.

You and Chelsey with Lex Tech Review recently published a buyer’s guide to eDiscovery solutions. Why did you decide to publish the guide, who do you anticipate using the guide and how do you anticipate them using it?

That’s easy! We simply had a passion to provide a comprehensive set of eDiscovery resources for a section of the market that is routinely overlooked – small and mid-size law firms.

For some reason, most lawyers think that “eDiscovery” is only something that huge law firms have to deal with, and that if you’re a small firm you don’t have to worry about it … so they don’t. But just because you’re in a small firm, doesn’t mean you don’t handle large, important litigation matters. And if you’re involved in litigation today, then you are absolutely involved in eDiscovery.

We have two goals for the eDiscovery Buyers Guide: First to serve as a literal buyers guide to help practitioners find the products they need to use, and second to serve as a resource to help lawyers be more competent about products available for their clients and to understand the products being used by opposing parties.

As an eDiscovery consultant, you work with a lot of clients.  What are the “hot” topics and issues that you find your clients are most consistently encountering?

Honestly, while I love talking out AI and predictive coding and statistical algorithms as much as next eDiscovery tech nerd, the vast majority of my clients are just trying to get their heads wrapped around the day-to-day practical aspects of eDiscovery.  The “hottest” topic I get asked about today is how to confidently and defensibly collect social media content from Facebook, LinkedIn, Twitter, etc. Email is still the primary source of potentially relevant electronic evidence today, but it’s unbelievable what folks will post in social media.

In a similar vein, not a week goes by that I don’t have a client asking how they can collect text messages from phones and mobile devices.  Beyond that, many of my clients are just seeking to learn about the best tools and products that they can use to get through an overwhelming number of emails, documents, and electronic files … which is one of the reasons we developed the eDiscovery Buyers Guide.

If you could provide legal practitioners out there with one or two key pieces of advice about better eDiscovery practices, what would you tell them?

Keep it simple and practical, but that doesn’t mean you can be ignorant about the tools you need to use as a litigator in today’s world. For example, I still find so many lawyers using their own Microsoft Outlook to review email collected from a client. And that just completely baffles me that they are co-mingling live electronic evidence with their own work product in the same software! That’s negligent and ripe with risk. If you’re a litigator today, you need to use legitimate document review tools that preserve the evidence, allow you to code, tag, & redact documents, and then produce them in the form or forms in which the opposing party requests.

Second I continue to encourage practitioners to just talk with the other side about eDiscovery processes and procedures. Sure I understand the “strategy” behind the reasoning to keep the opposing party in the dark for as long as possible, but this legacy approach to discovery doesn’t help you or your client when it involves electronic evidence. Cooperation is a dirty word for most lawyers, but it’s absolutely critical for swift and successful eDiscovery resolutions.

What would you like our readers to know about things you’re working on?

Thanks for asking! On the non-e-Discovery side, I offer a fantastic online course for lawyers that want to incorporate iPads into their practice at www.ipracticeonanipad.com. I also host a blog reviewing apps at www.appsinlaw.com.

My latest course is called TextExpander for Lawyers where you’ll learn about the most brilliant software utility that can type FOR you and save you time in your day.

Lastly, I encourage every reader to download the FREE eDiscovery Buyers Guide that features CloudNine amongst many other platforms and products. The website also has helpful blog posts and videos.

Thanks, Brett, for participating in the interview!

Also, we’re getting ever closer to the University of Florida E-Discovery Conference, which will be held on Thursday, March 29.  As always, the conference will be conducted in Gainesville, FL on the University of Florida Levin College of Law campus (as well as being livestreamed), with CLE-accredited sessions all day from 8am to 5:30pm ET.  I (Doug) am on a panel discussion at 9am ET in a session titled Getting Critical Information From The Tough Locations – Cloud, IOT, Social Media, And Smartphones! with Craig Ball, Kelly Twigger, with Judge Amanda Arnold Sansone.  Click here to register for the conference – it’s only $199 for the entire day in person and only $99 for livestream attendance.  Don’t miss it!

As always, please share any comments you might have or if you’d like to know more about a particular topic!

Sponsor: This blog is sponsored by CloudNine, which is a data and legal discovery technology company with proven expertise in simplifying and automating the discovery of data for audits, investigations, and litigation. Used by legal and business customers worldwide including more than 50 of the top 250 Am Law firms and many of the world’s leading corporations, CloudNine’s eDiscovery automation software and services help customers gain insight and intelligence on electronic data.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.