Processing

This is the tenth (and final) of the 2013 LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and generally asked each of them the following questions:

1. What are your general observations about LTNY this year and how it fits into emerging trends?
2. If last year’s “next big thing” was the emergence of predictive coding, what do you feel is this year’s “next big thing”?
3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Craig Ball.  A frequent court appointed special master in electronic evidence, Craig is a prolific contributor to continuing legal and professional education programs throughout the United States, having delivered over 1,000 presentations and papers.  Craig’s articles on forensic technology and electronic discovery frequently appear in the national media, and he writes a monthly column on computer forensics and eDiscovery for Law Technology News called Ball in your Court, as well as blogs on those topics at ballinyourcourt.com.

Craig was very generous with his time again this year and our interview with Craig had so much good information in it, we couldn’t fit it all into a single post.  Wednesday was part 1 and yesterday was part 2.  Today is the third and last part.  A three-parter!

Note: I asked Craig the questions in a different order and, since the show had not started yet when I interviewed him, instead asked about the sessions in which he was speaking.

What are you working on that you’d like our readers to know about?

I’m really trying to make 2013 the year of distilling an extensive but idiosyncratic body of work that I’ve amassed through years of writing and bring it together into a more coherent curriculum.  I want to develop a no-cost casebook for law students and to structure my work so that it can be more useful for people in different places and phases of their eDiscovery education.  So, I’ll be working on that in the first six or eight months of 2013 as both an academic and a personal project.

I’m also trying to go back to roots and rethink some of the assumptions that I’ve made about what people understand.  It’s frustrating to find that lawyers talking about, say, load files when they don’t really know what a load file is, they’ve never looked at a load file.  They’ve left it to somebody else and, so, the resolution of difficulties has gone through so many hands and is plagued by so much miscommunication.   I’d like to put some things out there that will enable lawyers in a non-threatening and accessible way to gain comfort in having a dialog about the fundamentals of eDiscovery that you and I take for granted.  So, that we don’t have to have this reliance upon vendors for the simplest issues.  I don’t mean that vendors won’t do the work, but I don’t think we should have to bring a technical translator in for every phone call.

There should be a corpus of competence that every litigator brings to the party, enabling them to frame basic protocols and agreements that aren’t merely parroting something that they don’t understand, but enabling them to negotiate about issues in ways that the resolutions actually make sense.  Saying “I won’t give you 500 search terms, but I’ll give you 250” isn’t a rational resolution.  It’s arbitrary.

There are other kinds of cases that you can identify search terms “all the live long day” and they’re really never going to get you that much closer to the documents you want.  The best example in recent years was the Pippins v. KPMG case.  KPMG was arguing that they could use search terms against samples to identify forensically significant information about work day and work responsibility.  That didn’t make any sense to me at all.  The kinds of data they were looking for wasn’t going to be easily found by using keyword search.  It was going to require finding data of a certain character and bringing a certain kind of analysis to it, not an objective culling method like search terms.  Search terms have become like the expression “if you have a hammer, the whole world looks like a nail”.  We need to get away from that.

I think a little education made palatable will go a long way.  We need some good solid education and I’m trying to come up with something that people will borrow and build on.  I want it to be something that’s good enough that people will say “let’s just steal his stuff”.  That’s why I put it out there – it’s nice that they credit me and I appreciate it; but if what you really want to do is teach people, you don’t do it for the credit, you do it for the education.  That’s what I’m about, more this year than ever before.

Thanks, Craig, for participating in the interview!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the tenth (and final) of the 2013 LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and generally asked each of them the following questions:

1. What are your general observations about LTNY this year and how it fits into emerging trends?
2. If last year’s “next big thing” was the emergence of predictive coding, what do you feel is this year’s “next big thing”?
3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Craig Ball.  A frequent court appointed special master in electronic evidence, Craig is a prolific contributor to continuing legal and professional education programs throughout the United States, having delivered over 1,000 presentations and papers.  Craig’s articles on forensic technology and electronic discovery frequently appear in the national media, and he writes a monthly column on computer forensics and eDiscovery for Law Technology News called Ball in your Court, as well as blogs on those topics at ballinyourcourt.com.

Craig was very generous with his time again this year and our interview with Craig had so much good information in it, we couldn’t fit it all into a single post.  Yesterday was part 1.  Today is part 2 and part 3 will be published in the blog on Friday.  A three-parter!

Note: I asked Craig the questions in a different order and, since the show had not started yet when I interviewed him, instead asked about the sessions in which he was speaking.

I noticed that you are speaking at a couple of sessions here.  What would you like to tell me about those sessions?

{Interviewed the evening before the show}  I am on a Technology Assisted Review panel with Maura Grossman and Ralph Losey that should be as close to a barrel of laughs as one can have talking about technology assisted review.  It is based on a poker theme – which was actually Matt Nelson’s (of Symantec) idea.  I think it is a nice analogy, because a good poker player is a master or mistress of probabilities, whether intuitively or overtly performing mental arithmetic that are essentially statistical and probability calculations.  Such calculations are key to quality assurance and quality control in modern review.

We have to be cautious not to require the standards for electronic assessments to be dramatically higher than the standards applied to human assessments.  It is one thing with a new technology to demand more of it to build trust.  That’s a pragmatic imperative.  It is another thing to demand so exalted a level of scrutiny that you essentially void all advantages of the new technology, including the cost savings and efficiencies it brings.  You know the old story about the two hikers that encounter the angry grizzly bear?  They freeze, and then one guy pulls out running shoes and starts changing into them.  His friend says “What are you doing? You can’t outrun a grizzly bear!” The other guy says “I know.  I only have to outrun you”.  That is how I look at technology assisted review.  It does not have to be vastly superior to human review; it only has to outrun human review.  It just has to be as good or better while being faster and cheaper.

We cannot let the vague uneasiness about the technology cause it to implode.  If we have to essentially examine everything in the discard pile, so that we not only pay for the new technology but also back it up with the old.  That’s not going to work.  It will take a few pioneers who get the “arrows in the back” early on—people who spend more to build trust around the technology that is missing at this juncture.  Eventually, people are going to say “I’ve looked at the discard pile for the last three cases and this stuff works.  I don’t need to look at all of that any more.

Even the best predictive coding systems are not going to be anywhere near 100% accurate.  They start from human judgment where we’re not even sure what “100% accurate” is, in the context of responsiveness and relevance.  There’s no “gold standard”.  Two different qualified people can look at the same document and give a different assessment and approximately 40% of the time, they do.  And, the way we decide who’s right is that we bring in a third person.  We indulge the idea that the third person is the “topic authority” and what they say goes.  We define their judgment as right; but, even their judgments are human.  To err being human, they’re going to make misjudgments based on assumptions, fatigue, inattention, whatever.

So, getting back to the topic at hand, I do think that the focus on quality assurance is going to prompt a larger and long overdue discussion about the efficacy of human review.  We’ve kept human review in this mystical world of work product for a very long time.  Honestly, the rationale for work product doesn’t naturally extend over to decisions about responsiveness and relevance.  Even though, most of my colleagues would disagree with me out of hand.  They don’t want anybody messing with privilege or work product.  It’s like religion or gun control—you can’t even start a rational debate.

Things are still so partisan and bitter.  The notions of cooperation, collaboration, transparency, translucency, communication – they’re not embedded yet.  People come to these processes with animosity so deeply seated that you’re not really starting on a level playing field with an assessment of what’s best for our system of justice.  Justice is someone else’s problem.  The players just want to win.  That will be tough to change.

We “dinosaurs” will die off, and we won’t have to wait for the glaciers to advance.  I think we will have some meteoric events that will change the speed at which the dinosaurs die.  Technology assisted review is one.  We’ve seen a meteoric rise in the discussion of the topic, the interest in the topic, and I think it will have a meteoric effect in terms of more rapidly extinguishing very bad and very expensive practices that don’t carry with them any more superior assurance of quality.

More from Craig tomorrow!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.

This is the tenth (and final) of the 2013 LegalTech New York (LTNY) Thought Leader Interview series.  eDiscoveryDaily interviewed several thought leaders at LTNY this year and generally asked each of them the following questions:

1. What are your general observations about LTNY this year and how it fits into emerging trends?
2. If last year’s “next big thing” was the emergence of predictive coding, what do you feel is this year’s “next big thing”?
3. What are you working on that you’d like our readers to know about?

Today’s thought leader is Craig Ball.  A frequent court appointed special master in electronic evidence, Craig is a prolific contributor to continuing legal and professional education programs throughout the United States, having delivered over 1,000 presentations and papers.  Craig’s articles on forensic technology and electronic discovery frequently appear in the national media, and he writes a monthly column on computer forensics and eDiscovery for Law Technology News called Ball in your Court, as well as blogs on those topics at ballinyourcourt.com.

Craig was very generous with his time again this year and our interview with Craig had so much good information in it, we couldn’t fit it all into a single post.  So, today is part 1.  Parts 2 and 3 will be published in the blog on Thursday and Friday.  A three-parter!

Note: I asked Craig the questions in a different order and, since the show had not started yet when I interviewed him, instead asked about the sessions in which he was speaking.

If last year’s “next big thing” was the emergence of predictive coding, what do you feel is this year’s “next big thing”?

I think this is the first year where I do not have a ready answer to that question.  It’s  like the wonderful movie Groundhog Day.  I am on the educational planning board for the show, and as hard as we try to find and present fresh ideas, technology assisted review is once again the dominant topic.

This year, we will see a change of the marketing language repositioning the (forgive the jargon) “value proposition” for the tools being sold continuing to move more towards the concept of information governance.  If knowledge management had a “hook up” here at LTNY with eDiscovery, their offspring would be information governance.  Information governance represents a way to spread the cost of eDiscovery infrastructure among different budgets.  It’s not a made up value proposition.  Security and regulatory people do have a need, and many departments can ultimately benefit from more granular and regimented management of their unstructured and legacy information stores.

I remain something of a skeptic about what has come to be called “defensible deletion.”  Most in-house IT people do not understand that, even after you purchase a single instance de-duplication solution, you’re still going to have as much of 40% “bloat” in your collection of data between local stores, embedded and encoded attachments, etc.  So, there are marked efficiencies we can achieve by implementing sensible de-duplication and indexing mechanisms that are effective, ongoing and systemic. Consider enterprise indexing models that basically let your organization and its information face an indexing mechanism in much the same way as the internet faces Google.   Almost all of us interact with the internet through Google, and often get the information we are seeking from the Google index or synopsis of the data without actually proceeding to the indexed site.  The index itself becomes the resource, and the document indexed a distinct (and often secondary) source.  We must ask ourselves: “if a document is indexed, does it ever leave our collection?”

I also think eDiscovery education is changing and I am cautiously optimistic.  But, people are getting just enough better information about eDiscovery to be dangerous.  And, they are still hurting themselves by expecting there to be some simple “I don’t really need to know it” rule of thumb that will get them through.  And, that’s an enormous problem.  You can’t cross examine from a script.  Advocates need to understand the answers they get and know how to frame the follow up and the kill.  My cautious optimism respecting education is function of my devoting so much more of my time to education at the law school and professional levels as well as for judicial organizations.  I am seeing a lot more students interested in the material at a deeper level, and my law class that just concluded in December impressed me greatly.   The level of enthusiasm the students brought to the topic and the quality and caliber of their questions were as good as any I get from my colleagues in the day to day practice of eDiscovery.  Not just from lawyers, but also from people like you who are deeply immersed in this topic.

That is not so much a credit to my teaching (although I hope it might be).  The greatest advantage that students have is that they have haven’t yet acquired bad habits and don’t come with preconceived notions about what eDiscovery is supposed to be.  Conversely, many lawyers literally do not want to hear about certain topics–they “glaze” and immediately start looking for a way to say “this cannot be important, I cannot have to know this”.  Law students don’t waste their energy that way. If the professor says “you need to know this”, then they make it their mission to learn.  Yesterday, I had a conversation with a student where she said “I really wish we could have learned more about search strategies and more ways to apply sophisticated tools hands on”.  That’s exactly what I wish lawyers would say.

I wish lawyers were clamoring to better understand things like search or de-duplication or the advantages of one form of production over another.  Sometimes, I feel like I am alone in my assessment that these are crucial issues. If I am the only one thinking that settling on forms of productions early and embracing native forms of production is crucial to quality, what is wrong with me?

I am still surprised at how many people TIFF most of their collection or production.

They have no clue how really bad that is, not just in terms in cost but also in terms of efficiency.  I am hoping the dialogue about TAR will bring us closer to a serious discussion about quality in eDiscovery.  We never had much of a dialogue about the quality of human review or the quality of paper production.  Either we didn’t have the need, or, more likely we were so immersed in what we were doing we did not have the language to even begin the conversation.

I wrote in a blog post recently about an experiment discussed in my college Introductory Psychology course where this cool experiment involved raising kittens such that they could only see for a few hours a day in an environment composed entirely horizontals or verticals.  Apparently, if you are raised from birth only seeing verticals, you do not learn to see horizontals, and vice-versa.  So, if I raise a kitten among the horizontals and take a black rod and put it in front of them, they see it when it is horizontal.  But, if I orient it vertically, it disappears in their brain.  That is kind of how we are with lawyers and eDiscovery.

There are just some topics that you and I and our colleagues see the importance of, but lawyers have been literally raised without the ability to see why those things matter.  They see what has long been presented to them in, say, Summation or Concordance, as an assemblage of lousy load files and error ridden OCR and colorless images stripped of embedded commentary.  They see this information so frequently and so exclusively that they think that’s the document and, since they only have paper document frames of reference (which aren’t really that much better than TIFFs), they think this must be what electronic evidence looks like.  They can’t see the invisible plane they’ve been bred to overlook.

You can look at a stone axe and appreciate the merits of a bronze axe – if all that you’re comparing it to are prehistoric tools, a bronze axe looks pretty good.  But, today we have chainsaws. I want lawyers demanding chainsaws to deal with electronic information and to throw away those incredibly expensive stone axes; but, unfortunately, they make more money using stone axes.  But, not for long.  I am seeing the “house of cards” start to shake and the house of cards I am talking about is the $100 to $300 (or more) per gigabyte pricing for eDiscovery.  I think that model is not only going to be short lived, but will soon be seen as negligence in the lawyers who go that route and as exploitive gouging by service providers, like selling a bottle of water for $10 after Hurricane Sandy.  There is a point at which price gouging will be called out.  We can’t get there fast enough.

More from Craig tomorrow!

And to the readers, as always, please share any comments you might have or if you’d like to know more about a particular topic!

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine Discovery. eDiscoveryDaily is made available by CloudNine Discovery solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscoveryDaily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.