All posts in Technology-assisted Review (TAR)

DOJ ANTITRUST DIVISION ISSUES NEW MODEL SECOND REQUEST, WITH NEW PREDICTIVE CODING INSTRUCTIONS

On November 28, 2016, the Department of Justice (DOJ) Antitrust Division issued an updated Model Second Request, aimed at revising and streamlining the model to conform to “current division practice.” The updated Model will be used for all Second Requests issued on or after December 12, 2016. The new model contains significant changes to merging parties’ obligations during a Second Request, as well as a substantial formatting overhaul.

Regarding the use of ediscovery technology during a Second Request from the DOJ, the predictive coding instructions were meaningfully modified.

First, the new model appears to signal an increased acceptance of use of predictive coding during a second request. Specifically, the searching and predictive coding instruction begins with the following new language, “Before using software or technology…” seemingly indicating that the Antitrust Division recognizes that it is not a matter of “if” parties are leveraging technology but “when” and “how” that technology will be used.

Second, the new model requires merging parties and their counsel to be more astute than ever before when it comes to ediscovery technology. For example, if search terms are used, merging parties must now submit a list of stop words and operators for the platform being used. Also, if predictive coding technology is used to identify or eliminate documents, merging parties must provide more than just a description of the methods being used. Under this new model, the Antitrust Division also is requiring information about the use of subject matter experts to review seed sets and training documents, effectiveness metrics (such as recall, precision and confidence-intervals) and validation protocols, including sampling protocols used to categorize non-responsive documents.

The new predictive coding and searching instruction is provided in full below:

November 2016 Version – DOJ Model Second Request

  1. Before using software or technology (including search terms, predictive coding, de-duplication, or similar technologies) to identify or eliminate documents, data, or information potentially responsive to this Request, the Company must submit a written description of the method(s) used to conduct any part of its search. In addition, for any process that relies on search terms to identify or eliminate documents, the Company must submit: (a) a list of proposed terms; (b) a tally of all the terms that appear in the collection and the frequency of each term; (c) a list of stop words and operators for the platform being used; and (d) a glossary of industry and company terminology. For any process that instead relies on predictive coding to identify or eliminate documents, you must include (a) confirmation that subject-matter experts will be reviewing the seed set and training rounds; (b) recall, precision, and confidence-level statistics (or an equivalent); and (c) a validation process that allows for Department review of statistically-significant samples of documents categorized as non-responsive documents by the algorithm.

As these new instructions reinforce, Second Requests are synonymous with sheer complexity. At Kroll Ontrack, we have leading technology backed by human experts who know how to successfully navigate a Second Request. Kroll Ontrack is uniquely equipped to help manage your document productions to the FTC, DOJ and other global competition bureaus.

TAR: Building a Better Playlist

TAR

“Sometimes it seems as if our Pandora and Netflix accounts know us better than we know ourselves, and can build a better play list…”

Brett M. Anders

Brett M. Anders, Jackson Lewis

In a recent article in Today’s General Counsel, Brett M. Anders of Jackson Lewis and my Kroll Ontrack colleague Rick Anderson, seek to debunk the misconception that human lawyers alone can build a better playlist when it comes to legal document review.

Human Review Is Not the Gold Standard

Practitioners shy away from predictive coding and technology assisted review (TAR) in part because of the myth that human review is superior to that done by a computer. However, this is not the case. Humans can be inaccurate: relevant documents can be missed and accuracy suffers. This position has been verified by studies and is generally accepted by the judiciary. The use of TAR has received much support from the courts in the cases where it has been an issue. However, at this point, no court has gone so far as to mandate the use of TAR.

Rick Anderson, Kroll Ontrack

Rick Anderson, Kroll Ontrack

Using Technology Assisted Review in Litigation

As of now, several judicial opinions have surfaced regarding the use of TAR. As noted above, the courts have supported it as a cost effective method for conducting discovery. The standard for discovery responses is “reasonable and proportional to the matter,” not perfection. Therefore, the accuracy offered by the use of TAR satisfies the standard for discovery production.

Speaking of proportionality, using TAR can place a party in a better position to make an argument about proportionality when litigation costs become too high. Because TAR prioritizes which documents are most likely to be relevant, a party who makes its way through the documents with the highest relevance has a basis to argue that additional discovery would not be “proportional to the needs of the case.”

Advantages of Using TAR

As discussed by Anders and Anderson, despite the apprehension to utilize this technology in the legal community, the fact remains that TAR has many advantages.

  • TAR costs a fraction of the expense it would take to review documents manually
  • TAR is typically faster than traditional document review, while also more accurate
  • Courts have approved the use of TAR; parties no longer need to worry about being the first to use this technology in a case
  • Courts are encouraging its use, while respecting the party’s wish to keep its seed set (used to train predictive coding technology) confidential

For more information on TAR, and how to utilize its benefits in litigation, be sure to read the full article, “Building a Better Play List with Technology-Assisted Review.”

Soft Data, Warm Data, Little Thread of Emails: The Big Bang Theory Meets ILTACon

The hit CBS show, The Big Bang Theory, returns this month and fans are eagerly awaiting developments with each character’s storyline – from relationships between Leonard, Penny, Sheldon and Amy to the impending arrival of baby Wolowitz (which may interfere with the gang’s annual Comic-Con plans).

One thing that cannot be interfered with is the legal tech industry’s own Comic-Con – better known as ILTACon. At ILTACon, the Big Bang Theory clan would be right at home, diving headfirst into technologies that improve law firm and corporate law/IT department responsibilities. Frankly, this tradeshow is known for being a total geek-fest. (And the lawyers, litigation support professionals and IT staff in attendance are cool with that.)

Bazinga! Just Another One of Kroll Ontrack’s Inventions

At ILTACon this year, Kroll Ontrack was excited to announce its newest gizmo to aid document review: Communication Insight.

To understand the benefits of Communication Insight, a Relativity-based application, it is worth elaborating on what viewing retrieved emails is like without it. Normally, when an email is reviewed, it is very difficult to piece together conversations and themes. Going through each file is time-consuming (read: expensive) and presents the risk that important information will be missed because the email is not viewed as part of a whole conversation. The bottom line – document review needs to be intuitive, as discussed at ILTACon by Kroll Ontrack product director Wendy King in a short video interview with LegalTech News’ Ian Lopez.

Kroll Ontrack Presents: Fun with Emails

At its core, Communication Insight makes it easier to view threads of emails and understand what happened in the entirety of the conversation. With this technology, the user is able to tell when the subject of an email changed, who was part of the conversation and if attachments were dropped. It provides a comprehensive overview of the conversation, so that complicated email chains can be read in context. Emails are displayed in a manner familiar to email users and the program notifies users if someone was added to or left the conversation. Through a logical display of conversation data and dynamic visual cues, reviewers can quickly make decisions and intuitively focus their review.

For Relativity users, Communication Insight is a new feature Kroll Ontrack is proud to bring to the platform. #ILTACon2016 attendees seeing Communication Insight in action noted that it gives them everything they need, front and center…and even a non-geek can appreciate that.

 

Global Predictive Coding: Gold, Silver & Bronze

This week, athletes from around the globe will gather in Rio to compete for gold medals and present years of hard work and dedication. With the Games as a backdrop, there is no better time than to explore our own global phenomenon in ediscovery – predictive coding technology.

Recently, Kroll Ontrack’s Michele Lange and Tracy Stretton co-authored an article for Bloomberg BNA’s Digital Discovery & e-Evidence in which they discussed the way predictive coding is taking the globe by storm.

A Global Predictive Coding Case Law Primer

Taking the gold medal for the first predictive coding judicial opinion, several years ago American courts approved the use of predictive coding technology in discovery, making it widely recognized and respected. In March 2015, Ireland earned the silver medal with its approval of the use of predictive coding in the discovery process in Irish Bank Resolution Corporation Ltd. & Ors v. Quinn & Ors. In February 2016, Master Matthews helped Britain win the bronze when he issued the first British opinion, known as Pyrrho Investments Ltd. v. MWB Property Ltd., which approved the use of predictive coding in High Court proceedings, partly relying on Magistrate Judge Peck’s opinion in Da Silva Moore v. Publicis Groupe. Even more recently in May 2016, Britain had its first contested case regarding predictive coding, David Brown v. BCA Trading. These opinions are just the start of what we will see on the issue of predictive coding in years to come. Case law will continue to develop as many of the same concerns are being raised globally.

What’s Next on the Horizon?

As discussed by Lange and Stretton in their article, in the US and UK, judges will continue to interpret the nuances of parties’ predictive coding practices and it will not be long until other European and Asian countries formally join the predictive coding games. In the meantime, technology is evolving so legal professionals must stay vigilant to meet the expectations of demanding international clients when producing electronic documents in global litigations or investigations.

Predictive Coding Technology: A Summer Blockbuster You Won’t Want to Miss

Predictive Coding Patent

What’s better than a summer blockbuster? Few things beat the heat better than an air-conditioned theater and a bucket of buttery popcorn while watching an engaging film.

Predictive Coding Patent Blockbuster:  Technology You Don’t Want to Miss

Kroll Ontrack has come out with its own “blockbuster” this summer: a predictive coding patent. Obtaining patents, like making blockbuster movies, takes years and ours was more than four years in the making. To accompany our patent, we have another mega-hit that deserves a second screening: Kroll Ontrack’s predictive coding guide. Our newly-patented technology, paired with our ever-relevant predictive coding guide, is a superhero team that the up-to-date ediscovery practitioner cannot do without.

2016 Summer Blockbusters: Movies You Don’t Want to Miss

Of course, there are other highly anticipated blockbusters arriving this summer. To start, the smart mouth turtles-turned-mutants of our youth are back in Teenage Mutant Ninja Turtles: Out of the Shadows. In Independence Day: Resurgence the people of earth once again are in critical struggle against an alien invader (hopefully with a better spaceship design). The daffy ghost hunters have returned in revival of the ’80’s classic movie Ghostbusters, this time with an all-female cast and the same catchy theme song. Star Trek Beyond continues the story of the USS Enterprise crew in the newest installment in one of popular culture’s longest-running franchises. Finally, the list would not be complete without the newest villain-themed movie from the Batman franchise: Suicide Squad.

Live Long and TAR!

Never Second Guess a Second Request

Second Request

Massive mergers are never a simple matter for organizations and their antitrust attorneys. The Second Request process can be a major burden for merging organizations since it requires that companies review, analyze and produce massive volumes of data in what can be a very short amount of time. If that doesn’t already cause panic, keep in mind that failing to fully comply can lead to substantial civil penalties and even the rejection of the merger transaction.

While Second Requests are a daunting task for any antitrust lawyer, my colleagues John D. Pilznienski and Sheldon A. Noel recently discussed the requirements counsel must meet to comply with a Second Request and how utilizing analytics like predictive coding can help simplify the process. The article written by Pilznienski and Noel, “Never Second Guess a Second Request: Leveraging Predictive Coding for Reviewing Documents in Antitrust Matters,” appeared in the March 2016 edition of Digital Discovery & e-Evidence, a Bloomberg BNA publication.

Read the article: Never Second Guess a Second Request

What is a Second Request?

For those of you not intimately aware of the corporate merger process, a Second Request is the issuance of a request by the Department of Justice (DOJ) or Federal Trade Commission (FTC) for “additional information and documentary material relevant to the proposed acquisition.” When corporations intending to merge meet a certain financial threshold, their proposed merger is subject to review by federal antitrust agencies. If the DOJ or FTC determines that more information is needed to ensure that there is no violation of federal antitrust laws during the initial review, they can request more information from the merging corporations. These requests can be extremely broad, requiring extensive resources, time and manpower to collect, process and produce the relevant documents and data.

Second Requests, Ediscovery and Predictive Coding: A Case Study

As discussed in the article by Pilznienski and Noel, in a recent technology sector merger the two companies utilized Kroll Ontrack’s predictive coding technology and review platform to respond to an FTC Second Request. The merger presented a number of hurdles – a large data set, multiple jurisdictions, complicated review guidance based on the document requests and an unexpectedly short production deadline – all of which were easily overcome by leveraging predictive coding. From the approximately 600,000 searchable documents, a random sample of approximately 2,300 documents was generated, sampled at a 95% confidence level and a 2% margin of error.  After applying a variety of review methods, the most relevant documents were reviewed first, aiding the merging companies in meeting the FTC’s deadline and significantly reducing the costs of review.

Be sure to read the full article, Never Second Guess a Second Request: Leveraging Predictive Coding for Reviewing Documents in Antitrust Matters, for a more in-depth discussion on the application of predictive coding to the Second Request process.

April Webinar: Got Data? Analytics to the Rescue!

Got Data Analytics to the Rescue

On April 19, 2016 join Kroll Ontrack experts Cathleen Peterson and Jim Sullivan, along with Kiriaki Tourikis from JP Morgan, as they discuss data analytics as both the lifeblood powering critical business operations and the kryptonite preventing the business from flexing its muscle. When investigations, litigation or compliance matters strike, organizations and their counsel that leverage analytics are more likely to win.

Register for the Data Analytics webinar today!

This session will feature hypothetical scenarios to explain the various analytics tools and how they fit into a case, data breach or investigation. At the end of the session, participants will understand how analytics can help:

  • Map the data collection and explore key points and related themes
  • Identify key players, timelines and communication patterns
  • Mine data for Personally Identifiable Information (PII)
  • Find redundant information and remove non-relevant, junk data

Plus Check Out These 2016 Webinar Recordings

Click the links below to watch Kroll Ontrack experts and panelists discuss the FRCP amendments and “dark data.”

January 2016: 2015 Year in Review: Ediscovery Case Law and Rules
February 2016: Turning on the Lights in a [Dark] Data Room

Podcast: Predictive Coding and ei-Recall

ESI Report

As an ediscovery specialist, if you have not yet become acquainted with predictive coding – carpe diem – there is no shortage of information to get you caught up. If you are already an avid predictive coding user, now is the time to hone your methods.

A New Approach: Ei-Recall

Astute legal teams seek ways to re-examine the metrics used to evaluate the effectiveness of predictive coding – precision, recall and f-measure. Recently, Ralph Losey went on a quest for the “Holy Grail” of recall calculations. After many hours and sleepless nights, he devised a new approach to recall methods, ei-Recall. You can see his in-depth discussion of ei-Recall in a three-part series on his blog.

Download the Podcast

To whet your appetite for all-things ei-Recall, Ralph joined my colleague, Michele Lange, on the February 2015 ESI Report podcast. Listen to this podcast, which includes a summary of ei-Recall, how it compares to other recall methods and some of the advantages of the new recall process. Also, be sure to watch for future editions of the ESI Report on the Legal Talk Network.

Predictive Coding’s Ripple Effect

There’s always a new video going viral. The best ones are often talked about over the office water-cooler, making the participants overnight sensations. Consider 5-year-old Noah Ritter of Pennsylvania. He was merely interested in the super slide at his county fair, but when a local news crew asked him how the ride was, his adorable interview immediately went viral, garnering more than 3 million views on YouTube. That single video sent a ripple effect through the internet (and likely his personal life), and his interview was featured in articles and news outlets as far away as India and earned him a spot on Good Morning America. And now, for the crème de la crème, his name even graces this fine blog.

This ripple effect of popularity got me thinking about predictive coding (no surprise there). Predictive coding certainly made a splash when the first Federal court approved its use in litigation in Da Silva Moore v. Publicis Groupe. But since then, savvy practitioners have seen predictive coding as more than a discovery tool and have devised new uses for this powerful technology. The idea of incorporating machine learning and predictive coding into everyday business tasks is not far-fetched. This ripple effect of predictive coding can be seen in many areas, but there are three specific areas where the ripple effect will likely soon be a wave.

First, early use of predictive coding can be used to narrow the scope of discovery and refine the precision of collection. In addition, using predictive coding in this way can effect settlement. Incorporating predictive coding into EDA creates value that at least one court has already recognized.

Second, predictive coding can be used as a compliance tool.  Predictive coding is a natural way to assess and detect risk patterns, and stop them from developing further. Therefore, predictive coding software could be trained to comb through all of an organization’s information to detect potential risks and enable counsel to take immediate corrective action.

Finally, predictive coding can be used to combat the trials of information governance. Predictive coding can be and should be implemented in record retention policies. Predictive coding can identify which documents should be kept – and those that can be defensibly deleted. As more and more businesses become bogged down by big data, a smarter, more automated approach to record retention will be the future of information governance.

For more information concerning these “ripples” of predictive coding, check out Kroll Ontrack’s latest infographic, The Predictive Coding Ripple effect.

The Latest ESI Report

ESI Report

In case you missed it, tune into the latest edition of the ESI Report at ediscovery.com which features two ediscovery experts discussing technology assisted review (TAR) and predictive coding. Recently, Cliff Nichols of Day Pitney and Tony Reichenberger of Kroll Ontrack joined the ESI Report for a conversation on a pay-to-play matter they worked on together, New Mexico State Investment Council v. Bland, 2014 WL 772860 (N.M. Dist. Ct. Feb. 12, 2014). Cliff and Tony executed an investigation leveraging TAR and predictive coding for the case.  This podcast includes an in-depth examination into the inner workings of the investigation.

Check out the podcast, which covers topics including the judge’s decision, the background of the case, the process by which Cliff and Tony decided to use and implement TAR and predictive coding in the investigation, and how they successfully used the technology. For an even deeper dive into the details of this case, be sure to also check out the case study  on this pay-to-play matter at ediscovery.com.

 
css.php