The Ediscovery Blog by Kroll Ontrack | Ediscovery. Rediscovered.

The Results are In: 2013 Ediscovery Magic Quadrant

by Thought Leadership Team on June 17, 2013 in Ediscovery with No comments

Gartner Ediscovery Magic Quadrant results

Industry analyst firm, Gartner, has just released its “2013 Magic Quadrant for E-Discovery Software”, an annual report examining the state of the electronic discovery industry. This year’s report provides an excellent snapshot of market trends and ranks providers in four quadrants – leaders, challengers, niche players, visionaries.

Gartner has long been known by IT professionals as the “go to” organization for quantifiable and objective technology recommendations. Less known in the legal industry, three years ago, Gartner stepped into the ediscovery market, with formal coverage of legal technology trends and providers.

The 2013 report is, once again, a comprehensive look at our industry – a direct result of the hard work of Gartner analysts Debra Logan, Sheila Childs, and Alan Dayley.

Kroll Ontrack is honored to be positioned by Gartner in the Leaders quadrant, confirming our quest to transform ediscovery into a repeatable, predictable process that gives legal teams more control while driving down cost.

For a complete, downloadable copy of the 2013 Magic Quadrant for E-Discovery Software visit: http://www.krollontrack.com/ediscovery-magic-quadrant/.

Rulemaking Efforts Move Forward

by Michele Lange on June 11, 2013 in Case Law, Ediscovery with No comments

Efforts to amend the Federal Rules of Civil Procedure took another step forward last week. On June 3, 2013, the Standing Committee on Rules of Practice and Procedure (“the Standing Committee”) approved a package of proposals to change several of the Federal Rules relevant to ediscovery. The package will be released to the bench and bar for a six-month public comment period on August 15, 2013.

The package approved by the Standing Committee was first adopted by the Civil Rules Advisory Committee (“The Rules Committee”) during its April 2013 meeting. The package combines nearly three years of initiatives handled by the Discovery Subcommittee and the Duke Subcommittee following the Duke Litigation Review Conference in May 2010. The package contains the following:

An amendment to Rule 1, addressing the issue of cooperation in discovery
Amendments to Rule 16 & 26(f), including preservation considerations as a component of discovery plans and scheduling orders
Amendments to Rule 26(b), adding proportionality to further define the scope of discovery
Amendments to Rules 30, 31, 33 and 36, imposing or reducing numerical limits on interrogatories, oral or written depositions, and requests to admit
An amendment to Rule 34, requiring greater specificity in objections to requests to produce
An amendment to Rule 26(c), further clarifying the courts authority to order cost-shifting
A replacement rule Rule 37(e), aiming to create a national, uniform standard for culpability with regard to spoliation sanctions

Following the comment period, public hearings will be held November 7, 2013 in Washington, January 9, 2014 in Phoenix and early February 2014 in Dallas. For the full text and comments about the Rules Package, check out the May 2013 Report of the Rules Committee to the Standing Committee.

Looking for more on this hot topic? Don’t miss Kroll Ontrack’s June 12^th webinar, Are Your Ediscovery Practices Ready for FRCP Amendments?, hosted by state and Federal rules expert Tom Allman. Additionally, Tom wrote this article with more details on the Rules Package.

Top 5 Ediscovery Case Summaries – June 2013

by Thought Leadership Team on June 4, 2013 in Case Law, Ediscovery with No comments

Read the very latest ediscovery case law summaries

Below are the top 5 ediscovery case law summaries for June, 2013.

Court Denies E. & J. Gallo Taxation Costs Spent Decanting ESI into Indexed, Flattened Form
Country Vintner of NC, LLC v. E. & J. Gallo Winery, Inc., No. 12-2074 (4th Cir. Apr. 29, 2013).

Race Tires America, Inc. Approach Adopted in Taxation Case
Phillips v. WellPoint, Inc., 2013 WL 2147560 (S.D. Ill. May 16, 2013).

Court Denies Taxation and Cost-Shifting of Ediscovery Costs
W Holding Co., Inc. v. Chartis Ins. Co. of Puerto Rico, 2013 WL 1352426 (D.P.R. Apr., 3, 2013).

Both Parties Cry Foul Regarding Production Adequacy, Cooperation; Court Says Work it Out
Procongps, Inc. v. Skypatrol, LLC, No. C 11-3975 (N.D. Cal. Apr. 1, 2013).

Mere Facebook Friendship Does Not Amount to Impartiality by Judge
Youkers v. State, 2013 WL 2077196 (Tex. App.—Dallas May 15, 2013).

To check out more ediscovery case summaries, visit our Case Law library.

Life at Kroll Ontrack

by Jen Wightman on May 28, 2013 in Ediscovery with No comments

I have been working at Kroll Ontrack since early 2002. Part of the fun of working at a company that’s been around as long as Kroll Ontrack are the stories and statistics. I have seen this company go to great lengths on behalf of our legal technologies and data recovery customers. For example, did you know that one winter was so bad that we had to call our neighborhood airport for jet fuel to power our data center through an extreme Minnesota blizzard? I honestly never imagined having a “good neighbor story” that swapped the normal cup of sugar with gallons of hydrocarbon, but there you go. Likewise, there are stories every day of memories saved or business-critical data recovered by our amazing Ontrack Data Recovery team, such as restoring data after Hurricane Sandy.

Here are some of my favorite “things you did not know” about Kroll Ontrack:

There is more to life than just PDFs and MS Word files.

Kroll Ontrack processed 7,000 different file types in the last two years alone. For many, this should emphasize that native file production format issues are real. For others, this is cold hard proof that there are actually other types of file types in existence besides the frequently used and technically questionable catch-all formats: ‘words,’ ‘sounds,’ and ‘videos.’

We can ship boxes of paper by the truckload – but fortunately no one asks anymore!

Kroll Ontrack can process 49 million pages of load files in a single day. My favorite story from back in the day was a US government agency that insisted that we send their production delivery as paper. “But it’s over a million pages,” we tried to warn them, “that will be several hundred boxes.” They insisted and so we started printing—when the first semi-truck load showed up and they had no room for even the first delivery, they cried ‘uncle’ and worked with us to determine an effective electronic delivery solution.

We have friends across the world.

Kroll Ontrack collected data from over 126 countries last year. We also provide critical data recovery services around the world, which means not only can we help when weather strikes in the US but also internationally. Kroll Ontrack’s legal technologies services have been supporting EU clients for approximately 20 years. In fact, just last year the UK branch processed a whopping 50 Terabytes of client data. In addition to the EU, Kroll Ontrack is quickly ramping up in Asia, processing and hosting more multilingual data than any other provider. In fact, 98% of our projects contain some amount of multilingual data!

We are part of the solution for taming the beast of electronic data.

It can feel like just some made up statistic when you hear that the amount of electronic data being generated and consumed in the world far exceeds a zettabyte. But it can be an expensive and painful truth when you are faced with litigation. Kroll Ontrack filtered over 2 billion documents in the past year and can filter over 30 million documents per day. How’s that for Big Data?

If you enjoyed these fun-facts, check out the complete infographic: “10 Things You Didn’t Know about Kroll Ontrack”.

Practical Tips for APAC Ediscovery

by Eric Robinson on May 20, 2013 in Ediscovery, International Ediscovery with Comments Off

In our ever-shrinking, interconnected world, it is imperative that legal practitioners establish expertise in international ediscovery law. One of the most dramatic evolutions in ediscovery is occurring in the Asia-Pacific region (APAC).

As in the U.S., several APAC countries have created special rules for the discovery of electronic data. Hong Kong, Singapore and Japan have (or are in the process of) adopted data privacy regulations. Hong Kong uses traditional English discovery law, which makes Hong Kong the APAC country most amenable to American ediscovery efforts. Singapore is equally as advanced, given its recent adoption of aggressive measures to become the premiere dispute resolution hub of the region. Japan has likewise begun deliberations on the implementation of ediscovery laws, but the Japan Privacy Act permits the conditional transfer of personal information from a corporate entity to a third party.

Perhaps the most challenging ediscovery environments are those of China and South Korea. In South Korea, ediscovery law is still relatively non-existent. China, on the other hand, deals with data protection and privacy issues on a piecemeal basis, and a central framework for governing ediscovery matters has yet to be established.

Handling these APAC ediscovery issues seems like a daunting task, but here are seven practical tips:

1. APAC Ediscovery Goes Beyond Translation
Even a U.S. attorney proficient in an Asian language will struggle with APAC ediscovery because of vast differences in legal systems. Most APAC companies cannot fathom why an American court would require a party to collect and exchange massive amounts of data.

2. Be Cautious of Nationalist Challenges
Strong nationalism may thwart U.S. litigation collection efforts, as parties question why APAC privacy considerations do not trump U.S. discovery laws.

3. Capture Full Forensic Images and Conduct Client Interviews
Because of geographical and nationalist challenges in APAC, a lawyer cannot risk an insufficient collection. As such, active data capture is not recommended in the APAC region. Along the same lines, it is important to ask custodians for all spelling variations of their name during the client interview.

4. Watch for International Data Nuances
In the APAC region, software is often vastly different from U.S. software. Furthermore, multilingual software platforms generate different metadata fields than U.S. software platforms. Finally, use of free email packages is more prevalent, and an attorney may need to collect ESI from several email systems.

5. APAC Companies Tend to Encrypt More Data
Build a workflow into collection and review for handling password protected documents. Keep a list of passwords found during document review, and be prepared to use password cracking software.

6. Don’t Overlook the Paper
Unlike in the U.S., APAC businesses still rely heavily on paper documentation. Pay special attention to paper in the APAC region, given that paper sizing and hole punching may be different. Optical Character Recognition (OCR) is often not available for many languages.

7. Make Friends and Work with Local Counsel
Perhaps most importantly, local counsel experienced in ediscovery collections or local service providers can greatly assist American attorneys by acting as a mitigating party, explaining sovereignty issues, integrating paper and data into one database, and collecting data before spoliation occurs.

Overall, APAC ediscovery law has proven to be extremely agile; thus, it is important for practitioners to keep afloat in these ever-changing waters.

For a top-notch overview of international ediscovery laws, check out this global ediscovery infographic.

Get Ready for New Ediscovery Rules in Minnesota

by Michele Lange on May 13, 2013 in Ediscovery with Comments Off

The month of July ensures some much needed warmth in Minnesota after an offensively wintery spring, but July also brings an exciting new set of amendments to the Minnesota Rules of Civil Procedure. From July 1, 2013 onward, Minnesota lawyers are obligated to cooperate and be more upfront during the discovery process. In an effort to mirror the crux of the 2006 amendments to the Federal Rules of Civil Procedure (FRCP), electronic discovery and proportionality will be the focus of the new Minnesota Rules of Civil Procedure, due in no small part to the resounding plea from lawyers all over the state for the judicial enforcement of cost-effectiveness and manageability in civil litigation.

Particularly interesting is the new Rule 1, which explicitly addresses proportionality – a welcome step that actually goes above and beyond the FRCP. The specific mention of proportionality takes the wiggle room away from litigants tempted to play fiscal “chicken” with opposing counsel via discovery costs. It puts the onus on each party to work across the aisle and limit discovery to a practicable scope.

The most expansive change comes in Rule 16, by which Minnesota lawyers must make an earlier investment on tailored ediscovery plans. Lawyers must conduct ediscovery management with more effort than ever by way of compulsory initial disclosures related to custodians, data sources, litigation holds, scheduling, production format, and everything in between. A strict new timeline will be enforced – thirty days after the answer to the complaint, the discovery conference must take place, and fourteen days after that, the parties must file a discovery plan with the court. No longer will “Yep, we talked about ediscovery” end the story, as the court will now be more involved in early planning to ensure that the discovery beast stays tame. In fact, no discovery will be allowed until after the initial disclosures and scheduling order pass the court’s muster. In essence, early preparation and a strengthened call for cooperation are certainly the themes of the new rules in Minnesota state court.

Updates to the Minnesota Rules come at the perfect time, when high-level discussions are occurring about similarly strengthening the FRCP to reflect the growing need to address proportionality in litigation. All in all, the updates reflect the legal system moving forward on a path towards the “North Star” in ediscovery.

The text of the new Minnesota Rules of Civil Procedure can be found here.

Top 5 Ediscovery Case Summaries – May 2013

by Thought Leadership Team on May 6, 2013 in Case Law, Ediscovery with Comments Off

Below are the top 5 ediscovery case law summaries for May, 2013.

Court Considers Possibility of Clawback Order in Undue Burden Assessment
In re Coventry Healthcare Inc., 2013 WL 1187909 (D. Md. Mar. 21, 2013).

Sanctions Granted for Social Media Spoliation
Gatto v. United Air Lines, Inc., 2013 WL 1285285 (D.N.J. Mar. 25, 2013).

Court Looks to Circumstantial Evidence to Rely on Evidence from MySpace
People v. Kucharski, 2013 WL 1281844 (Ill. App. 2^nd Dist. Mar. 29, 2013).

Proportionality is Key Principle in Predictive Coding Case
In re Biomet, 2013 WL 1729682 (N.D. Ind. Apr. 18, 2013).

Da Silva Moore Drama Dissipates
Da Silva Moore v. Publicis Groupe SA, 12-5020 (2d Cir. Apr. 10, 2013).

To check out more ediscovery case summaries, visit our Case Law library.

The ESI Report is Back in Business! New Ediscovery Podcast

by Michele Lange on May 1, 2013 in Ediscovery, Podcasts with 1 Comment

Computer Assisted Review Reference Model

After a brief hiatus, the ESI Report, sponsored by Kroll Ontrack and produced by the Legal Talk Network, is back on the air! We will bring you monthly ediscovery podcasts on the latest trends and case law that you can listen to anywhere.

Revving Up Your CARRM

In this new edition, George Socha of the EDRM and Tom Palladino of NightOwl Discovery join me to talk about the CARRM (Computer-Assisted Review Reference Model). Don’t miss the great interview with George and Tom – two of the contributors to the new CARRM. [For a quick primer on the CARRM and other technology assisted review (TAR) reference models, take a look at this blog post from Jennifer Wightman.]

An Ediscovery “Puzzler”: Real or Fake

Because we don’t always have to be serious on the ESI Report, the new Arguably Relevant segment on the podcast takes our guests on a quick jaunt through some of the more light-hearted subject matter in ediscovery. This month, we challenged our guests to identify a witty snippet from a real judicial opinion versus a fictitious judicial quote concocted by ESI Report writers.

See if you can guess which are real, and score your answers with the key at the bottom of the blog post. (Hint: three of the quotes are genuine.)

Real or Fake: “By destroying his computer with a sledge hammer, the plaintiff’s conduct was egregious enough to fortify his claims against the defendant; thereby, the court finds it appropriate to hammer these claims with a dismissal.
Real or Fake: “This is a case of too many cooks spoiling the broth . . . the litigation was conducted in an Inspector Clouseau-like fashion. However, unlike a Pink Panther film, there was nothing amusing about this conduct
Real or Fake: “Imagine a standup comic who delivers the punch-lines of his jokes first, a plane with landing gear that deploys just after touchdown . . . that’s what document production after trial is like—it defeats the purpose.”
Real or Fake: “Short of ordering a lobotomy—which, of course, this court cannot do—I strongly encourage the plaintiffs to forget these portions of the meeting notes and focus on documents that actually are discoverable.”
Real or Fake: “American lawyers engaged in discovery have never been accused of asking for too little. To the contrary, like the Rolling Stones, they hope that if they ask for what they want they will get what they need.”
Real or Fake: “Counsel could have included ‘Mickey Mouse’ or ‘Donald Duck’—both of which would have been about as useful as the information listed on the plaintiff’s privilege log.”

To hear how ESI Report guests, George Socha and Tom Palladino, scored on the Arguably Relevant ediscovery puzzler, download the full podcast here. Lastly, don’t forget to subscribe to the RSS feed, so that podcasts from the Legal Talk Network appear in your iTunes account each month for free!

Ediscovery Puzzler Answers:

Answers: 1. Fake; 2. Real (Coquina Invs. v. Rothstein); 3. Real (DL v. Dist. of Columbia); 4. Fake; 5. Real (McPeek v. Ashcroft); 6. Fake

Deploying Trainers in Technology-assisted Review (TAR) without “Spoiling the Broth”

by Jen Wightman on April 29, 2013 in Ediscovery, Technology-assisted Review (TAR) with 1 Comment

Leaving little room for interpretation, the court in Coquina Investments v. Rothstein, stated that the defendants’ litany of ediscovery project management pitfalls (which involved over 200 attorneys across two firms) culminated into a “case of too many cooks spoiling the broth.” While Coquina Investments involved format of production issues, the same rationale applies when deploying trainers in technology-assisted review (TAR) —too many trainers can lead to inconsistency and poor machine learning.

Taking Control of the Technology-Assisted Review Kitchen

Using TAR in litigation is strikingly similar to working in a professional kitchen. There are many parts moving on parallel tracks. Just like a pastry chef may begin working on dessert while a grill chef prepares the main dish, you may have reviewers allocated to train a recently found hard drive while a sub-team performs corrective training on a production set. And above all else, in either scenario, nothing leaves the kitchen without a taste test (quality control). But perhaps the most difficult task involves assigning appropriate roles to a diverse cast of employees during the stages of machine training.

Lead Attorney: The Chef de Cuisine—in charge of all things related to the kitchen. This role involves making executive decisions like when to stop review, how to provide additional training and who will train the machine.

Subject Matter Experts (SMEs): The Sous-Chefs—second-in-command to the Chef de Cuisine. These are attorneys that have a firm knowledge of the nature of the case and the issues involved. They are capable of making high-level decisions and have an expansive knowledge of the dispute.

Contract Attorneys: The Chefs de partie—line cooks responsible for certain areas of production. These are attorneys who are comfortable and trained on the issue at hand, but do not have the level of knowledge possessed by Subject Matter Experts.

Choose Your Recipe

The Chef de Cuisine works closely with the Sous-Chefs to ensure that everyone clearly understands the basics of the recipe so that when the Chef de Cuisine (the Lead Attorney) is out of the “kitchen” the quality of the output remains constant.

When it comes to dedicating a team of SMEs to train the system, the adage “less is more” carries the day. As discussed in a document produced by the TREC 2008 Legal Track, determining whether a document is responsive or not responsive is a deceptively subjective process. Lawyers “draw lines”—often at different places—across a number of determinations like “the nature of the risk posed by production, the party requesting the information” and the willingness of the production party to face a challenge for underproduction. Because the risk of inconsistencies in deciding responsiveness is exacerbated by the introduction of more trainers, rarely will you want more than five SMEs training the system. The restaurant owner mutters, “but my project is big, there is no way that I can rely on only five reviewers.” Generally, two to five reviewers can handle the targeted review load for even a very large project. The total amount of training documents will vary depending on if you plan to “seed” the system (and how much “seeding” you plan to do), the number of documents in your data set and your desired confidence level. Ultimately, responsiveness decisions made on this fraction of documents will be extrapolated to all remaining documents in the data set; it becomes critical that the SMEs are in sync with the goals and structure of the case.

Reduce and Stir

While the ideal structure for deploying this handful of SMEs is still up for debate, there is common consensus that there must be some process in place to arbitrate consistency when responsiveness disputes arise. I’ve seen some interesting hierarchical training structures over the years designed to handle training disputes. These are some of the most common:

Finally: Tasting the Broth

An effective document review and an efficient kitchen both rely upon QC measures to ensure quality and consistency of output. A well-designed plan for validating the automated technology-assisted review output is key to knowing when to stop training for quality and when the documents are ready for consumption at the next stage of the case. Where the Chef de Cuisine is responsible for ensuring that only quality dishes leave her kitchen, the Lead Attorney is also responsible for the quality of the data in her case. Only when quality control measurements reflect defensible levels of recall and precision will a Lead Attorney be in a position to move beyond first-pass review and plate the production for the requesting party—Bon Appetit!

To gain hands-on TAR experience, register now for the newest educational course offered by Kroll Ontrack, TAR Learning Labs. The next Learning Lab is coming up in Minneapolis, MN, in early June. Sign up soon, space is limited!

Math and Statistics for Ediscovery Lawyers Using Technology-assisted Review (TAR)

by Eric Robinson on April 22, 2013 in Ediscovery with Comments Off

Legal professionals learn a lot of complicated concepts and principles through education and/or practice… but they usually don’t have much—or anything—to do with math or statistics.

Then, along came Technology-assisted Review (TAR), which promises to revolutionize ediscovery by leveraging sampling techniques and advanced algorithms to predict whether documents are responsive to particular criteria. Simply put, these revolutionary technologies rely heavily on math and statistics—and modern practitioners need to be more tech- and math-savvy than ever before (or be willing to engage the appropriate predictive coding experts or other resources) in order to understand and leverage this new methodology to its highest potential.

However, to master the TAR process, legal professionals don’t need to dust off their slide rules and graphing calculators, crack open a stats book, or flock to the nearest college or university to enroll in a math class. Rather, they simply need to focus on understanding the key concepts and metrics necessary to manage the predictive coding process.

To help put you on the path to TAR technology mastery, here is a cursory overview of the metrics and processes you need to know.

Key Metrics in Effectiveness Reporting

After the technology has run machine learning, it will generate a report with raw data metrics and calculations that should look something like this:

Understanding the metrics on this report is key to analyzing your technology’s performance and determining what to do next.

True and False Positives/Negatives
In search or review exercises, “responsive” or “not responsive” classifications are reviewed. When the document is suggested as “responsive,” and the suggestion is correct, this is referred to as a True Positive; when it is incorrect (i.e., a non-responsive document is incorrectly coded as responsive), it is a false positive. Accordingly, when a document is suggested as “not responsive,” and the suggestion is correct, this is an example of a true negative; if the suggestion is incorrect (i.e., it should have been coded “responsive”), then it is a false negative.

Recall, Precision, and F-measure
While these three key metrics have been discussed previously, understanding these metrics is essential to successfully, effectively and efficiently employing predictive analytics for document review. Generally speaking, precision is the fraction of relevant documents within retrieved results—essentially a measure of exactness. Recall is the fraction of retrieved relevant documents, or the measure of completeness. F-measure is the harmonic average between the system’s recall and precision.

Accuracy
Accuracy incorporates how well the classifier did by identifying the fraction of correctly coded documents, essentially expressed as (True Positives + True Negatives) / (All Documents). While accuracy can be helpful, it should not drive decisions. Accuracy can be skewed upward if there is an overwhelming amount of either true positives or true negatives in the database.

Analyzing Technology-Assisted Review Metrics with Sampling

Sampling is one of the most versatile tools in your technology-assisted review arsenal. The sampling process examines a fraction of the document population to determine characteristics of the whole, further validating what you do or don’t have and strengthening the defensibility of your review processes and procedures. Notably, it is often used to perform quality control (QC), which can take place iteratively, or at the back end of a review to assess it.

Quality control rests on a simple principle: TAR predictions are not always right. Through sampling, various sets of the data are drawn, manually reviewed by a quality control team, and evaluated. Based on these results, teams can decide whether additional training is needed, or the team might conclude that the technology is categorizing documents so effectively that they are comfortable relying wholly on machine predictions and stopping manual review.

The Next Step: Mastery

While general knowledge of these predictive coding metrics is a great start, it is merely a drop in the bucket. To learn more about mastering the math behind the TAR technology, don’t miss the May 10^th webinar hosted by Kroll Ontrack and ACEDS, MATH & STATS 101: What Lawyers Need to Know to (Properly) Leverage TAR.