Podcast: Predictive Coding and ei-Recall

ESI Report

As an ediscovery specialist, if you have not yet become acquainted with predictive coding – carpe diem – there is no shortage of information to get you caught up. If you are already an avid predictive coding user, now is the time to hone your methods.

A New Approach: Ei-Recall

Astute legal teams seek ways to re-examine the metrics used to evaluate the effectiveness of predictive coding – precision, recall and f-measure. Recently, Ralph Losey went on a quest for the “Holy Grail” of recall calculations. After many hours and sleepless nights, he devised a new approach to recall methods, ei-Recall. You can see his in-depth discussion of ei-Recall in a three-part series on his blog.

Download the Podcast

To whet your appetite for all-things ei-Recall, Ralph joined my colleague, Michele Lange, on the February 2015 ESI Report podcast. Listen to this podcast, which includes a summary of ei-Recall, how it compares to other recall methods and some of the advantages of the new recall process. Also, be sure to watch for future editions of the ESI Report on the Legal Talk Network.

Bust these 4 Myths on Your Next Document Review

Coffe Cup

Conducting an effective and efficient legal document review requires a hybrid of rocket science, brain chemistry and hot coffee. With ediscovery technology and best practices constantly evolving, if your document review practices haven’t changed in the last five years, you are likely wasting time and money for your organization and/or your client. To ensure your practices meet today’s demands, bust these four document review myths on your next ediscovery project.

Myth #1: Document review just happens; you don’t really need a plan.

There is an outmoded notion that document review is a side-show to the main ediscovery attraction; however, that view could not be farther from the truth. Without question, document review is the most expensive aspect of discovery; accordingly, planning for the review should start at the same time the case team is thinking about preservation and collection. Failing to create a big picture plan and a thorough review manual can result in:

  • Multiple “side reviews” for special issues or newly discovered documents
  • Downtime for reviewers as they wait around for more work
  • Documents being touched two or three times

Lesson #1: Don’t let any member of your case team procrastinate when it comes to planning for review.

Myth #2: Any old attorney can conduct (or manage) a document review.

With formidable technology at their fingertips and millions of documents to wrangle, today’s document reviewers are anything but humdrum lawyers. Today’s document reviewers are not only licensed and highly qualified attorneys, but they often have specific training and certifications in various document review platforms. They understand how to employ the strictest quality control protocols to maximize time and minimize the cost of a review. Further, sometimes a review requires expertise in a language other than English or substantive knowledge in the following areas: banking and financial services, pharmaceuticals, insurance, telecommunications, technology, agriculture, transportation, oil and gas, securities, and more.

Lesson #2: Make sure you are equipped with the right people resources for your next review.

Myth #3: All document review technology is equal.

While most major review platforms function in a generally-similar manner, there are significant differences in how the capabilities are executed by users – which impacts the expected results of a specific function. Rock star document review professionals understand and can articulate the features and benefits of multiple review tools and know how and when to leverage features such as workflow, predictive coding, near de-duplication, topic grouping, smart searching, machine translation (just to name a few of the vital technologies key to any modern review). Running searches, batching and assigning documents, categorizing, and performing QC should be second nature for experienced reviewers, and they should know the nuances of these functions in any specific technology platform. If they cannot answer a question, they should know how to get a hold of the technology provider’s technical support team to lend a hand.

Lesson #3: You should not settle for anything less than document review technology superstars. 

Myth #4: It will be obvious when you can stop your review…when you run out of documents.

With the advent of predictive coding and other technology assisted review technologies, no longer do attorney’s eyes need to be placed on every document. As such, determining when to call the review “complete” is complex. Today’s document review professionals are skilled at interpreting metrics and reports generated by the document review technology and know when the numbers are showing a high-quality review versus an incomplete review.

Lesson #4: Dust off your math skills (or leverage a specialist) – it’s the only way a savvy document review professional knows when a review is finished.

Leverage Kroll Ontrack for Document Review

With Kroll Ontrack, your document review could not be in better hands, eyes or minds. For nearly a decade, Kroll Ontrack has offered advanced document review services in the US. Now, with a document review facility in London, Englandnewly opened in December 2014 – Kroll Ontrack’s document review expertise is global.

February 2015 Ediscovery Case Summaries

ediscovery case studies

Pure Speculation is Not Enough for Sanctions                                                       Advantor Systems Corp. v. DRS Technical Services, Inc., 2015 WL 403308 (M.D. Fl. Jan. 28, 2015).

Court Refuses to Impose Sanctions Where No Duty Exists                                   Ballai v. Kiewit Power Constructors, 2015 WL 423795 (Ks. Ct. App. Jan. 23, 2015).

Court Attempts to Strike Balance in Scope of Discovery                                 Webb v. Ethicon Endo-Surgery, Inc., 2015 WL 317215 (D. Minn. Jan. 26, 2015).

Failure to Implement a Proper Legal Hold Leads to Sanctions
Fidelity Nat. Ins. Co. v. Captiva Lake Invs., LLC., 2015 WL 94560 (E.D. Mo. Jan. 7, 2015).

Proportionality Depends on Parties Cooperating                                                 Johnson v. Allstate Property and Cas. Ins. Co., 2014 WL 7377198 (W.D. Wash. Dec. 29, 2014).

Top 10 List: #LTNY15 Wrap-up


LegalTech New York 2015 is history, and, if you’re like me, you probably needed this week to recover. Below is my top 10 list of memorable moments in New York City last week, in case you missed it.

(Make sure to click on the hyperlinks below to see some of the best articles, pictures, videos and tweets from #LTNY15.)

10: The cold never bothers us, anyway. For the second year in the row, the east coast was blanketed in snow, ice and sleet the week of LTNY. But just like last year, the weather did not put a damper on the event – attendance was stellar – despite some pretty horrid travel delays.

9: Missing Browning: Make no mistake about it, one formidable LTNY giant was missing this year. The conference was just not the same without Browning Marean.

8: The Changing of the Guard. The big story at LTNY was the retirement of Monica Bay, LTN editor. Monica’s 30+ years of covering legal technology news offered a well-deserved reason to turn LTNY15 into a giant going-away party. From her picture in Times Square to a tribute rock-concert, it was nice to toast Monica’s accomplishments.

 7: New Name, New Editor. With Monica’s departure, American Law Media took the opportunity to announce a new name and new editor for its flagship brand. LTNY was a perfect opportunity to showcase Erin Harrison and Legaltech News.

 6: Expanded Exhibit Hall. In addition to all the usual players, the exhibit hall was filled with up-and-coming legal technology start-ups. This year LTNY partnered with CodeX (The Stanford Center for Legal Informatics) to bring new legal technology applications to the conference.

 5: Swag. Everyone loves a good give-away! From prize wheels to ducks and shoe shines, no one walked away empty handed.

 4: Hot Parties!  As tradition, the first night of LTNY kicked-off with a welcome reception hosted by Kroll Ontrack. It was a pleasure to grab a drink, enjoy some swinging music and catch-up with old friends. For after-hours socializing, the young-at-heart made sure to get their kCura Relativity wrist-bands to secure their admission into the late night LTNY extravaganzas.

3: Killer Keynotes. Top-notch speakers with interesting commentary: cyber security, privacy, innovation, international corruption and more!

 2: IG and PC. Information governance and predictive coding dominated the educational tracks once again this year.

1: A True Tech-a-Thon. Above all, the chief reason to come to LTNY is to vet prospective solutions and look for new legal tech ideas. From early morning breakfast meetings to vendor announcements, this year did not disappoint.

See you next year for #LTNY16: February 2 – 4, 2016!  (Or for you real die-hards, see you at Legal Tech West Coast in San Francisco, July 13 – 14.)

LTNY 2015: See You Next Week!


Kick-off February with a bang!

The first few days of February are jam-packed full of fun.  The Super Bowl on February 1, Groundhog Day on February 2, and finally – the grand finale – Legal Tech New York 2015, February 3-5.

At LTNY there’s a lot to take in, but to help you set your schedule, we’ve put together a list of some of the can’t-miss sessions sponsored by Kroll Ontrack:

Succeed or Screw Up? How to Evaluate (and Validate) Results in Your Next Predictive Coding Project 10:30-11:45am, February 4

Make sure to join this session moderated by Jonathan Sachs, including a panel of Ralph Losey, Cliff Dutton, and Shannon Capone Kirk, on how to understand the methods and key metrics for evaluating effectiveness in your next predictive coding project.

Around the World in Less Than 80 Minutes – Delivering Successful Ediscovery Projects Across the Globe 2:00-3:15pm, February 4

Use your time wisely and check out this discussion on international ediscovery. Andrew Szczech will be moderating a discussion between Emily Cobb, Jerami Kemnitz, Brian Calla and Kate Chan on the quickly evolving best practices in international litigation and investigations.

Applying Technology to Information Governance: Data Management, Litigation Holds, Records Retention and Destruction 3:45-5:00pm, February 4

Attend this panel of Bennett Borden, D. Casey Flaherty, David Stanton and Jessica Escalera, moderated by Cathleen Peterson, on all things information governance. The panel will hit on such hot topics as how to develop information governance programs and what IG projects companies are undertaking today.

Lastly, if you want to top off Day 1 of the show, don’t forget to stop by the West Grand Ballroom on February 3rd from 5-7pm for the welcome reception sponsored by Kroll Ontrack, “Glitz and Glam New York Style.”

Find up-to-date information about what Kroll Ontrack will be doing this year (prizes, parties, exhibit hours, etc.) at our LTNY 2015 webpage. And, don’t forget to follow Kroll Ontrack on Twitter for live tweets, pictures and videos from the show floor!

January 2015 Ediscovery Case Summaries

ediscovery case studies

Court Imposes “New and Simpler Approach to Discovery,” Orders Search Terms
Armstrong Pump, Inc. v. Hartman, 2014 WL 6908867 (W.D.N.Y. Dec. 9, 2014).

Court’s Case Management Order Includes TAR, Provides Picky Production Requirements
Green v. Am. Modern Home Ins. Co., 2014 WL 6668422 (W.D. Ark. Nov. 24, 2014).

Failure to Produce Social Media Data Leads to Heavy Sanctions
Federico v. Lincoln Military Hous., 2014 WL 7447937 (E.D. Va. Dec. 31, 2014).

Gross Negligence in Destruction of Attorney Computer Leads to Adverse Inference Sanction
Dorchester Fin. Holdings Corp. v. Banco BRJ S.A., 2014 WL 7051380 (S.D.N.Y. Dec. 15, 2014).

Plaintiffs Not Obligated to Produce Data Maintained by Ediscovery Vendor
Ablan v. Bank of Am. Corp., 2014 WL 6704293 (N.D. Ill. Nov. 24, 2014).

Reduce Ediscovery Storage Costs through Nearlining


Nearline. If this term isn’t in your ediscovery vocabulary, it should be.

Nearlining is a feature that allows litigation teams to quickly and easily set aside irrelevant documents and keep them separate from the active document roster in a discovery database. By moving non-critical data from active to nearline in a document review tool, databases remain nimble and hosting costs are reduced, as nearline storage is less expensive.

Nearlining Promotes Speed

Ediscovery can move slowly at times, and most litigation teams are aware of the cost and time needed to effectively gather and isolate relevant documents in the ediscovery process. As such, they are eager to begin the process of reviewing (or at least compiling) documents sooner rather than later. Kroll Ontrack’s nearlining function allows those involved in the process to do just that, as described in a recent ediscovery.com Review case study. Documents can be added to the pool early on, regardless of whether search terms have been determined. This allows litigation teams to collect broadly at the start and then hone in on relevant documents later, without adding increased costs.

Keep All of the Data at Your Fingertips

Ediscovery is an ongoing process, and the definitions of what data is appropriate or relevant to a matter change frequently, causing undue delays or costs. As discussed in a recent ESI Report podcast, nearline technology in ediscovery.com Review allows users to set aside 100,000 documents in just 35 seconds with nothing more than the click of a mouse. The data is removed from the primary pool of relevant data but remains completely accessible. At any time, nearlined data can be brought back into the main pool, if the definition of what data is relevant or appropriate changes as the matter progresses.

Reduce Volume, Save in Data Hosting

A clutter-free data set is a beautiful thing. Nearlining streamlines the process of removing clutter by allowing users to quickly and easily identify and remove the obviously irrelevant “junk” data. This not only reduces the overall data footprint and saves on data hosting costs, but it also keeps the primary data pool smaller. Searches of that data pool will therefore be faster and much more focused.

Check out Kroll Ontrack’s latest case study and podcast to learn more about the savings that nearlining can bring in your next ediscovery project.

Further, to keep up with the latest ediscovery news and developments, straight from the mouths of the experts, download the current and archived episodes of the ESI Report, hosted by the Legal Talk Network.

5 Minute Quiz: Test your Ediscovery Fitness Level

ediscovery fitness quiz

To kick off 2015, take a minute to reflect on these New Year’s Resolution statistics:

  • 45: Percentage of Americans who usually make New Year’s resolutions
  • 66: Percentage of resolvers who set fitness goals as part of their resolutions

Did one of your 2015 resolutions involve fitness – losing weight or eating healthier? Are you feeling daunted by the goal in front of you? The good news? Not all fitness goals require a trip to the gym.

Ediscovery Fitness Quiz

Keeping with the theme of a new year and a new you, let’s talk about your ediscovery fitness levels. Do you know your way around the EDRM or have you just been introduced to ESI? Whether you are new to ediscovery or a seasoned pro, chances are you have some knowledge that you want to show off.

Looking for a chance to flex those ediscovery muscles that you have been building throughout 2014? You’re in luck! We set up a circuit training exercise for you to test your ediscovery knowledge, ranging from the EDRM to recent case law and everything in between.

Hop, skip and jump on over to take the quiz and see where you fall on the ediscovery fitness spectrum.

Win a FitBit from Kroll Ontrack

Need an incentive to start the workout? Those who are brave enough to participate in the 5 minute quiz will be entered for a chance to win a FitBit® Flex sponsored by Kroll Ontrack.

An opportunity to flex your ediscovery muscles AND win an awesome Internet of Things gadget? How could you say no to that challenge?  Test your knowledge today!

Thinking about awards season? Time to Vote for the National Law Journal’s “Best of 2015”!

NYLJ survey

As we approach awards season and you scramble to watch all of the nominated movies and TV shows (or add them to your to-do list), we here at Kroll Ontrack are looking toward something that hits a bit closer to home.

From now until February 5th the National Law Journal (NLJ) will be accepting votes for its own “People’s Choice” annual reader rankings survey. This survey is designed to give end-users the opportunity to rate the products and services they have been using in litigation.

Please take a moment to cast your vote today. You don’t need to be a National Law Journal subscriber to participate. The survey does take some time but you are not required to vote for every category.

While Tina Fey and Amy Poehler aren’t hosting the survey, and there probably won’t be a selfie that breaks Twitter involved, we encourage you to participate. Not only is your feedback invaluable as we work toward better serving our clients, but your participation will also provide valued information to your colleagues as they approach litigation with modern technology.

It has been an honor to find out that Kroll Ontrack has been nominated in the following categories:

  • Best end-to-end litigation consulting firm
  • Best end-to-end ediscovery provider
  • Best technology assisted review ediscovery solution
  • Best data and technology management ediscovery provider
  • Best date recovery solution provider
  • Best managed document review services
  • Best managed ediscovery and litigation support service provider
  • Best online review platform
  • Best case management software

To those of you who have already taken the time out of your day to cast a vote, thank you! We appreciate your feedback more than you know. To those of you who have yet to vote, we’ll see you at the polls!

Ediscovery Resolutions for the New Year!

ediscovery resolutions

Happy New Year and welcome to 2015!

As one year ends and a new one begins it’s an excellent time to take stock of our lives. Traditionally, the turn of a new year is accompanied by reflections on past accomplishments and resolutions and goals for the coming year and beyond. What is it that you are committed to achieving in 2015? Whether your personal resolution is to spend more time with family, learn how to cook or get to the gym more, we all have resolutions we want to focus on and goals that we want to achieve.

This rings true for our professional lives, as well. We recently had a chance to hear from 12 litigation professionals on what their 2015 professional resolutions are in the field of ediscovery. From better data preservation to a heavier use of predictive coding, these professionals have resolutions spanning the entire EDRM.

Here is a snippet of the 2015 New Year’s Resolutions “ESI Report” podcast, as recorded by the Legal Talk Network:

“My team and I are focusing on improving the attorney experience when working with litigation support and electronic discovery.” Danny Thankachan, Thompson & Knight, Dallas, Texas

“My ediscovery resolution is to use predictive coding on every document production that I receive from opposing counsel.” Cliff Nichols, Day Pitney, Stamford, Connecticut

“I want to help my clients use automated tools and processes to increase their consistencies in ways that are appropriate for their business and their processes.” James Sherer, Baker Hostetler, New York City

There is plenty in the field of ediscovery for all of us to look forward to in 2015. Below is a list of people who were interviewed on their 2015 ediscovery resolutions:

Ralph Losey, Jackson Lewis, Orlando, Florida

Anthony Diana, Reed Smith, New York City

David Yerich, United Health Group, Minneapolis, Minnesota

Shannon Capone Kirk, Ropes & Gray, Boston, Massachusetts

Danny Thankachan, Thompson & Knight, Dallas, Texas

Josh Zylbershlag, Paul Weiss, New York City

Cliff Nichols, Day Pitney, Stamford, Connecticut

David Baldwin, Choate Hall & Stewart, Boston, Massachusetts

Daniel Kavan, Kroll Ontrack, London, England

Sue Kaiser, Jones Day, Cleveland, Ohio

Joel Bothof, Medtronic, Minneapolis, Minnesota

James Sherer, Baker Hostetler, New York City

Don’t miss out on what these industry experts are striving for in 2015. Who knows, their ideas may even help you shape your professional resolutions. Download the latest ESI report and listen to it whenever and wherever you want. Cheers to the new year!