All posts tagged Sedona Conference

The Sedona Conference Commentary on Proportionality

As discussed in a blog back in January, 2017 is the year of proportionality.

The Sedona Conference Working Group on Electronic Document Retention & Production (WG1) recently released a new version of its publication, The Sedona Conference® Commentary on Proportionality in Electronic Discovery. This is the working group’s third iteration of this publication and is intended to address the 2015 amendments to the Federal Rules of Civil Procedure (FRCP).

This latest version focuses on the amendments to FRCP Rules 26 and 37, emphasizing the importance of proportionality in discovery. The authors note the balance of burden against benefit, stress that reasonableness depends heavily upon the facts within each case and underscore the importance of proactive communication between the parties throughout the litigation process. Additionally, the prominence of proportionality means that the amount in controversy is now only one of many factors that courts use to define the scope of discovery.

Specifically, the Sedona publication examines six proportionality principles:

  1. The burdens and costs of preserving relevant electronically stored information should be weighed against the potential value and uniqueness of the information when determining the appropriate scope of preservation.
  2. Discovery should focus on the needs of the case and generally be obtained from the most convenient, least burdensome and least expensive sources.
  3. Undue burden, expense or delay resulting from a party’s action or inaction should be weighed against that party.
  4. The application of proportionality should be based on information rather than speculation.
  5. Nonmonetary factors should be considered in the proportionality analysis.
  6. Technologies to reduce cost and burden should be considered in the proportionality analysis.

This updated document is another valuable resource for litigation teams dealing with the implications of the 2015 FRCP amendments. Commendations to the drafting team, and all those involved in the Sedona Conference!

Ediscovery 2011: Gibson Dunn’s Mid-Year Ediscovery Update

Ediscovery 2011: Gibson Dunn’s Mid-Year Ediscovery Update

The paramount role ediscovery plays in modern litigation is thoroughly emphasized by the 2011 Mid-Year Ediscovery Update released by Gibson Dunn. This comprehensive analysis of court decisions and trends over the first half of 2011 provides a guide to where ediscovery has been and where it may continue to go. We have read through Gibson Dunn’s excellent report and include below the highlights that caught our eye.

Preservation and Legal Holds

Preservation continues to be the number one issue plaguing corporations and practitioners alike. This challenging concept was further complicated following Judge Scheindlin’s Pension Committee decision of 2010. Throughout 2011, courts struggled with tension arising from Scheindlin’s bright-line rule in Pension Committee as demonstrated by the number of courts – including in Judge Scheindlin’s own district – that declined to adopt this rule. For example, Magistrate Judge James Francis in Orbit One Communications, Inc. v. Numerex Corp., respectfully disagreed with the holding in Pension Committee and held that sanctions would not be warranted unless a party could prove relevance of the lost information.

While it appears that in some cases an oral notification may be sufficient, the best practice for companies and attorneys who find themselves confronted with the need to preserve information is to issue legal holds in writing. In a similar vein, courts are re-emphasizing that merely distributing a legal hold alone may not be sufficient preservation. Instead, attorneys should ensure compliance with the hold and take steps to facilitate preservation along the way.

Information Disclosure and Software

An additional topic covered by the report was the use of software and its impact in case law. As attorneys continue to shift their reliance to comprehensive software and solutions to aid them in the ediscovery process, courts continue to contemplate whether litigants have met the “reasonable steps” requirement to prevent the disclosure of privileged information pursuant to Federal Rule of Evidence 502(b). In Sidney v. Focused Retail Property I, LLC, the Northern District of Illinois cited to a 2010 case from its district, Kmart Corp. v. Footstar, Inc., in which the court found he party failed to take reasonable steps to prevent disclosure by failing to mention whether software was used to prevent disclosure of privileged documents. Reputable ediscovery software can cut costs, save time, reduce headaches – especially associated with document review – and provide defensibility to your process and actions.

Keeping the (Good) Faith: FRCP 37, Cooperation and Karma

The 10th Circuit Appeals Court recently issued an opinion, Lee v. Max International, LLC, that imparts valuable insight regarding the cause and effect of poor discovery behavior. In this case, the plaintiffs committed three repeated discovery violations that compelled the court to write, “[b]ut there is such thing as discovery karma. Discovery misconduct often may be seen as tactically advantageous at first. But just as our good and bad deeds eventually tend to catch up with us, so do discovery machinations.” The court affirmed the lower court’s dismissal of the case as an appropriate sanction. Considering the rise of sanctions cases identified in the Gibson Dunn report, parties must continue to educate themselves regarding ediscovery best practices and implement strategies to approach discovery efficiently and cost-effectively. Otherwise, you risk incurring sanctions that may impact you monetarily or even result in the dismissal of your case – before it even gets to the merits of the underlying litigation.

The Report also highlighted the growing trend within the judiciary to emphasize the importance of cooperation in the ediscovery process. Courts are increasingly endorsing the Sedona Conference® Cooperation Proclamation and are urging practitioners (and sanctioning them if they do not comply) to meet and confer in good faith to resolve disputes. For example, the District of Puerto Rico recently concluded an opinion that followed the filing of over 25 motions related to the discovery process with a call to litigants and the general Bar to adopt civility as a standard of professional conduct, noting that it the concept of civility is indeed not in direct contravention with zealous advocacy and that by adopting this standard, litigants can provide a better service to their clients. Baez-Eliza v. Instituto Psicoterapeutico de Puerto Rico, 2011 WL 2413051 (D. Puerto Rico June 16, 2011).

The Gibson Dunn report covered many other areas that are important in terms of case law and the ediscovery process. We here at Kroll Ontrack urge everyone involved in ediscovery to continue reading resources such as the report and resources found on our website in order to further cultivate their ediscovery knowledge. Once you better understand the case law, processes and best practices behind this exciting area of the law, you will be better equipped to navigate the murky waters effectively.

Case Law: Haraburda v. Arcelor Mittal USA, Inc

Case Law

Court Orders Defendants to Issue Litigation Hold Before Rule 26(f) Conference

Haraburda v. Arcelor Mittal USA, Inc., 2011 WL 2600756 (N.D. Ind. June 28, 2011). In this employment discrimination suit, the plaintiff requested the court order the defendant to preserve e-mail evidence, claiming the defendant previously deleted e-mails from the plaintiff’s account without her permission and refused to issue a litigation hold prior to the Fed.R.Civ.P. 26(f) meet and confer. The defendant argued the plaintiff’s request was premature as Rule 26(d)(1) prohibited a party from seeking discovery before the Rule 26(f) conference. Disagreeing with the defendant’s argument, the court noted Rule 26(d)(1) prohibited requesting production – not compelling preservation – and stated that ruling to the contrary would leave a party with knowledge of an intent to destroy evidence without a remedy. Accordingly, the court found the plaintiff could suffer measurable prejudice based on the suit’s heavy reliance on e-mails if evidence was destroyed and ordered the defendant to implement a litigation hold.


Although the defendant put forth a novel argument, it failed to acknowledge the obligations each party has to proactively preserve evidence upon the reasonable anticipation of litigation. As the Sedona Conference® notes, the duty to preserve evidence “includes an obligation to identify, locate, and maintain, information that is relevant to specific, predictable, and identifiable litigation”. The Sedona Conference®, Commentary on Legal Holds: The Trigger and the Process.

However, the issue of when the duty actually arises is often a challenging one as different courts have found various triggers to be applicable. Generally, courts recognize that the “mere possibility of litigation” does not trigger the duty to preserve because litigation is “an ever-present possibility” in modern society. Cache La Poudre Feeds, LLC v. Land O’ Lakes, Inc.

Recently, the District of Colorado determined that the defendant’s duty to preserve evidence triggered when the plaintiff filed a formal complaint at work (racial discrimination lawsuit), which put the defendant on notice to preserve all existing and future video that included the plaintiff. The defendant failed to do so, and the court found the plaintiff was prejudiced by the defendant’s willful destruction of video recordings. Accordingly, the court issued both mandatory and permissive adverse inference instructions as well as attorney fees relating to the spoliation motion. McCargo v. Texas Roadhouse, Inc., 2011 WL 1638992 (D. Colo. May 2, 2011). In the 2009 case, Goodman v. Praxair Services, Inc., the District of Maryland concluded that the defendant’s duty to preserve triggered following receipt of a letter informing the defendant the plaintiff had consulted attorneys. Further, a ruling from the Southern District of New York found the duty to preserve arose no later than the lawsuit’s filing. Green v. McClendon.

As demonstrated by the sampling of cases above (which by no means present an exhaustive list of possibilities), it is no wonder parties are confused as to when the duty to preserve arises. It is better for parties to be safe than sorry by implementing a written legal hold sooner rather than later if litigation appears to be on the horizon. To increase defensibility, parties should maintain detailed notes of the preservation protocol followed, which includes when the hold was issued, what details were included in the hold, to whom the hold was issued and the efforts taken to continually monitor compliance.

Case Law: In re Facebook PPC Adver. Litig.

Case Law

Court Orders Parties to Meet and Confer to Develop Discovery Plan and Resolve Production Disputes

In re Facebook PPC Adver. Litig., 2011 WL 1324516 (N.D. Cal. Apr. 6, 2011). In this breach of contract litigation, the plaintiffs sought resolution of various production disputes including a disagreement regarding the development of an Electronically Stored Information (ESI) Protocol. Rejecting as speculative the defendant’s concern that entering an ESI Protocol would frustrate and slow the discovery process, the court ordered the parties to meet and confer to develop an ESI Protocol in light of the “clear thrust of the discovery-related rules, case law, and commentary,” including the Federal Rules of Civil Procedure, case law and the Sedona Conference® that suggest communication is crucial to a successful discovery process. Turning to the production disputes, the court prohibited the defendant from using a secure and restrictive document-viewing website in lieu of actual production, finding the website was unduly burdensome, inefficient and unnecessary in light of a two-tiered protective order. The court also ordered the defendants to reproduce unusable, non-searchable files – including an 18,000 page customer complaint database – in their native format, disclose relevant source code, and meet and confer to determine the appropriate method of production for proprietary documents relating to the dispute.


In this case, rather than producing certain documents, the defendant uploaded them onto website, that provided the plaintiff with viewing access, but gave the defendant the ability to limit it in significant ways: restrict access, track viewer activity, set document expiration dates and prohibit searching, annotating and printing. The defendant’s justification for the use of this service in lieu of actual production was that it was necessary to protect the disclosure of “extremely sensitive and confidential” documents. As noted in the summary above, the court denied these justifications and noted that the two-tiered protective order (which the defendant agreed to be bound by) provided sufficient protection to obviate the website’s use. Accordingly, the court ordered the defendant to stop using the website and produce the documents in had uploaded to the plaintiff in a fully accessible form.

Although this opinion is relatively brief in length and scope, it involves certainly a creative attempt to circumvent the more traditional methods of production. After all, cases involving parties withholding documents altogether are very common, but a party offering only limited access via a secure portal is a relatively novel development in the ediscovery world.

This case demonstrates the importance of reaching an agreement regarding the use of these solutions at the meet and confer. The court’s opinion seems to indicate that if the parties had agreed to use the restricted viewing technology before discovery began, it may have been permitted. Taking the time to meet and confer and negotiate mutually beneficial production agreements can pay dividends by potentially avoiding unnecessary, expensive discovery disputes and protecting important party-specific interests.

Want to learn more about this case? Listen to our recent ESI Report podcast that delves into case facts, specifics and lessons to be learned

Metadata – You Be the Judge

Metadata – You Be the Judge

As we all know well, ediscovery is a relatively young area of the law that is changing and developing rapidly. The objective of this series is two-fold: (1) continue highlighting changes and developments and (2) actively solicit feedback on the substantive legal issues from our readership. With the introductory remarks taken care of, let’s move on to today’s discussion – metadata.

Metadata is essentially data about data. It “describes when a file was created, where it was stored, and what programs the computer uses to help access the file.”[1] When in “native” format, all electronically stored information carries associated metadata. Some of this metadata details user activity on a file, such as edits and save dates, while other forms consist of background data necessary to the operation of the file but hidden from view. As the ediscovery process continues to evolve, this inconspicuous data is becoming increasingly important. Depending upon the circumstances of the case, metadata can be anywhere from relatively insignificant to practically crucial. For example, metadata is critical to determining the integrity and authenticity of files.[2] Metadata is also essential to conducting effective searching during the review process. Yet, the current Federal Rules of Civil Procedure do not expressly require mandatory disclosure of this information, and absent court order or party agreement, appear to allow parties to exclude it from their production.[3] Indeed, Rule 34(b)(2)(E) states that if production format is not specified, parties may produce ESI in the form “in which it is ordinarily maintained or in a reasonably useable form.” (emphasis added). This generally allows parties to produce ESI in TIFF or PDF – which permit Bates-stamping – without any duty to include accompanying load files, for example.

Parties certainly have the right to request metadata prior to production under Rule 34(b), but at a time when so many attorneys are simply unaware of its importance, or existence all together, it is often overlooked until it is too late.[4] With the growing importance of metadata to the ediscovery process, the question becomes whether metadata should be considered an essential and mandatory part of initial disclosures.

Not To Produce Metadata

In Autotech Technologies Limited Partnership v., Inc., the Northern District of Illinois noted that “[o]rdinarily, courts will not compel the production of metadata when a party did not make that a part of its request.”[5] This holding was echoed in the more recent case of Chapman v. General Board of Pension and Health Benefits of the United Methodist Church, Inc., noting that “parties who do not specifically request metadata are not typically entitled to it if the responding party has already produced the documents in another reasonably useable form.”[6] The District of Columbia reached the same conclusion in D’Onofrio v. SFX Sports Group, Inc.,[7] holding that “[b]ecause no such request [for metadata] has been made concerning the Business Plan, the Court will not compel the defendant to produce it in its original form with accompanying metadata.” Indeed, the approach is consistent with the Sedona Principles interpretation of the Federal Rules of Civil Procedure, which noted that in developing the 2006 Amendments, the Advisory Committee rejected proposals to require production of metadata.[8] Instead, the Advisory Committee preferred the parties to discuss production and reach their own agreement, as directed under Fed.R.Civ.P. 26(f).[9]

Produce Metadata

In contrast, other courts have found that the growing importance of metadata renders it crucial to discovery. Judge Shira Scheindlin, author of the seminal Zubulake opinions, held “it is well accepted, if not indisputable, that metadata is generally considered to be an integral part of an electronic record.

Produce Metadata

In contrast, other courts have found that the growing importance of metadata renders it crucial to discovery. Judge Shira Scheindlin, author of the seminal Zubulake opinions, held “it is well accepted, if not indisputable, that metadata is generally considered to be an integral part of an electronic record.” National Day Laborer Org. Network v. United States Immigrations and Customs Enforcement Agency, 2011 WL 381625 (S.D.N.Y. Feb. 7, 2011). Although Nat’l Day Laborer limited this declaration to requests for public records under the Freedom of Information Act, much of the discussion regarding the production of metadata seemed to include general admonitions. Indeed, much of the legal community’s commentary regarding the opinion has believed it is intended to have a more general scope, or at least lay the groundwork for such a trend. Indicative of this is the fact that the court took note of The Sedona Principles: Second Edition’s about-face from the 2005 Sedona Principles, in which the Sedona Conference® now embraces the production of metadata even absent party agreement.[10]

The District of Colorado took this approach in Chevron Corp. v. Stratus Consulting, Inc.[11] when it considered the issue as a matter of first impression. Comparing Aguilar v. Immigration & Customs Enforcement [12] (heavily cited by Nat’l Day Laborer) and White v. Graceland College Center for Professional Development,[13] the court found the latter to be more applicable because the timing and authenticity of the documents were at issue in the case, and thus required production of metadata to be “reasonably useable” under Rule 34.[14]


Although Chevron and White ordered metadata production because the specific circumstances of the case required it, very few cases do not depend upon the authenticity of authorship or timing of documents to some degree. Further, the “reasonably useable” requirement has been read to include efficiency considerations,[15] and because the availability of metadata can enhance the usefulness of most document review technologies, it can lead to reduced costs and a more effective review process. While leaving it up to the parties to address the issue in the Rule 26(f) meeting may seem to be an acceptable compromise, the practical realities illustrated in ediscovery case law have made it clear that facilitating an effective meet and confer session is difficult and rare enough already, without trusting the average, non-tech savvy attorney to explicitly request this relatively inconspicuous but extremely important information. However, expanding Rule 34 to require production of metadata also broadens the duty to preserve, which could increase discovery disputes and offset any gains realized from the change.

Cooperation & Proportionality – Essential Pieces to the Ediscovery Puzzle

Cooperation & Proportionality – Essential Pieces to the Ediscovery Puzzle

Like an unstoppable force meeting an immovable object, the rapid pace of technology and the staunchly conservative nature of the law could hardly be in greater conflict.

As the two forces continue to clash, the fallout can be seen in attorneys – doing what they believe is best – battling relentlessly while costs continue to soar. Caught in the middle of it all are the clients and courts, desperately searching for a resolution to the madness. In the midst of this chaos, two familiar concepts have emerged in an attempt to restore order – cooperation and proportionality.

Love it or hate it, electronic discovery is here to stay and parties must work together to curb the exorbitant (and increasing) costs involved in locating, reviewing and producing data in a lawsuit or regulatory matter. U.S. courts’ strong appetite for liberal discovery is tempered by the concept of proportionality, which recognizes that at some point ongoing discovery results in diminishing returns. Aside from its common law roots, proportionality is embodied in Federal Rule of Civil Procedure 1, stating that the rules should be “construed and administered to secure the just, speedy, and inexpensive determination of every action and proceeding,” as well as Rule 26, which permits courts to limit discovery if they determine the burden outweighs the benefit. As for proportionality’s counterpart, cooperation, its spirit can be found throughout legal opinions, scholarly commentary and model codes such as the American Bar Association’s Model Rules of Professional Conduct – particularly in Rules 3.2 and 3.4.1

With both of these concepts’ illustrious presence and history in litigation, one would think their application in ediscovery should be fairly straightforward. Yet court-ordered sanctions and costs related to ediscovery continue to rise at a startling pace.2 It is evident that although the basic framework to support proportionality and cooperation is in place, we still need to fill in the gaps. Thankfully, commentators and scholars from both the bench and bar, along with industry experts, have provided the missing pieces – we just need to put them together.3

Fear of the Unknown

In the ediscovery context, knowledge is power and ignorance is disabling. Unfortunately, the latter is far more common. No matter your professional background, the e-discovery process can be complex and daunting. Litigators who do not fully understand the process and intricacies are far more likely to make mistakes that can significantly prolong litigation and increase costs. Fear of the unknown and resulting defensiveness can be as (or more) problematic than being uneducated in this space. All attorneys naturally fear the fatal mistake of disclosing the proverbial “smoking gun” that sinks their client’s case.4 However, when discovery involves millions of pages of documents – as e-discovery commonly does – it is no wonder why costly production and spoliation disputes dominate e-discovery jurisprudence.

Thankfully, the solution is relatively simple and something attorneys are no stranger to – education. In the past few years, many organizations have been formed to encourage and help attorneys, clients and the judiciary to work together and better understand the nuts and bolts of ediscovery. For example, the Seventh Circuit’s Electronic Discovery Pilot Program’s Principles Relating to the Discovery of Electronically Stored Information has recognized the need for cooperation, proportionality and education “to reduce the rising burden and cost of discovery . . . brought on primarily by the use of electronically stored information (ESI) in today’s electronic world.”5 Important concepts from this effort include Principle 1.02, which states “[a]n attorney’s zealous representation of a client is not [emphasis added] compromised by conducting discovery in a cooperative matter” and also acknowledges that the “failure of counsel . . . to cooperate in facilitating and reasonably limiting discovery requests and responses raises litigation costs.” In addition, Principle 1.03 instructs that the application of the proportionality standard in discovery will be furthered by crafting discovery requests and responses that are “reasonably targeted, clear, and as specific as practicable.” Further, Principle 3.01 stresses the pressing need for education by calling for judges, attorneys and parties to “become familiar with the fundamentals of discovery of ESI,” and Principle 3.02 establishes a duty to pursue continuing education in this regard. According to the Phase One report from May 2010, “92 percent of the judges agreed that the Principles [set forth in the program] had a positive effect on counsels’ ability to resolve discovery disputes before requesting court involvement.” Also according to this report, Principle 1.02 was often viewed as incredibly useful and allowed counsel to cooperate with each other while fulfilling their obligation to zealously advocate on behalf of their clients.6

It is no secret that a lack of knowledge is at the root of many ediscovery failures and is a necessary precondition to effective cooperation, yet the calls for increased education have focused on short-term solutions. Familiarity with rules and best practices is not enough to solve a systemic problem. A long-term solution in the form of more in-depth, formalized education is necessary. Law schools must educate future attorneys on ediscovery principles, and knowledgeable practitioners must share their ediscovery expertise with their colleagues through continuing legal education (CLE) courses.

Building a Solid Foundation

While education may help foster cooperation, it is only one piece of the puzzle. In the meantime, courts still need practical tools to address the current lack of cooperation and skyrocketing discovery costs.

One of the more novel approaches has been the use of phased discovery. In a recent discovery order from the Northern District of Illinois, Magistrate Judge Nan R. Nolan (chair of the Seventh Circuit Pilot Program, discussed earlier) ordered the parties to engage in a “phased discovery schedule.”7 After familiarizing themselves with ediscovery principles including cooperation, the parties were ordered to engage in “cooperative discussions to facilitate a logical discovery flow.” Consistent with principles of proportionality and the Federal Rules of Civil Procedure, this flow was to begin with completing Rule 26(a) initial disclosures, followed by a narrowing of the discovery scope to “claims most likely to go forward” and, finally, prioritizing discovery efforts according to sources that are least “expensive and burdensome.”8 Last, and arguably most important, Judge Nolan dictated that “nothing in this Order shall prejudice the parties from conducting all forms of discovery” if necessary in the future, making clear that phased discovery does not mean limited discovery.

The concept of proportionality in the context of document production is well-suited to address many of the cost problems. Although relatively few courts have expressly ordered it, there is little conflict over its value. In the production context, proportionality is logical, cost-effective and in harmony with the underlying principle of liberal discovery.9 Unfortunately, production is only one part of the ediscovery process.

Preservation is no less important to the ediscovery process than production; however, its relationship with proportionality may be slightly more dubious. In Orbit One Communications v. Numerex Corp., the Southern District of New York rejected the standard of “reasonableness and proportionality” advocated in Victor Stanley II and Rimkus Consulting Group as “too amorphous to provide much comfort to a party deciding” what information to retain. Instead, the court favored the Zubulake IV standard of retaining “all relevant documents . . . in existence at the time the duty to preserve attaches.”10 Courts are understandably reluctant to grant too much leeway to parties to determine what information is worth saving on the reasonableness and proportionality criteria dictated throughout case law and the rules. Proportionality and “phasing” have become critical because litigants have been unable to effectively manage their growing masses of ESI. Data proliferation has shown no sign of slowing down, so there is little reason to believe that proportionality and phased approaches to ediscovery can be a viable long-term solution when neither addresses the underlying issue – proper ESI management.

Technology is designed to make difficult tasks easier. Many corporations are in a quandary, however, because they have adopted technology to create information more efficiently without also implementing technology to manage it more efficiently. In regard to ESI preservation, enterprise archiving technology that can manage vast stores of data with remarkable efficiency is readily available. In addition, hosted discovery repositories provide a cost-effective and secure location to preserve, filter and search large volumes of ESI that are, or may likely become, relevant to litigation and investigations. On the other end of the ediscovery process, technology-aided document review tools are poised to revolutionize this part of the process, which is typically the most expensive and time-consuming. Historically, there simply may not have been a strong enough countervailing incentive to manage the growing stores of information, but it is a safe bet that the interests of justice will soon fill that void and litigants will be expected to responsibly manage their information throughout its life cycle.


No doubt cooperation is imperative to managing an effective ediscovery process, but court orders and sanctions only go so far. Fostering cooperation in the long term will require a paradigm shift in the way litigators approach the ediscovery process. A key piece of this lies in ensuring that jurists and litigants understand this complicated process. As for proportionality, novel approaches such as phased discovery may be sufficient to stave off the tradition of liberal discovery for now, but there is a demonstrated need to revolutionize the process in terms of technology and efforts used to manage data more cost-effectively and efficiently. While the best answer may be unclear, what is perfectly evident is that we already have all the pieces to solve the puzzle – we just need to put them all together.

1 See American Bar Association Model Rules of Professional Conduct, Rules 1.1, 3.2 and 3.4, available at:

2 See Kroll Ontrack, Fourth Annual ESI Trends Report (2010) (Finding 13: On average, companies spend $1.25 million per year on discovery); also The Kroll Ontrack Year in Review 2010, available at:

3 The Sedona Conference® has provided numerous commentaries on proportionality and cooperation that practitioners should read. See The Sedona Conference® Cooperation Proclamation; also The Sedona Conference® Commentary on Proportionality in Electronic Discovery, available at:

4 See Mt. Hawley Ins. Co. v. Felman Prod., Inc., 2010 WL 1990555 (S.D.W.Va. May 18, 2010) (Plaintiff’s failure to perform critical quality-control sampling and other discovery failures resulted in the inadvertent production of an alleged “smoking gun” e-mail).

5 Available at:

6 Seventh Circuit Electronic Discovery Pilot Program Phase One Report available at

7 Tamburo v. Dworkin, 2010 WL 4867346 (N.D. Ill. Nov. 17, 2010).

8 Id. at 3.

9 Fed.R.Civ.P. 26(b)(1) (“Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party’s claim or defense”).

10 2010 WL 4615547 (S.D.N.Y. Oct. 26, 2010).

Note: The above post appeared in the May 2011 issue of the free, monthly e-newsletter, Case Law Update & Trends published by Kroll Ontrack. This newsletter is designed to help busy legal professionals keep pace with case law and information pertaining to electronic evidence. Subscribe and gain valuable and timely information on new ESI court decisions, as well as informative articles and tips for both the corporate and law firm audience.

Two Truths and a Fib About Intelligent Categorization

Two Truths and a Fib About Intelligent Categorization

Time is money, and linear document review is almost prohibitively expensive due to the surge in electronic data volume over the past several years and the corresponding increase in resources required to review the data. Besides time and costs, having a multitude of attorneys reviewing and categorizing documents for (potentially) months on end can yield inconsistent results. Innovative technological advances have arrived on the document review scene, but concerns about overall effectiveness persist as the legal industry remains hesitant to explore new technology.

Enter Intelligent Categorization (iC). Intelligent Categorization is the third component of Intelligent Review Technology (IRT) that analyzes and learns from category decisions made by human reviewers, then identifies and elevates documents most likely to be relevant and suggests categories for documents not yet reviewed. Along with Automated Workflow and Intelligent Prioritization, the other two legs of IRT, reliance on Intelligent Categorization technology is on its way to becoming a well-established practice in 2011. Differing ideas and opinions associated with this technology have been tossed around, giving rise to certain ideas and misconceptions about what iC is, what iC is not and what iC can do for electronic discovery. Today we will dissipate the confusion and set the record straight by exploring two important truths and a common fib associated with Intelligent Categorization.

Defensible? True.

First and foremost, Intelligent Categorization is defensible. One of the early qualms about iC was that until the technology became court-tested, it was too risky to use. That simply is not the case. In fact, such fears have preceded the acceptance of all new technology, including features such as advanced searching and sampling, which are now embraced by jurists and litigants alike.[1] Case law supports the use of a systematic, well-documented and repeatable process, and Intelligent Categorization is specifically designed to increase accuracy and effectiveness while decreasing review time. Indeed, when using all three components of Intelligent Review Technology, it is possible to save 50 percent on review costs.

Intelligent Categorization also supports the notions of proportionality set forth in Rule 1 of the Federal Rules of Civil Procedure, with the goal of proceedings to be “just, speedy and inexpensive.”[2] As an integrated component of IRT, iC is fully transparent with real-time metrics and analytics available throughout the review process. In addition, experts can explain the technology to judges, opponents, clients and staff if necessary.

Further, the Sedona Conference® has endorsed the use of automated methods (although has not endorsed particular technologies to do so). The Sedona Conference Best Practices Commentary on the Use of Search and Information Retrieval Methods in Ediscovery, Practice Point 1 states:

[R]eliance solely on a manual search process for the purpose of finding documents may be infeasible or unwarranted. In such cases, the use of automated search methods should be viewed as reasonable, valuable, and even necessary.[3]

In addition, The Sedona Conference Commentary on Achieving Quality in the EDiscovery Process advises practitioners to utilize technology that “reasonably and appropriately enable a party to safely and substantially reduce the amount of ESI that must be reviewed by humans.”[4] These commentaries stress the use of technology to realize the important goal of achieving proportionality in the electronic discovery process that has unfortunately spiraled out of control in recent years.

Effective? True.

Closely linked to defensibility is effectiveness. Before investing in new technology, legal teams must be confident that the new feature will work and is worth the change. With sufficient training data, supervised learning can target documents most likely to be relevant. Using supervised learning to identify and pull responsive documents into categories early reduces the time spent organizing documents responsive to particular requests, and helps reviewers and the legal team better understand the case early on. Also, related documents can be dealt with more efficiently as a group and can even be assigned to a reviewer with expertise in a particular category.

The effectiveness of this technology may also be tested through sampling. Sampling is the key to measuring, monitoring, controlling and correcting potential errors in categorization, and is useful in any review to validate results. The technology can systematically and iteratively test the data to evaluate the accuracy of iC (in addition to Intelligent Prioritization). Without the use of sampling, some courts have concluded a party did not take reasonable steps to prevent disclosure.[5] With the flexibility to conduct as much or as little sampling as desired, iC not only reduces the time needed to complete a review, it improves the consistency of and confidence in category determinations.[6]

Independent studies are also proving that the use of Intelligent Review Technology (including Intelligent Categorization) is more effective than traditional, manual review processes. The Ediscovery Institute released a survey that showed using the technology equivalent of Intelligent Categorization resulted in reduced review costs by 45 percent or more.[7] In addition, the TREC Legal Track study from 2008 demonstrated that a “Boolean keyword search found only 24% of the total number of responsive documents in the target data set” while automated searching methods found 76 percent of the responsive documents.[8]

Devoid of Human Control? False.

Intelligent Categorization is not a process devoid of critical human insight and control. In some instances, this new technology has been pitched as a purely hands-off, eyes-off solution. In reality, Intelligent Review Technology as a whole does not replace human reviewers, nor should it. iC works by “learning” from human decisions and applying human logic when suggesting document categories. Human input is required so the technology has data sets with applied classifications from which to learn, and the system learns from both responsive and non-responsive decisions of human reviewers. As more documents are received and sorted, legal teams can rely on technology to continually improve the model while human reviewers can focus their efforts on the content and substance of the documents. In addition, because the tool was designed to increase consistency and accuracy, it affords the flexibility and scalability to give the ediscovery team more control over the review and to leverage as much or as little human input and oversight as is appropriate for the project. Thus, iC is not a substitute for skilled lawyers; rather, it enhances and compliments the work they do.

The question of whether it is reasonable to omit review of some documents altogether is an as-yet undetermined legal question. From a technical standpoint, however, IRT systems can support a range of approaches to selective review, such as extracting documents with a sufficiently low probability of responsiveness from review, guiding a review to read just the most important portions of long documents or focusing extra review on documents likely to belong to sensitive categories.

In short, Intelligent Categorization is a defensible, effective, cost-saving measure that leverages the work of talented attorneys to decrease the time required to complete document review. It is designed to meet flexibility and repeatability needs of the client, and is proving to be the key differentiator in the ability to respond to electronic discovery demands quickly and proportionately.

Note: The above post appeared in the April 2011 issue of the free, monthly e-newsletter, Case Law Update & Trends published by Kroll Ontrack. This newsletter is designed to help busy legal professionals keep pace with case law and information pertaining to electronic evidence. Subscribe and gain valuable and timely information on new ESI court decisions, as well as informative articles and tips for both the corporate and law firm audience.

[1] See, e.g., William A. Gross Constr. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 2009 WL 724954 (S.D.N.Y. Mar. 19, 2009).

[3] The Sedona Conference® Best Practices Commentary on the Use of Search and Information Retrieval Methods in Ediscovery. (Published August 2007).

[4] The Sedona Conference® Commentary on Achieving Quality in the Ediscovery Process, available for download at (Published May 2009).

[5] Mt. Hawley Ins. Co. v. Felman Prods. Inc., 2010 WL 1990555 (S.D.W.Va. May 18, 2010). 

[6] The Sedona Conference® Commentary on Achieving Quality in the Ediscovery Process, Principle 2, states: “In the ediscovery context, statistical sampling can serve as a check on the effectiveness of…automated tools in identifying responsive information and on the reviewers’ ability to correctly code documents.”

[7] See “Ediscovery Institute Survey on Predictive Coding,” available at

[8] For complete results from TREC Legal Track, visit

Technology Empowers Legal Review Teams and Increases Document Review Efficiency

Technology Empowers Legal Review Teams and Increases Document Review Efficiency

In his recent New York Times article, John Markoff highlights how “advances in artificial intelligence,” (ediscovery software in particular) have created a shift in the way law firms and corporations conduct legal discovery. While Mr. Markoff agrees that technology provides greater efficiencies and cost savings in document review, he claims that a heavier reliance on technology will ultimately result in a decreased demand for lawyers. While we agree that litigants are relying more heavily on technology during discovery, the suggestion that this will have negative repercussions for the practice of law is misguided.

Document review is, by far, the most expensive part of the legal discovery process. Today, for every $1 spent on processing data, $3 or more is spent on document review. This disparity makes traditional document review economically impractical and counter to notions of proportionality and reasonableness. Furthermore, recent court decisions and learned commentary from reputable organizations such as The Sedona Conference® have opened the door for technology to bring efficiencies to the costly practice of document review.

Kroll Ontrack has a long history of developing groundbreaking technology solutions calculated to make the discovery process less expensive and more efficient. These innovations include advanced search technologies, multilingual character recognition and early data assessment. Most recently, Intelligent Review Technology (IRT), which integrates smart technology with legal expertise, has been developed and implemented in the award-winning Ontrack® Inview™ document review tool. IRT expedites document review decisions by prioritizing and suggesting categories for yet-to-be-reviewed documents. IRT learns from lawyers while they work, strengthens the defensibility of document production decisions and ultimately empowers attorneys to focus on what they do best – devising case strategy and making tactical decisions. Most importantly, studies show that this powerful technology can reduce the expense of document review by more than 50 percent.

“Litigants and their counsel are relentlessly pursuing more efficient, cost-effective legal discovery processes,” said George May, vice president of product strategy, Kroll Ontrack. “It’s inevitable that reliance on technology will continue to increase as clients strive to reduce the total cost of litigation.”

Case Law: Alford v. Rents

Case Law

Court Sanctions Attorneys for “Scorched-Earth Approach to Discovery”

Alford v. Rents, 2010 WL 4222922 (S.D. Ill. Oct. 20, 2010). In this employment discrimination litigation, the court reviewed the magistrate judge’s recommendation advocating sanctions against both defendants’ counsels individually for discovery misconduct. Upon review of the record, the court found substantial and repeated violations of both the Federal Rules of Civil Procedure and the Illinois Rules of Professional Conduct based on the filing of over 14 discovery-related motions, incessant “accusation-laced, uncivil correspondence,” the need for judicial supervision of depositions and the appointment of a special master to decide discovery disputes. In light of counsels’ “scorched-earth approach to discovery” in which the attorneys “embarked upon a course entailing a conscious effort to maximize litigation and to make certain [it] was as time-consuming, difficult, unpleasant, and expensive as possible,” the court adopted the magistrate judge’s recommendation and held the two attorneys personally liable for sanctions in the amount of $3,750 each, to be paid without reimbursement from the law firms or clients.


Cooperation in the discovery process has been increasingly stressed by courts around the country, as judges continue signing onto the principles expressed in the Sedona Conference® Cooperation Proclamation, which notes that zealous advocacy is consistent with cooperation in discovery. Despite these notable efforts and several court rulings admonishing counsel to simply get along with regard to discovery, the cultural change of a “reveal nothing, gain everything” mentality by attorneys has yet to occur on a significant scale. When will this shift happen? Perhaps movement on the cooperation front will not begin until clients start mandating to their counsel that cooperation in discovery is essential in order to prevent disputes and other roadblocks that only increase time spent in the discovery process which translates into a higher bill for the client.

Or, perhaps more ground will be gained through efforts such as the Seventh Circuit’s Electronic Discovery Pilot Program which seeks to develop and improve pretrial litigation procedures that provide fairness and justice to parties while reducing the cost and burden of ediscovery. Although only Phase One has been completed and reported on, the early results remain promising. According to the Phase One report, “92 percent of the judges agreed that the Principles [set forth in the program] had a positive effect on counsels’ ability to resolve discovery disputes before requesting court involvement.” Also according to this report, Principle 1.02 – which addresses cooperation – was often viewed as incredibly useful and allowed counsel to cooperate with each other while fulfilling their obligation to zealously advocate on behalf of their client.

Whatever the path shall be, it is clear that judges will not tolerate uncooperative counsel that disrupt the process with numerous discovery disputes and time-consuming arguments. As one prominent magistrate judge recently noted, he did not become a judge to resolve discovery disputes. Thus, don’t waste the court’s and your client’s time (and inherently money) by waging discovery warfare and losing sight of the big picture – the just, speedy, and inexpensive determination of every action and proceeding