This is a notable entry from the Justis International Law & Technology Writing Competition 2019 in the category of The Future of Legal Technology, by Haitham Salman of City Law School, City, University of London. Find out more about our next competition’s topics here.
Humans have a notoriously poor track record of predicting the future. The increasing prevalence and use of predictive technologies may help mitigate the consequences of this historical shortcoming.
For practitioners and users of legal services, this technology would enable them to predict the optimal course of action in any given situation. Casey and Niblett argue that, as this technology gets better, failure to use it will become a per se violation of a legal standard.
The gradual necessity of technological assistance in law is currently most pronounced in areas where the limits of human cognitive capability are overwhelmed by the sheer size and complexity of data, such as in electronic disclosure.
Indeed, the rise of automated and technology/computer assisted review (“TAR/CAR”) represents the most mature use of technology within legal practice. At odds with the innate conservatism of the profession, this anomaly is directly attributable to the massive proliferation of electronically stored information in recent decades.
It is desirable that document review produce the highest precision at a cost proportionate to the value of a case. The growing cost of litigation can partly be attributed to increasing amounts of information. From 2013 to 2020, the total amount of digital information is expected to grow from 4.4 trillion to 44 trillion gigabytes. Lawyers, and more generally humans, will struggle to keep up. In particular, litigation involving large document volumes raise the question whether the use of TAR should soon become mandatory.
It is important to clarify what is meant by TAR, also referred to as predictive coding, as the use of machine-learning (“ML”) algorithms to iteratively learn from human judgement to review and categorise a large number of documents. The technical methods across systems vary and include neural networks, naïve Bayes classifiers, and genetic algorithms.
Technobabble aside, there have been few roadblocks in the way of TAR’s introduction into the courtroom. The first judicial approval of TAR appeared in the US by 2012 with Moore v Publicis Groupe wherein Judge Peck lauded the benefits of the technology, while stating that this was ‘not a case of machine replacing humans.’
In Moore, the issue in question was the appropriate methodology by which to evaluate approximately 3,000,000 emails the defendants had gathered from their custodians. On the basis of ‘(1) the parties’ agreement, (2) the vast amount of ESI to be reviewed, (3) the superiority of computer-assisted review to the available alternatives, and (4) the need for cost effectiveness and proportionality’, the court upheld the defendant’s proposed use of TAR.
The Irish High Court also endorsed the use of TAR in Irish Bank Resolution Corporation v Quinn. Unlike Moore, the parties had not agreed to use TAR. In support of his judgment, Fullam J referred to a study by Grossman and Cormack demonstrating the shortcomings of manual review. Moreover, he noted that even if TAR were no more precise than a manual review, its use would still enable a ‘more expeditious and economical delivery process.’
By 2016, TAR arrived in UK courts with Pyrrho Investments v MWB Property wherein the judge echoed his American and Irish counterparts in recognising the significant time and cost benefits of this technology. The decision went further in stating that a full manual review of each document in the case would be unreasonable within paragraph 25 of Practice Direction B to Part 31. In effect, if suitable automated alternatives exist at lower cost, it is beyond the ambit of reasonableness not to use TAR.
This gradual development towards standardisation appears in later decisions. In Brown v BCA Trading Ltd, the Registrar went so far as to conclude outright that ‘based on cost that predictive coding must be the way forward.’
The recent decision of Triumph v Primus continues this trend but sheds light on a few caveats of the technology. For the time being, all TAR systems rely on supervised learning from the input of an expert human reviewer. In other words, if you tell the ML to look for junk, junk is what it will find. The lack of apparent oversight by a senior lawyer who had ‘mastered the issues of the case’ in Triumph raised concerns whether the system had been sufficiently ‘educated.’ The plaintiffs assessed 220,000 documents with TAR which predicted that only 0.38% of the documents would be relevant. Yet their failure to disclose the use of the system itself, and how it arrived at its prediction, was considered insufficiently transparent nor independently verifiable.
If TAR systems are inhibited by weak supervision, it compromises the precision of the technology and no longer provides the optimal course of action. The risk is that the blind or unilateral use of TAR reduces its usefulness and thus eliminates its necessity.
Perhaps this is why in Triumph, Coulson J is quick to commend the TeCSA/TECBAR eDisclosure protocol, which contemplates the use of TAR in appropriate cases, for its guidance. Co-operation, agreement, and the criteria for relevance should be established at an early stage, unless there is a very good reason to do otherwise. Ensuring a transparent process in utilising predictive technology will allow us to maneuver around what Daniel Katz describes as ‘important and well-known human defects.’
Even expert reviewers are beset by well-documented cognitive biases. Often, we do not know when we are considering irrelevant factors and subconscious influences. At the same time, studies have shown that human review only achieves about 50% consistency. Is it then inevitable that advanced TAR becomes a de jure standard in electronic disclosure?
Not so fast. In Hynes v City of New York, Judge Peck once again had to consider TAR. This time whether the plaintiff could force the defendant to use TAR instead of their preferred method of keyword searching. His response was telling: ‘there may come a time where TAR is so widely used that it might be unreasonable for a party to decline to use it. We are not there yet.’
That prediction may be enough for some, but we’re only human after all.
Haitham is a Bahraini student with an LLB from the University of Nottingham and has recently completed his LLM/LPC at The City Law School. He has experience working in Middle Eastern jurisdictions as well as the UK. Haitham is particularly interested in the developing intersection between law and technology.
 Anthony J. Casey & Anthony Niblett “The Death of Rules and Standards” (2015) University of Chicago, Public Law Working Paper No. 550 p.5.
 “The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things” EMC with Research and Analytics by IDC available at https://www.emc.com/leadership/digital-universe/2014iview/index.htm (accessed 29/11/18)
 Monique da Silva Moore, et al., v. Publicis Groupe SA & MSL Group, No. 11 Civ. 1279 (S.D.N.Y Feb. 24, 2012)
 Irish Bank Resolution Corporation Ltd v Quinn  IEHC 175
 Maura R. Grossman & Gordon V. Cormack “Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review”, 17 Rich. J.L. & Tech (2011) available at http://scholarship.richmond.edu/jolt/vol17/iss3/5 (accessed 29/11/2018)
 n. 5.
 Pyrrho Investments Limited and Another v MWB Property Limited & Ors  EWHC 256 (Ch)
 Brown v BCA Trading Limited & Ors  EWHC 1464 (Ch)
 Triumph Controls UK Ltd & Anor v Primus International Holding Co. & Ors  EWHC 176 (TCC)
 Daniel M. Katz “Quantitative Legal Prediction -or- How I Learned to Stop Worrying and Start Preparing for the Data Driven Future of the Legal Services Industry” Emory Law Journal, Vol. 62, 2013.
 Katey Wood & Brian Babineau “Predictive Coding: The Next Phase of Electronic Discovery Process Automation” (2011) available at http://docplayer.net/3639140-The-next-phase-of-electronic-discovery-process-automation.html (accessed 29/11/2018)
 Hynes v City of New York et al., No. 1 Civ. 03119 (S.D.N.Y 2016)