OpenText™ Discovery (formerly Recommind) recently partnered with Ari Kaplan Advisors to poll partners at leading law firms regarding e-discovery issues. The results are captured in “How Law Firms Are Incorporating Efficiency, the Cloud and Technology to Meet Enhanced Client Demands,” which is available at recommind.com/survey. The report provides an enlightening window into the varying approaches of in-house counsel and their outside firms in terms of technology solutions. Below, Ari Kaplan and OpenText’s Hal Marcus preview the findings. Their remarks have been edited for length and style.
MCC: Data security around e-discovery is a major issue, with 88 percent of respondents to the 2016 Law Firm Survey expressing concern and only a few being able to articulate specific security protocols or features. What can OpenText Discovery do to address this important education gap?
Marcus: There are so many protocols and standards – it can be very confusing for legal practitioners to get a handle on. We try to educate our clients around the practical considerations that have a real impact, like encryption in motion and at rest, intelligent management of encryption keys, multifactor authentication, and limiting the copying and distribution of discovery data.
Kaplan: As you mentioned, 88 percent of respondents expressed concerns about transferring their data, but the truth is that 100 percent of them have concerns. Some have a higher level of confidence, but there were a number of interesting responses referencing the Panama Papers or firms that have hired a chief security officer. Ultimately, cybersecurity is part of the nature of doing business. Many of the partners knew that their firms had advanced protocols, but they trusted their security teams that those protocols were sufficient for the data being protected.
MCC: The vast majority of respondents are using predictive coding, though a large segment, 33 percent, is less than fully satisfied. Talk about the state of predictive coding and specifically the concern over back and forth with opposing counsel on the front end. Is this is a serious problem, and how might it be addressed?
Marcus: There’s a growing amount of experience with predictive coding, also known as technology assisted review (TAR), but much of it has been on the more rigid, so-called “stabilized” model, which some are calling “TAR 1.0.” That model prompts an awful lot of back and forth negotiation in order to agree on protocols, and there’s been a lot of judicial intervention around requiring extensive transparency and documentation.
By contrast, continuous machine learning can be used in a more flexible, nuanced way, on an issue–by-issue basis. It can work and adjust to every matter-almost any size data set. Most of the back and forth is obviated; parties don’t need to engage in lengthy protocols. You don’t need an extraordinary effort by a subject matter expert to create a “perfect” seed set for training. Continuous systems adapt and learn as you do. They amplify the human review effort in a very natural way.
Kaplan: Everyone has basically used predictive coding because firms place a premium on efficiency, and any way that you can leverage technology to be more efficient is attractive. In addition, we saw that the nature of litigation is changing, so there’s a greater need to evaluate data more quickly, more effectively.
Of course, we received some really interesting comments. One lawyer said, “I trust the artificial intelligence more than I trust human reviewers” and discussed saving 40 percent of the cost of a traditional review.
There is a question as to whether lawyers are willing to invest the time in using this technology. What we’re seeing – especially in comparison to last year, with corporations being very interested in having outside counsel leverage technology to the fullest extent possible – is a trend toward leveraging overall analytics tools to make a matter move more smoothly while also providing answers that may not have been as easy to access in the past.
MCC: Regarding e-discovery metrics concerning efficiency and staffing, are firms tracking the right thing? What does OpenText Discovery suggest as a best practice for metrics tracking?
Marcus: First, participants in the discovery process have different needs. The best approach for them is to have all of the key data available and let them mine that data as they see fit: They can create their own reports and focus on the metrics that actually matter to them.
However, there is some agreement in the industry about what to care about; namely, the efficiency of the review process, which can be summarized with a simple metric: Of all the documents that receive human review, how many are relevant? If it is one out of 10, you’re spending a lot of time and capital looking at irrelevant content. The better you cull, analyze and prioritize, the more you’re going to spend your time with the relevant content. If you can get to, say, 60 percent of the reviewed documents being relevant, that’s a pretty good efficiency score in most scenarios.
Kaplan: From a trends perspective, law firm leaders are increasingly keeping a significant number of metrics, really trying to respond to the needs of their clients while being able to share those details.
Marcus: Seeing metrics across cases, across an entire portfolio of matters that you’re working on, makes sharing that much more valuable because you can compare the results. For example, if a corporation can see that certain outside counsel and service providers are achieving better efficiency scores than others on a regular basis, or that certain matters are responding better to certain discovery tactics, that information is really valuable for your choices on the next matter. Turning this into a repeatable process requires that you be able to see and share this information across different matters.
MCC: Even though 80 percent of firms reported consolidating their approach to e-discovery, cloud-based solutions continue to draw mixed results, with 60 percent of firms being enthusiastic about adoption, but many still lagging. That contributes to relatively low cloud usage. What’s the state of the cloud today, and what are the barriers to faster adoption? Is it all about security?
Kaplan: For law firms specifically, the cloud represents a more efficient, highly mobile future with fewer technology issues. Your data is accessible to everyone at the firm, especially with an increasingly remote work force. Some firms are evaluating the cloud in anticipation of adopting it after existing contracts run out, or they are waiting to see what to adopt when a new generation of leaders takes over.
The cloud is clearly seen as a net positive – but for some there are concerns over security. The truth is, firms are starting to realize that they’ll get more comfortable with security as the cloud becomes more common. We saw that 56 percent reported using cloud-based tools, and, as you mentioned, 60 percent said that their willingness to adopt cloud solutions in the coming year ranked pretty highly. The cloud has a lot of promise at firms, and once it gets through this period of adoption, it’s likely to be quite widespread.
Marcus: While that general trend is encouraging, I still see a significant delta in the results that we got from our Corporate Legal Survey in 2015 compared with the Law Firm Survey in 2016. For corporations, the concerns over security are prompting them to take control of discovery data, do less copying and distributing, and put the data into a cloud-consolidated environment that meets all of their security protocols, so that they have a consistent set of standards and greater control.
Law firms, however, are looking inward, as indicated by some of the comments the partners gave in their survey interviews. Their data security concerns may be prompting them to keep more data in-house, beef up their firewalls and be less adoptive of cloud solutions. Of course, coming out of a year in which we had high-profile data breaches targeting law firms like Cravath, Weil Gotshal, and obviously Mossack Fonseca, firms have to understand that maintaining the most sensitive data in-house puts a target on them. If hackers are looking for inside information on a merger or acquisition, they know to target the law firm handling it. With cloud consolidation providers, data is less centralized, so there’s less of a target on a specific firm’s data.
So while corporations and law firms have similar concerns about data security, they may be reacting to them very differently.
MCC: There seems to be a disconnect between firms and clients on reporting. More than three out of four firms are generating reports but many don’t share them with clients. Moreover, firms want efficiency scores, and clients want multi-mattered dashboards. Please discuss this disconnect and whether it’s symbolic of deeper issues in firm-client relations related to e-discovery.
Kaplan: Talented legal operations leaders in corporations are fueling this revolution in tracking and consistently monitoring their cases by reporting and maintaining metrics. Recommind had the vision to study legal operations leaders last year, and now law firm partners this year, but what you’re seeing is the law firm partners maintaining a number of metrics and reports internally so that they can share information with one another to gauge the profitability or success rates of their matters. You’ll continue to see them also taking the metrics they have and providing them to their clients, who are clearly interested in these kinds of details, because it’s become part of the approach that they’ve taken and the success that they’ve seen.
Marcus: In 2015, only 28 percent of the corporate legal operations execs were satisfied with the level of visibility that they had into the discovery practices of their outside counsel. In 2016, we have the law firm partners indicating that 76 percent of them are generating reports and metrics around their discovery process. In all honestly, we’re not sure what to make of that. Is it that, a year later, law firms are delivering more transparency to their clients? Are firms generating lots of reports but not sharing them enough, as was indicated? Or are the reports not providing what the corporations need to see to really understand what’s going on? There’s a disconnect that’s either being rectified or being perpetuated, and it’s hard to gauge which from the data at hand.
Anecdotally, people in the industry often make comments like, “There isn’t enough uniformity in the standards being tracked.” In our surveys, review efficiency was the least tracked aspect by corporate legal operations in 2015 and the most tracked aspect by law firms in 2016, so clearly there’s some misalignment. With the advent of efficiency scoring, a simple metric that shows the cumulative effect of a wide range of good and bad e-discovery practices, we’re seeing more uniformity start to emerge in how people can look at these processes.
Kaplan: There could just be a practical element here too, one of firms adopting technology that has built in a better way to keep track of metrics and provide reports, so now they’re able to generate those reports. As the technology evolves and becomes more advanced, there is an opportunity to pull data that will help firms make decisions. It could be that as firms continue to adopt new technology, the technology provides them with information that they can use internally and also share in a way that’s particularly effective. The technology providers are creating this in response to what the corporations are asking for, so you see it’s very circular.
Marcus: That’s certainly the goal of our development efforts. I love that optimistic vision, and I hope to see it fulfilled with law firms and their clients being on the same page on these fronts.
MCC: What was the single most surprising finding from the survey and why?
Marcus: The most interesting finding for me was the one area that was perfectly aligned between the two surveys. In 2015, 84 percent of corporate legal operations execs indicated that despite their significant data security concerns around discovery, they rarely or never audited the technological competency of outside counsel. This year, only 16 percent of the law firm partners indicated that they are audited by their clients.
That near perfect alignment across a year in time strongly suggests to me that auditing and tracking the wide range of outside counsel is simply not a viable path to higher security. That’s one of the reasons that we’re seeing more focus from corporations on consolidating data into cutting-edge, secure cloud environments. There’s no perfect security in the world, but consolidating data into a unified environment – one to which they manage the encryption key and can open up access to outside counsel and service providers as needed – puts them in the greatest control over their data.
Kaplan: For me, it was that 88 percent of the law firms were outsourcing some of their e-discovery. A handful do it themselves, and it’s working for them, but most firms either don’t have the capabilities in-house, due to the size or sophistication of a matter, or often a client has specifically determined that the law firm use a provider with which it has a preferred relationship.
This statistic represents what people have been talking about for many years in that there is greater control by the corporation in deciding the nature and structure of e-discovery. There’s a real partnership growing now between law firms, their clients and the outside providers who are trusted to handle certain types of matters or cases of certain sizes. You’re starting to see a fundamental collaboration that’s holistic in its approach to how firms are handling e-discovery.
Published November 7, 2016.
