Today, litigation, investigations and regulatory compliance regularly involve hundreds of thousands, or more, of documents that need to be reviewed. In most cases, the clock is ticking. Corporate legal teams, unable to handle the sheer volume of documents requiring review, parse the project out to contract attorneys, their lead litigation counsel or both.
Good lawyers are not necessarily good project managers (nor should they be), and many times, to meet tight deadlines, they staff up with bodies, believing that the more bodies the faster the review will go, the more rapidly the facts will be uncovered, and the fewer dollars will be spent. This approach can throw a huge monkey wrench into the review—and the litigation itself. In reality, the larger the review team, the higher the chance of misalignment that results in coding inconsistencies, the more inefficient the review, and the greater the chance for suboptimal results.
For example, take a large review project using technology-assisted review (TAR) staffed with 300 law firm associates and contractors. On average, the reviewers find half the documents to be responsive. However, 50% of the reviewers code a document as relevant while the other 50% code a similar document as non-relevant. Such wide coding inconsistencies can result from a number of factors, from uneven training among disparate teams, how long an individual reviewer is staffed on the project, individual human judgment and reviewer fatigue from scrolling through page after page of documents. As the review team grows in size, the risks of missing relevant information, potentially exposing sensitive documents or even waiving privilege are magnified.
Solving the big review team conundrum
Many legal departments, managing increased workloads with fewer resources and smaller budgets, are looking at how to best optimize their resources, both internal and external. Rather than sending cases “over the fence” for expensive law firm associate review, they are partnering with their law firms and together leveraging experienced managed review teams that align the interests of the corporation (e.g., save costs and minimize risk) that of their outside counsel (e.g., meet deadlines and provide the best results). At the same time, they are limiting eyes-on review of documents.
“By partnering with OpenText, it has allowed our attorneys to provide cutting-edge legal services at a fraction of the cost. Utilizing the OpenText continuous active learning technology in addition to their talented document review team allows us to save client financial resources by quickly and more accurately identifying responsive documents.” Robin Stewart, Head of e-discovery, Litigator and Partner, Kutak Rock
Here are four ways to solve the big review team (and big cost and risk) conundrum in 2021:
1. Align your team early on—it’s critical to success and saving costs
Early alignment between the managed review team, you and your outside counsel is critical at the onset of any project, and the upfront investment can result in measurably more reliable and accurate coding, with repeatable and consistent QC processes employed for privilege, responsive, non-responsive, PII, PHI, hot documents and more. When robust QC processes are employed at the start of a project, the managed review team can reduce QC by outside counsel—often from 20% of the documents reviewed—to 5% or less. This can result in substantial cost-savings for the corporation—in the order of tens or even hundreds of thousands of dollars per project like one global technology company realized.
2. Exploit technology to control costs
Technology-assisted review based on continuous active learning or continuous machine learning significantly improves reviewer consistency by surfacing likely responsive documents to the top of the pile to review. TAR continuously ranks documents to take advantage of additional judgments by reviewers in real-time. As training continues, the algorithm and document rankings continuously improve so the review team finds relevant documents faster. Moreover, TAR can be used stop and validate review at a reasonable recall rate (how many of the relevant documents in a collection have been actually found), saving time and costs of reviewing unnecessary or extra documents.
3. Enhance alignment with smaller, focused teams
Take the same project referenced above with 300 reviewers on any single day. With optimized technology, including technology-assisted review, and an aligned team for enhanced QC, total project reviewers can be reduced from 300 to just 60, average responsiveness rates increase to 60% and the error rate (inconsistent coding calls) is reduced to just 11%. Again, resulting in better results and substantial cost savings.
4. Automate workflows to enhance the review
Because every minute and every dollar counts in e-discovery, the approach to a review must be agile based on the need—while automating workflows to minimize eyes-on documents.
Often, lawyers have to do a responsive review for production, which could entail a straight TAR review to minimize the number of documents reviewed. Sometimes legal teams just need to get the documents out the door in the most expedient but defensible way possible, as is often the case in a second request or third-party subpoena, requiring a highly automated Cut Point review to expedite a large review for production. Or the situation might warrant an investigative review in which a small team uses analytics to provide quick, accurate results at exceptionally high recall and precision (how many of the documents retrieved are actually relevant, or on target), along with much greater insight into the substance of the documents.
In any review scenario, when large volumes are involved, the solution is never to throw more bodies at the project. Rather, a small, aligned team using optimized technology and comprehensive QC processes will get the job done faster, with measurably better results and at substantially lower cost.
Published February 10, 2021.
