Anyone who has had contact with the document review side of eDiscovery over the past 10 years can tell you how much the industry has changed. Today, the majority of the changes most people would describe center around technology: with ever-growing volumes of documents subject to review, savvy corporate clients are frequently on the lookout for a technological silver bullet to help slay the beast of dramatically rising costs. Necessity has birthed advancements such as predictive coding, near duplicate and email detection and threading, concept clustering, as well as increasingly intuitive and robust document review platforms.
Nevertheless, it is imperative to complement these advancing technologies with the necessary expertise so as to leverage the tools to accomplish the best use in the often rough-and-tumble world of eyes-on document review. That said, smart corporate counsel looks for both advancements in technology as well as deep expertise in review management operations. Just as technology-assisted review (TAR) companies like Equivio and kCura have introduced game-changing elements to the eDiscovery process, cutting-edge managed review vendors have implemented their own quiet revolution in how attorney review operates. While less attention-grabbing than the technological advancements mentioned above, these developments in review management operations and philosophy can have an equal if not even greater effect on document review efficiency, quality and cost savings.
The “Bare Bones” Review Dilemma
Just five years ago it was not unusual for a document review to simply be handled by a large group of contract reviewers placed in a single room, in “sweatshop”-type conditions. Reviewers were given a printed review protocol and, if they were lucky, a half-hour talk from outside counsel associates outlining the background of the case. From that point onward, minimal supervision was provided beyond taking attendance; any substantive questions were fielded by largely unqualified staff attorneys or managers who were often contract attorneys with only slightly more experience on the matter at issue.
The fact is that many of these practices still continue today, particularly on document reviews that take place in lower-cost markets or offshore. Many vendors and firms tout tremendous cost savings offered by such “bare bones” contract attorney review. The real question is, can reviews performed with such simplicity cause the quality of the review to suffer, leading to errors such as incorrect tagging and missed privilege calls? The amount of re-review and quality control necessary to make a defensible production, particularly that subsequently performed by first- and second-year associates from the firm running the litigation or investigation, becomes so great that the final cost of review can become far more than it would have been had the initial review been performed using a better-managed review methodology rather than taking a bare bones approach. Too many decision makers are penny-wise and pound foolish.
Sophisticated in-house counsel know that a review staffed with competent, vetted contract attorneys overseen by highly qualified review managers implementing proven workflows, both linear and TAR-based, can provide document review services that outshine even that of a law firm associate review and at a fraction of the cost.
Creating a Modern Review Team
First things first: Who will be performing the review? The client should always pose this question of their review vendors. While vendors have mixed reputations, using contract document reviewers to complete a first-pass review is still the most cost-effective means of performing an eyes-on attorney review. While every vendor has a system in place for vetting potential contract reviewer candidates (i.e. background checks, resumes and interviews), they need to have qualified personnel performing these evaluations as well. Unfortunately this is rarely true, and nearly all vendors still rely on non-attorney recruiters with no document review or legal experience. Thus, these recruiters are likely unqualified to perform their gatekeeper function.
A top-quality review vendor will have review manager attorneys as an integral part of their recruiting and staffing process. Even when the client or outside counsel themselves have a hand in the recruiting process, qualified review managers with hands-on document review experience should be prescreening resumes and offering guidance. Furthermore, review management teams should provide feedback on each reviewer after every engagement to keep an updated performance record for future staffing purposes. Currently, there is an oversaturated market for attorneys nationwide, making for a pool of exceptionally qualified attorneys that are readily available to staff any review, particularly in larger markets such as New York.
This is even truer for the review managers who will be running the document reviews. Corporate counsel should demand that review vendors be able to provide top-level talent, whether full-time staff or contract attorneys themselves, for anyone they place in a review management role. RVM stands by this statement, and each RVM review manager is a graduate from a T-14 law school with associate experience in an Am Law 100 firm or the equivalent.
The Role of the Review Manager
A review manager should be a well-seasoned eDiscovery professional and credentialed attorney. Attorneys without actual training in the review management role should not be placed in the position of a review manager. Too often, vendors will simply promote a top-performing contract attorney from a previous review to a management position with very little guidance or formal instruction. This can severely limit the value of having (and for the client, paying for) a review manager at all. Review management training should include, at minimum: knowledge transfer and quality control workflows, advanced review tool administrative functions, privilege log construction techniques (particularly with regards to Microsoft Excel proficiency), and basic TAR functionality. This training can take place during prior reviews as potential review managers gain experience.
Why is all this important? In order to create the most value for a client, a modern review manager needs to be a true document review consultant. A qualified and trained review manager will have a level of expertise and experience that exceeds all but the most savvy clients or associates. During the initial review planning stages, a review manager should be able to recommend the most appropriate workflows (including TAR-based) for any given engagement while anticipating the vast majority of problems that may otherwise elicit costly re-review in the future. These dedicated individuals are the most frequent point of contact for outside counsel and serve as the communication hub among counsel, the review team, and the eDiscovery hosting and production team. A qualified review manager knows what metrics to track and report that best map review performance and point out fixable inefficiencies. This individual should be able to perform all common review platform functions such as document batching and coding pane modification, eliminating the communication overhead that leads to frequent delay or error. For any tasks beyond the depth of their expertise, a qualified review manager will be able to redirect or escalate any requests to the proper channels in the most efficient way possible.
Smarter Document Review Workflows
A capable review team paired with expert review management equates to far more accurate, efficient and less expensive document review regardless of whether TAR is leveraged.
Managing the order in which documents are reviewed is one easy example of how a more advanced workflow can greatly increase the speed and quality of a review. Less sophisticated reviews follow simple custodian-based prioritization with batches of documents assigned to the review attorneys generally sorted by date. A smart review manager can increase review speed by 10 to 15 percent through the application of efficient review batching techniques alone, utilizing technologies such as near duplicate and email threading and concept clustering. Batch creation with predictive coding-based prioritization can also help identify the most relevant documents early on in the review, which allows the associate team to identify important issues that they may not have otherwise considered and refine the review protocol in the early stages before costly re-review becomes necessary.
Proper batching techniques can also help with knowledge transfer, particularly early on in a review. One of the biggest problems with contract attorney review is getting the review team up to speed with the often complex substantive issues of a new matter. This becomes less of an issue when the same attorney or attorneys are looking at all of the documents from a particular email thread or dealing with a particular issue or concept. Furthermore, when the review team is looking at more relevant documents early on in the review, the review manager can more easily attack the learning curve in Six Sigma-type fashion, addressing early on the usually small number of key issues that are causing the majority of confusion and error.
Beyond first-level review, there are now a plethora of advanced quality control techniques that are employed by sophisticated review managers. Some review vendors still rely entirely on statistical sampling-based error correction as their recommended method of QC. This is not only overly simplistic but fairly indefensible when it comes to certifying a production and protecting a client’s sensitive and privileged information. Statistical sampling is a useful tool for validation of results and discovery of possible sources of error. Judgmental sampling, using key terms related to common sources of error, is the next and necessary step to ensure a sound production. Many more advanced quality control techniques are now available as well. These techniques include near duplicate and email thread-based consistency checks (particularly useful for redactions); domain parsing analysis to eliminate privilege false positives due to third-party disclosure; and false positive/negative analysis based on relevancy score clustering. Unfortunately, these advanced techniques are sorely underused. The expertise offered by a quality document review vendor that can implement all of these time- and cost-saving workflows, customized to the schedule and cost tolerances of any review matter, is crucial to success.
The Face of Better Managed Review
New eDiscovery technologies, while still not wholly embraced, have earned a great deal of attention over the past few years and with good reason. For the foreseeable future, however, eyes-on managed attorney review will still be at the heart of any document review effort. No client can afford to stay with a vendor that employs archaic managed review methodologies any more than they can afford to ignore changes in technology.
Managed review and discovery analytics technology will consistently advance hand in hand. The changing labor market for highly qualified attorneys has allowed the most aggressive and forward-seeing review vendors to grow and develop in seamless concert. Vendor attorneys and managers with the proper expertise can now replace costly law firm associate billable hours more than ever, resulting in cheaper, faster and better managed review.
Published February 19, 2015.
