Ms. Brooks, a member of the California Bar Association based in Cypress, CA, and KPMG Director Priya Keshav worked with a client to prepare the White Paper "Software-assisted Document Review: An ROI Your GC Can Appreciate ." It compares actual legal case data reviewed through a software-assisted technology review followed by one pass of human review, with the results of an earlier human review where software-assisted technology for relevance ranking was not used.
The KPMG White Paper can be found at the following link: http://www.kpmg. com/US/en/IssuesAndInsights/ArticlesPublications/Pages/software-assisted-document-review-roi.aspx.
Editor: The White Paper describes the enhanced cost benefits of properly utilizing e-discovery software to improve data analysis and document defensibility for corporate legal departments and law firms. Why is this important to the marketplace?
Brooks: Corporate law departments and their law firms are looking for ways to reduce the overall spend of the e-discovery process and are targeting the evidence review process - by far the most expensive step in getting evidence ready for either litigation or an investigation.
The White Paper demonstrates that if you enhance traditional human review with technology that ranks documents for relevance, you can reduce costs by making the review process more accurate and efficient.
Editor: Why is there such a focus on e-discovery software?
Brooks: Continued increases in e-discovery costs have been driven by the ongoing explosion of electronic data (email, Sharepoint, digital voicemail, etc.). To control the soaring costs and improve efficiency in the review process, most companies and law firms are filtering data based on date ranges and keywords or doing a more targeted collection of evidence in the first place. But they have found only limited cost reductions in doing simple things like de-duplication and date range and keyword filtering. For example, in the area of keyword filtering, both sides must negotiate to agree on what keywords will be applied. One side tends to want a broad keyword list, and other side wants it to be narrow. Often, an extensive list of keywords results when both sides cannot reach agreement.
So those more traditional ways of filtering down the evidence provide only limited cost relief. This is where software-assisted review comes into play.
Editor: What advantages are there to having the right match of software and internal process in order to determine the appropriate documents for discovery?
Brooks: Since the vast majority of evidence resides in an electronic format, there has been an ongoing search for technology to assist in reviewing it. The White Paper points out that such technology now exists to increase efficiency and therefore reduce cost when coupled with the important human element of review.
The most important human ingredient is the subject matter expert responsible for "training" the software. These subject matter experts include the highly qualified attorneys who will take the case to trial or respond to government or internal investigations. Those attorneys know what issues might raise red flags.For instance, if the situation involves a highly technical field like intellectual property, you will enlist an attorney who specializes in that field. You can't eliminate their roles, and nobody would want to eliminate them.
Now, appropriate technology can make the human reviewers more efficient and more accurate, and can complement the human review.
Editor: What qualities should good e-discovery review software offer?
Brooks : The goal of any e-discovery software technology is to get at the relevant evidence as quickly, accurately and cost effectively as possible. It also must be defensible so that if it is challenged, you can describe the process from start to finish.Therefore, when we talk to our clients about the capabilities of particular software, it must provide an audit trail so that they can say this is the evidence that we started with, and, at the end of the process, we ended up with a subset of that evidence that was relevant to our search.
You also need software that is user friendly because folks are going to be using it on a day-in-and-day-out basis to look at the evidence that is displayed in the software. It has to have the ability to achieve the kinds of things that the user needs - the ability to filter the data, to be able to apply issue codes to it, the ability to redact if something needs to be redacted, and the ability to put confidentiality watermarks on it - all of those things. Therefore, it not only has to have the functionality and the flexibility to do what the user wants it to do in a robust fashion, but it also must be easy to use. And, of course, one of the overarching things that any client wants is affordability.
In doing the study on which the White Paper is based, we relied on KPMG's Discovery Radar software platform, which integrates Equivio> Relevance technology. What the software does at the outset is to randomly sample and create document sets to be turned over to the subject matter expert who looks at those documents and makes relevance decisions "yes," "no" or "do not know." In the course of reviewing a sufficient number of documents, the software starts to understand how the documents should be ranked as to degree of relevance.
In effect, the software will over time come to a point of stability where it tells you, "Okay, I understand why you are marking something 'yes' and why you're marking something 'no.'" It can apply that knowledge to the larger population of documents and give those documents a relevance score. With this information, you can then decide that your traditional team of human reviewers will review only that subset of documents above a certain relevance cut-off score.
The White Paper contrasts this with the expense and inefficiency of the traditional approach of hiring lots and lots of reviewers. In many cases, these are contract lawyers or lawyers right out of law school who look at a huge population of documents, which they go through one by one.
Once they've completed their review, then a more advanced set of attorneys with more experience gets involved in the review process. They do a second-pass review to make sure that everything that was marked in first-pass review as something to be produced would, in fact, be relevant and should be produced.They might also spot check to see if things that were marked as irrelevant were, in fact, irrelevant. Because everybody wants to make sure that what is going to be produced is accurate, you might even have a third level of review.
It's the multiple layers of review by different teams of reviewers that drives up the cost of discovery. Contrast this traditional approach with the ease of using technology for that first-pass review - something that is done by the software after it has been trained by a subject matter expert. This usually takes 8 to 16 hours. When you bring in the team of human reviewers, they need only look at the smaller population that the technology has pulled with the assistance of the subject matter expert.
Some folks might ask how they would know if results of applying technology were really accurate. What we also talk about in the White Paper is the ability to test and validate the results by statistical sampling.
Editor: Is cost the only benefit?
Brooks: No, there are other benefits as well. The White Paper points out that the software was able to find and pull in documents that were relevant even when those documents had been missed by human reviewers. Software doesn't have that subjectivity and limited attention span of human reviewers. It consistently follows and sticks with a pattern, which increases accuracy in the form of fewer false negatives.
It also makes for a more efficient use of time. If you have a tight deadline and need to be able to streamline the process, throwing a bunch of bodies onto the review is not as efficient.
Software-assisted e-discovery also leads to an earlier focus on the content of evidence.Knowing the evidence of your matter earlier in the litigation or investigation process is important in developing the strategy and addressing important steps such as the timing and content of depositions, motions to dismiss and the consideration of settlement.
Editor: Do you have any closing comments?
Brooks: To summarize, in today's economy, clients and their law firms need ways to trim the fat by reducing cost and increasing efficiency. You can greatly increase efficiency with software-assisted, first-pass review supplemented thereafter by more traditional human review. Accuracy also increases. You don't want to cut cost at the expense of accuracy. What our study found is that you get all of the benefits that you've been looking for, reduction in cost and increases in efficiency and accuracy. Also, your subject matter experts get familiar with the evidence earlier, rather than later. This is a tremendous benefit that had not even been contemplated when we started our study.
Published October 4, 2011.
