Editor: What types of legal matters does your law department handle?
Stone: From our division office in Greenville, North Carolina, we
handle the majority of the legal matters associated with the company's
operations in North and South America, including employment law, transactional
documentation such as customer and supplier agreements and operating agreements,
import/export compliance, non-products liability litigation and a great deal
more. We are not generally involved, from the division office, in product
liability, insurance, intellectual property, corporate and tax matters.
Lauer: Besides the usual corporate matters, the legal issues here at
Global Compliance involve privacy matters, regulatory matters, some litigation
and counseling.
Carr: The in-house lawyers at FMC Technologies are responsible for all
of the company's legal matters except tax. We outsource litigation and
intellectual property preparation and prosecution.
Peters: The lawyers at the Turnpike Commission advise and counsel our
clients and handle general commercial and technology matters, litigation
management, employment and labor law matters and property acquisition.
Editor: Do you use metrics to manage those matters?
Peters: Yes, we do. The organization is in the midst of implementing
enterprise resource planning software, but that software does not include a
matter management system, so we have to "start over" with new processes and
metrics.
Carr: We use metrics vigorously in the FMC law department.
Lauer: Having joined Global Compliance only recently, I have not begun
to use metrics in my work. I've used metrics in previous in-house positions to a
limited degree. In addition, during the time that I consulted with law
departments, I worked with clients on projects that had metrics-related
components and that experience provided me with insights into this topic.
Stone: We don't use metrics in the formalized sense of the term that
many people use. We do, however, collect and review data on different items at
different times. For example, we track the cost to defend particular types of
litigation, the turn-around time for processing contracts, the number of
contracts reviewed in a particular time period and other metrics-related
data.
Editor: How do you determine which metrics to apply?
Carr: We focus on two types of metrics. We look at benchmarking data
to see how we're doing in comparison to companies that we think of as our peers.
We also analyze our operations to determine what metrics would assist us to
drive performance improvement. Our driving focus is on metrics that matter -
metrics that are meaningful from a performance standpoint.
Peters: We use inventory, cycle time, and productivity to give us a
"snapshot" of operations, but we deal with a small and varied inventory of
matters so we haven't gone beyond that point. Our use of metrics is therefore
somewhat limited.
Stone: In determining what data to collect and how to conduct metrics
analysis, we first need to understand why we want that analysis. For example, do
we need to fill a corporate request, determine cost structures, establish or
demonstrate service levels, etc.? We also consider who will see the data. Will
that information remain in the legal department? Will we share it with others in
our company and, if so, with whom or which departments and why? Will we share it
with individuals outside the company? We need to understand how the data we
might collect could implicate any privileges we might have.
Lauer: The metrics that I used previously came from surveys and some
internal analysis. Based on some individual benchmarking with attorneys in other
law departments, the other in-house attorneys I worked with, I determined how
best to measure our work. I also worked for over five years as a consultant to
law departments and had occasion to conduct benchmarking research for clients
for a number of projects.
Editor: How do you collect the data for your metrics analysis?
Stone: We may collect data in different ways, such as by reviewing
outside published surveys, reviewing internal data or by reviewing budget
information and conducting reviews of litigation and other projects which
involve outside counsel on a regular basis.
Carr: We collect some data from various external survey sources, like
Hildebrandt, the General Counsel Roundtable, Altman Weil and Serengeti.
Internally, all our metrics come directly from our Serengeti matter management
system ("Tracker").
Lauer: For many law departments, the e-billing capability built into
their matter-management systems or that they have on a stand-alone basis
provides them most of their metrics-related data.
Peters: We use data that we can find in our internal files.
Editor: Do you find useful information in surveys that are available from
other sources?
Peters: To some extent we do.
Carr: Yes, we review surveys from the sources that I named a moment
ago.
Stone: Some information from external surveys may be useful sources
for certain "big picture" metrics.
Lauer: Speaking from my prior experience and my consulting work, I
found existing surveys to have very limited utility. Often, the age of the data
that underlay a survey's conclusions was not easy to discern. Second, I
frequently could not determine whether the law departments that responded to the
surveys uniformly or consistently interpreted the questions and some of the
terms used. For example, how does an in-house attorney qualify as a "management"
attorney? If the answers to that question differed among respondents, the
results of the survey would not provide reliable information.
Those issues led Nena Wong and me to found the Open Legal Standards
Initiative a couple of years ago. We hope that OLSI will develop common metrics
for corporate law departments to enable in-house lawyers to more comfortably and
reliably determine their success in managing their companies' legal affairs. Law
departments will have more comparable information and be better able to
correlate their own efforts with those of other departments.
Editor: Can you reliably use data from those external sources to manage
your law department? Are those data consistent with the types of information you
need in your department's practice?
Stone: I think this type of data may be useful for comparison
purposes, some level of benchmarking, but internal management of the legal
function is very specific to your own company and its own operations. Only to a
limited extent are those surveys relevant to our needs.
Carr: Yes and no. The formats and questions differ from survey to
survey, as Steve suggested, which really impairs their utility. Also, the scope
of an outside survey can be too limited for our purposes.
Peters: We use the data from such surveys on an aggregate level for
comparison purposes, but our organization's operations, like those of
Meredith's, are unique on some issues so we can't make very detailed comparisons
using that information.
Lauer: I agree with Al and Meredith that the reader must apply the
results from a survey to his or her own department very carefully, especially in
light of the uncertainty about definitions, etc. Many in-house lawyers have told
me that they find some of the survey information useful, usually in terms of
things like the number of administrative staff per attorney and other, clearly
objective numbers.
Editor: Are existing surveys consistent with respect to the types of
metrics that they report and how they present the information? Should surveyors
try to maintain more consistency in their efforts in order to provide more
useful information to corporate law departments?
Peters: Our basic challenge is that we handle a small number of
matters but those matters vary tremendously. For that reason, we find it hard to
generalize from those surveys. Analyzing the Commission's legal service also
presents other challenges and opportunities. We face difficulties demonstrating
the benefits of our preventive law efforts, though we analogize from the
estimating experience of accountants in valuations and appraisals. Second, we
face the difficulty of finding the "right" metric: sometimes you need to create
specialized metrics because standard ones don't fit your needs. Finally, we try
to remember that one of our roles is that of an advocate for our department, our
work and even our careers. We try to determine the role that metrics can play in
telling those stories.
Carr: In the order of your questions: no and yes. Unfortunately, the
proliferation of surveys steals time away from me and my people. The most useful
thing that can be done from the perspective of metrics would be to develop a
single source for data input. This would allow us and other in-house lawyers to
contribute data one time. The various consultants could then pull the
information that they need from that pool. We would input data once and only
once.
As the data collection system is currently structured, there is a real risk
that law departments such as ours will refuse to participate, undermining the
utility of the surveys created. The other critical factor is that we need to
focus on the metrics that truly matter - a survey with 100 data points is
neither useful nor likely to be filled out. On the other hand, certain areas are
not probed at all - outside counsel evaluations, real settlement/judgment costs
by case/jurisdiction/type of case; legal expenses by type of
matter/location/complexity.
Stone: Survey results may not be consistent. The way a question is
worded, the types of responding companies, the demographics of those companies
and other factors impact the survey results and the value of that information.
Formal surveys of this type may be helpful for benchmarking, to see if your
company is away from or near the "norm," but not necessarily definitive to a
company or a legal department's own operations. There are too many subjective
factors unique to each of our companies.
It might be more useful to develop surveys by industry, geography and/or by
size of law department. For example, metrics from large law departments may not
be that useful to small departments and vice versa. More targeted surveys for
small law departments and small departments (but not small companies) without
consistent repetitive litigation would probably add more value.
Lauer: Many in-house attorneys find it hard to determine whether
various surveys are consistent. They find that the surveys' users are at the
mercy of the survey respondents' interpretations of the definitions in the
surveys, which can vary in ways that are not apparent. Greater consistency among
surveys would render them all much more useful and provide much more valuable
information than they do now.
OLSI will tackle those difficulties in developing consistent approaches to
the measurement of corporate legal service. With the many substantive challenges
that in-house lawyers face in their companies' legal matters, they don't need
additional difficulties when they try to manage those matters more closely.
Editor: What does OLSI do?
Lauer: Last year, with the support of the Association of Corporate
Counsel and the help of the Practising Law Institute and West Legal Ed, OLSI put
on a series of online symposia on metrics. Jeff, Nena and I wrote an article
about metrics that appeared in the ACC Docket that described OLSI's efforts.
OLSI also launched a survey on metrics for corporate law departments to
establish some common language and understanding of the subject. We are putting
together an in-person conference on the topic for later this year. If anyone has
questions about OLSI and its activities or wants to get involved in its various
committees and activities, I urge them to contact us at
href="mailto:[email protected]"
target=_new>[email protected].
Published June 1, 2007.