THE DIRTY SECRET OF DOCUMENT REVIEW



The Dirty Secret of Document Review

A REVIEW OF 100,000 POTENTIALLY RESPONSIVE DOCUMENTS WAS CONDUCTED IN RESPONSE TO LITIGATION. THE LEGAL TEAM "SUCCESSFULLY" REPORTED THAT THE REVIEW WAS 99.7% ACCURATE WHEN SAMPLING THE WORK PRODUCT, AND THEY WERE CONFIDENT THAT ALL RESPONSIVE DOCUMENTS HAD BEEN PROVIDED TO THE REQUESTING PARTY. ON FURTHER REVIEW, HOWEVER, IT WAS DISCOVERED THAT 300 DOCUMENTS WERE IMPROPERLY CODED AND SEVERAL HIGHLY CONFIDENTIAL E-MAILS DAMAGING TO THE CASE WERE INADVERTENTLY PRODUCED TO THE REQUESTING PARTY.

When it comes to electronic discovery document review, one discovers something surprising and a bit scandalous — the average document review contains a lot of bad review calls. These bad review calls can result in documents being incorrectly identified as relevant vs. irrelevant, inaccurately issue-coded or, worse, improperly tagged as responsive when they should have been marked as confidential or privileged.

A certain number of mistaken review calls are unavoidable, especially in the context of complex discovery projects and demanding deadlines. But that should be no excuse for the questionable quality of the document review process. Most of these mistakes are avoidable with the application of smarter review processes, sound technology and more diligent management controls.

The Truth Hurts

THERE ARE SEVEN COPIES OF AN E-MAIL MESSAGE WHERE A SALES MANAGER REQUESTED ADVICE FROM HIS EXECUTIVE TEAM AND GENERAL COUNSEL ON COMPETITIVE PRICING ISSUES. STUDYING THE REVIEW CALLS ON THE SEVEN DOCUMENTS REVEALED THAT TWO OF THE E-MAILS WERE CATEGORIZED AS "PRIVILEGED," AND FIVE WERE CATEGORIZED IT AS "NOT PRIVILEGED."

Okay, so maybe the bad-review-call factor is not surprising for document review veterans. Misguided calls on documents have been committed as long as document review has existed. And when you consider the scope of managing a major discovery project, imperfection is inevitable. When the documents are complex, the case multi-faceted, and the document analysis requires a great deal of expertise, there is often no objectively "correct" way to review and code a document. As with everything in life, it's a matter of interpretation. Combine that with the realities of dozens of attorneys reviewing hundreds of thousands of documents to respond to a litigation or investigation, it is only reasonable that there will be some variance in the assessment of the documents.

But still, the bad-review-call factor could justifiably be seen as at least mildly astonishing. After all, the effort mobilized to review electronic documents is usually extensive and expensive. It typically involves technical teams disrupting custodians to collect files from their computers, attorneys spending thousands of hours reviewing documents, and litigation support specialists managing databases and document productions.

And yet, there's that e-mail, which is clearly a privileged communication, marked as "responsive/to be produced" by a review team member already receiving the warm embrace of the adversary.

What are the consequences of bad review calls? Often, the results are harmless and result in the producing party incurring nominal costs for "tiffing," or producing the documents in tiff format, and handling the evidence. In other instances, the consequences can be much more severe. Take for example the counsel that produced that highly confidential e-mail and failed to protect sensitive client information. What if that e-mail contained important information, thus waiving privilege or disclosing vital intelligence to the requesting party?

Ideally, the legal team would have agreements in place enabling them to retrieve such inadvertently-produced privileged documents. But even then, the inadvertent production can be a wasteful, time-consuming and damaging event.

Seven Steps to Reinvigorate Review Quality

MOST BAD REVIEW CALLS ARE UNDERSTANDABLE MISTAKES. LENGTHY MEMOS OR VAST SPREADSHEETS OR EXTENDED E-MAIL MESSAGE THREADS MAKE IT DIFFICULT FOR REVIEWERS TO DECIDE ON RELEVANCE, PRIVILEGE, AND RESPONSIVENESS TO CERTAIN ISSUES. ON THE OTHER END OF THE SPECTRUM, HOWEVER, INATTENTIVENESS AND SLOPPINESS ARE USUALLY THE CULPRITS.

There are seven steps to help attorney teams catch inconsistencies or incompleteness in the document review, thus reducing risk and improving the quality of the output as follows:

1. Conduct case training. It's common practice to hold an orientation session for the document review team. Make it count. If you have attorneys joining the review who have not been working on the matter, take the appropriate time to ensure they have the necessary background and information. Bring up specific sample documents for discussion. Record the session, so you can show it to others who may get added to the review team later in the process. Using a multimedia file is simple to create and re-use the training materials.

2. Create case documentation. It's also standard operating procedure to have a document review binder or project manual. Most of the content is obvious — a copy of the complaint, key list of players, organizational charts, information on products or transactions in question, etc. Again, make this as complete and as specific as possible, and keep it updated with new information.

3. Design the review workflow. After determining the key objectives of the document review, create a workflow diagram. Will there be a first pass and a second pass review of documents? Will you identify potentially privileged documents and assign those to higher level attorneys? Do you need to identify documents to prepare for depositions? Include, as necessary, the actions that will take place for documents needing redaction, documents with technical issues, documents needing translation, etc. Formalize these actions in a document, and share it with all relevant parties.

4. Choose good technology. A good review tool is invaluable for automating the review workflow and document decisions. It will help eliminate many mistakes before they have a chance to happen. Among many other capabilities, this tool should offer customizable logic in the coding functionality to help minimize inconsistent calls; the ability to automatically group similar documents together when assigning out documents to the review team leads for more accurate and efficient decision-making; easy access to e-mail chains, e-mails and associated attachments; and the ability to bulk code these "document families."

5. Create good documentation on using review tool. Part of the value of reviewer protocol documentation is in forcing the team leaders to think through and clarify what actions they want the review team to take when reviewing documents. It should be utterly clear to the review team what review calls are required on each document, as well as how they should use other functions such as annotation and redaction. There is some up front work involved, but it's not extensive. For example, the document may be 10 pages with screen shots and may take several hours to create, plus 95 percent of the time, there will be updates and revisions, which is to be expected.

6. Implement substantive quality control procedures. The first few days of the review represents a critical opportunity for lead attorneys to upgrade the consistency and accuracy of the document review staff. Encourage questions from the review team on case issues, and publicly communicate the answers. If at all possible, have lead attorneys spend time reviewing documents and auditing the coding of the rest of the team, especially if you are using contract attorneys. Often, decisions will be made that are curious and sometimes shockingly wrong. Share feedback and insights in group sessions. This will be easier if the team is together, but can be disseminated through e-mail or a number of other vehicles.

7. Implement technical quality control procedures. Most electronic discovery review tools offer a handful of good approaches for ensuring completeness and consistency among review calls. These are critical. You won't be able to check every document decision, but good technology will allow you to cast a wide net for catching questionable review calls. It's important to analyze and document the most important quality control procedures and implement them before each production set is created. Plan extra time for these checks because often there will need to be additional legal review to resolve the inconsistent coding and ensure accurate production.

These processes and procedures require additional time and effort, but the payoff is substantial. With a relatively small number of hours spent coordinating and managing the review, bad review calls can be banished, ensuring more successful projects and defensible outcomes that minimize risk and lower overall costs.

This article appeared in the February 2007 issue of ALM's e-Discovery Law & Strategy newsletter ().

Peter McLaughlin

Engagement Manager, Review Management Services

Fios, Inc.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download