Technology Advantage

Part Two of ESI Basics: Processing

July 25th, 2018
Part Two of ESI Basics: Processing

This is Part Two of a continuing series on ESI basics. In this series, we cover some of the terms used most often on the tech-side of e-discovery. In Part One, we provided an overview of PSTs. You can find that article here. Whether this is an introduction to you or a refresher, and whether you are an attorney, member of an in-house team or data analyst, this information may come in handy in your practice.

What Is Processing and Why Do You Need It?

Let’s say you have a new case and you request proposals from various e-discovery vendors (or your discovery counsel firm) for preparing the documents for review and production. Maybe you get a proposal back with a very large number next to it called "ESI processing" or “Metadata Extraction.” What do those terms mean and what are you actually getting for your money?

In general terms, ESI processing and metadata extraction involve collecting electronic data and making it useable. In more specific terms, ESI processing takes all of your native documents and runs them through software that parses out the data it reads from the header and extracted text into searchable fields. These searchable fields are often referred to as metadata (data about the data). Metadata allows you to search, put documents in chronological order and otherwise analyze your documents in a way that is efficient and consistent across software applications. Note that there are methods for forensic metadata and extraction but this blog will be confined to more common non-forensic analysis.

ESI Processing – Items to Consider

1. Processing is only as good as the tool that is being used. Some advanced processing tools can handle a very large number of file types and can extract the maximum amount of information available. They can also do it quickly. Other more basic processing tools can handle the most common file types and may be sufficient for your needs. Some popular processing tools are Law, Nuix, IPRO, Relativity Processing and other proprietary tools that are created by database hosting vendors.

2. The software provider will have a list of file types that can be processed and what the tool considers an "exception." Exception files are usually proprietary file types used with specific software or are files that don't extract (for example,  some system files and database files). Ask for the processible file list from your vendor to make sure that their processing platform can process your files, particularly if you use 3D modeling software, accounting software or legacy files that no longer have software support. Also, keep in mind that metadata extraction processes may differ depending on the type of device from which the data was collected.

3. You should be able to request a list of the specific pieces of data that are extracted. There can be hundreds of items depending on the file type and you won't necessarily need them all for the purposes of production but it is good to know what can be provided.

4.  You’ll want to ask about the handling of duplicates, attachments and compressed files. Will documents be de-duped on a global or custodial basis at the time of processing? Does the processing platform automatically unzip container files? Does it do that with other compressed file applications, like 7 Zip or Pkzip? Can it open Encase files?

5. Also, you may need to ask how the processing platform manages password-protected or corrupt files. Will it log them for you so that you can review a report and consult with the client regarding a solution? Most platforms should be able to provide you with such a report.

File Compression and Data Expansion

In most cases, your data will expand after processing, which means the data size collected does not end up being the data size hosted. Data expansion occurs most often with email because processing pulls out any attachments as separate files and extracts the metadata from them. It can also pull out embedded files and unwrap various containers, including the zip files mentioned above. While the expanded size can be estimated, it largely depends on the original systems used and how they were set up. An Outlook .pst file will expand to 1.75 - 2.00 times the original .pst size. However, Lotus Notes .nsf files can expand to much more than that, sometimes 3-4 times the original size because of the way the files are compressed. So, when budgeting for a case, it’s important to understand what type of data you have in order to reasonably estimate the potential processing and hosting charges.

File Type Filtering

During the early stages of processing (the “pre-processing” stage), you may have the option of narrowing down your document collection by file type before full processing begins. Simple filtering can include de-NISTING, which uses a list provided by the US government called the NIST list, and filters out files by file extension. The chief purpose of NIST filtering is to remove file types that are unlikely to be useable or responsive, such as system files. Some processing providers also use their own common list of non-document file types that can be removed prior to processing and review. Examples include .db, temp and .bak files.

Post-Processing

You should request a summary report after the documents are processed, which will show you the various file types that were in the processed data set. This report is very handy if you have to go back to the client to ask about specific file types that couldn’t be processed. It will also reveal the number of emails, Word documents, spreadsheets, Adobe PDFs, etc.,  so that you can get an idea of how long and detailed the review may be.

After processing is complete, you will be able to run keywords to cull the data set needing review. Keywords can be set to run across metadata as well (which we have found to be extremely helpful). You would provide a list of relevant search terms and then receive a report displaying the number of documents identified, by term. You can then tweak the terms to get to the document set you intend to review.

In Conclusion

It’s helpful to discuss processing options with your service provider up front so that you are aware of any impact to timelines. In addition, you’ll want to have realistic expectations when you discuss discovery deadlines and ESI specifications with opposing counsel. Asking the right questions before and during processing will help you avoid document-related issues down the road, particularly during the review and production phases.

 

 

DISCLAIMER: The information contained in this blog is not intended as legal advice or as an opinion on specific facts. For more information about these issues, please contact the author(s) of this blog or your existing LitSmart contact. The invitation to contact the author is not to be construed as a solicitation for legal work. Any new attorney/client relationship will be confirmed in writing.

Topics: E-Discovery Data Processing KTLitSmart LitSmart project management

Subscribe to the E-Discovery Newsletter

Related Posts

  • Data Mapping - Why is it Important for Successful E-Discovery?

    A data map is usually something that is generated along with a data retention plan and is a great starting point if and when an organization becomes involved in litigation and discovery. Long before litigation and discovery, however, it is important to understand the relevant data sources that may be in play and where those can be found within the organization’s various data systems. This blog explains what a data map is, what it contains and how it is helpful.

  • Pitfalls of Complex Search Protocols in ESI Agreements

    ESI Agreements cover the full gambit of e-discovery issues, from preservation expectations to production specifications. Sometimes, these agreements include an overview of the process by which the parties will identify the universe of potentially responsive documents by using specific date ranges, identifying priority custodians and developing proposed search terms. Adding another layer of complexity, this discussion is often had in a vacuum before the parties even know how much data their clients have. Coming up with criteria that identifies the relevant documents, but is not overly broad, in this vacuum can be difficult. However, the process is important because the date range, custodians and terms will inevitably dictate how much data is collected, processed and reviewed, which can significantly affect any litigation budget.

Newest Posts

  • When The Timing of Your Spoliation Motion Can Be As Important As Its Substance

    A motion for an adverse inference was denied in Pratt v. Robbins, et al. where Defendants failed to preserve or produce a video that might have contained pivotal evidence going to the heart Plaintiff’s civil rights claim for excessive force.  Plaintiff argued that Defendants spoliated evidence by failing to produce the video footage that may have recorded the use of force at issue.   A party seeking spoliation sanctions bears the burden of proving all of the elements of Rule 37(e), and under Fourth Circuit precedent is generally held to a clear and convincing standard. Plaintiff’s decision to suddenly cry foul on the eve of trial did not go over well with the Court.

  • PLAY IT AGAIN, SAM . . . BUT WAIT – WHAT IS HE PLAYING, AND WHO IS SAM?: Website Session Replay Software and Wiretapping

    The trend of class action cases alleging wiretap statute violations against website operators for the use of session replay software to collect information of website users continues to be a concern for companies in 2024.  While many of these cases have turned on whether website operators disclosed and obtained consent from website users prior to deploying any session replay software, a recent decision in a pending class action brought under the Pennsylvania Wiretap Act indicates that consent alone may not be the end of the story in these cases.

  • Utilizing E-Discovery Tools in Innovative Ways

    As part of the e-discovery process, we use digital means to identify relevant information for use in a legal proceeding. The proceeding may be a large-scale medical malpractice lawsuit, a patent infringement case, a government investigation, or countless other legal actions. However, the skills and resources used in those types of matters can also be used in other areas.