Legacy Data and Media Conversion: A Guide for Records Managers Inheriting an Old System

The situation is common enough to have a familiar shape. A new records manager or recorder takes office and finds, in a back room, the machine that holds twenty or thirty years of document images and the index that makes them findable. The company that installed it has changed hands at least once. The one employee who knew how to restart it has retired. There's a shelf of optical cartridges nobody has read in a decade and a support contract that costs more at every renewal. Everything still works, mostly. That "mostly" is where the risk sits.

This guide is for the person who has inherited that system. It covers how to decide what to do, how to run a conversion so you can account for every record afterward, and how to retire the old system without losing anything you are required to keep. It's written from the owner's side of the table, whether you do the work in-house or hire a contractor.

Why conversion stops being optional

Legacy systems rarely fail all at once. They decay.

• Hardware outlives its support. Drives for older optical and tape media are no longer made, and spare parts come from secondhand dealers. When the last working drive fails, the media becomes unreadable in practice even if the data on it is intact.
• Keeping it alive gets expensive. Extended support contracts, an unpatched operating system that IT wants off the network, and one-off repairs all add up.
• Slow access is a legal problem. Public records laws set response deadlines, and discovery requests and litigation holds don't wait for a jukebox to load a platter. A record you can't produce in reasonable time is close to a record you don't have.
• Organizations change shape. Merged departments, shared-service agreements and regional consolidations leave offices running two or three systems that should be one.

When to convert: a checklist

If two or more of these apply, start planning now rather than waiting for a failure to force the decision.

• Your office is merging with, or taking over records from, another office or agency.
• Offices that share work run different systems and swap records by export, email or paper.
• You are replacing the application software, the server hardware, or both.
• The vendor has announced end of support or no longer responds.
• Records sit on optical platters, tape or other media that only one aging drive can read.
• Service calls on the same equipment keep coming back.
• You've received data from another body on media or in formats your systems can't open.
• You are changing archival or off-site storage providers and must move data between them.
• The people who understand the system are within a few years of retirement.
• Title searchers, attorneys or the public complain about retrieval times.

Step 1: Inventory everything

You can't convert what you haven't described. Build the inventory before talking to any vendor, because it becomes the basis for the scope of work and for every reconciliation that follows.

For land records, give the index special attention: grantor and grantee names, instrument numbers, book and page, recording date and time, document types, legal descriptions, and the cross-references that tie a release to the mortgage it releases. In my experience the index is both the most valuable part of the system and the hardest to move cleanly. Images are bulky but simple. Indexes carry decades of inconsistent data entry, abandoned fields and local conventions nobody wrote down.

Step 2: Migrate, emulate or export to a standard

There are three broad paths, and many projects combine them.

A sensible default for permanent records is to export to standard formats as a preservation set first, then load that set into whatever application you choose. You end up with a vendor-neutral copy that will outlive the next system too. Our article on file formats for long-term preservation covers the target formats, and the legacy formats and media field reference covers the old media and formats you're likely to meet along the way. For the storage that receives the export, the National Digital Stewardship Alliance's Levels of Digital Preservation, published at ndsa.org, are a practical yardstick.

Step 3: Treat image, index and metadata as one record

A conversion that moves images but loses their link to the index has produced a pile of pictures. Every record has to travel with three parts intact.

1. The image. Every page, in the right order, at the original quality. Don't recompress bitonal images into a lossy format to save space.
2. The index. Every field, including the odd ones nobody uses anymore. Keep the original values alongside any cleaned-up versions.
3. The metadata. Source system identifiers, original file names, capture dates, page counts, checksums, and a note of any transformation applied during conversion.

Write the mapping down, old field to new field, with every transformation described. That mapping document belongs in the permanent record of the project.

Step 4: Keep a chain of custody you could defend

Assume that someday someone will ask you to prove that a document in the new system is the same one recorded in the old. Build the evidence as you go.

• Barcode every piece of media at intake and log the label, type, date, who received it and where it went.
• Keep source media in a locked, environmentally stable space, with access limited and logged.
• Compute a checksum (SHA-256 is a common choice) for every extracted file the moment it is read, and carry it through to the destination.
• Encrypt data in transit and at rest, including on drives that travel to and from a contractor.
• Record who did what and when in an audit trail that is itself kept with the project records.
• Return or destroy media formally, with a signed receipt or certificate of destruction, and only when your retention authority allows.

Step 5: Reconcile every batch

A reconciliation report answers one question: what went in, what came out, and what happened to the difference. Every conversion has a difference. The ones that look perfect usually weren't counted properly.

A hypothetical report for a land-records conversion might look like this:

Every exception needs a reason code and a resolution, and the report goes into the permanent project file. If something truly can't be recovered, say so in writing and record which originals or microfilm stand in for it.

Step 6: Validate by sampling before, during and after

Automated counts and checksums catch a great deal, but not everything. Good validation pairs machine checks with human eyes.

• Before: run a pilot on a small but awkward batch, such as a period of known index problems, a suspect platter or a set of oversize plats. Fix the process before scaling it.
• During: check every batch automatically (record counts, page counts, checksums, required fields present, valid dates) and pull a random sample for visual review. Enlarge the sample for batches drawn from problem media or problem periods.
• After: search the new system the way its users will. Pick known documents and follow a chain of title through them. Run the same queries on both systems and compare the results.

Some projects borrow acceptance-sampling tables such as ANSI/ASQ Z1.4 to set sample sizes. Whatever method you choose, write the rule down before you start and stick to it.

Step 7: Decommission carefully

Switching off the old system is a records decision, not only an IT one.

1. Run in parallel for an agreed period, with staff working in the new system and checking against the old when in doubt.
2. Get written sign-off from the records custodian, IT and, where needed, legal counsel.
3. Keep a final export of the old database and image store, read-only and documented, even if you never expect to open it.
4. Check for legal holds before destroying anything. A hold on any record in the system can reach its source data too.
5. Close licenses and contracts in writing, including the vendor's duty to return or destroy any copies it holds.
6. Dispose of hardware and media securely, with certificates, once your retention authority permits.

Keeping the old data

The converted copy may or may not be the official record under your state's law. Some states have specific rules about which medium counts as the record copy and when source media may be destroyed after reformatting. Ask your state archives and keep the answer in the project file. Our article on writing a retention schedule departments will actually follow explains how to bring conversion source data into the schedule so its eventual destruction is authorized rather than improvised.

Be honest in the plan about limits. Some media will be unreadable, some index entries will be wrong in ways nobody can fix, and conversion cost is driven more by the condition of the source than by its size. Build contingency for both, and keep the original volumes and microfilm wherever they exist. They are the backstop when the digital record turns out to have a hole in it.