This article is part of a series examining the key considerations that shape successful SCADA modernization projects in oil and gas pipeline operations.
A pipeline operator’s process historian holds something that cannot be recreated. Years of pressure, flow, temperature, and equipment data, recorded second by second, form a continuous record of how the pipeline has actually behaved over its operating life. That record informs engineering decisions, supports regulatory reporting, establishes the baselines that make anomalies visible, and provides the evidence trail that any incident investigation depends on.
When a SCADA modernization changes the platform underneath that data, the operational record needs a deliberate plan to carry it forward. Historian migration is the part of a modernization most easily treated as secondary, because the new system can be fully operational from go-live without it. Yet the data at stake represents an asset accumulated over years, and protecting it is a matter of planning that is well understood.
Why the Historian Deserves Specific Attention
In most modernization projects, the priority understandably falls on getting the new system running: configuring the platform, migrating tags, rebuilding displays, and validating control. The historian occupies a different position. The new historian can begin recording from go-live regardless of what happens to the historical data, which makes it easy to treat the migration of that historical record as something to address later.
The difficulty is that “later” can arrive after the legacy system has been decommissioned. Once the old platform is retired and its hardware repurposed or removed, data that was not exported and validated beforehand can become difficult or impractical to recover. This is entirely avoidable. The key is to treat the historical operational record as a defined project deliverable from the outset, with the same standing as the new system configuration.
The Regulatory Dimension
For pipeline operators, historian data is not only an engineering asset. It is a compliance obligation. Industry guidance for pipeline SCADA recommends that operators retain one second of process data for a minimum of two years to satisfy PHMSA integrity-management requests and TSA security-incident reviews. That retention expectation does not pause because a platform is being replaced.
The broader regulatory trend reinforces the point. PHMSA’s recordkeeping requirements have expanded considerably in recent years, with a strong emphasis on traceable, verifiable, and complete records. An operational history that is fragmented or partially lost during a platform change works directly against that standard. A migration planned with retention obligations in mind keeps the operator on solid footing.
Deciding What Happens to the Historical Data
A well-planned historian migration begins with a clear decision about the disposition of the existing data. Industry practice recognizes three established approaches, and the right choice depends on data volume, the source historian format, and how the historical data will be used.
Full migration imports the historical data into the new platform’s historian so the complete record lives in one place going forward. Modern SCADA platforms support this path with tools for large-scale imports from legacy solutions. It delivers the cleanest long-term outcome and is most straightforward when data volumes are manageable and the source format is well understood.
A read-only legacy archive retains the historical data in its original system or an exported archive that remains queryable, while the new historian records everything from go-live forward. This is a practical choice when data volumes are very large or the source format does not translate cleanly.
A combination migrates the most recent and most frequently accessed data into the new platform while archiving older data in a retained legacy format. Practitioners often express this as migrating the most recent few years into the new historian and archiving older data in its native form, balancing continuity against the effort of moving very large volumes.
None of these is inherently correct. The decision is made early and documented, so everyone understands where the operational record will live and how it will be accessed.
The Migration and Its Validation
Historian data migration is a defined technical process, and its reliability rests on treating it as one.
The starting point is a baseline of the legacy data, capturing counts, time ranges, and the tags represented before anything moves. This baseline becomes the reference against which the migrated data is later verified. Because source and target historians rarely share an identical internal structure, migration involves mapping the source data to the target model: tag naming, engineering units, timestamp handling, and data types all need to align, so that a value recorded years ago carries the same meaning in the new environment that it did in the old. Timestamp fidelity deserves particular care in pipeline operations, where the exact time a reading was recorded is fundamental to trend analysis and incident reconstruction. For very large datasets, migration is typically staged in defined segments that can each be validated before proceeding.
Validation is the step that gives a migration its credibility. Record count reconciliation confirms that the number of records in the target matches the source for each tag and time range, with any difference traced to an explicit reason rather than silent loss. Checksum or hash comparison provides a stronger integrity signal, detecting any change to the underlying values. Field-level spot checks and statistical sampling confirm that values, ranges, and formats match across the dataset, and aggregate comparisons of sums, averages, and extremes confirm the data means the same thing in the new system as in the old. A particularly effective technique is parallel comparison: querying the same information from both historians for a defined period and confirming the results agree. Throughout, the validation results are documented, becoming part of the operational record and supporting audit readiness.
Sequencing the Cutover
The order in which migration and cutover occur is what protects the data. The legacy system is retained until the historical data has been exported, migrated, and validated, and until the operator has confirmed that the record in the new environment is complete and accurate. Decommissioning follows that confirmation rather than preceding it.
Running the two systems in parallel through this period is a common and effective practice. It provides the window in which validation can be completed against a still-live source, and if any question arises about the migrated data, the original remains available to resolve it. Only once the operator is satisfied that the operational record is fully preserved does the legacy system reach end of life.
Bringing It Together
Operators who protect their operational record through a modernization approach it consistently. They treat the historical data as a defined project deliverable with acceptance criteria, not as a task that follows go-live. They make a deliberate, documented decision about whether data is fully migrated, archived in a read-only legacy form, or handled through a combination of both. They plan the migration with attention to tag mapping, units, and timestamp fidelity. They validate the result with record counts, checksums, sampling, and parallel comparison. And they retain the legacy system until that validation is complete.
The operational history held in a pipeline’s historian is one of the most valuable assets a modernization touches, and one of the few that cannot be rebuilt if it is lost. Carried forward with a deliberate plan, that record continues seamlessly into the new system, preserving the baselines, the compliance trail, and the engineering insight that years of operation have produced.
PipeCom is a certified SCADA systems integrator serving pipeline operators across Canada, the United States, and Latin America. For more insights on SCADA integration and operational technology, visit pipecom.com.