Your control system has been running for 25 years and it is now time to modernize. Before you can start your modernization, you need a clear picture of everything running in your current system, so you start looking for your documentation. At this point, you may realize you do not have any documentation. Or you may find that your documentation reflects the system as it was when it was installed. While someone may have thought to scratch a few notes in the margins of the design drawings or add a redline on a termination drawing for a point that was added years ago, can you trust that it is current?
To develop a complete picture of all the devices included and functions performed within your control system, you may need to implement some serious detective skills to reverse engineer the system especially since there is no magic bullet for getting what you need from the system. This blog provides our advice on how to reverse engineer your control system by taking what you know about it, no matter how little information that may be, and working from the outside-in to determine the functionality of each algorithm. From this sleuthing, you can then assemble the documentation you need to be prepared to properly maintain or modernize your control system.
Searching for Clues: Gathering the Information about your System
In our experience, most older systems outside of those in regulated industries, like Life Sciences, do not have functional descriptions or process narratives describing how a control system operates. On top of that, commenting that could otherwise help in the reconstruction of a functional description is not always available or complete within a legacy configuration. Maybe that level of code documentation was never there or maybe the comments were lost over time.
The first step in any reverse engineering effort is to take stock of what information is available about your system. Here are some of the places we suggest looking:
Documentation such as P&IDs or electrical drawings, even if out-of-date, make for a great starting point for the physical upgrade of the controller and associated I/O and communication modules. These drawings can also help fill in the gaps in other places as well. Even out of date electrical drawings and P&IDs can help in the development of a preliminary system device list (motors, valves, analog transmitters, etc.). Electrical drawings are also a nice starting point to getting comments back into your legacy configuration. By filling in the information related to the inputs and outputs, the nature of the function blocks and sequence logic is put in better context.
An HMI allows the operator to interface with the process through the controller. The configured screens within an HMI may include a graphical representation of the process, system setpoints, and parameters. HMIs also include a tag database and an alarm configuration. All this information can be used to fill in additional comments within the legacy configuration and help in the development of the system alarm and device lists, as well as provide some level of understanding of any sequencing and recipes that are configured.
Panel Buttons, Lights, and Indicators
If your control system does not have an HMI, panel buttons, indicators, and lights should be labeled. These are also physical I/O so the wires can be traced to an I/O point to, again, help with adding comments into the legacy code.
A Panalarm annunciator or set of alarm lights can also be useful in the development of a system alarm list. The chances are good that in any modernization these lights will be eliminated or supplemented with some level of alarm management in your SCADA system.
Field Instruments and Wiring
Once you have reviewed the more obvious sources of information, it is a good idea to go into the field and trace the wires from the device to the input or output at the control system terminal. If documentation existed and there was an HMI from which to learn about the process this will help confirm the information that was gathered to this point. If documentation was sparse this may be the best starting point to help develop an I/O list and fill comments in your control system configuration.
During this investigative process, it is also a good idea to interview the people who running the system. If you have experienced operators, this is where the institutional knowledge arrives. They can tell you what works and what doesn’t, what has been decommissioned, what has moved, and generally how they run the process. Our suggestion in this case would be to center a discussion like this around the HMI by going through all the screens to determine what they know about each of them.
Keep in mind that this can be a somewhat iterative process and a bit like logic puzzles or a game of Sudoku. For example, once the configuration is partially commented you can go back through the routines and start to fill in more information by examination of the logic elements that are used. This can be useful as you reconstruct the functional description.
Additional Information to Gather for Batch Processes
If you are running a batch process, we recommend following the hierarchy model provided in the ISA-88 standard to gather information about the system. This includes determining what falls under the procedural, physical, and process models and identifying the units and procedures used within the system. If a process narrative is not available, then having someone with operational experience walk you through the sequence will help in the initial development of a description of the batch process. This information will help provide more context and a deeper understanding during a code review of the configuration.
With a batch or sequence process, it is also a good idea to look at the notifications that the current system is providing, such as step numbers, step descriptions, or message prompts. For example, if you have sequence steps that appear on the HMI, you should be able to determine how your process equipment is used in each step.
Develop Your Features List, Control System Narrative, and Additional Documentation
Once you have used your detective skills to determine your I/O and program structure and have devices and code that are fairly well commented, you can begin the process of developing your control system narrative and/or the design/spec documentation you will need to perform a modernization in the future. This should include I/O, interlock lists, and operator prompts, especially if you are operating a batch system, and information on dead components, code, and equipment that may have been abandoned in place.
If your intent is to perform a modernization, you should also create the lists of all the additional features that will need to be replicated. We recommend the following lists:
- Recipe parameters
- HMI screens
- Historical data or trending requirements
- Understanding plant hierarchy
For batches and sequences, you should also include charts or descriptions of steps, sequences, or procedures.
When Should You Start Reverse Engineering Your System
While we focused on reverse engineering a way to prepare for a modernization project throughout this post, there is no reason to wait until a modernization is upon you to get started. Looking into some of this information will not cost much and can help you run and troubleshoot your system now, as well as prepare you for a modernization when the time comes. However, since the process of reverse engineering a control system, and creating the subsequent documentation, is not straightforward, you may want to start by talking to the experts at ACE. Our experienced engineers are ready to help you perform the detective work required to help you better understand your control system.
Saying Goodbye To Your Favorite Control System
If you are like me, you loved your first car. Nothing since has felt so solid. So reliable. You...
Improved Safety and Production Performance through Better Alarm Management
Ever misplace something important? Ever have too many important things on your to-do list and can’t...
Improving Cybersecurity Resilience
The first thought most people have when it comes to industrial, or OT, cybersecurity relates to the...