Disaster Recovery at the Plant Level? Why?

Let’s start by defining what Disaster Recovery consists of.

There are many existing definitions of “Disaster Recovery” or its translation into Spanish: “Recuperación ante Desastres” and all of them are closely related to the world of computing (that other “world” that is ahead of us in terms of recovery strategies), among them we find:

Disaster Recovery:

“…It is part of a larger Business Continuity plan that includes the processes and solutions aimed at restoring critical applications, information, hardware, communications and networks, and other infrastructures of information and technology systems.”

Source: www.neverofftechnology.com

“The Disaster Recovery Institute International (DRII) states that disaster recovery is the area of business continuity that deals with the recovery of technology rather than the recovery of business operations.”

Source: blog.celingest.com

“Disaster Recovery is the area of security planning that is responsible for protecting a company from the effects of various negative events…”

Source: whatis.techtarget.com

We might think that Disaster Recovery is only applicable to computer systems at the level of business and office applications (in other words, to office automation and computing as we know it), but this is not the case. From this last definition, what technologies can we mention in our industrial “world”? Among them we find: PLCs, HMIS, SCADAs, drives, robots, etc.

Data acquisition and transfer technologies have advanced so much in the last 20 years that it is now very unlikely that a computer is not part of a manufacturing process. In addition, the various control devices (PLCs, DCS, drives) are all interconnected and this is due, among other things, to the need to take more data to make better decisions.

As the phrase “you cannot control what you cannot measure” indicates, it is no longer a secret to anyone that corporate decisions are increasingly based on data.

Image by D.Fletcher

How to Apply these Recovery Strategies in an Industrial Environment?

Are there technologies for this? The answer is yes. Several of these technologies and solutions are aimed at guaranteeing minimum unscheduled downtime and increasing our productivity (and all those KPIs that will then be analyzed by decision-makers in “dashboards”).

Among the solutions that allow us to easily recover from a disaster (that is, adopt Disaster Recovery strategies), we can mention the following:

1. Redundant communication networks.
2. SCADA software with redundancy possibilities at the application, historical and visualization levels.
3. Software to create backups and version history of industrial device programs (PLCs, SCADAs, HMIs), so that if, for example, a PLC breaks down, we can recover the last known good copy since it is stored on a server. In this way, we will only have to connect the new hardware and download the program.
4. Hardware and software solutions so that your application servers are fault-tolerant, which will allow systems to continue functioning even if a hardware failure occurs, for example.
5. Visualization terminals based on “zero clients” which will allow you, for example, to replace an operation terminal in less than 5 minutes.

These solutions are aimed at facilitating operations at the plant level and do not require advanced computer knowledge.

If any of the following situations sound familiar, I think it’s a good time to consider some strategy of what I would dare to call: Industrial Disaster Recovery:

1. I don’t know where the latest copy of the program of the main PLC(s) is. If one failed, I don’t know how long it would take to recover the system.
2. A PC on the network has been infected with a virus that spread throughout the plant, we had to stop production. It hasn’t been easy to recover.
3. The server that hosts the database that stores the traceability data had a hard drive failure, we have lost very important information during all the time it took us to recover it.

I could cite a few more scenarios, although I am sure that at this very moment, many more are occurring to you…