Author: Nop Srinara
The digital economy has dramatically affected the way that organisations think about disaster recovery. What was once a ‘nice to have,’ is now a ‘must’ for many businesses, in line with customer demands and evolving legislation. Issues around privacy and the way we handle personal data, cyber-attacks, online system failures, human error and natural disasters have been a key driving force. But how often are these systems actually put to the test? How can we be sure they are going to save the day if the worst happens?
For managed service providers, Disaster Recovery (DR) testing should be considered essential. Regular testing is the only way to be certain you can restore customer operations quickly following an outage. Managed services success is all about delivering a stellar customer experience.
DR testing is a key part of [that]. If you are serious about delivering excellent customer service, DR testing should be a priority.
Here are five tips that can help ensure your testing efforts are effective:
CHOOSE TECHNOLOGY THAT FACILITATES TESTING
Not too long ago, SMB disaster recovery was all about local disk backups with offsite tape. Remote replication was far outside the typical SMB’s budget, requiring massive hardware investments. However, restores from tape are slow, incurring significant business downtime. Additionally, tape backups typically only occurred once a day, so any information created between backups was vulnerable to data loss.
Today’s DR systems take frequent image-based backups and replicate server images to the cloud. In the event of a primary server outage, operations can be restored directly from a backup instance of a virtual server on premises or in the cloud. This approach is commonly referred to as “instant recovery,” or “instant virtualisation”. This technology fundamentally changed how DR testing is performed by allowing users to easily spin up virtual machines (VMs) and test the ability to restore operations. The testing process will vary depending on the backup system that you choose. However, here are some general guidelines you should follow.
First, always test cloud virtualization without networking to verify VMs have basic functionality. Then, start VMs with limited/internal networking. Once the VM is booted, test the functionality of your essential services. Remember, some services (Exchange, certain database applications) may not be able to start with limited networking. Next, log out of the device and log in as a different user to ensure that the local VM is available. Remember - If your customers are relying on outdated or ineffective technology, it is your responsibility as their managed services provider to recommend gear that enables DR testing.
DEFINE THE SCOPE OF TESTING
Are you testing the ability spin up a virtual machine locally? In the cloud? Both? Is the test conducted in a cloud-based environment that mirrors the production environment? Or, is the scope broader than that? Other tests might go beyond IT—testing an emergency generator, for example. There is no single “right” approach. However, some types of testing can introduce the risk of data loss or corruption, for instance, unplugging a server or other technology to simulate a power outage. When determining the scope of your DR testing, you’ll need to balance the specific needs of your customers, how much disruption they can tolerate during testing, and the amount of time and resources required for you to conduct tests.
There’s no magic number for how frequently you should perform DR tests. Again, it’s a matter of balancing customer needs with your time and resources. For example, you might conduct local spin up tests quarterly and a more comprehensive cloud failover twice a year. You should however consider performing DR tests following any significant changes to your production environment. Neglecting to do so has been the undoing of many a DR plan.
There are many tools available document networks, disaster recovery plans, and DR testing results. These tools range from basic and narrowly focused to highly comprehensive and flexible. It is important to remember that documentation goes beyond IT components. You should also have detailed contact lists for your customers, technology vendors, and any other pertinent information you might need following a disaster event.
REPORT THE RESULTS OF YOUR TESTS
Sharing the results of your DR testing gives you the opportunity to illustrate the value you deliver to your customers. One popular approach is to share results of your DR testing during a quarterly business review (QBR) with your customers. Remember: the purpose of DR testing is to identify issues BEFORE they impact your ability to restore. Reporting after the fact gives you the ability to present testing in a way that highlights your value.