• Home
• Articles Archive
• Features List

• Media Pack
• Contact Us
• Subscribe

CONTINUITY OR COLLAPSE? THE CHOICE IS YOURS

From STORAGE Magazine Vol 7, Issue 1 - February 2007

When it comes to disaster recovery-business continuity, you are only as safe as the systems put in place to cope with such an event. So, If a serious incident were to strike your business, how well would it cope? Editor Brian Wall speaks to those in the know about what measures all organisations should be taking to avert a data disaster

Why is it that, despite the potential havoc that a data disaster might wreak on the business, some organisations fail to take adequate steps to protect that most precious of assets? Is it sheer bloody-mindedness or an attitude that borders on the cavalier?

"While cavalier may be a little strong, some organisations seem to believe that 'it will not happen here' or hope it will not, knowing that they could be better prepared," says Dominic Hill, consultant, Siemens Insight Consulting. "Others have gone through the effort of creating plans, only to leave them untested and unmaintained, doing little other than providing a false sense of security."

Realistically, of course, disasters do happen, without prejudice to the type of organisation or its state of preparedness. Often it is the implausible or an unpredictable sequence of events that culminate in a disaster, Hill adds. So what should a business do? There are a number of steps he recommends:

• Get buy-in from everyone, but especially at board level. Without this, it will be an uphill struggle to get suitable resource
• Identify the 'critical' functions that derive the greatest business benefit (whether financial or intangible, such as reputation and goodwill) and the impact of their loss - preferably in financial terms, as it helps justify the costs
• Where possible, design the provision of critical functions to minimise the single points of failure, giving a resilient and available solution. Alternatively, develop a recovery strategy that meets the business requirements for availability
• Develop recovery plans that provide appropriate cover for the critical functions, identifying who needs to do what, when and with whom. Write the technical recovery procedures for a defined skill level, and in a way that can easily be maintained.

Change happens, states Hill, whether it be a new business direction or an application patch that changes the recovery process. "Rigorous change control will capture any alterations and good discipline will ensure that they are reflected in the plans in a timely manner. Test the plans regularly, ensuring that those who need recovery experience get it. Don't have a disaster, if you don't need to. Understand what is important and protect it in a way that is fit for the business."

According to Gary Preston, managing director, north and south EMEA, Tandberg, true disaster planning has become seriously implemented in most mission-critical large organisations. "At Buncefield, it wasn’t the large corporations such as BP and Shell that suffered astronomical data loss - it was the small/medium businesses on the neighbouring industrial parks that took the risk and are now paying the price for having no backup for their lifeblood data," he says. "True disaster scenarios of a magnitude enough to make headline news are less common than the fact that most businesses who suffer an astronomical loss of data suffer at the hands of human error or deliberate attack by a disgruntled employee."

Even smaller companies should consider implementing a true watertight disaster recovery strategy that minimises the risk of data loss. "The trick is to consider the likely threats and weight them accordingly, and incorporate these into the IT infrastructure," argues Preston. "Donšt just consider your data centre - consider if you have small remote offices implementing applications that can easily and cost effectively protect remote offices centrally.

"Consider the impact of downtime on each of the businesses' core daily activity and rank them accordingly. Donšt neglect compliance and legal obligations to retain information. Even the smallest companies have compliancy requirements. Understand that true disaster recovery is an ongoing and evolving process that grows with the organisation."

Frequently, this is where companies fall short; plans made 18 months ago are usually grossly inadequate for any company experiencing even average growth in its business and resultant data.

Preston also offers a number of tips for minimising downtime and data loss:

• Expect the unexpected - assign a dedicated member of staff to be responsible for DR
• Be familiar with your procedures
• Test the data recovery window
• Perform monthly trial restores
• Back up the entire system and keep backup logs to enable fast drill down for file restores. Consider complementary disc and tape technology - disc for speed of access, and restore and tape to take off-site
• Keep at least one off-site copy in a secure environment
• Keep an-up-to-date record (off-site) of what you have - system configurations, network settings, disc configurations, application and software versions.

Certainly, a well thought-out disaster recovery plan is built around the requirements and circumstances of an individual business, as well as the potential damage, and degree of risk and exposure, to which its IT infrastructure might be prey.

One of the first contingency planning tasks to be undertaken when planning for disaster recovery-business continuity is to prepare a comprehensive list of the potentially serious incidents that could affect the normal operations of the business. This list should include all possible incidents, no matter how remote the likelihood of their occurrence.

Yet despite the fundamental good sense of having such a recovery plan in place, and keeping it up to date, there are many common mistakes made by companies when it comes to disaster recovery - the first and most crucial being not having a plan in the first place!

"Some companies leave this solely to the IT deparTment, and therefore the business and IT requirements are not linked," points out Gill Borniche, director, EMC Software Marketing, Rest of Europe. Where there is a plan, sometimes the plan isn't current and hasn't taken into account new applications, upgrades, increased storage capacities and new personnel. "Furthermore, companies do not test their plan, thus it doesn't address the right risks and value of information.

“For instance, in a consolidated environment, with different back-up systems, the recovery management policy doesn't work. Other common mistakes include not putting enough emergency-trained staff in place and providing employees with little or no information."

Live testing and trial run situations are a vital aspect of any DR-BC plan, says Borniche, "not only regarding the internal systems and services, but also human and environmental aspects. For example, following a London bombing in recent years, a financial institution had difficulty following the failover from their principal site to their disaster recovery site. No one had anticipated not being able to get their staff from one site to the other due to the paralysis of the London Underground and roads."

A standard disaster recovery-business continuity plan should, he adds, start with a project manager and appointed staff from different departments of the company to:

• Collect data - what is the value and how critical is it to the company or department?
• Conduct risk analysis - ie, an airline would need to analyse "what if check in for flights fail?"
• Define exposure and strategies for: network recovery, application recovery, data recovery, user recovery
• Plan staffing/training
• Plan testing/maintenance.

And what about outside help - should that be considered as part of the overall strategy? "A company should not hesitate to consult an expert in disaster recovery," he says, "for example, EMC's consulting services."

A major step in creating a DR plan is understanding the data you are protecting and the need to take a top-down approach to make sure you are attaching the right level of protection to the right information. The better the data management, the greater the data protection and recovery. There are a number of steps that need to be taken, including the following:

• Assess the current state of data management
• Agree a detailed tiered storage strategy for DR
• Evaluate how virtualisation may - or may not - help
• Look at a top-down approach to data
• Identify the key steps of an effective data protection strategy
• Establish a workflow for data
• Investigate how applications should be designed for DR.

Of course, a DR plan is only as good as its ability to provide business resumption. Getting data back - and in a useable form - is the main goal of a DR plan. And businesses are generating more data than ever before, as Mike Walters, consulting systems engineer at NetApp, remarks. "IT managers need to understand their business and know which data is valuable and which is not. At the same time, storage policies need to be implemented to protect all data from loss or corruption, regardless of type (current or archived), format (file or block) or locale (local or remote DAS, NAS, SAN).

"Disaster recovery solutions should be simple to implement and operate, and with minimal demand on IT and network administrations resources. These solutions should seamlessly integrate into existing infrastructures, providing immediate protection, without changing how a business works."

Walters reasons that data recovery solutions today are flexible enough to address a broad range of recovery point objectives, ranging from zero data loss, to one hour, to one day. "Customers are enabled to make the trade-off between cost and data loss exposure, based on their data recover and budget needs."

An important aspect when implementing any disaster recovery solution, he adds, is the ability to include it within a tiered storage or Information Lifecycle Management (ILM) policy. "For example, solutions need to include the ability to flexibly replicate data between primary or tier 1 systems and secondary or tier 2 systems. Many solutions today do not offer this flexibility, so some businesses are only implementing disaster recovery solutions for their top five to ten per cent of mission-critical applications."

Once a DR plan has been developed, it must be subjected to rigorous testing. The testing process itself must be properly planned and carried out in a suitable environment to reproduce authentic conditions as closely as is possible. Crucially, it is those who would undertake such activities who must test the plan, if it is to have any bearing on reality.

Moreover, the test procedures should be well documented and the results carefully recorded. This is important to ensure that feedback is obtained for fine-tuning the plan and to ensure that all of those who are party to the procedure have documented evidence of any changes, so they are in agreement right thorough to final ratification. Equally, it is important to audit both the plan itself, and the contingency and back-up arrangements supporting it. No shortcuts can be permitted at any juncture.

This stage is dependent upon the development of the plan, and the successful testing and audit of the plan's activities. Everyone within the organisation must be made aware of the plan and its contents, as well as his or her own related duties and responsibilities.

Again, it is important that all personnel take the disaster recovery planning seriously, even if the events that would trigger the plan seem remote and unlikely. It makes sense to get feedback from staff in order to ensure that responsibilities and duties are fully understood, and the potential gravity of the situation recognised and acknowledged. This is particularly important where an individual, or group, has a close dependency on the actions that are taken by others.

On-going, any plan must always be kept up to date and applicable to current business circumstances. This means any changes to the business process, or to the relative importance of each part of the business process, must be properly reflected. Also, someone must be assigned responsibility for ensuring the plan is maintained and updated regularly. And that means ensuring information concerning changes to the business process are properly communicated across the organisation.

Equally vital is that any changes or amendments be fully tested. Personnel should also be kept abreast of such changes, in so far as they affect their duties and responsibilities.

The advantages and benefits of disaster recovery planning are manifold, affecting numerous aspects of the business, including its ability to:

• Maintain, or resume, operational trading
• Safeguard its reputation, brand and image
• Reduce downtime through the mitigation of disasters
• Prevent loss of customers to competitors, due to an inability to trade
• Ensure the long-term confidence of its clients, investors, business partners and own personnel.

Ultimately, in whatever way an organisation puts together an effective disaster recovery strategy, the most important aspect is getting buy-in from the senior management team, in order to - as EMC Software Marketing's Gill Borniche states - "at least protect the company's most valuable assets". ST