• Home
• Articles Archive
• Features List

• Media Pack
• Contact Us
• Subscribe

EMAIL ARCHIVING - THE VITAL LINK

From STORAGE Magazine Vol 6, Issue 4 - May 2006
 

When it comes to email archiving, businesses need a system that will store, manage and allow access to all emails over their lifecycle - and eventually allow for their destruction. But which are the best methodologies to embrace? What solutions will work best for your particular scale or type of operation? Editor Brian Wall investigates

The information contained in your organisation's emails may play a pivotal role in shielding you from potential liabilities, assuring compliance with government regulations, or providing knowledge that exists nowhere else in your business. In fact, emails have now become a dominant method of business communication. However, the way they are managed is often overlooked by organisations. And while their growing volume imposes a huge strain on mail server storage, compliance dictates that such messages must nevertheless be kept and made available on demand.

One of the many increasingly heavy demands being placed on IT departments across the world is the management of archived information. Organisations are now deeply aware of the need to keep business-critical, or business-pertinent, information for regulatory and compliance purposes. They are beginning to see the scale of the problem, given the inexorable growth of structured, unstructured and indeed semi-structured content like email. They look to the CIO and his or her department to take ownership of this very pressing need.

This, says Dave Gingell, vice president software marketing, EMEA, EMC, is why IT organisations are required to manage archived information and make it readily available for enterprise-wide search and discovery, whilst continuing to hammer on costs and performance issues. "Email archiving software is a category that has gained much prominence in the last few years as the IT department has tried to cope with the rapid growth in email and the largely unchecked use of locally stored files (.nsf and .pst files). Many of these email archiving applications are quite sophisticated, managing the full lifecycle of an email, from classification and categorisation, through storing email commensurate with its value and importance, through to its eventual long-term archival or disposition."

The next wave of innovation, he states, is already at hand and has emerged to solve an even bigger problem - that of enterprise archiving. "This is recognition that, although email is an important piece of content, it is only part of a more comprehensive and all-encompassing enterprise archiving requirement. Email must be archived, along with other relevant information, to get a full picture of the transaction; the customer or the other content relating to an important business process. These archives are being driven by many sources of data, ranging from email, reports, transactional data and unstructured content, such as contracts, documents and rich media."

Organisations need to think beyond just their needs for email archiving alone and look at an enterprise-level archiving platform that brings silos of archived information together in unified way, according to Gingell.

"The convergence of email archiving and content management technology going forward will set the stage for a new wave of technology fusion that will blend collaborative tools like Voice over IP [VoIP], real-time video conferencing, content management, instant messages and email into one common environment, thereby allowing users to share information and collaborate in a more real-time, online environment.

"A unified archiving platform will enable customers to employ a common set of services for collecting, retaining, migrating, securing and discovering all types of information. As a result, customers are better equipped to search and retrieve information for compliance and legal discovery, content reuse, improved decision making, and improving the cost and operational efficiencies of their archiving applications." EMC is already delivering on this requirement, he adds, with the recent release of Documentum Archive Services for E-mail, an enterprise-class email archiving software solution based on a unified archiving platform.

Certainly, one of the biggest challenges that organisations face when it comes to implementing the correct procedure for email archiving are setting internal policies and motivating employees to comply. However, many companies have yet to develop guidelines, states Alasdair Kilgour, managing director, CommVault UK, leaving employees to make their own decisions on what emails they should keep or delete.

"Commonly, pre-determined storage quotas dictate how much can be saved in a user's inbox, which encourages employees to delete messages in bulk, regardless of content, as soon as they are prevented from sending an email. As compliance becomes every IT manager's most vivid nightmare, and ensuring that data can be 'found on demand' becomes more important than ever, it makes sense for organisations to turn to external suppliers, not only to set best-practice guidelines around the storage of email, but also to implement data management strategy incorporating backup and archiving systems, thereby ensuring data is easily recovered. In fact, a good data management strategy system should reduce administration; allow a search through data stores to be completed quickly and preserve - or delete - data at the right time. Get these fundamentals right and half the battle is already won."

Despite the varied way in which companies do business, there is a common misconception that there must be a 'one size fits all' solution to email archiving. Legislators demand different cradle-to-grave lifecycles for information, depending on the business and the focus of individual departments. CommVault, says Kilgour, has taken these different business demands into account with its DataArchiver and DataMigrator solutions. "CommVault understands that each customer's business is unique, so it offers flexible, easy to configure solutions because - like fingerprints - no two systems are exactly the same! By working closely with each client, the professional services team develops an understanding of, and solution to, the business pain points, whilst also ensuring a rapid return on the investment."
What is most concentrating people's minds is the way in which email is now being used. Once what Andrew Graham, CEO, EASY SOFTWARE, describes as
"a revolution in written correspondence", email now contributes to an ever-increasing number of corporate bottlenecks and the trend is set to continue, with a staggering forecasted 30% increase in email traffic over the next couple of years.

"Apart from the telephone, electronic mail traffic is undoubtedly our most important communication medium. As mail servers strain under the weight, and regulations and compliance burdens to retain and safeguard emails increase, there are now sound reasons to acquire an extensive archiving solution that will not merely store emails, but also logically manage them, providing a foundation for multiple information management."

Speed is the key benefit of email, Graham points out, "from simple memos through to sending information internally and externally, right up to invoices, orders and customer documentation - all this is transmitted to the respective recipient in seconds."

Meanwhile, the number of sensitive corporate emails and legally significant attachments is increasing at a disproportionate rate, an application scenario that the original mail system was designed to cope with. So the sheer flood of disorganised information sweeping the inbox is overloading our mail servers.

"This problem can be remedied, though," adds Graham. "Intelligent email management tools that directly integrate with Outlook Exchange and Lotus Notes/Domino take remedial action at various levels and deliver revision-proof storage of emails, while meeting all the legal requirements ensuring compliance and reducing corporate risk. Simple and fast full text search in all texts and email attachments boosts productivity which, in turn, reduces costs by saving a significant amount of time.

"The original email, once archived in a compliant system, has greater evidential weight than the printed version. The printed version could have been changed and then printed. This legal anomaly is intrinsic in most electronic document formats, e.g. Word, Excel.

"Intelligent email management tools remove the clutter that is not permanently required from databases. The net result is that businesses' databases remain organised, clean and fast, because the email server is no longer used as a storage location, but only for transmitting information between your workstation and the electronic archive. In this way, intelligent email archiving solutions are the basis for modern information management, making your enterprise-wide communications flow once again."

Compliance is, of course, the most newsworthy driver for email storage, but there is also an increasing need to improve the capacity management of email and improve efficiencies in email performance and backup/recovery.

"The amount of email being processed each day has enormous impact on the latter here, and that's where archiving comes into its own," says Nigel Tozer, consulting director, CA. "The problem is that, while email servers are capable of handling large volumes of data, they are not designed for storing it for long periods of time and can't function properly if they are used in this way.

There are several ways in which companies might try to address this, such as creating archive PST files or equivalent on individual users' desktops, or storing them on offline backup media, such as tape or CDs. Both of these have drawbacks, not least in the fact that they aren't readily searchable. This produces a litigation issue, as a court of law will now no longer accept an organisation's inability to produce subpoenaed emails due to lack of email retention policies."

A centralised, easily searchable repository is the best option, he argues, which will allow for not only easy access to any email, but also the effective management of each one's lifecycle. However, alongside the technology, companies need to consider their retention management policy. Email needs to be treated as a corporate record, but different content will require different retention periods - and the management of this needs to be built in to a company's guidelines in line with any regulatory requirements.

"Different solutions on the market today offer differing takes on 'email archiving' and it's important to evaluate what your company's needs are and what the products offer," adds Tozer. "CA's Message Manager, for example, works across Exchange email servers, but also the others, such as Notes and UNIX, which make up the other 50% of the market. There is also an issue of storage. Estimates suggest that, in 2009, email servers will process about 45MB of email per user, per day, so the ability to scale high enough is essential."

In most organisations, the primary role of the email backup is to enable an operational recovery in the event of a system failure. The critical word here is 'operational', stresses Guy Bunker, chief scientist, Symantec. "The IT department will be protecting the operational integrity of the email infrastructure. They will be taking regular backups so that, if necessary, they can recover an operational system to a specific point in time. Recovery, or following of an email trail across a period of time, is a completely different problem.

Bunker offers two scenarios. First, a person in your organisation sends a defamatory email to a competitor and then immediately deletes the copy of the email from the sent box and the deleted items folder. One month later, you receive a notice of libel from your competitor, citing this email as evidence. You need to see what was actually sent. Does the email exist in your system? "The answer is 'probably not'," says Bunker. "It may exist in a deleted items cache (on a backup) but, given the elapsed time, this will in all likelihood have been purged."

In the second scenario, your purchasing department has been negotiating a vital contract and, to speed things up, a lot of the negotiation takes place via email. The negotiation is concluded over a period of one month. Two years later, the contract is in dispute and a court of law asks for all evidence supporting your claims about the contract. Email is a critical part of your case and you need to find all relevant email. Can you find the emails?

"This time the answer is 'it depends'," states Bunker. "Your IT department may well have a policy that says they recycle the backup media. Maybe you are fortunate and the IT policy is to keep all tapes. Now the problem turns into something akin to finding multiple needles in multiple haystacks (and you have to rebuild each haystack first)!"

In both instances, according to Bunker, the way out of the problem is to access and search the complete collection of (email) records of historical interest - ie, an archive. The archive needs to have not just those emails that are 30, 60 or 90 days old, but every email. This means implementing a solution that catches every email through the system, via the journal, not just archiving an individual's mailbox after a period of time.

"It's time we all more clearly understood that backup and archive are not the same thing," he stresses. "This way, we can reduce corporate risk and, as a by-product, decrease the costs and overheads associated with protecting operational systems.

"In the case of email, there is historical email (which doesn't change) and new email. The historical email, even if it is only two minutes old, belongs in an archive, which may have a different retention policy, but where it can be preserved for historical access, along with the 'traditional' view of the archive, which has email that has been removed from the mail system after a period of time.

Backup of the email system is still vitally important for disaster recovery purposes, but, by using archiving, the quantity of data that needs to be backed up can be significantly reduced. In making this distinction, the overheads associated with operational backup can be drastically reduced and the ability to support business processes regarding corporate record is greatly enhanced."

How well is the vendor community responding to these needs? Simon Pearce, vice president EMEA, Quest Software, believes that existing email storage solutions are, in the main, inadequate. "There can be no justification for technology that forces the IT team to manage 100% of email when only 2% changes on a daily basis. Nor for relying on users to manage their own in-boxes - a strategy resulting in haphazard deleting that undermines compliance policies and jeopardises corporate information.

"However, organisations looking at email archiving solely to resolve the storage problem are in danger of missing the point. Simply using archiving technology to move email data away from the inbox and into a repository may resolve the storage problem, but it also locks away vital business information.

"Used correctly, email is a core component of business success. The email archive should be treated accordingly to ensure all enterprise information is available to every user and every application across the business. To achieve this goal, email needs to be stored in a fully retrievable, auditable and searchable manner. And, critically, this must be done without creating a massive storage overhead or requiring user intervention."

Underpinning this shift, Pearce adds, must be a recognition that the traditional hierarchical storage mechanism within email servers is constraining access to corporate information. "While undoubtedly an excellent messaging system, the email server was never designed as an information store. By using a relational database email management solution alongside existing mail servers, organisations can solve the storage problem and, critically, unlock vital business information.

The relational model overcomes the conflict between retaining all emails and escalating storage costs. By modelling emails in an information store that was designed for this purpose, the process becomes extremely efficient. "Once every email is stored within a relational database structure, organisations finally have rapid access to this key corporate asset. By integrating the email data store with core applications, from CRM and finance to HR and ERP, and providing remote access via mobile devices, organisations have a complete enterprise information view for the first time." ST