The challenge is how to maintain or, better yet, improve performance while reducing the total cost of ownership. Do you have enough knowledge about your network to efficiently manage its day-to-day operations and precisely plan and execute major network projects such as server consolidations or platform migrations?

It's impossible to efficiently manage what you don't fully understand, and trying to do so will only be frustrating for you and costly for the organization. But you no longer need to depend on guesswork to manage your e-mail infrastructure. This paper describes how an enterprise-level messaging traffic management solution can provide the detailed quantitative traffic information you need to tackle even your most difficult messaging management challenges.

The critical e-mail management issues

Corporate e-mail usage is growing at an astounding rate, with e-mail now the primary medium for all business communications. In fact, IDC has predicted the number of worldwide e-mail mailboxes will jump 138 percent between 2000 and 2005 -- from 505 million to 1.2 billion. Not only are there more mailboxes, but e-mail volume per user is also increasing. IDC predicts that the number of person-to-person e-mails sent on an average day will exceed 60 billion worldwide by 2006. Add to this the growth in use of inbound advertising spam, HTML/graphic-based content, large distribution lists, and opt-in e-newsletters, and it's not surprising that messaging infrastructures are drowning in a flood of e-mail.

When e-mail isn't flowing efficiently, everyone in the company, as well as customers and suppliers, are immediately impacted. Your IT organization is tasked with managing this mission-critical application while trying to keep costs under control, or even reduce them. The challenge is how to do it all: reducing the total cost of ownership (TCO) while improving quality of service (QoS) and system reliability.

Let's look at the main factors impacting messaging TCO and QoS today.

Rising IT costs

Users are sending more documents and attachments than ever before and making greater use of distribution lists. In addition, corporations are facing extensive new SEC regulations for e-mail retention. As a result, storage and archiving requirements are growing at an alarming rate, WAN and LAN expenses are mushrooming, and mail servers are becoming increasingly overloaded. A typical response to these increasing demands has been to throw more powerful servers, disk subsystems, and network bandwidth at the problem. Without adequate insight into what's causing these increases, there's no real alternative to spending more to maintain performance -- whether in new equipment or additional staff.

Service level demands

Business units are demanding higher service levels -- lower delivery latency, higher uptimes, faster access when working remotely, etc. Users often complain that their e-mail is "slow," but without the ability to monitor and analyze e-mail delivery service levels systematically across the network, validating such claims and identifying whether or not e-mail is being delivered efficiently -- and why -- is a daunting task.

Misuse and abuse

Wasteful or inappropriate user behavior can have serious cost implications. Indiscriminate mass mailings, frequent sending of large messages, and excessive mailing to external domains for personal purposes all consume unnecessary infrastructure resources and eventually lead to higher costs as the infrastructure must be over-sized to support this excess traffic.

Unclear ROI from strategic network projects

Many organizations undertake major network projects in an effort to reduce costs and improve performance -- projects such as server consolidations, platform migrations to the latest release or to a "less costly" platform, or topology redesigns. But without the information necessary to plan and execute these projects, you're "flying blind" in many cases and must depend on guesswork, ending up with topologies that aren't well suited to actual usage patterns, unbalanced loads, and poor performance.

The dynamics and challenges of constant network change

Although your IT organization tries to keep your messaging network stable, business needs dictate otherwise. Business acquisitions, mergers, divestitures, layoffs, and new business initiatives are just some of the business activities that put messaging networks in turmoil, forcing you to adapt to these changes by merging networks and domains, often at greater cost and lower performance.

Going beyond health and availability

Two classes of system management solutions

After your messaging infrastructure is in place, you need to keep it performing reliably, with a high quality of service and low TCO. Two classes of system management products are essential:

• Health and availability products --These products provide your administrators with tools to keep individual servers performing. They provide detailed information about the overall status of the selected server (or small group of servers) and focus on monitoring individual server characteristics such as CPU, RAM and disk availability, network bytes sent/received, tasks, and processes. The end-results focus on server availability and component problem identification. Examples of such products include BMC Patrol, Tivoli Analyzer, Candle IntelliWatch, and NetIQ AppManager.
• Efficiency and usage solutions -- These products provide your IT organization with solutions to manage and optimize e-mail traffic on an enterprise-wide basis. Focusing on the efficiency of the entire messaging network and how it's being used, they monitor, analyze, and report on e-mail traffic, usage, and performance. The end-results provide the critical message-level information needed to lower operational costs, increase service levels, and ensure major project success. Traffic management solutions, such as DYS CONTROL!, fall into this category.

• How much mail is being sent across the entire enterprise?
• How busy are all my servers, networks, and disk systems?
• Which locations have increasing mail loads?
• What are users doing? Where are they located? How much mail are they sending? To whom are they sending mail?
• Where are resources being stretched, constrained, or wasted?
• Are there any bottlenecks being created by improperly balanced servers and bandwidth?
• How long and how many hops is it taking messages to get from origin to destination?
• What end-to-end delivery service levels can the infrastructure sustain?
• Are there any potential security breaches and abuses (e.g., excessive personal use, e-mail to/from competitors)?

• Message size -- You've probably already taken steps to educate users and establish policies to discourage or even prevent large messages. However, DYS has found that medium-size messages (1 to 5MB) typically pose a much bigger problem. For example, using an efficiency and usage product, the IT group at an international bank discovered that one employee was sending out a 1.5MB corporate newsletter to 15,000 employees on a weekly basis. Upon further investigation, they discovered that the newsletter contained a 1.4MB corporate logo. By substituting a smaller format logo, the newsletter went from 1MB to 100KB, resulting in over 1 Terabyte reduction per year in storage alone and saving the company hundreds of thousands of dollars.
• Distribution lists -- Communicating with large groups of employees is becoming more common in globally distributed organizations. This form of collaboration is easy to do but costly, especially when most of the recipients on large distribution lists don't even read the mail sent to them. And there are better ways for groups to communicate than using large distribution lists. With clear, objective metrics that illustrate the impact of rampant distribution list use, you can educate users and encourage them to use other collaborative solutions and send messages to large groups only when absolutely necessary.
• Mail broadcasts -- Many organizations have applications that periodically broadcast messages. Based on its customers' experience, DYS has found that the usefulness of these broadcasts doesn't justify the costs and impact on system performance. After you've identified these high-volume situations, discussions with the application owners about alternative communication methods (intranet postings, for example) can dramatically reduce the cost of these broadcasts.
• Binge mailers -- Individual users can have a significant impact on e-mail resources. With just a few mouse clicks, a single user can easily consume costly resources by sending one large attachment to hundreds of recipients. Education, alternative collaboration methods, and policy backed by continual monitoring to ensure compliance with documented usage policies can dramatically lower overall costs.
• Misuse/abuse -- Excessive use of the corporate messaging infrastructure for personal purposes drives up costs. Personal e-mail users may be binge users sending attachments (vacation photos, for example) to dozens of friends. Even worse, several DYS customers have spotted employees running a business after hours using the company's messaging system or even sending unauthorized e-mails to competitors. Once these employees are identified, the users can be educated and, if necessary, disciplined.

• Server loads -- Comparing enterprise-wide server metrics will identify over- or under-utilized servers. With this information, your administrators can more accurately balance loads across the other servers simply by modifying the routing plans or by reassigning users to under-utilized servers. Measuring this information over time will further enable you to identify growth trends and put resources in place only when needed.
• Bandwidth utilization -- The situation is similar with bandwidth. Metrics on daily network traffic enable you to identify trends in bandwidth utilization and to gauge the need for bandwidth upgrades. In many cases, such measurements can help you avoid unnecessary bandwidth upgrades altogether, by simply modifying routing schemes to redirect more traffic onto under-utilized network links.
• Storage consumption -- Measuring daily, weekly, or monthly storage consumption over time helps you assess the need to acquire additional disk storage capacity. Through measurement, you can spot imbalances and reassign users to other mail servers, as well as identify wasteful usage behavior. Both of these can help defer investments in new storage, resulting in considerable savings. With storage TCO estimates of somewhere between $300/year/GB (McKinsey/Merrill Lynch) and $2,000/year/GB (Gartner), you can potentially save hundreds of thousands of dollars per year in storage costs if you're able to reduce storage consumption significantly.

• Uncontrolled and unplanned evolution from small pilot implementations -- In many older cities, tens of thousands of motor vehicles per day battle to squeeze through what is essentially an inadequate system of ancient cow-paths from the city's early roots, gradually paved over and slightly widened over years of population (and traffic) growth. So is the case with many e-mail networks. Routing paths that were configured during early pilot phases or when the company was much smaller and e-mail usage was minimal are no longer suited to the overall mailing patterns of the corporation today. Such messaging networks tend to be inefficient, have lots of bottlenecks, and eventually tend to require extreme measures such as indiscriminate hardware and bandwidth upgrades to address emergency performance problems as they arise.
• Mergers, acquisitions and migrations -- Many of today's corporate e-mail networks have evolved as a patchwork of disparate network "pockets" quickly patched together as a result of mergers, acquisitions, and migrations. Like some of the world's largest cities that have gradually "swallowed up" isolated towns and villages, such patchwork e-mail networks tend to have highly inefficient routing paths, with mail routing through far too many hops and in some cases, traveling across poor bandwidth links unnecessarily. These networks tend to provide low quality of service at an unnecessarily high cost.

• It can identify the most heavily loaded servers and network links and then provide a detailed analysis of the traffic that's routing through these infrastructure components. By identifying the sources and destinations of this traffic, it identifies traffic that could (or should!) route through alternative paths. A common example, due to the way routing was historically set up, is mail sent from one site in Asia to another in Asia that routes through corporate headquarters in the United States, unnecessarily consuming precious trans-Pacific bandwidth twice for each message.
• It can analyze site-to-site and server-to-server traffic volumes. Understanding which site is sending how much mail to which site helps determine infrastructure and routing requirements. Specifically, the right solution will show you where to place servers and what paths mail should take to make optimal use of the infrastructure.
• It can analyze server-to-server and site-to-site service levels for mail delivery to identify inefficient mail routes. A good efficiency and usage product will analyze the routing paths between these sites and identify the bottlenecks.

• Load balancing -- How should users be distributed on the new servers? Users are not all alike. Some users send and receive massive amounts of e-mail while others hardly send or receive any. These trends often run along departmental lines. Marketing and HR departments, for example, tend to send larger volumes than finance. If you migrate users to a smaller number of servers without a load balancing plan, loads on the consolidated servers will most likely end up unbalanced. Some of the new servers will bear a much higher routing and storage load than others. As a result, the overloaded servers will require hardware upgrades, maintenance, storage upgrades, etc. much more frequently than others. With an efficiency and usage solution, you can analyze comparative usage and load trends in the existing environment before the consolidation and use this information to plan the best way to distribute users onto the consolidated servers in a way that will balance the routing and storage loads.
• Planning hardware -- What aggregate routing and storage loads will the new servers have to support? How much mail will the new servers route in a typical day? What will the peak hours be? How many messages and what volume of mail will typically be routed during peak hours? How much messaging storage will be impacted on a typical day? Answering these questions is crucial in ensuring that you've planned adequate processing power and sufficient storage on the consolidated servers. Failure to address them in the planning phase can lead to under- or over-sizing of the hardware. Using an efficiency and usage solution, you can accurately project these aggregate loads in advance, prior to the consolidation, by calculating the answers to these questions based on the traffic and usage trends in the existing environment.
• Planning bandwidth -- What will be the impact of the consolidation on WAN traffic volumes? When mail servers from remote sites are consolidated into centralized data centers, Wide Area Network (WAN) traffic between the remote and the central sites increases -- usually substantially. The reason is simple -- every time a user at the remote site sends a message, that message must travel over the WAN from the user's client machine at the remote site to the server at the data center. Every time a user receives a message, that message must travel over the WAN from the server at the data center to the user's client machine at the remote site. Every message sent from one user to another user at the same site must travel twice over the network -- once to the server and once back. Such intra-site messages, which often constitute the majority of mail at any particular site, aren't carried on the WAN prior to the consolidation. They're the source of the increase in network traffic. It's important to understand what the increase in bandwidth needs will be for two reasons: a) ensuring that bandwidth costs are taken into consideration when deciding to move servers out of any particular site, and (b) ensuring that sufficient bandwidth is planned and procured for the post-consolidation environment. By using an efficiency and usage solution to carefully analyze mail usage trends into and out of any particular site (or sites), you can project, with greater accuracy, what the increased bandwidth needs will be and help ensure that adequate bandwidth is in place.

• Definition -- Defining messaging service levels in terms of metrics for delivery-times of e-mail messages will enable you to contract for different delivery times for small and large messages and for different delivery priorities. You can even allow for variances in service levels based on geography and quality of network links.
• Measurement -- Objectively measuring how well service levels are being met on an ongoing basis is critical to managing SLA performance. End-to-end service levels across the network, sorted by segment, geography, server, message sizes, etc. can identify chronic weak links and facilitate troubleshooting of the source of the problems.
• Monitoring -- Monitoring actual message performance across the network will identify bottlenecks and broken links and help ensure service levels are met in day-to-day operations.

• Driving down costs -- Regardless of whether or not the organization actually implements charge-back to departments or not, associating a cost to e-mail usage provides a uniform standard by which all departments can be measured. After departments are made aware of the costs that their users incur and are shown a comparison with other departments' costs, they can be given explicit incentives -- financial or other -- to reduce consumption. This reduction can be in the form of traffic reduction initiatives as described earlier in this paper. All in all, senior management gains clearer and tighter control of messaging costs at their source.
• Fair billing increases customer satisfaction -- In addition to driving costs down, organizations that do charge back messaging costs will find that a usage-based charge will be perceived as equitable and fair. No longer will departments or business units feel that they're being charged unjustly or subsidizing other groups.

• Develop a usage-based cost allocation program.
• Analyze weekly or monthly traffic and determine the aggregate impact on the infrastructure, broken down by user or department.
• Produce both aggregate and detailed billing reports by department.

• Is scalable enough to handle volumes of data across the entire organization.
• Has a network perspective, not a per-server perspective.
• Is able to identify and measure the various steps in the propagation dynamics of messages throughout the network.
• Has the flexibility to be easily tailored to your organization's particular network and mail-routing topologies.
• Can provide targeted reports to different audiences -- whether they be IT, executive management, or at the business unit level.
• Is easy to deploy and imposes minimal incremental overhead to your network.