A Critical Analysis on the Short Term Roles of ATM in a Campus Networking Environment

John Leong
Technical Director, Computing Services Carnegie Mellon University
john.leong@cmu.edu

August, 1995

This paper takes a critical look at ATM from the points of view of an infrastructure provider charged with upgrading the campus production network infrastructure. For this task, parameters such as product stability, availability, cost effectiveness and timing is every bit as important as, and arguably more so than the raw capabilities of the competiting technologies.

Introduction and Summary

"Life is what happens while you're busy making other plans" - John Lennon

Carnegie Mellon University has one of the largest and most heavily used heterogenous local area network complexes in the world. We have pioneered the concept of inverted and collapsed backbone in the late 80's and have made a number of significant contributions to the industry particularly in the area of network management. In early 1991, in order to provide more bandwidth to our community and to handle the potential increase in multimedia applications, we began exploring options for a comprehensive upgrade to our network infrastructure.

Our initial focus was with ATM (Asynchronous Transfer Mode). We have followed the evolution of UNI (User Network Interface) [1], the debates of rate versus credit based flow control schemes [2] and the development of LAN emulation [3] etc. with great interest. We have also acquired a small ATM switch 2 years ago for experimentation with the view of deploying it as a LAN backbone. While ATM has great potential and technically interesting, as an operating rather than a research unit, we have to factor in mundane but very important parameters such as install base, migration strategy, product availability, cost-effectiveness, as well as the often overlooked but considerable effort required to make changes to protocol and application software.

Our conclusion is to back away from that ATM for now in favour of fast Ethernet. We will continue to apply ATM technology for niche areas and will revisit it again particularly when the OC-12 class of links are required for carrying aggregated traffic. The following describes our observations leading to this change. We will examine the communication requirements of multimedia applications and look at the various potential markets for ATM. We will conclude with a brief description of our network architecture evolution leading to the plan for our next generation, fast Ethernet and router based network which we are in the process of deploying.

Multimedia and Communciations

While the demand for network bandwidth has been steadily increasing over the years, the next big leap forward in demand is expected to come from multimedia applications.

While multi-media means different thing to different people, it most often implies the integration of audio and video into traditional text only objects. While it is clear that audio and video elements will add to the demand on communication networks, the question is how much? Will it require dedicated 100Mbps or OC-3 pipes? Will it need isochronous support as provided by CBR (Constant Bit Rate) class of traffic of ATM [4]?

Multi-media material can be roughly divided into 2 major categories - pre-generated and real time material. Since video is the medium people most often associated with multimedia and because it is also arguably the most demanding medium, we will use that as the benchmark for the following analysis.

Pre-Generated Multimedia Material

The bulk of multi-media applications use pre-generated material, such as video segments. Such pre-generated material is typically stored in a compressed format. Given that video element can often tolerate some loss without significantly impacting the information content, higher efficiency but a potentially lossy compression algorithm such as MPEG2 [5] can be used. Indeed, experimentation has shown that reasonable VHS quality video can be compressed for carriage over an ADSL (Asymmetric Digital Subscriber Line) [6] link of 1.5Mbps. For this category of video material, in term of bandwidth, the ubiquitous 10Mbps Ethernet is more than adequate particulalry if we are dealing with switched or dedicated Ethernet.

Quality of service demand is another matter. In general, an application environment involving compressed video does not require isochronous communication as provided by CBR of ATM or any point-to-point, unshared circuit. The exception will be in the highly unlikely situation, at least for the data communications world, that the client station does not have the capability to carry out decompression, requiring the server to do all of the work and feed it with uncompressed bits.

Compressed video may or may not need bounded delay support depending on the nature of the application. If the video application is such that the video element is being displayed in real time from the server, then bounded delay is required for the compressed bit stream. With ATM this is provided by VBR-RT (Variable Bit Rate - Real Time). Providing guarranteed bounded delay service is not something Ethernet can do readily. However, progress is being made to provide similar service at higher level protocols, such as the experimental ST-II (Internet Stream Protocol Version 2) [7] and RSVP (Reservation Setup Protocol) [8] in the IP world. On the other hand, if the application is in the form of multimedia E-mail, then bounded delay is not required. It is really nothing more than a file transfer operation.

Finally, while applications for pre-generated video is starting to grow, the relatively large storage demand and the high cost of production and/or licence will likely to keep the growth rate relatively modest. Within the next few years, the multimedia element that is going to have a true impact in bandwidth is likely to be the less glamorous high resolution colour imageries used extensively on popular services such as the Web.

Real Time Multimedia

The other major application category is real time multimedia such as the video phone and video conferencing. In these cases, the video data are not pre-generated and any compression has to be done on the fly or not at all. Given that most of the more efficient compression algorithms are asymmetric in nature and that compression is often significantly more processor and time consuming than decompression, using them on the fly can lead to unacceptable latency. The potential solutions are brute force compression with special hardware, use a low efficiency compression algorithm, use a lossy algorithm, sacrifice resolution, do with a smaller frame size, reduce the number of frames per second or finally, assume that we will find the bandwidth to transmit the uncompressed data without making any compromises.

It is interesting to note that virtually all of the video conferencing equipment makers are targeting the wide area network market where affordable high bandwidth is on the order of megabits per second and is not likely until phone companies have made a major paradigm shift away from deriving most of their revenue from voice service. Even though bandwidth is much less of a problem in a LAN environment, it is arguable as to whether a video conferencing application is cost effective in a local context. This is more of a social rather than a technical problem, that is the problem of rendezvous. Any user of a phone system is likely to notice the increasing failure in being able to reach the called party. Most of phone calls ended up being picked up by an answering machine or voice mail. Hence, while there are phones in virtually every office, alternative modes of communications that do not require real time rendezvous such as E-mail and Fax , have been gaining popularity by leaps and bounds. Video mail is likely to be just as popular. And, as mentioned earlier, video mail is really just a file transfer operation and does not demand real time communication support.

Video conferencing requires the real time rendezvous of multiple parties. The chance of doing that sucessfully, without pre-scheduling, is even less likely than video phone. Hence in spite of their conceptual appeal, it will be hard to justify the major expenditure required to provide explicitly for such a service in a local context. On the other hand, wide area video conferencing can usually be justified by saving on travel expense as well as time. In this environment, equipment makers would be foolish not to consider bandwidth or, the lack there of, as an important and serious design constraint for some time to come.

The ATM Market Segments

We will now change gear and look at the 3 major potential market segments for ATM. They are: network for the desk top, backbone network, and WAN (Wide Area Network). Since we are not a multi-site campus and our Internet connectivity is provided by commercial network service providers, our main interests in ATM are for desktop connectivity and as the backbone for our LAN complex.

Network for the Desktop

ATM's advantages over traditional LANs are a high speed family of access links, high aggregate bandwith, and the ability to reasonably guarantee Quality of Service (QoS).

In terms of access link for the desktop, as mentioned in the multimedia section, with compression, even the demanding video applications can live quite happily with relatively low bandwith. The difference in bandwidth between OC-3 and fast Ethernet is marginal. It is beyond OC-3 that ATM becomes interesting. However, desktop applications requiring OC-12 links today are definitely very special cases . OC-12 has a good potential role as an aggregating link within the next few years.

Most data communications protocol families do not assume the support of QoS in the underlying network. In order to take advantage of that, significant modifications to the higher level protocol stacks, new API and/or applications software will have to be provided. This is a major problem that has often been overlooked. Indeed, even though priority reservation schemes exist for today's LAN technologies, such as token ring [9] and FDDI [10], they are rarely used. Again, for applications requiring QoS, developments, such as ST-II, weighted fair queueing and RSVP, are being made to provide such services at existing higher layer protocols without requiring increase in capabilities by the link layer.

A related problem is that the installed base of software assumes a connectionless network with multicast capability, as in the case of traditional LANs. ATM presents a very different model. This also calls for significant changes in the large install base of network layer protocols which most network users would prefer not to have to deal with. ATM forum acknowledged this problem and has spent considerable effort coming up with LAN emulation. This is somewhat of an irony as it serve to strengthen the traditional LAN paradigm by providing it with the support of yet another transport network - that of a stripped down ATM.

ATM's main competiton for the desk top is the ubiquitous Ethernet and the rapidly emerging fast Ethernet [11]. Except for the PC, almost all desktop computers today come equipped with Ethernet and quite a lot of them have the Ethernet chip set built right onto the mother board for cost saving and potential performance advantages. This huge installed base gives the backward compatible fast Ethernet a big advantage in the battle for the desk top. By simply replacing the 10 Mbps Ethernet chip set with the 100Mbps version, a computer vendor have a machine that can serve both the current Ethernet as well as the new fast Ethernet markets. ATM will most likely be made available as an option. The history of FDDI suggests that optional add on network technology typically does not thrive for the desktop market in the presence of a "free" and adequate alternative.

Based on the above obervations, we belive that ATM will remain a niche player in the desktop market place. We belive existing Ethernet will continue to have an important presence for the coming few years. At the next level, switched Ethernet will be applicable for applications with substained bandwidth demand of less than 10Mbps. Fast Ethernet will be appropriate for applications requiring higher but bursty bandwidth. Finally, switched fast Ethernet should satisfy most of the high bandwidth applications till the end of the century.

ATM in Wide Area Networking

ATM comes from the telecommunication world. It is designated as the integrating switching engine for all voice, data and video traffic of the B-ISDN world. This vision is not likely to be fulfilled for some time to come. The telecommmunications world moves relatively slowly due to the massive install base and long tax write off period for the equipment. The slow pace of standards setting by the CCITT does not help. Furthermore, designing commercially viable thousands by thousands ports ATM switch is considerably more complex than the small switches available today. Technical problems such as flow control over long haul networks is still challenging even though some settlement has been reached for now in the rate versus credit base flow control schemes. As a result of the above, the probability of seeing a commercial ATM based switch replacing Central Office switches such as the AT&T 5ESS or the Northern Telecom's DMS100 before the turn of the century is almost nill.

For the next decade, wide area ATM network will appear as a niche network operating in parallel to its massive voice brethren, much like X.25, Frame Relay and the Internet today.

In the data specific wide area applications such as the vBNS, the new NSF (National Science Foundation) funded research network, ATM is mostly used as a wide area trunking network making use only of PVCs (Permanent Virtual Circuits) for the connection of routers. In such application, ATM is arguably redundant. One could just as easily, and more efficiently, connect the routers directly with the underlying Sonet links.

Finally, given the high cost of DS-3 links today, unless there is drastic reduction in the pricing structure, wide area OC-3 links will remain unaffordable to all but the very well heeled. Note that prices for high bandwidth network services would unlikely to drop substantially until the telecommunications industry has made major paradigm shift to wean itself from its heavy dependency on voice revenue.

ATM as Backbone for LANs

Switched network, subject to design, is preferred to the traditional LAN as a backbone because it is potentially scalable. Whereas LAN has a pre-defined maximum aggregate bandwidth, the aggregate bandwidth of a switched network can be increased by providing more cross points. This is, of course, applicable only to switches that are designed to be scalable. Another valuable attribute for ATM is its family of high bandwidth access links.

ATM was particularly attractive to us two years ago when fast Ethernet was not yet available. Our approach was to connect our routers and bridges with an ATM switch. For that purpose, the then still unresolved UNI interface for SVC (Switched Virtual Circuit) was not an issue because all we needed was the much simpler PVC (Permanent Virtual Circuit). Given the relatively small number of routers and bridges we have to deal with, static configuration of PVCs will serve our purpose. The main problem we encountered was the slow adoptation of ATM by router and bridge vendors. We considered side stepping the router ATM interface availability problem by using workstations running routing software such as GateD, but the performance was not up to par. While the progress of the ATM forum slowed down as the size of membership increased, fast Ethernet, which was not a player 2 years ago, has caught up. Fast Ethernet switch is simpler, suffers fewer compatiblity problems and is more cost effective than ATM switches. Note that the potential QoS advantage for ATM as a backbone will not really be applicable if the Ethernet dominated desktop network environment does not understand and cannot make use of this feature.

Network Evolution at Carnegie Mellon

When we first started building the network complex back in the early 80's, there was really no architecture to speak of. Ethernet and, in particular, fibre Ethernet was barely working. The backbone topology was roughly a bus as suggested by the Ethernet specification. Subnets were connected to bridges and home grown routers in an ad hoc manner over fibre cables.

We soon find this to be an unmanageable mess. We then partnered with a start up company called SynOptics (now Bay Networks), arguably the founder of the hub business, to reconfigure the network into a rooted tree. We also created the concept of the inverted backbone by bringing the subnets back to a central location to be connected together rather than having the backbone reachs out to connect the subnets. The motivation of that approach is reduce the number of times we have to send technicians out to the field by having all the connecting routers in a central location as well as having an instance of every subnet at that location. This makes for considerable easier network management - particularly when remote management tools such as SNMP (Simple Network Managment Protocol) had yet to be developed.

The next major evolution was the creation of the "collapsed" backbone in the late 80's when it occured to us that the backbone was really the connected bus inside of the Ethernet hub at the root of the tree and that most bus, being parallel device, could easily do significantly better than 10Mbps. The first incarnation of the collapsed backbone was a Cisco AGS+ offering a maximum theorectical bandwidth of 530Mbps.

In the early 90's, we explored an upgrade of the "collapsed" backbone with an ATM switch using PVC to provide higher speed links to the routers at the next level of the tree. Failure to get ATM interfaces for the routers and poor performance of the alternative workstation based router approach postponed the deployment of this plan.

In the mid-90's, we decided to revamp the whole campus network infrastructure from the desktop up [see diagram]. Buildings will still be equipped with standard Ethernet hubs. Ethernet switches and fast Ethernet hubs will be deployed in areas requiring higher bandwidth support. All the hubs in the wiring closets of a building will be connected to a fast Ethernet switch located in the main wiring closet or the Building Entrance Facility (BEF). 100Base-Fx links will connect buildings back to the routers in the central location. The routers there will be connected together by the root of the tree which will be another fast Ethernet switch. A parallel ATM subnet will be deployed to support applications requiring ATM features and allow us to keep tabs on the development of this promising technology. When the bandwidth demand for interbuilding connection exceeds 100 Mbps or when the applications requiring ATM features become significant, we will reconsider ATM for the leading role. Note that this migration plan is evolutionary in that it does not require us to radically change the network architecture or topology. It also allows us to preserve our installed base of equipment until they are no longer needed.

Last Words

Finally, to illustrate that technological excellence is not necessary the main driving parameter when it comes to service provisioning, the following is an interesting problem we are confronting.

Back in the 80's, with IBM's assistance, we have wired up the whole campus with IBM's high performance STP (Shielded Twisted Pairs) based cabling system. Over 12,000 outlets no less. For high speed communications, STP is superior to UTP (Unshielded Twisted Pairs) in numerous aspects. Hence most 100 Mbps over copper standards such as CDDI, 100BaseTx etc. were established first with STP, followed by Category 5 UTP and finally for Category 3 cables. When we were asked whether our cable plant was "good" enough to handle 100Mbps class of communications, our answer used to be "sure". As the fast Ethernet products start coming into the market place, we are having some second thoughts - the cable plant may have the problem of being "too good". Given that there are much more Category 5 and Category 3 cables out there, almost all of the hub vendors are engineering their products for the lower quality but much more popular UTP cable plants. In order for us to use those products with the STP cabling system, we have to purchase additional adaptors (Baluns). Native STP hubs may yet become available, but due to the lower market volume, they will likely to command a premium. The bottom line is that we ended up having to pay more for the same service because we have a better cable plant! Oh well, live and learn.

References

[1]User Network Interface Specification, Version 3.0 Prentice-Hall, 1993

[2a] Bonomi F., Fendick K., "The Rate Base Flow Contromework for the Available Bit Rate ATM Services", IEEE Network Mach/April, 1995

[2b] Kung H., Morris R., "Credit Based Flow Control for ATM Networks", IEEE Network Mach/April, 1995

[2c] Ramakrishnan K.K., Newman P., "Integration of Rate and Credit Schemes for ATM Flow Control", IEEE Network Mach/April, 1995

[3] "LAN Emulation Over ATM : Draft Specification - Rev 5", ATM Forum Technical Committee, LAN Emulation Sub-working Group.

[4] Siu K.Y., Jain R., "A brief Overview of ATM : Protocol Layers, LAN Emulation, and Traffic Management", ACM Comp Comm Review, April 1995

[5] Ang P.H., Ruetz P.A., Auld D., "Video compression makes big gains", IEEE Spectrum, October 1991.

[6] Waring D.L., "The Asymmetrical Digital Subscriber Line (ADSL): a new transport technology for delivering wideband capabilities to the residence" IEEE GLOBECOM '91.

[7] Zhang L., Deering S., Estrin D., Shenker S., Zappala D., "RSVP: A New Resorice ReSerVation Protocol", IEEE Network Magazine, September 1993

[8] Topolcoc C., "Experimental Internet Stream Protocol: Version 2 (ST-II)", Internet RFC 1190.

[9] IEEE Standard 802.5, Token Ring

[10] ANSI Standard X3T9.5 Fibre Distributed Data Interface (FDDI)

[11] IEEE Standard 802.3u, 100BASE-T Fast Ethernet (Draft)