Subject: News for and about the Internet2 community
I2-NEWS: Linux Users on Internet2 Networks Enjoy the Benefits of Logistical Networking
- From: Greg Wood <>
- Subject: I2-NEWS: Linux Users on Internet2 Networks Enjoy the Benefits of Logistical Networking
- Date: Mon, 17 Nov 2003 11:03:42 -0500
Contact: Micah Beck,
LINUX USERS ON INTERNET2 NETWORKS ENJOY THE BENEFITS OF LOGISTICAL NETWORKING
PHOENIX, AZ – November 17, 2003 – Linux users on Internet2 networks are enjoying the benefits of a new approach to high performance content distribution, called Logistical Networking, which will be on display this week in the Internet2 booth at SC2003 in Phoenix, AZ. Developed by a research team from the Logistical Computing and Internetworking (LoCI) Laboratory at the University of Tennessee, Logistical Networking (LN) combines state-of-the-art data transfer technology with storage resources provisioned throughout the network to create a convenient and powerful new paradigm for distributed data management.
To test this technology, the LoCI team has used the 22 terabyte (TB) testbed of LN "depots" deployed across Internet2 networks to create an ad hoc content distribution network for distributing 650 megabyte (MB) CD-images (called "ISO's") of Linux and FreeBSD software. Users are now employing the Logistical Runtime System (LoRS) tools to download these ISO's at speeds of 30 to 80 megabits per second (Mbps)—roughly tens times faster than from traditional HTTP or FTP mirror sites. Downloads for gigabit Ethernet connected users can exceed 150 Mbps.
"We think that this kind of network storage infrastructure paves the way for a new era in content distribution," said Dr. Micah Beck, Co-Director of LoCI Laboratory and the chair of Internet2's Network Storage Special Interest Group. "For example, although using multiple copies and multiple TCP streams to increase transfer speed is similar to what some peer-to-peer systems do, with our fixed but shared infrastructure of well connected nodes, you can scale up the size of the content without sacrificing performance."
What makes this unique combination of flexibility and performance possible is an XML encoded metadata file called an exNode. A content publisher who uploads a file to the testbed of LN depots, which is called the Logistical Backbone (L-Bone), receives an exNode containing metadata that maps the segments of the file's content to L-Bone storage allocations, which are time-limited to make them more shareable. A single exNode can represent content that has been fragmented across multiple depots to accommodate large sizes, replicated to ensure fault tolerance, or both replicated and geographically dispersed to improve accessibility and performance. A single exNode used to distribute a Linux ISO represents eight copies of the ISO's content, which has been broken up into 8-20MB chunks and spread across L-Bone depots nationwide.
Publishing content that has been stored in the L-Bone is as simple as sharing the exNode that represents it. Since exNodes are text files, they can be published via HTTP, sent as e-mail attachments, or passed on a floppy disk. When they are posted on a Web site, as with the exNodes for Linux ISOs, the result is called an exNode Distribution Network, or XDN. To access the content in an XDN, users simply retrieve the relevant exNode from the site, and then use them with the LoRS tools to download the content. The LoRS tools are freely available and easy to set up, have a convenient GUI, and run on Microsoft Windows, Apple OS X, and all common variants of UNIX and Linux. The LoRS tools make fast, mulitsource/multistream downloads routine for Internet2 users when the content is suitably replicated, as in the Linux XDN,
"As compared to some of the other things we're doing with Logistical Networking, like managing the data sets from supercomputer simulations or accelerating remote browsing of massive image databases, putting up an XDN is a pretty simple application that anyone on Internet2 can do," explains Dr. Beck. "And prototype applications like IBPvo show that there are some easy variations on XDN that can automate different parts of the process."
IBPvo is an Internet2-enabled personal video recorder (PVR) service created to show the flexibility of LN technology. Like TiVo or ReplayTV, IBPvo can be scheduled in advance (via a Web interface) to record standard television programs. But unlike any other PVRs, the recorded video is automatically uploaded and replicated in the L-Bone, and the user receives an exNode for the video content in e-mail. Once the user has the exNode, the LoRS tools can be used for a high performance, multisource/multistream download.
Members of the research and education community who want to set up their own XDN now have a substantial infrastructure to work with. Worldwide there are over 23 TB of storage space available on more than 300 depots in 20 countries. A portion of the L-Bone is the National Logistical Networking Testbed, funded by a grant from the National Science Foundation (NSF) and a donation from Yotta Yotta, a leading Canadian storage company. Other depots use the resources of PlanetLab, a collaborative research infrastructure located at universities throughout the U.S. and other countries. PlanetLab, which was seeded by funding from Intel, has now received additional funding from the NSF and a contribution of 30 more nodes from research and technology collaborator Hewlett Packard. As with the Internet itself, some L-Bone resources are provisioned by the nation's leading colleges and universities in order to support research and education applications. The Department of Energy (DOE) also provisions some private Logistical Networking resources in support of science projects based at the National Laboratories.
For more information:
Logistical Networking -- http://loci.cs.utk.edu
Linux/FreeBSD XDN -- http://loci.cs.utk.edu/lors
IBPvo -- http://loci.cs.utk.edu/ibp/files/IBPvo_final1.pdf
The Logistical Computing and Internetworking (LoCI) Laboratory of the Computer Science Department of the University of Tennessee is devoted to research on information logistics for distributed computer systems and networks. Information logistics studies architectures and strategies for the flexible coscheduling of the physical resources that underpin computer systems — storage, computation, and data transmission. Formed in 2001 with support from UT's Center for Information Technology Research, and led by Assoc. Profs. Micah Beck and James S. Plank of UT's Computer Science Dept., LoCI Lab has pioneered in the application of the Internet model of scalable resource sharing to physical storage, creating a unified communication infrastructure that can support advanced applications not adequately served by the conventional model of Internetworking. Its work is funded by grants from the NSF and the DOE.
For list utilities, archives, subscribe, unsubscribe, etc. please visit the
ListProc web interface at
- I2-NEWS: Linux Users on Internet2 Networks Enjoy the Benefits of Logistical Networking, Greg Wood, 11/17/2003
Archive powered by MHonArc 2.6.16.