News for and about the Internet2 community

[Laura Wolf <>]

June 4, 2007

GLORIAD: USA-RUSSIA LIGHTPATH ENABLES FAST DATA TRANSFER OF TERABYTE-SIZED 
SCIENTIFIC DATASETS
 
Knoxville/Oak Ridge, Tenn. -- Scientists from the National Center for Data 
Mining (NCDM) at the University of Illinois at Chicago and the Geophysical 
Center at the Space Research Institute, Russian Academy of Sciences, Moscow, 
demonstrated a new method for distributing extremely large volumes of 
scientific information across the world. They successfully moved 1.4 
TeraBytes (TB) of data in about 4.5 hours over a 1 Gbps lightpath between 
Chicago and Moscow as part of the Teraflow Network initiative. This event, 
which represents the highest performance information transfer ever recorded 
between these two countries, was made possible by a unique international 
organizational partnership.
 
Although the amount of science information is growing rapidly, the ability to 
move it on the regular Internet is still very limited. NCDM partnered with 
Russia’s Geophysical Center to demonstrate a new capability for moving 
science data by transferring the Sloan Digital Sky Survey (SDSS) dataset 
between their two sites, using a specialized international communications 
facility.
 
This new capability uses two integrated innovations. One requires placing 
information directly on lightwaves while avoiding the slower services that 
are used by the traditional Internet. The other uses specialized 
communications technologies (new network protocols) for high performance 
streaming to avoid the limitations of standard Internet communications.  
 
Using NCDM’s open-source, high-performance network transport protocol UDT 
(UDP-based Data Transfer) on the Teraflow Network, researchers were able to 
quickly transfer the SDSS astronomy catalog data, between Chicago and Moscow. 
The 2.5 TB catalog is compressed to 1.4 TB, split into 60 files and, is 
distributed to astronomers around the world from the NCDM in Chicago.  Using 
UDT, the 1.4 TB was transferred over a 1 Gbps lightpath and then decompressed 
in Moscow to its original size.  It now resides on a local server 
<www.skyserver.ru> in Moscow.
 
This data transfer had a sustained rate of 711 Mbps  and a peak rate of 844 
Mbps, and took about 4.5 hours  to complete.  This is about the speed that 
the data could be moved across the city of Chicago over a 1Gbps network, 
which graphically illustrates how barriers of distance are being eliminated 
by the new communications infrastructures and technologies. These techniques 
are   required for research and experimentation for many science disciplines, 
and in the future it may also be used for many types of data intensive 
commercial applications.
 
This accomplishment was made possible through a unique partnership among 
organizations in eleven countries that have created international advanced 
communication facilities at locations literally around the world.  GLORIAD, 
the Global Ring Network for Advanced Applications Development, is a 
consortium of several countries, notably the USA, Russia, China, Korea, 
Canada, the Netherlands, and the Nordic countries (Denmark, Sweden, Norway, 
Finland and Iceland), that are contributing networking capabilities to build 
a global 10 Gbps optical network around the northern hemisphere of the globe 
in support of   advanced science and engineering. In the USA, GLORIAD is 
supported by the National Science Foundation’s International Research Network 
Connections (IRNC) program, which also funds a 10 Gbps path between Chicago 
and Amsterdam called TransLight/StarLight. GLORIAD has been provided with a 3 
Gbps path on TransLight/StarLight to allow a direct high-performance 
connection between the USA and Europe.
 
GLORIAD’s Russian partners recently installed a 10 Gbps path from Amsterdam 
to Moscow, provided by the Russian Research Center “Kurchatov Institute.” 
This allowed a 1 Gbps lightpath to be dedicated to the Teraflow Network, from 
Chicago (the StarLight facility), to Amsterdam (the NetherLight facility) and 
then on to Moscow (the MoscowLight facility). GLORIAD participants are part 
of a global initiative called the Global Lambda Integrated Facility (GLIF), 
which promotes the paradigm of lightpaths, or lambda networks, for 
data-intensive scientific research and applications.
 
This science demonstration was also supported by NCDM’s Teraflow Network, an 
international facility designed to develop innovative technologies to stream 
massive distributed datasets over high-performance networks, at 1 Gbps, 10 
Gbps and multiple 10 Gbps. The TeraFlow Network is being used as a 
next-generation platform, capable of supporting data-intensive applications, 
including many requiring information transfers that cannot be supported by 
traditional networks. The TeraFlow Network is developing techniques that will 
be required by future global applications.
 
“This is the latest in a string of demonstrations that proves that it is now 
practical for the working scientist to efficiently access terabyte size 
datasets from anywhere in the world. All it takes are today’s 
high-performance networks and new network protocols, such as UDT,” said 
Robert Grossman, NCDM director at the University of Illinois at Chicago. 
“With the technology now available, there is no reason for scientists not to 
have access to the latest data available in order to advance their research.”
 
“We look forward to using these new technologies to share and mine very large 
databases in global change, space weather and remote sensing studies,” said 
Mikhail Zhizhin, head of the Telematics Lab at the Geophysical Center in 
Moscow, “and to applying the technologies from the Teraflow Network to the 
larger GLORIAD infrastructure. In particular, the Research Group is working 
with the USA National Geophysical Data Center (NOAA) on the Space Physics 
Interactive Data Resource (SPIDR), and is working with Microsoft Research 
Cambridge on the Environmental Scenario Search Engine (ESSE). Additionally, 
there is strong demand to transmit real-time data streams and high-resolution 
images, which has not previously been possible. ”
 
”This is a significant achievement between USA and Russian scientists,” 
stated Alexey Soldatov, co-director of GLORIAD/Russia and director of the 
Institute of  Information Systems, Russian Research Center “Kurchatov 
Institute” (RRC “KI”), - "GLORIAD/Russia, based at RRC “KI”, provides support 
and development of the networking infrastructure for scientist and educators. 
In addition, our Research Center is one of the leaders of the nationwide 
Russian Data Intensive Grid program, that will use GLORIAD's advanced 
networking infrastructure to support data-intensive projects and frontier 
experiments in high-energy physics, nanotechnology, gravitational wave 
research, digital astronomy and molecular genomics."
 
“The ability to move multi-terabyte datasets internationally in a matter of 
hours, and ultimately minutes, has been based on the cooperation and efforts 
among many international teams and it builds a solid foundation for future 
international science projects,” said Natalia Bulashova, GLORIAD/USA 
co-principal investigator.
 
“Lessons learned on the Teraflow Network can be expanded to the entire 
GLORIAD community, and ultimately other GLIF international partners, said 
Greg Cole, GLORIAD/USA principal investigator. “No matter how fast we 
increase capacity and services on the GLORIAD network, the various science 
groups out there are moving faster. It’s a real challenge for us, but it’s a 
good challenge.”
 
The GLORIAD/USA team worked with teams from Russia (GLORIAD member - the 
Russian Research Center “Kurchatov Institute”; MoscowLight/RBNET/Russian 
Institute for Public Network (RIPN); Institute of Space Research at the 
Russian Academy of Sciences; the Geophysical Center at the Russian Academy of 
Sciences; and, the Visualization Laboratory at Moscow State University); the 
USA (National Center for Data Mining at the University of Illinois at 
Chicago; the International Center for Advanced Internet Research at 
Northwestern University; TransLight/StarLight; StarLight; Johns Hopkins 
University; and the GLORIAD UT-ORNL JICS); and, the Netherlands team (GLORIAD 
member - NetherLight/SURFnet; SARA; and the University of Amsterdam).
  
# # #
 
CONTACT:
 
Natalia Bulashova
Co-Principal Investigator, GLORIAD
UT-ORNL JICS

www.gloriad.org

---------------------------------- 

About GLORIAD <www.gloriad.org>
The GLORIAD (Global Ring Network for Advanced Application Development) 
advanced science internet network was launched in January 2004 by the USA, 
Russia and China, expanded its reach in 2005 – to Korea, Canada and The 
Netherlands – and in 2006 to the five Nordic countries of Denmark, Sweden, 
Finland, Norway and Iceland. GLORIAD provides an optical network ring 
encircling the northern hemisphere of the globe with individual network 
circuits providing up to 10 Gbps – promoting new opportunities for 
cooperation for scientists, educators and students. The GLORIAD project is 
supported by the Ministry of Science and Education of Russian Federation, the 
National Science Foundation of USA, the USA Research & Education (R&E) 
network National LambdaRail, the Chinese Academy of Science, the Ministry of 
Science and Technology of Korea, Canadian non-profit association CANARIE, the 
national R&E network of Netherlands SURFnet, the national R&E network of the 
Nordic Countries NORDUnet, as well as a number of other organizations 
representing countries which participate in the project. GLORIAD/USA is based 
at the University of Tennessee – Oak Ridge National Laboratory, Joint 
Institute for Computation Science. GLORIAD/Russia is based at the Russian 
Research Center “Kurchatov Institute”. GLORIAD/Netherlands is based in 
Amsterdam and managed by SURFnet.
 
About the Teraflow Network <www.teraflowtestbed.net>
The Teraflow Network (TFN), under the leadership of the National Center for 
Data Mining (NCDM) at the University of Illinois at Chicago and the 
International Center for Advanced Internet Research (iCAIR) at Northwestern 
University, is both: 1) a unique infrastructure enabling scientists to share 
large scientific datasets with their collaborators around the world; and, 2) 
an international experimental facility designed to create and test advanced 
networking technologies, new protocols and services, and integrated 
middleware to support distributed, high-performance data-intensive 
applications. The TFN supports 1 Gbps, 10 Gbps and multiple 10 Gbps data 
streams and connects computer clusters distributed over three continents. The 
TFN allows scientists to contribute terabyte datasets to repositories as 
shared resources - enabling colleagues around the world to use them as the 
basis for new discoveries. With the TFN, it is easy to explore, integrate and 
analyze datasets contributed by colleagues. The TFN is sponsored in part by 
the National Science Foundation of the USA.

About The Geophysical Center, Russian Academy of Sciences 
<www.gcras.ru/index_e.html>
The Geophysical Center of the Russian Academy of Sciences was founded in 1958 
by a special Decree of the USSR Council of Ministers with the mandate to 
archive and exchange data resulting since the first International Geophysical 
Year. Its primary goal is to conduct basic and applied research in the fields 
of geophysics and geoinformatics.
 
About The Russian Research Center “Kurchatov Institute” <http://www.kiae.ru>
The Russian Research Center “Kurchatov Institute”, Russia’s leading research 
and development institution in the field of nuclear energy, was founded in 
1943. The Kurchatov Institute is funded through the Ministry of Education and 
Science of the Russian Federation and through international cooperation and 
commercial projects. It is divided into 15 institutes and six scientific and 
technological divisions. The Institute conducts research on controlled 
thermonuclear fusion, plasma physics, solid state physics, superconductivity, 
molecular physics, physical and inorganic chemistry, chemical physics, 
chemistry, safety of new technologies, ecology and health, biology, genomics 
and bioinformatics, biotechnology, element basis of microelectronics, 
material science, nanotechnology, networking and information science. The RRC 
“KI” is the founder of Russia’s Internet and the Russian Research and 
Education network, and the Russian Institute for Public Networks (RIPN). 
GLORIAD/Russia is based at the RRC “KI” and the Russian Institute for Public 
Networks (RIPN).
 
About The Russian Institute for Public Networks (RIPN) <http://www.ripn.net>
Russian Institute for Public Networks (RIPN) was founded in 1992 by the 
Science and Higher School Committee of Russia, Russian Research Center 
"Kurchatov Institute" and the Computer Center of the Kurchatov Institute. 
RIPN operates the Russian Backbone Network and MoscowLight facility, supports 
Technical Center of national domain TLD .RU and develops public Internet 
Exchanges in the largest cities of Russia. RIPN is GLORIAD/Russia’s network 
operator.
 
About The Russian Backbone Network (RBNet) (since 1996) <http://www.rbnet.ru>
Russian Backbone Network (since 1996) is a backbone that serves regional, 
specialized and corporate networks for Research and Education. At present, it 
is a high-speed IP network with Points-of-Presence (POPs) in all Federal 
Regions of Russia, and is migrating to a Hybrid Network Infrastructure.
2006-January: Open the GLIF Open Lambda Exchange (GOLE) Facility, 
MoscowLight, to support the Hybrid Network Infrastructure
2007-January: Establish 10 Gbps link from Moscow (MoscowLight) to Amsterdam 
(NetherLight)
2007-April: Establish lightpath between Moscow, Amsterdam and Chicago
2007-2008: Plan to establish 2.5Gbps - Moscow - Khabarovsk link, with a 
connection to Hong Kong (HKOEP-HKLight)  and Korea (KRLight).
 
About The Institute of Space Research, Russian Academy of  Sciences 
<http://www.iki.rssi.ru/eng/index.htm>
Institute of Space Research, Russian Academy of Sciences (RAS), is the 
leading RAS organization in scientific research areas such as: Outer Space, 
Solar System planets and other objects of the Universe. The Space Research 
Institute (IKI) is primarily in charge of long-range planning and elaboration 
of space research programs, of which a considerable part is performed within 
the framework of international space research cooperation.
 
About The National Center for Data Mining at the University of Illinois at 
Chicago <http://www.ncdm.uic.edu>
The National Center for Data Mining (NCDM) at the University of Illinois at 
Chicago (UIC) was founded in 1998 as a national resource for high-performance 
and distributed data mining. The Center performs research, coordinates 
standards, operates testbeds, and engages in outreach. The Center coordinates 
the development of the Predictive Model Markup Language (PMML), a standard 
for statistical and data mining models, and operates the Teraflow Network, a 
network for distributing large e-science datasets.
 
About TransLight/StarLight <http://www.startap.net/translight>
In cooperation with USA and European National Research & Education Networks, 
TransLight/StarLight is implementing a strategy to best serve established USA 
and European production science, including usage by those scientists, 
engineers and educators who have persistent large-flow, real-time, and/or 
other advanced application requirements. TransLight/StarLight currently 
provides two connections between the USA and Europe for production science: a 
routed connection that connects the pan-European GÉANT2 to the USA Internet2, 
National LambdaRail and ESnet networks, and a switched connection that is 
part of the LambdaGrid fabric being created by participants of the Global 
Lambda Integrated Facility (GLIF) and used by the GLORIAD community and the 
Teraflow Network project. Major funding is provided by the USA National 
Science Foundation International Research Network Connections (IRNC) program, 
award OCI-0441094 to the University of Illinois at Chicago (UIC), for the 
period February 2005 - January 2010.

About StarLight <www.startap.net/starlight>
StarLight is an advanced optical infrastructure and proving ground for 
network services optimized for high-performance applications. StarLight is 
the GLIF Open Lightpath Exchange (GOLE) in Chicago. Operational since summer 
2001, StarLight has 1GE and 10GE switch/router facilities and true optical 
switching for wavelengths. StarLight is being developed by the Electronic 
Visualization Laboratory (EVL) at the University of Illinois at Chicago 
(UIC), the International Center for Advanced Internet Research (iCAIR) at 
Northwestern University, and the Mathematics and Computer Science Division at 
Argonne National Laboratory, in partnership with Canada's CANARIE and the 
Netherlands' SURFnet. StarLight (sm) is a service mark of the Board of 
Trustees of the University of Illinois.

About UT-ORNL JICS <www.gloriad.org/gloriad/team/usa/index.html> and 
<www.jics.utk.edu/jics_about.html>
The University of Tennessee (UT) and Oak Ridge National Laboratory (ORNL) 
established the Joint Institute for Computational Sciences (JICS) to advance 
scientific discovery and state-of-the-art engineering and to further 
knowledge of computational modeling and simulation by taking full advantage 
of the terascale and beyond computers in the National Center for 
Computational Sciences housed at ORNL, and by educating a new generation of 
scientists and engineers well-versed in the application of computational 
modeling and simulation for solving for the most challenging scientific and 
engineering problems. GLORIAD/USA based at UT-ORNL JICS.
 
About NetherLight/SURFnet <www.netherlight.net/info/about/introduction.jsp> 
and <www.surfnet.nl>
NetherLight is the GLIF Open Lightpath Exchange (GOLE) in Amsterdam, The 
Netherlands and has been operational since January 2002. It is an advanced 
open optical infrastructure and proving ground for network services optimized 
for high-performance applications, especially those that require Grid 
infrastructure. NetherLight provides multiple specialized international 
high-performance communication services in partnership with other advanced 
research networks. NetherLight has supported multiple landmark data-intensive 
science experiments and demonstrations and is pioneering new concepts for 
architecture that may find their way into other GOLEs worldwide and the 
traditional telecommunications world as they move to next-generation 
architectures. SURFnet enables breakthrough education and research. SURFnet 
develops and operates the national SURFnet6 network and provides innovative 
services in the areas of security, authentication and authorization, group 
communication and video. SURFnet is a subsidiary of the SURF organization in 
which research universities, universities of applied sciences and research 
institutions collaborate nationally and internationally on innovative ICT 
facilities. SURF provides the foundation for the excellence of higher 
education and research in the Netherlands. GLORIAD/Netherlands is based in 
Amsterdam and managed by SURFnet.
 

Acknowledgment
Joe Mambretti< 
>,
 Robert Grossman 
<>,
 Yunhong Gu 
<>,
 Thomas DeFanti  
<>,
 Maxine Brown 
<>,
 Alan Verlo 
<>,
 Linda Winkler 
<>,
 Alex Szalay 
<>,
 Greg Cole  
<>,
 Natalia Bulashova 
<>,
 Evgeny P.Velikhov 
<>,
 Alexey Soldatov 
<>,
  Alexey Platonov 
<>,
 Mikhail Zhizhin 
<>,
 Ravil Nazirov < 
>,
 Anton Korotin 
<>,
 Alexander Ilin 
<>,
 Michael Boyarsky 
<>,
 Veniamin Konoplev < 
>,
 Olga Starostina 
<>,
 Kees Neggers 
<>,
 Cees de Laat 
<>,
 Erik-Jan Bos 
<>,
 Paola Grosso 
<>,
 Bram Peeters 
<>,
 J.P. Velders 
<>,
 Wounter Huisman 
<>,
  John Lankford 
<>,
  Peter de Boer 
<>,
  Thomas Tam 
<>,
  Bill St.Arnaud 
<>,
 Rene Hatem 
<>,
 NetherLight NOC 
<>,
 RBnet NOC 
<>,
  StarLight NOC 
<>.