| Yannis, Thanks for your feedback.
I was involved in another discussion this morning that really illustrated the tradeoffs we have to make when working on different scales.
At one end of the continuum you have residential broadband type situations, where most of the locations on your network have extremely limited equipment and capacity, and you are looking to establish very general or coarse usability measurements. In this environment, a measurement probe needs to be very in-expensive, possibly just a small application or service that runs in the background on a cable modem, or wireless router.
At the other end of the continuum, you have networks like ESnet, Internet2 & GEANT where all of the nodes include big expensive equipment that are connected by multiple 10GE links. In this environment we are concerned with highly accurate measurement tools that can help us debug subtle problems in high bandwidth circuits, some of which are very expensive.
So it makes sense for me to deploy several rackmount servers in a pop where I have $500,000 worth of routing & switching equipment. Clearly this doesn't make sense in all environments.
The recent changes in the perfSONAR-PS toolkit packaging methods should make it significantly easier to use those tools in an imbeded environment running a standard redhat OS. Of course the host administrative type stuff in there will probably need to be tweaked to deal with differing device names, and enabling the 'right' set of services. But this should be fairly straight forward.
--Joe
On Dec 15, 2010, at 2:18 PM, Yannis Mitsos wrote: Dear Joe,
Thank you for your enlightening response. It is obvious that I had a different scenario in my mind. Namely, how can put cheap (?) probes all around our network. Bear in mind that according to our policy, for each customer we provide a L2 CPE, operated by us to monitor the status of the fibre up to our customer's premises. Planting a probe (I was thinking for instance a stick-like embedded system) in each CPS would provide us very comprehensive and accurate data for the whole network.
Now, talking about porting, allow me to clarify something: I was considering standard embedded platforms that are equipped with typical components and may run *standard* linux distribution. Not embedded systems that we would have to port the kernel or re-compile the applications.
From your words, I understand that there is no substantial problem in having perfsonar-ps executed in a "weaker" platform. I will try to elaborate a little bit more in my free time and if I come-up with concrete results, I will update the list.
Best Regards,
Yannis On Dec 14, 2010, at 17:37 , Joe Metzger wrote: Yannis, Somebody has suggested this idea at least twice a year for the last 2-3 years. The 'green' aspects of this are worth considering, and it might improve the value of perfSONAR to communities concerned with inexpensive low bandwidth networks.
The reason it hasn't taken off is that the primary perfSONAR contributors are concerned with improving the ability measuring across very high performance wide area networks, and diagnose problems on paths across multiple wide area networks. The small, cheap 'plug computers' have limited processing power which could reduce the accuracy of the measurements they take, and make diagnosing problems more difficult. Also, porting perfSONAR tools to this type of platform and supporting it would require resources that we don't have at this time, and the personnel costs of porting & maintaining it greatly outweigh the potential capital costs savings.
So, my response has always been that yes, I think it is a great idea. And the perfSONAR collaboration will gladly provide you all the help we can, if you are willing to invest the time and resources to do this.
I suspect that the 'green computing' and energy efficiency issues will heat up and the capabilities of the plug computers will continue to grow to the point where it will make economical or political sense for the principal contributors to invest the resources to port and maintain perfSONAR on that type of platform in a couple of years.
--Joe
On Dec 14, 2010, at 3:15 AM, Yannis Mitsos wrote: Dear Nina, On Dec 14, 2010, at 11:02 , Nina Jeliazkova wrote:
On 14 December 2010 10:34, Yannis Mitsos <> wrote:
Dear All,
Speaking about hardware requirements, I was wondering if one has considered "porting" perfsonar-ps to a compact embedded platform. Instead of having big servers scattered around the network, a small and cheap device might be connected directly to an ethernet port of a CPE switch providing a good view of the network performance.
Can't help replying ... a wonderful idea, although I wonder if the
current XML protocol will fit well in an embedded platform.
Can you please elaborate on the latter. How is the XML correlated to the hardware? When I refer to embedded platform, I have in my mind hardware that is fully supported by the Linux kernel (no uClinux stuff) with adequate memory.
Best regards,
Yannis
Best regards,
Nina Jeliazkova
Regards,
Yannis On Dec 8, 2010, at 21:49 , Alan Whinery wrote:
Since I hadn't actually done much with them yet, I was keeping to
myself, but I just recently bought and deployed 4 of these as
PerfSONAR throughput machines:
PowerEdge R210 Chassis w/up to 2 Cabled HDs and Quad-Pack LED
Diagnostics, Unit Price $1,074.96
Operating System No Operating System
Memory 4GB Memory (4x1GB), 1333MHz, Single Ranked UDIMM
Processor Intel® Xeon® X3440, 2.53 GHz, 8M Cache, Turbo, HT
Network Adapter Intel PRO 1000PT 1GbE Single Port NIC, PCIe-1
Also separate built-in NIC for IPMI
Hard Drive 500GB 7.2k RPM Serial ATA 3Gbps 3.5-in Cabled Hard Drive
3Yr Basic Hardware Warranty Repair: 5x10 HW-Only, 5x10 NBD Onsite
I don't buy optical drives for machines that are never going to see
a CD or DVD after initial install. I would like to see flash devices
(i.e. SATA or IDE SATA CF adapters) edge out optical. I have to buy
a plane ticket to get to most of my devices.
Although they are intended as 10GbE nodes, I haven't bought 10 GbE
for these yet, as most locations aren't tooled for it. Performance
test seems to be a niche dominated by Myricom and Chelsio, at the
moment. Our two existing NICs are Myri.
========================
Also considered HP Proliant, which turned out to be more expensive
for very similar machine.
========================
I also have 10 of these coming for use as PerfSONAR delay-oriented
machines (and possible CAIDA Ark fillers):
Super Micro
1RU Chassis SYS-5015A-EHF
-
Intel® Atom™ D510 processor (DMI)
2x Intel 82574L Gigabit LAN
1 PCI-e x4 slot
real RS-232 serial port (critical for CDMA timing)
IPMI
Crucial 2GB Kit SO-DIMM DDR2 667MHz unbuffered
ST31000520AS 1 TB Low Power HDD
200 watt power supply.
I required the PCI-e slot as a fall-back in case I don't like the
DMI-connected Intel LAN
Total $530 each, incl all taxes and fees -- through open public bid.
Although they haven't actually arrived yet. I'm nose deep in hard
drives and RAM, which has limited entertainment, with no computers
attached. Maybe in time for Christmas!
===================================
My current operational 10 GbE bwctl/NDT/NPAD box, which I have seen
push between 7 & 8 Gbit/sec to Los Angeles:
Intel MB with dual Intel(R) Pentium(R) D CPU 3.20GHz 1 M cache
1 GB RAM
30 GB HDD
Myri-10G Dual-Protocol NIC (10G-PCIE-8A)
Which I didn't buy, but a colleague's desktop that he left behind
when he moved to another university.
==============================
-Alan
On 12/8/2010 6:26 AM, Buraglio, Nicholas D wrote:
No problem. Here is the brief synopsis of what I ordered:
PowerEdge R210 Chassis
Intel Xeon X3430, 2.4 GHz, 8M Cache
4GB Memory
500Gig Disk
DVD-ROM
3Yr Basic Hardware Warranty Repair: 5x10 HW-Only, 5x10 NBD Onsite
Total cost was a bit under $1100 per box and it should meet or exceed the requirements. I'm also going to test this config with an Intel 10G card that I already have just to see how well it performs.
For what it's worth, I purposely priced these out for cost savings over redundancy so that I could get more of them and instrument our networks in more locations. The small footprint was nice as well since rack space is always a premium commodity.
nb
---
Nick Buraglio
Network Engineer
University of Illinois CITES / ICCN
GPG key 0x2E5B44F4
Phone: 217.689.4254
On Dec 7, 2010, at 8:12 PM, Thomas Tam wrote:
Nicks,
Thanks for the information! It would be much appreciated if you could share your hardware config.
Regards,
-Thomas
On 10-12-07 5:16 PM, "Buraglio, Nicholas D" wrote:
We're in the middle of doing this both on our campus and on our statewide optical network, so I'm by no means an expert, and only experienced in that I have a handful of test boxes out there. I'm more in the same type of situation as you, "new deployment" status.
I've started with a handful of 1G boxes to add to the few cobbled together test boxes we already have and plan to add a handful of 10G boxes specifically for throughput testing. I used the ESnet KB as well as the page Jason posted for a starting point for my 1G hardware build: http://fasterdata.es.net/ps_hardware.html
I've priced out a handful of slightly customized (more RAM, better processor) Dell PowerEdge R210, which are pretty inexpensive, have a small footprint and exceed the minimum requirements for a 1G perfSonar box. Generally speaking, I've always stuck with Intel or Myricom 10G cards for my other applications (Intel has the best support for what I've used them for in the past, which is mostly pf_ring packet capture stuff).
I'd be more than happy to share my hardware config if you're interested.
nb
---
Nick Buraglio
Network Engineer
University of Illinois CITES / ICCN
GPG key 0x2E5B44F4
Phone: 217.689.4254
On Dec 7, 2010, at 2:15 PM, Thomas Tam wrote:
Hi
I am planning to deploy couple of perfonser servers in our network. At first, we are looking into deploying a 1GE connection to the network, but will migrate to a 10GE connection in a few months. I would love to hear recommendations of what server platforms and 10GE interface cards to be used from experienced perfsonar users.
Your information is greatly appreciated.
Best regards,
-Thomas
CANARIE NOC
|
