perfSONAR User Q&A and Other Discussion

[Hyojoon Kim <>]

Thanks for all the suggestions and comments!

To answer some: 
* I did remove the "ondemand option" as I saw that note somewhere too before 
I ran the test. I’m sure it did something but I still get around 550Mbps 
bwctl. 
* I did the Linux Tuning *after* I got 550Mbps at my first trial, hoping it 
would fix it. No luck. 
* Interestingly, I get around 940Mbps when I initiate the bwctl test *from* a 
Dell node *to* the LIVA X box. But the other direction still gives me around 
550Mbps. 
* One thing I noticed is that: 
  - When I initiate the test from the Dell node, it opens an ephemeral port 
(e.g., 45250) on the dell node.
      - local LIVAXBOX port 5593 connected with DELL port 45240
  - From LIVAX box to Dell server is:
      - local DELL port 5220 connected with LIVAXBOX port 5220
    

Thanks,
Joon 

> On Dec 22, 2015, at 1:43 PM, Uhl, George D. (GSFC-423.0)[SGT INC] 
> <>
>  wrote:
> 
> When I was researching the Liva X capabilities I ran across an email from 
> Larry Blunk of Merit that he posted on the list this past October.
> 
>   One thing to note is that Ubuntu enables the "ondemand" init script
> by default which puts the CPU in "powersave" mode.   In testing the
> LIVA, I found that this seems to limit throughput a bit in performance
> tests.  I get around 900Mbps TCP throughput vs. 940Mbps in
> "performance" mode.  Also saw some packet loss when doing UDP tests
> with 1500 byte datagrams at 1 Gbps in powersave mode.  You can disable
> the script with the following which will leave the CPU in performance
> mode.
> 
> update-rc.d -f ondemand remove
> 
> 
> 
> -Larry Blunk
>   Merit
> 
> Disabling the “ondemand” mode capability might work for you.   I did it and 
> I was able to achieve 940 Mbps.
> 
> -George
> 
> From: 
> <>
>  on behalf of Brian Tierney 
> <>
> Reply-To: 
> ""
>  
> <>
> Date: Tuesday, December 22, 2015 at 1:12 PM
> To: Daniel Doyle 
> <>
> Cc: Hyojoon Kim 
> <>,
>  
> ""
>  
> <>
> Subject: Re: [perfsonar-user] Bandwidth measurement with LIVA X 2GB/32GB 
> eMMC
> 
> 
> 
> On Tue, Dec 22, 2015 at 12:00 PM, Daniel Doyle 
> <>
>  wrote:
>> Hi Joon,
>> 
>> Sorry to hear about that. A couple of questions / debugging ideas:
>> 
>> - Are you seeing this in both directions?
>> - Have you tried using the same port with a different device?
>> - If you're using / not using jumbo frames has the device been configured 
>> accordingly?
>> 
>> fwiw, I did not apply any tunings out of the box on a LIVA X and got 
>> >900Mbps on a local network. It's possible some of those tunings might not 
>> be appropriate for a machine like that, but I have't dug into it.
> 
> I agree that might be the issue. The tunings on fasterdata might not be 
> right for a small node.
> See if the original default debian settings are better.
> 
>  
>> 
>> -Dan
>> 
>> > On Dec 22, 2015, at 11:58 AM, Hyojoon Kim 
>> > <>
>> >  wrote:
>> >
>> > Hi all,
>> >
>> > We decided to play with a LIVA X 2GB/32GB eMMC node with perfSONAR 3.5. 
>> > However, I am getting around 535Mbps instead of 940+Mbps when I initiate 
>> > a bwctl test to another local perfSONAR node we have. Between two Dell 
>> > R610 servers with perfSONAR, we do get over 950Mbps, so it’s likely not 
>> > a network problem.
>> >
>> > I have generally followed the instructions here 
>> > (http://docs.perfsonar.net/install_debian.html), and installed three 
>> > packages:
>> > apt-get install perfsonar-testpoint
>> > apt-get install perfsonar-toolkit-security
>> > apt-get install perfsonar-toolkit-sysctl
>> >
>> > I also did the Linux Tuning, which is indicated here:
>> > http://fasterdata.es.net/host-tuning/linux/
>> >
>> > Does anyone know what I am missing? I do have Ubuntu 14.04.3 Desktop OS 
>> > on this. Maybe switching to the well-tested Ubuntu 12.04 is better?
>> >
>> > ===
>> > $ lsb_release -a
>> > No LSB modules are available.
>> > Distributor ID: Ubuntu
>> > Description:    Ubuntu 14.04.3 LTS
>> > Release:        14.04
>> > Codename:       trusty
>> >
>> > $ uname -a
>> > Linux perfbox-livaxtest-01 3.19.0-39-generic #44~14.04.1-Ubuntu SMP Wed 
>> > Dec 2 10:00:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>> > ===
>> >
>> > Thanks,
>> > Joon
>> 
>> Dan Doyle
>> GlobalNOC Software Developer
>> 1-812-856-3892
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Brian Tierney, http://www.es.net/tierney
> Energy Sciences Network (ESnet), Berkeley National Lab
> http://fasterdata.es.net
>