perfSONAR User Q&A and Other Discussion

[Zafar Gilani <>]

Hi Jason,

I've changed the NTP servers to closest ones but don't know of the closest to Pakistan. I will try to get the list and add those as replacement. We've blocked SSH on these machines so I won't be able to run bwctl tests remotely. Will let you know on these. Thanks for all the help! I really appreciate this.

On Fri, Nov 26, 2010 at 6:06 PM, Jason Zurawski <> wrote:

Hi Zafar;

On 11/26/10 6:32 AM, Zafar Gilani wrote:

Hi Jason,

Checked server time, it seems to be fine.

One thought that springs to mind is that these hosts are physically located in Pakistan?  The 'default' NTP servers that come with the toolkit are located in the US on R&E networks.  The NTP peer status can be found in this manner:

[root@lab236 ~]# ntpq -p
    remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
-otc2.psu.edu    147.84.59.145    2 u  565 1024  377   23.018    0.010   0.110
*navobs1.oar.net .USNO.           1 u  697 1024  377    7.271   -0.125   0.093
+2001:468:1:12:: 130.207.244.240  2 u  669 1024  377   27.101   -0.005   0.137
-2001:468:2:12:: 64.57.16.34      2 u  590 1024  377   30.220   -0.251   0.081
+2001:468:6:12:: .PPS.            1 u  611 1024  377   18.816   -0.165   0.076

In the case above, the 'delay' field shows how close the target NTP servers are to my host.  Having these be small values is good.  The latency to synchronize over the distance from the US to Pakistan (more than 300ms when I did a ping) will have an effect on the accuracy of the measurement tools.  This could contribute to the error you report below.

If you know of NTP servers that are 'closer' it would be good to add them to the NTP configuration file manually.  Edit '/etc/ntp.conf' to add lines like this:

 server NTP_SERVER_IN_PAKISTAN iburst

And either comment out or delete the other server lines.  Try to find at least 3 (preferably 5) in that region of the world.   Restart NTP (sudo /etc/init.d/ntpd restart) after making any changes.

The databases have data for
both owamp and bwctl tables for LA and BW nodes respectively. However
the tests return the following:

For bwctl "remote peer cancelled test". However it does say server
listening on xy port and binding to local address. For owamp tests I see
first and last results but it also says 0 sent, 0 lost. This is strange.

What do you think might be the problem? I do see data which suggests the
firewall might not be the culprit.

If you see data, then firewalls are probably not to blame.  Try to implement the suggestion to NTP above, but you can also try this test immediately:

 bwctl -c BW_HOST_IN_SCHEDULE -a 5
 bwctl -s BW_HOST_IN_SCHEDULE -a 5

The 'a' option tells BWCTL to allow an extra bit of offset (+ or - the specified value) when calculating the measurement start/stop.  Its normally used in cases where the clock may be off between hosts.  Raise the value to 10 or even 20 and see if that allows the tests to complete.

I also think upgrading to 3.2 is a good idea, but lets solve this problem first :)

Thanks;

-jason

On Thu, Nov 25, 2010 at 10:31 PM, Jason Zurawski <

<mailto:>> wrote:

   Hi Zafar;


   On 11/25/10 12:26 PM, Zafar Gilani wrote:

       Hi Jason,

       Thanks for the reply. I'm not able to access the machines from
       my home,
       I'll try tomorrow morning and let you know. Possibility of a
       firewall
       also comes to my mind. I also checked system clock and it seemed
       to be
       correct. Addresses:

       BW: 115.186.131.105
       LA: 115.186.131.107



   I am able to reach both, one thing I am noticing is that these are
   both 3.1.2.  I would suggest upgrading to 3.2 if at all possible.

   In any event, report back if you are seeing data when you get a
   chance - also if you suspect a firewall may be an issue, try some
   simple tests to some of the hosts that are in the regular testing
   groupings, e.g.:

     owping HOST_IN_LATENCY_TESTS
     bwctl -c HOST_IN_BW_TESTS
     bwctl -s HOST_IN_BW_TESTS

   If these fail for any of the hosts, firewalls may be an issue.

   Thanks;

   -jason


       These machines are deployed at NUST, Islamabad, Pakistan.

       On Thu, Nov 25, 2010 at 5:48 PM, Jason Zurawski
       < <mailto:>

       <mailto: <mailto:>>>

       wrote:

           Hi Zafar;

           I am CCing the performance-node-users list on this response.
             Comments below:


           On 11/25/10 5:57 AM, Zafar Gilani wrote:

               Hi,

               I've deployed new BW and LA PS nodes but can't seem to
       view any
               graphs.
               Whenever I go to Service Graphs -> Throughput or Service
       Graphs ->
               One-Way Latency, I get the following:

               Problem Handling Request
               MA */http://localhost:8085/perfSONAR_PS/services/pSB/*
       did not
               return
               the expected response, be sure it is configured and
       populated
               with data.

               For OWAMP I try to view graph and receive following:


                   Internal Error - Service returned data, but it is not
               plotable for
                   this measurement pair.

               Anyone having any ideas? I've already checked the
       scheduled tests.
               Everything seems to be correct. I also checked
       http://psps.perfsonar.net/toolkit/FAQs.html#Q19 but I don't
               think this
               is the case as it has been more than 24 hours since the
       nodes
               were deployed.


           What is the address of this performance node, is it world
       accessible
           or does it have a local address?

           I would suggest looking in the database by hand to see if
       there is
           recent data:

              mysql -u root

           Once at the mysql prompt try these commands:

              use owamp;
              show tables;

           Look for a table that has data for 20101125_* or similar.
         Try the
           same for bwctl:

              use bwctl;
              show tables;
              exit;

           There should be data for 201011_*.  If you don't see recent data
           tables, you are not getting data from the regular tests
       (even though
           you did set them up).  This could be because your system
       clock is
           not correct/not running, firewalls are preventing the
       BWCTL/OWAMP
           tests, or perhaps some other reason.

           Let us know what you see and we can debug further, thanks;

           -jason