Skip to Content.
Sympa Menu

perfsonar-user - Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????

Subject: perfSONAR User Q&A and Other Discussion

List archive

Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????


Chronological Thread 
  • From: Aaron Brown <>
  • To: Pedro Henrique Diniz Da Silva <>
  • Cc: Andrew Lake <>, "" <>
  • Subject: Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????
  • Date: Wed, 1 Apr 2015 12:25:02 +0000
  • Accept-language: en-US
  • Authentication-results: cern.ch; dkim=none (message not signed) header.d=none;
Hey Pedro,

From the logs, it looks like the problem may have cropped up when
ps02.cat.cbpf.br is the receiver. Try running: bwctl -T iperf3 -t 20 -i 1 -c
ps02.cat.cbpf.br -s perfsonar-bw.cern.ch

Cheers,
Aaron


> On Mar 31, 2015, at 9:33 AM, Pedro Henrique Diniz Da Silva
> <>
> wrote:
>
> Andy,
>
> After 3 attempts, all of them was successfull as you can see below:
>
> [gridlafex@ps02
> ~]$ bwctl -T iperf3 -t 20 -i 1 -s ps02.cat.cbpf.br -c perfsonar-bw.cern.ch
> bwctl: Using tool: iperf3
> bwctl: 27 seconds until test results available
>
> SENDER START
> Connecting to host 128.142.223.246, port 5594
> [ 14] local 152.84.101.142 port 47612 connected to 128.142.223.246 port 5594
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 14] 0.00-1.00 sec 617 KBytes 5.05 Mbits/sec 0 49.5 KBytes
> [ 14] 1.00-2.00 sec 1021 KBytes 8.36 Mbits/sec 0 115 KBytes
> [ 14] 2.00-3.00 sec 2.06 MBytes 17.3 Mbits/sec 0 230 KBytes
> [ 14] 3.00-4.00 sec 2.30 MBytes 19.3 Mbits/sec 0 409 KBytes
> [ 14] 4.00-5.00 sec 5.00 MBytes 42.0 Mbits/sec 0 727 KBytes
> [ 14] 5.00-6.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.27 MBytes
> [ 14] 6.00-7.00 sec 17.5 MBytes 147 Mbits/sec 0 2.35 MBytes
> [ 14] 7.00-8.00 sec 16.2 MBytes 136 Mbits/sec 0 4.15 MBytes
> [ 14] 8.00-9.00 sec 17.5 MBytes 147 Mbits/sec 7 2.55 MBytes
> [ 14] 9.00-10.00 sec 12.5 MBytes 105 Mbits/sec 0 2.55 MBytes
> [ 14] 10.00-11.00 sec 12.5 MBytes 105 Mbits/sec 0 2.55 MBytes
> [ 14] 11.00-12.00 sec 12.5 MBytes 105 Mbits/sec 0 2.55 MBytes
> [ 14] 12.00-13.00 sec 13.8 MBytes 115 Mbits/sec 0 2.55 MBytes
> [ 14] 13.00-14.00 sec 12.5 MBytes 105 Mbits/sec 0 2.55 MBytes
> [ 14] 14.00-15.00 sec 12.5 MBytes 105 Mbits/sec 0 2.56 MBytes
> [ 14] 15.00-16.00 sec 12.5 MBytes 105 Mbits/sec 0 2.56 MBytes
> [ 14] 16.00-17.00 sec 12.5 MBytes 105 Mbits/sec 0 2.56 MBytes
> [ 14] 17.00-18.00 sec 12.5 MBytes 105 Mbits/sec 0 2.56 MBytes
> [ 14] 18.00-19.00 sec 11.2 MBytes 94.4 Mbits/sec 0 2.56 MBytes
> [ 14] 19.00-20.00 sec 12.5 MBytes 105 Mbits/sec 0 2.56 MBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 14] 0.00-20.00 sec 208 MBytes 87.4 Mbits/sec 7 sender
> [ 14] 0.00-20.00 sec 190 MBytes 79.7 Mbits/sec
> receiver
>
> iperf Done.
>
> SENDER END
> [gridlafex@ps02
> ~]$ bwctl -T iperf3 -t 20 -i 1 -s ps02.cat.cbpf.br -c perfsonar-bw.cern.ch
> bwctl: Using tool: iperf3
> bwctl: 27 seconds until test results available
>
> SENDER START
> Connecting to host 128.142.223.246, port 5595
> [ 15] local 152.84.101.142 port 46154 connected to 128.142.223.246 port 5595
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 15] 0.00-1.00 sec 554 KBytes 4.54 Mbits/sec 0 52.3 KBytes
> [ 15] 1.00-2.00 sec 1.07 MBytes 8.97 Mbits/sec 0 134 KBytes
> [ 15] 2.00-3.00 sec 2.37 MBytes 19.9 Mbits/sec 0 315 KBytes
> [ 15] 3.00-4.00 sec 3.73 MBytes 31.3 Mbits/sec 0 583 KBytes
> [ 15] 4.00-5.00 sec 7.50 MBytes 62.9 Mbits/sec 0 1.02 MBytes
> [ 15] 5.00-6.00 sec 13.8 MBytes 115 Mbits/sec 0 1.88 MBytes
> [ 15] 6.00-7.00 sec 17.5 MBytes 147 Mbits/sec 0 3.33 MBytes
> [ 15] 7.00-8.00 sec 20.0 MBytes 168 Mbits/sec 0 5.51 MBytes
> [ 15] 8.00-9.00 sec 31.2 MBytes 262 Mbits/sec 0 8.51 MBytes
> [ 15] 9.00-10.00 sec 46.2 MBytes 388 Mbits/sec 0 11.3 MBytes
> [ 15] 10.00-11.00 sec 61.2 MBytes 514 Mbits/sec 0 14.0 MBytes
> [ 15] 11.00-12.00 sec 73.8 MBytes 619 Mbits/sec 0 16.6 MBytes
> [ 15] 12.00-13.00 sec 87.5 MBytes 734 Mbits/sec 0 19.4 MBytes
> [ 15] 13.00-14.00 sec 97.5 MBytes 818 Mbits/sec 0 21.5 MBytes
> [ 15] 14.00-15.00 sec 100 MBytes 839 Mbits/sec 0 21.5 MBytes
> [ 15] 15.00-16.00 sec 62.5 MBytes 524 Mbits/sec 9 10.8 MBytes
> [ 15] 16.00-17.00 sec 52.5 MBytes 440 Mbits/sec 0 10.8 MBytes
> [ 15] 17.00-18.00 sec 51.2 MBytes 430 Mbits/sec 0 10.8 MBytes
> [ 15] 18.00-19.00 sec 51.2 MBytes 430 Mbits/sec 0 10.8 MBytes
> [ 15] 19.00-20.00 sec 33.8 MBytes 283 Mbits/sec 93 2.68 MBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 15] 0.00-20.00 sec 815 MBytes 342 Mbits/sec 102 sender
> [ 15] 0.00-20.00 sec 799 MBytes 335 Mbits/sec
> receiver
>
> iperf Done.
>
> SENDER END
> [gridlafex@ps02
> ~]$
> [gridlafex@ps02
> ~]$
> [gridlafex@ps02
> ~]$
> [gridlafex@ps02
> ~]$
> [gridlafex@ps02
> ~]$ bwctl -T iperf3 -t 20 -i 1 -s ps02.cat.cbpf.br -c perfsonar-bw.cern.ch
> bwctl: Using tool: iperf3
> bwctl: 27 seconds until test results available
>
> SENDER START
> Connecting to host 128.142.223.246, port 5596
> [ 15] local 152.84.101.142 port 34871 connected to 128.142.223.246 port 5596
> [ ID] Interval Transfer Bandwidth Retr Cwnd
> [ 15] 0.00-1.00 sec 608 KBytes 4.98 Mbits/sec 0 60.8 KBytes
> [ 15] 1.00-2.00 sec 1.02 MBytes 8.55 Mbits/sec 0 136 KBytes
> [ 15] 2.00-3.00 sec 2.34 MBytes 19.6 Mbits/sec 0 294 KBytes
> [ 15] 3.00-4.00 sec 3.68 MBytes 30.9 Mbits/sec 0 563 KBytes
> [ 15] 4.00-5.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.10 MBytes
> [ 15] 5.00-6.00 sec 15.0 MBytes 126 Mbits/sec 0 2.05 MBytes
> [ 15] 6.00-7.00 sec 16.2 MBytes 136 Mbits/sec 49 3.16 MBytes
> [ 15] 7.00-8.00 sec 8.75 MBytes 73.4 Mbits/sec 70 1.66 MBytes
> [ 15] 8.00-9.00 sec 5.00 MBytes 41.9 Mbits/sec 10 919 KBytes
> [ 15] 9.00-10.00 sec 2.50 MBytes 21.0 Mbits/sec 98 232 KBytes
> [ 15] 10.00-11.00 sec 0.00 Bytes 0.00 Mbits/sec 0 235 KBytes
> [ 15] 11.00-12.00 sec 1.25 MBytes 10.5 Mbits/sec 0 238 KBytes
> [ 15] 12.00-13.00 sec 1.25 MBytes 10.5 Mbits/sec 0 240 KBytes
> [ 15] 13.00-14.00 sec 1.25 MBytes 10.5 Mbits/sec 0 243 KBytes
> [ 15] 14.00-15.00 sec 1.25 MBytes 10.5 Mbits/sec 0 246 KBytes
> [ 15] 15.00-16.00 sec 1.25 MBytes 10.5 Mbits/sec 0 249 KBytes
> [ 15] 16.00-17.00 sec 0.00 Bytes 0.00 Mbits/sec 0 252 KBytes
> [ 15] 17.00-18.00 sec 1.25 MBytes 10.5 Mbits/sec 0 255 KBytes
> [ 15] 18.00-19.00 sec 1.25 MBytes 10.5 Mbits/sec 0 257 KBytes
> [ 15] 19.00-20.00 sec 1.25 MBytes 10.5 Mbits/sec 0 260 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bandwidth Retr
> [ 15] 0.00-20.00 sec 73.9 MBytes 31.0 Mbits/sec 227 sender
> [ 15] 0.00-20.00 sec 54.9 MBytes 23.0 Mbits/sec
> receiver
>
> iperf Done.
>
> SENDER END
>
>
> Cheers,
> Pedro Diniz
> From: Andrew Lake
> []
> Sent: Tuesday, March 31, 2015 12:25 PM
> To: Pedro Henrique Diniz Da Silva
> Cc:
>
> Subject: Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????
>
> Hi Pedro,
>
> Looks like BWCTL is not completing when run by the regular testing. The
> command you ran had some variations from what is actually run by regular
> testing so may not be hitting the same limits, etc. What happens when you
> run the following:
>
> bwctl -T iperf3 -t 20 -i 1 -s ps02.cat.cbpf.br -c perfsonar-bw.cern.ch
>
> Thanks,
> Andy
>
> On Mar 27, 2015, at 10:37 AM, Pedro Henrique Diniz Da Silva
> <>
> wrote:
>
>> Hi Andy,
>>
>> #1 - Sorry, I made a mistake. The "Traceroute Test Between WLCG Bandwidth
>> Hosts" are being conducted, as you can see on
>> https://myosg.grid.iu.edu/pfmesh/mine/hostname/ps02.cat.cbpf.br . What is
>> strange is when I try to re generate the config, as you can see:
>>
>> [root@ps02
>> ~]# sudo -u perfsonar
>> /opt/perfsonar_ps/mesh_config/bin/generate_configuration
>> The compute_all_applicable_attributes method has been deprecated. Use
>> get_all_attributes instead.
>> at /usr/lib64/perl5/vendor_perl/Class/MOP/Deprecated.pm line 165
>>
>> Class::MOP::Class::compute_all_applicable_attributes('Moose::Meta::Class=HASH(0x2d26298)')
>> called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Config/Base.pm
>> line 106
>>
>> perfSONAR_PS::MeshConfig::Config::Base::unparse('perfSONAR_PS::MeshConfig::Config::TestParameters::Traceroute=...')
>> called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/perfSONARRegularTesting.pm
>> line 151
>> eval {...} called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/perfSONARRegularTesting.pm
>> line 129
>>
>> perfSONAR_PS::MeshConfig::Generators::perfSONARRegularTesting::add_mesh_tests('perfSONAR_PS::MeshConfig::Generators::perfSONARRegularTesting...',
>> 'HASH(0x2e6b580)') called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Agent.pm
>> line 347
>> eval {...} called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Agent.pm
>> line 346
>>
>> perfSONAR_PS::MeshConfig::Agent::__configure_host('perfSONAR_PS::MeshConfig::Agent=HASH(0x5842638)')
>> called at
>> /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Agent.pm
>> line 101
>>
>> perfSONAR_PS::MeshConfig::Agent::run('perfSONAR_PS::MeshConfig::Agent=HASH(0x5842638)')
>> called at /opt/perfsonar_ps/mesh_config/bin/generate_configuration line
>> 113
>>
>>
>> ################################################################
>>
>> #2 - Of course I can send, they follow attached. What I could detect about
>>
>> is below. Maybe can it be happenning because the server is too busy? But
>> as you can see in
>> https://ps02.cat.cbpf.br/toolkit/gui/psTracerouteViewer/index.cgi?mahost=http%3A%2F%2Flocalhost%2Fesmond%2Fperfsonar%2Farchive%2F&stime=yesterday&etime=now&tzselect=America%2FSao_Paulo&epselect=%2Fesmond%2Fperfsonar%2Farchive%2F2523fe31fea44b3caeadc84c146db997%2Fpacket-trace%2Fbase,
>> shouldn't CERN appear on https://ps02.cat.cbpf.br/serviceTest/psGraph.cgi
>> as the others?
>>
>> 2015/03/25 17:28:19 (21185) ERROR> EsmondBase.pm:53
>> perfSONAR_PS::RegularTesting::MeasurementArchives::EsmondBase::__ANON__ -
>> Error writing metadata (1) Error running test from perfsonar-bw.cern.ch
>> tops02.cat.cbpf.br with output bwctl: start_endpoint: 3632650383.492841
>> bwctl: run_endpoint: receiver: 152.84.101.142
>> bwctl: run_endpoint: sender: 128.142.223.246
>> bwctl: Unable to initiate peer handshake with [152.84.101.142]:6038 -
>> canceling
>> bwctl: stop_endpoint: 3632650383.492841
>> 2015/03/25 17:28:19 (21185) ERROR> MeasurementArchiveChild.pm:209
>> perfSONAR_PS::RegularTesting::Master::MeasurementArchiveChild::handle_results
>> - Problem storing results: Error writing metadata: Error running test
>> from perfsonar-bw.cern.ch to ps02.cat.cbpf.br with output bwctl:
>> start_endpoint: 3632650383.492841
>> bwctl: run_endpoint: receiver: 152.84.101.142
>> bwctl: run_endpoint: sender: 128.142.223.246
>> bwctl: Unable to initiate peer handshake with [152.84.101.142]:6038 -
>> canceling
>> bwctl: stop_endpoint: 3632650383.492841
>> 2015/03/25 17:28:19 (21185) ERROR> MeasurementArchiveChild.pm:125
>> perfSONAR_PS::RegularTesting::Master::MeasurementArchiveChild::__ANON__ -
>> Problem handling test results: Problem storing results: Error writing
>> metadata: Error running test from perfsonar-bw.cern.ch to ps02.cat.cbpf.br
>> with output bwctl: start_endpoint: 3632650383.492841
>> bwctl: run_endpoint: receiver: 152.84.101.142
>> bwctl: run_endpoint: sender: 128.142.223.246
>> bwctl: Unable to initiate peer handshake with [152.84.101.142]:6038 -
>> canceling
>> bwctl: stop_endpoint: 3632650383.492841
>>
>> #2 Continuing - Another host with problems is "ps-bw.sciencedmz.on.br".
>> It also doesn't appear on
>> https://ps02.cat.cbpf.br/serviceTest/psGraph.cgi. We can make manual tests
>> without problems as below but there is no error in
>> /var/log/perfsonar/regular_testing.log related to this host.
>>
>> [root@ps02
>> ~]# bwctl -c ps-bw.sciencedmz.on.br -f m
>> bwctl: Using tool: iperf
>> bwctl: 30 seconds until test results available
>>
>> RECEIVER START
>> ------------------------------------------------------------
>> Server listening on TCP port 5122
>> Binding to local address 200.20.223.252
>> TCP window size: 0.08 MByte (default)
>> ------------------------------------------------------------
>> [ 15] local 200.20.223.252 port 5122 connected with 152.84.101.142 port
>> 53837
>> [ ID] Interval Transfer Bandwidth
>> [ 15] 0.0-10.1 sec 1103 MBytes 917 Mbits/sec
>> [ 15] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
>>
>> RECEIVER END
>>
>> Cheers,
>> Pedro Diniz
>>
>> From: Andrew Lake
>> []
>> Sent: Thursday, March 26, 2015 4:45 PM
>> To: Pedro Henrique Diniz Da Silva
>> Cc:
>>
>> Subject: Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????
>>
>>
>> On Mar 25, 2015, at 7:31 AM, Pedro Henrique Diniz Da Silva
>> <>
>> wrote:
>>
>>> Hi Andy,
>>>
>>> Two problems that I think that can be correlated:
>>>
>>> #1 - The traceroute data for those tests is being collected by an old
>>> mesh config that is not being held anymore, it is called "Traceroute Test
>>> Between WLCG Bandwidth Hosts". As these traceroutes tests are provided by
>>> a mesh config I don't know how to disable it, is there a way to do that?
>>>
>>
>> If you no longer want to participate in the mesh, you can remove the
>> <mesh></mesh> from
>> /opt/perfsonar_ps/mesh_config/etc/agent_configuration.conf. That being
>> said, I don't think this has anything to do with the problem you are
>> trying to debug in #2 (i.e. missin throughout data) other than its
>> clogging up your interface and making it harder to tell what throughput
>> tests are actually working.
>>
>>
>>
>>> #2 - The main problem is that I have other bw tests that have been held
>>> (e.g. perfsonar-bw.cern.ch, but it's not the only one) but is not being
>>> graphed. That's why I thought that cleaning the MA could solve the
>>> problem.
>>>
>>> * For example, based on "/var/log/perfsonar/owamp_bwctl.log" the
>>> connections seem to be working properly, right?
>>
>> Those log messages could be just from the bwtraceroute command, not
>> necessarily iperf. They also look more like other hosts connecting to you
>> rather than vice versa. Maybe you could send me
>> /opt/perfsonar_ps/regular_testing/etc/regular_testing.conf and
>> /var/log/perfsonar/regular_testing.log? That should hopefully give a
>> better idea of what your host is configured to run and what types of
>> results it gets when it tries.
>>
>>
>>
>>
>>>
>>> [gridlafex@ps02
>>> ~]$ cat /var/log/perfsonar/owamp_bwctl.log | grep "Mar 25" | grep "cern"
>>> Mar 25 00:14:57 ps02 bwctld[22298]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:56275)
>>> Mar 25 00:29:41 ps02 bwctld[24769]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:53194)
>>> Mar 25 00:29:42 ps02 bwctld[24769]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:53194)
>>> Mar 25 01:14:57 ps02 bwctld[483]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:46295)
>>> Mar 25 01:32:12 ps02 bwctld[4133]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:56909)
>>> Mar 25 01:32:12 ps02 bwctld[4133]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:56909)
>>> Mar 25 01:34:35 ps02 bwctld[4582]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:57052)
>>> Mar 25 01:34:36 ps02 bwctld[4582]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:57052)
>>> Mar 25 01:35:57 ps02 bwctld[4844]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:6183) from (perfsonar-bw.cern.ch:49859)
>>> Mar 25 01:35:57 ps02 bwctld[4844]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:6183) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:49859)
>>> Mar 25 02:14:57 ps02 bwctld[11916]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:36320)
>>> Mar 25 02:27:24 ps02 bwctld[14098]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:59846)
>>> Mar 25 02:27:24 ps02 bwctld[14098]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:59846)
>>> Mar 25 02:29:46 ps02 bwctld[14511]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:60246)
>>> Mar 25 02:29:46 ps02 bwctld[14511]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:60246)
>>> Mar 25 03:14:57 ps02 bwctld[22734]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:54659)
>>> Mar 25 03:27:40 ps02 bwctld[24944]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:35327)
>>> Mar 25 03:27:41 ps02 bwctld[24944]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:35327)
>>> Mar 25 03:31:55 ps02 bwctld[25690]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:36115)
>>> Mar 25 03:31:56 ps02 bwctld[25690]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:36115)
>>> Mar 25 03:35:34 ps02 bwctld[26319]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:6176) from (perfsonar-bw.cern.ch:43720)
>>> Mar 25 03:35:34 ps02 bwctld[26319]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:6176) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:43720)
>>> Mar 25 03:41:34 ps02 bwctld[27535]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:6154) from (perfsonar-bw.cern.ch:49137)
>>> Mar 25 03:41:35 ps02 bwctld[27535]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:6154) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:49137)
>>> Mar 25 04:14:57 ps02 bwctld[1208]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:44801)
>>> Mar 25 04:29:17 ps02 bwctld[4262]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:39159)
>>> Mar 25 04:29:18 ps02 bwctld[4262]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:39159)
>>> Mar 25 04:30:43 ps02 bwctld[4611]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:6172) from (perfsonar-bw.cern.ch:48749)
>>> Mar 25 04:30:43 ps02 bwctld[4611]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:6172) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:48749)
>>> Mar 25 04:31:07 ps02 bwctld[4690]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:39623)
>>> Mar 25 04:31:08 ps02 bwctld[4690]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:39623)
>>> Mar 25 05:14:57 ps02 bwctld[12735]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:38553)
>>> Mar 25 05:27:06 ps02 bwctld[14844]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:42857)
>>> Mar 25 05:27:06 ps02 bwctld[14844]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:42857)
>>> Mar 25 05:36:06 ps02 bwctld[16444]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:43875)
>>> Mar 25 05:36:06 ps02 bwctld[16444]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:43875)
>>> Mar 25 06:14:57 ps02 bwctld[23454]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:56949)
>>> Mar 25 06:27:30 ps02 bwctld[25552]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:46630)
>>> Mar 25 06:27:31 ps02 bwctld[25552]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:46630)
>>> Mar 25 06:38:10 ps02 bwctld[27431]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:6132) from (perfsonar-bw.cern.ch:35187)
>>> Mar 25 06:38:10 ps02 bwctld[27431]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:6132) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:35187)
>>> Mar 25 06:40:25 ps02 bwctld[27785]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:47871)
>>> Mar 25 06:40:25 ps02 bwctld[27785]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:47871)
>>> Mar 25 07:14:57 ps02 bwctld[1738]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-omd-dev.cern.ch:46971)
>>> Mar 25 07:32:51 ps02 bwctld[3893]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:50859)
>>> Mar 25 07:32:51 ps02 bwctld[3893]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:50859)
>>> Mar 25 07:43:01 ps02 bwctld[9198]: FILE=sapi.c, LINE=353, Connection to
>>> (ps02.cat.cbpf.br:4823) from (perfsonar-bw.cern.ch:51742)
>>> Mar 25 07:43:01 ps02 bwctld[9198]: FILE=sapi.c, LINE=510,
>>> ControlSession([ps02.cat.cbpf.br]:4823) accepted from
>>> userid(nil):([perfsonar-bw.cern.ch]:51742)
>>>
>>> * Based on "/var/log/esmond/esmond.log" it doesn't seem that ESMOND has
>>> problems.
>>>
>>> [gridlafex@ps02
>>> ~]$ cat /var/log/esmond/esmond.log | grep "2015-03-25"
>>> 2015-03-25 02:30:01,396 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 02:30:01,398 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 02:30:01,410 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 02:30:01,423 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 02:30:01,424 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 02:30:01,434 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:23:46,446 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:23:46,448 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:23:46,459 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:12,584 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:12,586 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:12,596 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:13,785 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:13,787 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:13,796 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:14,762 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:14,764 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:14,774 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:33,834 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:33,836 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:33,845 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:34,571 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:34,573 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:34,582 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:24:57,833 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:24:57,835 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:24:57,844 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:30:13,079 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:30:13,081 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:30:13,092 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:30:13,844 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:30:13,846 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:30:13,857 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:30:31,111 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:30:31,113 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:30:31,122 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:40:48,656 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:40:48,712 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:40:48,761 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:40:49,886 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:40:49,890 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:40:49,905 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>> 2015-03-25 07:46:30,561 [INFO] /opt/esmond/esmond/cassandra.py:
>>> Checking/creating column families
>>> 2015-03-25 07:46:30,564 [INFO] /opt/esmond/esmond/cassandra.py: Schema
>>> check done
>>> 2015-03-25 07:46:30,578 [INFO] /opt/esmond/esmond/cassandra.py: Connected
>>> to ['localhost:9160']
>>>
>>> * Is there any other place that we could look for errors?
>>>
>>> Thank you very much,
>>> Pedro Diniz
>>>
>>> From: Andrew Lake
>>> []
>>> Sent: Tuesday, March 24, 2015 10:04 AM
>>> To: Pedro Henrique Diniz Da Silva
>>> Cc:
>>>
>>> Subject: Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????
>>>
>>> Hi Pedro,
>>>
>>> It looks like the script worked correctly, but you are still collecting
>>> traceroute data for those tests. Using the example given there is no
>>> BWCTL or OWAMP data, but traceroute is still being updated. The graphs do
>>> provide a link to the traceroute data if available, so I think this is a
>>> subtlety of how that initial page determines what to display. You need to
>>> disable the traceroute tests and then you can either re-run the script in
>>> a few days or they will finally disappear on their own.
>>>
>>> Thanks,
>>> Andy
>>>
>>> On Mar 23, 2015, at 7:24 PM, Pedro Henrique Diniz Da Silva
>>> <>
>>> wrote:
>>>
>>>> Hi Andy,
>>>>
>>>> Yes, there was an extra comma :|
>>>>
>>>> I tried all the steps you told and it seemed that the data could be
>>>> cleaned.
>>>>
>>>> But even after doing all the steps, the error seems to be the same.
>>>>
>>>> * Nodes that we aren't doing more tests continue to appear on
>>>> http://ps02.cat.cbpf.br/serviceTest/psGraph.cgi
>>>>
>>>> * On the graph pages above, the error is still the same as following:
>>>> http://ps02.cat.cbpf.br/serviceTest/graphWidget.cgi?source=81.180.86.41&dest=152.84.101.142&url=http%3A%2F%2Flocalhost%2Fesmond%2Fperfsonar%2Farchive%2F
>>>>
>>>> Any idea?
>>>>
>>>> Cheers,
>>>> Pedro Diniz
>>>> From: Andrew Lake
>>>> []
>>>> Sent: Monday, March 23, 2015 11:06 AM
>>>> To: Pedro Henrique Diniz Da Silva
>>>> Cc:
>>>>
>>>> Subject: Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????
>>>>
>>>> Hi Pedro,
>>>>
>>>> That's my fault, there was an extra comma in the config file I sent you.
>>>> See attached for a valid file.
>>>>
>>>> Thanks,
>>>> Andy
>>
>> <regular_testing.conf><regular_testing.log>


  • Re: [perfsonar-user] How to clean esmond/perfsonar/archive ????, Aaron Brown, 04/01/2015

Archive powered by MHonArc 2.6.16.

Top of Page