Skip to Content.
Sympa Menu

perfsonar-user - RE: [perfsonar-user] personar fills up the disk in esmond_latency_localhost and esmond_latency_traceroute directories

Subject: perfSONAR User Q&A and Other Discussion

List archive

RE: [perfsonar-user] personar fills up the disk in esmond_latency_localhost and esmond_latency_traceroute directories


Chronological Thread 
  • From: Zhi-Wei Lu <>
  • To: Aaron Brown <>
  • Cc: "" <>
  • Subject: RE: [perfsonar-user] personar fills up the disk in esmond_latency_localhost and esmond_latency_traceroute directories
  • Date: Fri, 19 Dec 2014 16:18:46 +0000
  • Accept-language: en-US
  • Authentication-results: spf=none (sender IP is ) ;

Hi Aaron,

I removed those files, once I start regular_testing, the disk will be filled
soon.

> "address" : "64.124.161.242",
> "override_parameters" : {
> "type" : "bwtraceroute",
> "force_ipv4" : "1"
> },
> "description" :
> "\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
>
> Tons of backslash, Do you know what process creates these *EMjQ files?
>

How could I get out of this self-destructive circle?

Thank you.

Zhi-Wei Lu
IET-CR-Network Operations Center
University of California, Davis
(530) 752-0155

-----Original Message-----
From: Aaron Brown
[mailto:]

Sent: Tuesday, December 16, 2014 10:40 AM
To: Zhi-Wei Lu
Cc:

Subject: Re: [perfsonar-user] personar fills up the disk in
esmond_latency_localhost and esmond_latency_traceroute directories

Hi,

The regular testing daemon creates those files whenever a test can’t be
written to esmond, so that it can be retried later. Note, this is different
from a test just failing. Whether a test fails or succeeds, something gets
written to esmond.

You should be able to remove them without issue, albeit you’ll lose whatever
data might have been in there.

Cheers,
Aaron

> On Dec 16, 2014, at 1:35 PM, Zhi-Wei Lu
> <>
> wrote:
>
> Thanks Jason.
>
> After I stopped the regular testing and Cassandra, I still cannot use the
> "configure tests" at all,
>
> Gateway Time-out
> The gateway did not receive a timely response from the upstream server or
> application.
> Apache/2.2.15 (CentOS) Server at p51.noc.ucdavis.edu Port 443
>
> The files in
> /var/lib/perfsonar/regular_testing/esmond_traceroute_localhost/failed/data/1/O
> drwxrwxrwx 3 perfsonar perfsonar 4096 Dec 15 13:34 ..
> -rw-rw-rw- 1 perfsonar perfsonar 536892791 Dec 16 09:32
> 50.20141216173234486173.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 6265 Dec 16 09:32
> 50.20141216173247099266.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892828 Dec 16 09:33
> 50.20141216173323571178.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892869 Dec 16 09:33
> 50.20141216173353526886.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892819 Dec 16 09:33
> 50.20141216173355090130.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892842 Dec 16 09:35
> 50.20141216173536544323.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892805 Dec 16 09:35
> 50.20141216173539200618.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892906 Dec 16 09:37
> 50.20141216173749520175.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 805328325 Dec 16 09:37
> 50.20141216173752649319.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:39
> 50.20141216173957571983.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 805328326 Dec 16 09:40
> 50.20141216174025363413.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892843 Dec 16 09:40
> 50.20141216174045157625.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892879 Dec 16 09:44
> 50.20141216174410299038.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892862 Dec 16 09:44
> 50.20141216174425301435.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 6265 Dec 16 09:44
> 50.20141216174456831046.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 5817 Dec 16 09:44
> 50.20141216174459716445.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892915 Dec 16 09:46
> 50.20141216174603118253.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892831 Dec 16 09:46
> 50.20141216174631392221.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892831 Dec 16 09:46
> 50.20141216174633547370.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:48
> 50.20141216174841901446.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:49
> 50.20141216174912131552.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892888 Dec 16 09:49
> 50.20141216174933282328.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892888 Dec 16 09:49
> 50.20141216174937306445.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892904 Dec 16 09:51
> 50.20141216175133693082.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892904 Dec 16 09:52
> 50.20141216175219732395.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892859 Dec 16 09:52
> 50.20141216175235915105.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892859 Dec 16 09:54
> 50.20141216175410064202.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:54
> 50.20141216175411768689.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:54
> 50.20141216175438239629.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:56
> 50.20141216175617544033.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:57
> 50.20141216175659829503.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:58
> 50.20141216175822195902.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:58
> 50.20141216175825088636.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:59
> 50.20141216175901787207.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:59
> 50.20141216175908392282.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 10:00
> 50.20141216180048019864.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:00
> 50.20141216180052676480.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:02
> 50.20141216180234081082.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:02
> 50.20141216180241641765.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:03
> 50.20141216180315834811.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:03
> 50.20141216180327139412.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:05
> 50.20141216180517718112.EMjQ
> -rw-rw-rw- 1 perfsonar perfsonar 1796 Dec 16 10:05
> 50.20141216180522445352.EMjQ
>
> It filled with
> ...
> "address" : "64.124.161.242",
> "override_parameters" : {
> "type" : "bwtraceroute",
> "force_ipv4" : "1"
> },
> "description" :
> "\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
>
> Tons of backslash, Do you know what process creates these *EMjQ files?
>
> Thank you.
>
> Zhi-Wei Lu
> IET-CR-Network Operations Center
> University of California, Davis
> (530) 752-0155
>
> -----Original Message-----
> From: Jason Zurawski
> [mailto:]
>
> Sent: Monday, December 15, 2014 4:18 PM
> To: Zhi-Wei Lu
> Cc:
>
> Subject: Re: [perfsonar-user] personar fills up the disk in
> esmond_latency_localhost and esmond_latency_traceroute directories
>
> Hi Zhi-Wei;
>
> If you can get a terminal going, stop the regular testing daemon and
> cassandra:
>
>> sudo /etc/init.d/regular_testing stop
>> sudo /etc/init.d/cassandra stop
>
> That should stop the ride long enough for you to make changes to the
> regular testing configuration.
>
> Thanks;
>
> -jason
>
> On Dec 15, 2014, at 5:03 PM, Zhi-Wei Lu
> <>
> wrote:
>
>> Hi all,
>>
>> We just noticed that one of our perfsonar box (3.4.2) was using a
>> temporary 192.168* address as its main address, thus all configured tests
>> failed a sometime. I disabled the “internal testing” address and
>> restarted the system. The system came back and it is busy generating
>> files in above directories ( */failed subdirectory). Could someone advise
>> us how we can stop this downward spiral?
>>
>> I could not login into “Configure Tests” as the system is busing writing
>> disks and filling up the disk quickly.
>>
>> Thank you
>>
>> Zhi-Wei Lu
>> IET-CR-Network Operations Center
>> University of California, Davis
>> (530) 752-0155




Archive powered by MHonArc 2.6.16.

Top of Page