perfSONAR User Q&A and Other Discussion

[Zhi-Wei Lu <>]

Thanks Jason.

After I stopped the regular testing and  Cassandra, I still cannot use the 
"configure tests" at all, 

Gateway Time-out
The gateway did not receive a timely response from the upstream server or 
application.
Apache/2.2.15 (CentOS) Server at p51.noc.ucdavis.edu Port 443

The files in 
/var/lib/perfsonar/regular_testing/esmond_traceroute_localhost/failed/data/1/O
drwxrwxrwx 3 perfsonar perfsonar      4096 Dec 15 13:34 ..
-rw-rw-rw- 1 perfsonar perfsonar 536892791 Dec 16 09:32 
50.20141216173234486173.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar      6265 Dec 16 09:32 
50.20141216173247099266.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892828 Dec 16 09:33 
50.20141216173323571178.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892869 Dec 16 09:33 
50.20141216173353526886.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892819 Dec 16 09:33 
50.20141216173355090130.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892842 Dec 16 09:35 
50.20141216173536544323.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892805 Dec 16 09:35 
50.20141216173539200618.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892906 Dec 16 09:37 
50.20141216173749520175.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 805328325 Dec 16 09:37 
50.20141216173752649319.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:39 
50.20141216173957571983.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 805328326 Dec 16 09:40 
50.20141216174025363413.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892843 Dec 16 09:40 
50.20141216174045157625.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892879 Dec 16 09:44 
50.20141216174410299038.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892862 Dec 16 09:44 
50.20141216174425301435.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar      6265 Dec 16 09:44 
50.20141216174456831046.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar      5817 Dec 16 09:44 
50.20141216174459716445.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892915 Dec 16 09:46 
50.20141216174603118253.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892831 Dec 16 09:46 
50.20141216174631392221.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892831 Dec 16 09:46 
50.20141216174633547370.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:48 
50.20141216174841901446.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:49 
50.20141216174912131552.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892888 Dec 16 09:49 
50.20141216174933282328.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892888 Dec 16 09:49 
50.20141216174937306445.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892904 Dec 16 09:51 
50.20141216175133693082.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892904 Dec 16 09:52 
50.20141216175219732395.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892859 Dec 16 09:52 
50.20141216175235915105.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892859 Dec 16 09:54 
50.20141216175410064202.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:54 
50.20141216175411768689.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:54 
50.20141216175438239629.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892852 Dec 16 09:56 
50.20141216175617544033.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:57 
50.20141216175659829503.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:58 
50.20141216175822195902.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892825 Dec 16 09:58 
50.20141216175825088636.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:59 
50.20141216175901787207.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 09:59 
50.20141216175908392282.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892864 Dec 16 10:00 
50.20141216180048019864.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:00 
50.20141216180052676480.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:02 
50.20141216180234081082.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536893528 Dec 16 10:02 
50.20141216180241641765.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:03 
50.20141216180315834811.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:03 
50.20141216180327139412.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar 536892785 Dec 16 10:05 
50.20141216180517718112.EMjQ
-rw-rw-rw- 1 perfsonar perfsonar      1796 Dec 16 10:05 
50.20141216180522445352.EMjQ

It filled with 
...
            "address" : "64.124.161.242",
            "override_parameters" : {
               "type" : "bwtraceroute",
               "force_ipv4" : "1"
            },
            "description" : 
"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Tons of backslash, Do you know what process creates these *EMjQ files?  

Thank you.

Zhi-Wei Lu
IET-CR-Network Operations Center
University of California, Davis
(530) 752-0155

-----Original Message-----
From: Jason Zurawski 
[mailto:]
 
Sent: Monday, December 15, 2014 4:18 PM
To: Zhi-Wei Lu
Cc: 

Subject: Re: [perfsonar-user] personar fills up the disk in 
esmond_latency_localhost and esmond_latency_traceroute directories

Hi Zhi-Wei;

If you can get a terminal going, stop the regular testing daemon and 
cassandra:

> sudo /etc/init.d/regular_testing stop
> sudo /etc/init.d/cassandra stop

That should stop the ride long enough for you to make changes to the regular 
testing configuration.  

Thanks;

-jason

On Dec 15, 2014, at 5:03 PM, Zhi-Wei Lu 
<>
 wrote:

> Hi all,
> 
> We just noticed that one of our perfsonar box (3.4.2) was using a temporary 
> 192.168* address as its main address, thus all configured tests failed a 
> sometime.  I disabled the “internal testing” address and restarted the 
> system.  The system came back and it is busy generating files in above 
> directories ( */failed subdirectory).  Could someone advise us how we can 
> stop this downward spiral?
> 
> I could not login into “Configure Tests” as the system is busing writing 
> disks and filling up the disk quickly.
> 
> Thank you
> 
> Zhi-Wei Lu
> IET-CR-Network Operations Center
> University of California, Davis
> (530) 752-0155