perfSONAR User Q&A and Other Discussion

[Raul Lopes <>]

Hi,

Thanks for your quick reply.

pscheduler result doesn't give any hint of error. It's actually my goto and I hadn't seen anything and I don't see now.

I have checked a few urls in two and I don't see errors. And, yet, they are not publishing.

======================================

2021-07-06T11:52:13Z on ps01-ban.rl.ac.uk and ps19-ban.chobs.rl.ac.uk with

  iperf3:

throughput --source ps01-ban.rl.ac.uk --dest ps19-ban.chobs.rl.ac.uk --duration

  PT30S

* Stream ID 5

Interval       Throughput     Retransmits    Current Window

0.0 - 1.0      458.00 Mbps    1014           698.54 KBytes  

1.0 - 2.0      891.29 Mbps    0              880.11 KBytes  

2.0 - 3.0      901.78 Mbps    0              1.03 MBytes    

3.0 - 4.0      639.63 Mbps    4              361.75 KBytes  

4.0 - 5.0      524.29 Mbps    0              539.15 KBytes  

5.0 - 6.0      754.98 Mbps    0              729.04 KBytes  

6.0 - 7.0      901.78 Mbps    0              999.31 KBytes  

7.0 - 8.0      796.92 Mbps    4              677.75 KBytes  

8.0 - 9.0      880.79 Mbps    0              862.09 KBytes  

9.0 - 10.0     901.78 Mbps    0              1.02 MBytes    

10.0 - 11.0    618.66 Mbps    4              363.13 KBytes  

11.0 - 12.0    534.77 Mbps    0              539.15 KBytes  

12.0 - 13.0    744.49 Mbps    0              726.26 KBytes  

13.0 - 14.0    901.78 Mbps    0              997.92 KBytes  

14.0 - 15.0    786.43 Mbps    2              676.37 KBytes  

15.0 - 16.0    880.80 Mbps    0              860.71 KBytes  

16.0 - 17.0    891.29 Mbps    0              1.02 MBytes    

17.0 - 18.0    754.98 Mbps    92             668.05 KBytes  

18.0 - 19.0    870.32 Mbps    0              852.39 KBytes  

19.0 - 20.0    901.77 Mbps    0              1.01 MBytes    

20.0 - 21.0    796.91 Mbps    4              662.51 KBytes  

21.0 - 22.0    870.34 Mbps    0              846.85 KBytes  

22.0 - 23.0    901.78 Mbps    0              1.01 MBytes    

23.0 - 24.0    817.89 Mbps    2              644.49 KBytes  

24.0 - 25.0    859.79 Mbps    0              831.60 KBytes  

25.0 - 26.0    901.82 Mbps    0              990.99 KBytes  

26.0 - 27.0    786.38 Mbps    11             630.63 KBytes  

27.0 - 28.0    849.40 Mbps    0              819.13 KBytes  

28.0 - 29.0    901.78 Mbps    0              981.29 KBytes  

29.0 - 30.0    817.88 Mbps    1              623.70 KBytes  

Summary

Interval       Throughput     Retransmits    Receiver Throughput

0.0 - 30.0     801.35 Mbps    1138           799.10 Mbps

Archivings:

  To esmond, Finished

    2021-07-06T12:52:52+01:00 Succeeded

---

From: Mark Feit <>
Sent: 06 July 2021 14:25
To: Raul Lopes <>; <>
Subject: Re: Node not reporting to Maddash

 

Raul Lopes writes:

 

 I have a couple of nodes in a 20 nodes mesh that run the tests, but do not report to Maddash.
 
  The logs are full of lines like
 
  Jul  5 19:19:25 ps03-em1 journal: runner INFO     489461: Posted result to https://ps03-lat.rl.ac.uk/pscheduler/tasks/0203bb86-10af-4d77-a607-a51de117f9cf/runs/defa4e86-a17
  7-41a5-aebc-e1256683bd07
 
  I can use pscheduler on the command line and get latency and bandwidth run with success. However, they do not send anything to Maddash. There is nothing going from that host to the Maddash server.

 

The usual first step would be to grab a few of the run URLs for the posted results and ask pScheduler how it thinks the archvings went:

 

pscheduler result --archivings RUN-URL

 

At the bottom, you’ll see something that looks like this that should provide diagnostic information like this contrived example:

 

Archivings:

 

  To failer, Unfinished

    2021-07-06T13:05:51Z      Possibly-permanent archiver error: Program failed to start after 3 tries: Program exited 1: Forced to fail badly.

 

 

However also shows

 

2021-07-06 09:58:14,944 [INFO] /usr/lib/python3.6/site-packages/pycassa/logging/pool_logger.py: Connection 139755773060824 (localhost:9160) in pool 139755772886656 failed:
TSocket read 0 bytes

 

esmond/django.log has an error

 

2021-07-06 10:50:25,567 [ERROR] /usr/lib/esmond/lib/python3.6/site-packages/django/core/handlers/exception.py: Internal Server Error: /esmond/perfsonar/archive/4dbdcd5297fa41eaa01b5474ff5b186f/

Traceback (most recent call last):

…

  File "/usr/lib64/python3.6/site-packages/thrift/transport/TSocket.py", line 143, in read

    message='TSocket read 0 bytes')

thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

Has anyone seen a situation like that?

 

I haven’t seen that specifically, but zero-byte reads are often a sign that the server closed the connection while the client was expecting it to send something.  Andy might have something more specific since he’s our Esmond maven.  He’s just back from vacation and should get to this while digging out from under a week’s worth of mail.

 

--Mark

 

Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under company number. 05747339, VAT number GB 197 0632 86. Jisc’s registered office is: 4 Portwall Lane, Bristol, BS1 6NB. T 0203 697 5800.

Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under company number 02881024, VAT number GB 197 0632 86. The registered office is: 4 Portwall Lane, Bristol, BS1 6NB. T 0203 697 5800.

Jisc Commercial Limited is a wholly owned Jisc subsidiary and a company limited by shares which is registered in England under company number 09316933, VAT number GB 197 0632 86. The registered office is: 4 Portwall Lane, Bristol, BS1 6NB. T 0203 697 5800.

For more details on how Jisc handles your data see our privacy notice here: https://www.jisc.ac.uk/website/privacy-notice