perfSONAR User Q&A and Other Discussion

[Casey Russell <>]

Group,

     Sometime in the last few weeks, it appears something's broken in our perfsonar installation.  We can no longer view the jitter graphs for IPv4 and IPv6.

     When we click on the links in the main page, we see the normal looking mesh page (grid showing senders and receivers) but when we click in one of the boxes to show the graphs, we get the following error:

The image "http://ps-data.kanren.net/toolkit/gui/owamp_jitter/owampplot_histogram.cgi?sender=ps-ku-lt&sender_address=164.113.32.17&receiver=ps-ksu-lt&receiver_address=164.113.32.25&duration=120&plot_min=0&y_min=0&y_max=15" cannot be displayed because it contains errors.

I don't know if it's related, but we do see this in the owamp_bctl logs:

Sep  3 14:11:01 ps-bryant-lt powstream[18418]: _OWPReadFetchAck: Unable to read from socket.
Sep  3 14:11:01 ps-bryant-lt powstream[18418]: _OWPWriteFetchSession called in wrong state.
Sep  3 14:11:01 ps-bryant-lt last message repeated 10 times
Sep  3 14:11:01 ps-bryant-lt powstream[18411]: _OWPReadFetchAck: Unable to read from socket.
Sep  3 14:11:01 ps-bryant-lt powstream[18418]: _OWPWriteFetchSession called in wrong state.

I restarted services on the collection box (ours is a non-standard install with distributed PS nodes and a single data collection point), but it seemed to have no effect.

The error message seems to point to the inability to open a socket, or something else network related, And a "netstat" on one of the nodes shows open (ESTABLISHED) sessions on port 861 to each of the other PS nodes

I don't know enough about it PS and it's normal behaviors, to know if that's normal or where to start looking for the problem.

Any thoughts or ideas as to what might be causing the problems?

Casey Russell

Network Engineer

Kansas Research and Education Network

1405 Wakarusa Dr., Ste B
Lawrence, KS 66049

(785)856-9820  ext 9809