perfSONAR User Q&A and Other Discussion

[Brian Candler <>]

On 16/02/2016 15:36, Garnizov, Ivan (RRZE) wrote:

Hi Brian,

 

This is very much clear from your example.

“For one particular burst, the source port is all the same. It might be for example:
source port 36334 -> dest port 44445
source port 36334 -> dest port 44446
source port 36334 -> dest port 44447
source port 36334 -> dest port 44448
... etc
”

 

I read it as: Sender is using UDP port 36334, on reply back use UDP port 36334 as destination. Where do you suggest the intermediate routers should reply?

 

That's not the problem. At the sending side, iptables has a "permit anything outbound" policy which creates state, and the response packets will be allowed in as part of an "ESTABLISHED" session.

This is as per the link you sent:
http://www.iptables.info/en/connection-state.html#UDPCONNECTIONS

1)      If your destination host firewall rejects the packages to 4444?, then a reply is formed by the end host.

If the destination iptables rejects the packets to 4444?, then it will send back either an ICMP "Admin Prohibited" or "port unreachable" message to the sender. This is not too bad, except I don't think it will be helpful in determining the MTU of the last hop (i.e. I'm guessing that "Admin Prohibited" takes precendence over "fragmentation needed")

I have now opened ports >44444 on the Cisco firewall in front of the server.

Testing from 192.168.5.5 to 10.0.0.4:

$ tracepath -n 10.0.0.4
 1?: [LOCALHOST]     pmtu 1500
 1:  192.168.5.1       0.511ms pmtu 1426
 1:  10.0.0.4          4.035ms reached
     Resume: pmtu 1426 hops 1 back 1

During this time, tcpdump at the sending side shows:

16:01:01.561019 IP 192.168.5.5.38706 > 10.0.0.4.44444: UDP, length 1472
16:01:01.561443 IP 192.168.5.1 > 192.168.5.5: ICMP 10.0.0.4 unreachable - need to frag (mtu 1426), length 36
16:01:01.561557 IP 192.168.5.5.38706 > 10.0.0.4.44445: UDP, length 1398
16:01:01.564267 IP 10.0.0.4 > 192.168.5.5: ICMP 10.0.0.4 udp port 44445 unreachable, length 556

Hmm. "port unreachable" could also mean the packet passed the firewall. Let me check the rules:

# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
perfSONAR  all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain perfSONAR (1 references)
target     prot opt source               destination
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           icmp type 255
ACCEPT     icmpv6--  0.0.0.0/0            0.0.0.0/0
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:80 state NEW,ESTABLISHED
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:443 state NEW,ESTABLISHED
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:123 udp
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:8090
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:33434:33634
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:8000
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:8001:8020
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:843
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:7123
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:3001:3003
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:861
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:8760:9960
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:4823
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:6001:6200
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5000:5900
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5000:5900
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5001:5300
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5001:5300
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5001:5300
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5001:5300
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5301:5600
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5301:5600
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5301:5600
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5301:5600
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5601:5900
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5601:5900
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:5601:5900
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:5601:5900
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:6001:6200
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:6001:6200
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:8760:9960
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:8760:9960
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpts:8760:9960
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpts:8760:9960
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

Very strange, the policy is ACCEPT on all chains and there are no REJECTs... this means the iptables firewall is useless!

But when I look on an older perfsonar box (installed much earlier, Jan 2015) I see:

# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  52.19.226.164        0.0.0.0/0           tcp dpt:10050 state NEW,ESTABLISHED
fail2ban-SSH  tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:22
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW,ESTABLISHED tcp dpt:22
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           state NEW udp spt:547 dpt:546
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8080
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:5060
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:8080
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:5060
perfSONAR  all  --  0.0.0.0/0            0.0.0.0/0
REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain fail2ban-SSH (1 references)
target     prot opt source               destination
REJECT     all  --  202.106.211.99       0.0.0.0/0           reject-with icmp-port-unreachable
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

... followed by the perfSONAR chain.

So for some reason, the more recently installed boxes have not activated their main firewall policies. They were just direct installs from the perfsonar installer on USB, whatever was current in November last year.

I installed 7 perfsonar boxes then. I've just checked four of them and they are all the same: perfSONAR chain exists but with almost-empty INPUT chain and ACCEPT policy.

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
perfSONAR  all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

All have auto-updates enabled. So it looks like this is something I'll have to check this on every new perfsonar install, as well as working out how to fix these ones :-(

Regards,

Brian.