Hi Brian,
See comments inline: On February 16, 2016 at 11:15:54 AM, Brian Candler () wrote: On 16/02/2016 15:36, Garnizov, Ivan (RRZE) wrote:
Hi Brian, This is very much clear from your example. “For one particular burst, the source port is all the same. It might be for example:
source port 36334 -> dest port 44445
source port 36334 -> dest port 44446
source port 36334 -> dest port 44447
source port 36334 -> dest port 44448
... etc
” I read it as: Sender is using UDP port 36334, on reply back use UDP port 36334 as destination. Where do you suggest the intermediate routers should reply?
That's not the problem. At the sending side, iptables has a "permit anything outbound" policy which creates state, and the response packets will be allowed in as part of an "ESTABLISHED" session. This is as per the link you sent: http://www.iptables.info/en/connection-state.html#UDPCONNECTIONS1) If your destination host firewall rejects the packages to 4444?, then a reply is formed by the end host.
If the destination iptables rejects the packets to 4444?, then it will send back either an ICMP "Admin Prohibited" or "port unreachable" message to the sender. This is not too bad, except I don't think it will be helpful in determining the MTU of the last hop (i.e. I'm guessing that "Admin Prohibited" takes precendence over "fragmentation needed") I have now opened ports >44444 on the Cisco firewall in front of the server. Testing from 192.168.5.5 to 10.0.0.4: $ tracepath -n 10.0.0.4 1?: [LOCALHOST] pmtu 1500 1: 192.168.5.1 0.511ms pmtu 1426 1: 10.0.0.4 4.035ms reached Resume: pmtu 1426 hops 1 back 1 During this time, tcpdump at the sending side shows: 16:01:01.561019 IP 192.168.5.5.38706 > 10.0.0.4.44444: UDP, length 1472 16:01:01.561443 IP 192.168.5.1 > 192.168.5.5: ICMP 10.0.0.4 unreachable - need to frag (mtu 1426), length 36 16:01:01.561557 IP 192.168.5.5.38706 > 10.0.0.4.44445: UDP, length 1398 16:01:01.564267 IP 10.0.0.4 > 192.168.5.5: ICMP 10.0.0.4 udp port 44445 unreachable, length 556 Hmm. "port unreachable" could also mean the packet passed the firewall. Let me check the rules:
This is to be expected and just a function of how tracepath works. In some ways this is the equivalent of a success message. There is nothing listening on the UDP port so you will get an unreachable message. What tracepath cares more about is whether you get something else like an MTU exceeded message or similar. I am not sure if the “prohibit” message is preventing informational messages like that from getting back or not, hence the open question as to whether we really “need” to open the ports for tracepath to function at it’s full capability or not.
# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
perfSONAR all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain perfSONAR (1 references)
target prot opt source destination
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 255
ACCEPT icmpv6-- 0.0.0.0/0 0.0.0.0/0
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 state NEW,ESTABLISHED
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 state NEW,ESTABLISHED
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:123 udp
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:8090
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:33434:33634
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:8000
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:8001:8020
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:843
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:7123
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:3001:3003
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:861
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:8760:9960
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:4823
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:6001:6200
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5000:5900
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5000:5900
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5001:5300
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5001:5300
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5001:5300
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5001:5300
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5301:5600
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5301:5600
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5301:5600
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5301:5600
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5601:5900
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5601:5900
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:5601:5900
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:5601:5900
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:6001:6200
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:6001:6200
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:6001:6200
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:8760:9960
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:8760:9960
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpts:8760:9960
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpts:8760:9960
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Very strange, the policy is ACCEPT on all chains and there are no REJECTs... this means the iptables firewall is useless!
But when I look on an older perfsonar box (installed much earlier, Jan 2015) I see:
# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- 52.19.226.164 0.0.0.0/0 tcp dpt:10050 state NEW,ESTABLISHED
fail2ban-SSH tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:22
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 state NEW udp spt:547 dpt:546
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:8080
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:5060
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 state NEW udp dpt:8080
ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 state NEW udp dpt:5060
perfSONAR all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain fail2ban-SSH (1 references)
target prot opt source destination
REJECT all -- 202.106.211.99 0.0.0.0/0 reject-with icmp-port-unreachable
RETURN all -- 0.0.0.0/0 0.0.0.0/0
... followed by the perfSONAR chain.
So for some reason, the more recently installed boxes have not activated their main firewall policies. They were just direct installs from the perfsonar installer on USB, whatever was current in November last year.
I installed 7 perfsonar boxes then. I've just checked four of them and they are all the same: perfSONAR chain exists but with almost-empty INPUT chain and ACCEPT policy.
Chain INPUT (policy ACCEPT)
target prot opt source destination
perfSONAR all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
All have auto-updates enabled. So it looks like this is something I'll have to check this on every new perfsonar install, as well as working out how to fix these ones :-(
Somebody else reported this as well a couple days ago and I have seen it before as well. Basically when the ISO is installing, the netfilters kernel module is not loaded so the post step to setup the rules is failing. I wonder if their was a recent update in anaconda or some other package that has recently made this surface, because it was definitely fine when the 3.5 release was made. You can run "/opt/perfsonar_ps/toolkit/scripts/configure_firewall install” to setup the rules. We are currently testing 3.5.1 of the toolkit which installs a boot script to run that command as well in case the initial install fails to do so.
|
