perfSONAR User Q&A and Other Discussion

[Antoine Delvaux <>]

I think another quick way to optimise the iptables rules, and still use 
connection tracking, is to move the ESTABLISHED state tracking as the very 
first single rule of the set and disregard port number in this rule.

This very first rule (even before the ICMP rule) would be:

-A perfSONAR -m state --state ESTABLISHED -j ACCEPT

and all following rules will not use the ESTABLISHED state, like:

-A perfSONAR -m tcp -p tcp --dport 80 -m state --state NEW -j ACCEPT

up to

-A perfSONAR -m state --state NEW -m tcp -p tcp --dport 5000:5900 -j ACCEPT

That way, most packets will go through the very first rule and only the new 
connections will need to go through additional checks.

I don’t have a Liva to try this, but it would be worth exploring.

BTW, if anybody has more understanding of what is happening in the linux 
kernel with the iptables rules and the limited throughput on low end devices, 
it might be a good topic to submit to this BoF session at netdev 1.1: 
http://www.netdevconf.org/1.1/bof-network-performance-bof-jesper-dangaard-brouer.html

Antoine.

> Le 5 janv. 2016 à 00:12, Ed Brown 
> <>
>  a écrit :
> 
> I'm not sure there is penalty for just having the conntrack modules loaded, 
> but I've definitely seen significant throughput improvements in another 
> context and in perfsonar testing, on regular server hardware, by 
> eliminating _unnecessary_ iptables STATE checks in the rules. Eliminating 
> iptables rules altogether can further improve throughput, but may not be an 
> option or a good option.
> 
> Beyond Mark's suggestions in his bug report, I'd suggest that there is 
> little real security value in having state checks (-m state ...) AT ALL in 
> the ruleset ( 
> https://github.com/perfsonar/toolkit/blob/master/etc/perfsonar_firewall_settings.conf
>  as referenced by Mark in his bug report ), and that eliminating them will 
> have the most impact.  They basically only exclude INVALID traffic, and 
> [perhaps incorrectly] RELATED traffic.  Dropping INVALID traffic might make 
> some sense in certain situations (when it might be a way past source 
> restrictions), but when NEW traffic is being permitted from anywhere 
> anyway, it buys very little (imho).
> 
> Maybe a little harder to script than concatenating individual rules, but 
> another optimization might be building multiport rules for TCP and UDP, e.g:
> 
> -A perfSONAR -p tcp -m multiport --dports 
> 80,443,843,861,3001:3003,4823,5000:5900,6001:6200,7123,8000:8020,8090 -j 
> ACCEPT
> 
> -A perfSONAR -p udp -m multiport --dports 
> 123,5000:5900,6001:6200,8760:9960,33434:33634 -j ACCEPT
> 
> Ed
> 
> 
> On 01/04/2016 01:52 PM, Hyojoon Kim wrote:
>> Hello Mark,
>> 
>> Thank you for this explanation!
>> 
>> One possible way to have *some* level of security could be through just
>> using stateless iptables rules. Adding *stateless* iptables rules does
>> not enable the nf_conntrack kernel modules, thus does not degrade the bw
>> performance of the LIVA box (i.e., iptables rules without '-m state'
>> option).
>> 
>> However, I am not sure if stateless iptables rules are enough and what
>> are the consequences when the box only has stateless rules :-P
>> 
>> Thanks,
>> Joon
>> 
>>> On Jan 4, 2016, at 10:43 AM, Mark Feit 
>>> <
>>> <mailto:>>
>>>  wrote:
>>> 
>>> Hyojoon Kim writes:
>>> 
>>>    “lsmod" command shows that nf_conntrack-related kernel modules
>>>    become loaded after the perfsonar security package installation,
>>>    along with several iptables rules. After this happens, bwctl
>>>    measurement will never go over 700Mbps when the test is initiated
>>>    from the LIVA X box. Flushing the iptables rules and disabling the
>>>    nf_conntrack-related modules fixes the issue, restoring the bwctl
>>>    measurement to 940Mbps.
>>> 
>>>    Can someone confirm this?
>>> 
>>> 
>>> I can’t confirm it for the LIVA because I don’t have one, but iptables
>>> is well-known to be a drag in high-traffic situations.  You won’t
>>> notice this as much on bigger machines because they tend to have
>>> enough horsepower that the extra processing time doesn’t matter.  Your
>>> change effectively disables all of it, so the increase in throughput
>>> makes perfect sense.  The tradeoff is that the machine is no longer
>>> protected from the outside.
>>> 
>>> Iptables processes the rules in each chain in the order they appear,
>>> and the perfSONAR chain isn't arranged to get time- and rate-sensitive
>>> traffic processed as quickly as possible.
>>> 
>>> I wrote a bug on this: https://github.com/perfsonar/toolkit/issues/82.
>>> 
>>> —Mark