perfSONAR User Q&A and Other Discussion

[Scot Colburn <>]

Hello Ivan,

Thanks for the reply.

We found that installing the perfsonar-toolkit-sysctl did NOT effect the TCP performance. 

It was only when the perfsonar-toolkit itself is installed that the TSO is turned off and the performance degrades.

We searched for the package that would cause the performance degradation by installing individual packages and testing after each one. We installed the prerequisites for perfsonar-toolkit hoping to find the one that caused the issue, but none did. It was only the final install of the toolkit itself invoked the problem. Here's that final (debian) installation invocation:

root@debian:/home/ansible-mgmt# apt install perfsonar-toolkit

Reading package lists... Done

Building dependency tree... Done

Reading state information... Done

The following NEW packages will be installed:

  perfsonar-toolkit

0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.

It is this final invocation that turns off TSO. I'm still curious about why TSO is turned off... 

The TSO is disabled by /etc/init.d/perfsonar-configure_nic_parameters : 

root@debian:~$ cat /etc/init.d/perfsonar-configure_nic_parameters

<SNIP>

# always set these on perfsonar hosts
DISABLE_TCP_OFFLOAD=1

<SNIP>

    if [ $DISABLE_TCP_OFFLOAD ]; then
        log_action_begin_msg "Disabling TCP offload for interface $interface"
        TSO_OFF $interface
        if [ $? -eq 0 ]; then
            log_action_end_msg $?
        else
            log_action_end_msg $?; ret=1
        fi
    fi

<SNIP>

--Scot

On Tue, Mar 18, 2025 at 7:46 AM Ivan Garnizov <> wrote:

Hello Scot,

 

The package in the pS Toolkit, which deals with system optimization is: perfsonar-toolkit-sysctl

 

I hope this is what you are looking for.

 

Regards,

Ivan

 

 

From: <> On Behalf Of Scot Colburn
Sent: Friday, 14 March 2025 01:38
To:
Subject: [perfsonar-user] Poor 10GE iperf3 performance after toolkit installed?

 

Hi Perfsonar gurus!

 

Why does the perfsonar-toolkit installation turn off TSO?

 

Our 10GE Debian perfsonars connected back-to-back get only 3-4 Gbps performance on 1500 byte MTUs after installing the perfsonar-toolkit. We get good 9.5 Gbps iperf3 TCP performance until the perfsonar-toolkit is installed. We can restore 9.5 Gbps performance by enabling TSO on the transmitting 10GE interface, like this:

root@debian:/home/ansible-mgmt# ethtool -K enp1s0f1 tso on
root@debian:/home/ansible-mgmt# ethtool -k enp1s0f1 | grep tcp-segmentation-offload
tcp-segmentation-offload: on

root@debian:/home/ansible-mgmt# ethtool --driver enp1s0f1
driver: i40e
version: 6.1.0-31-amd64
firmware-version: 9.00 0x8000d299 1.3256.0
expansion-rom-version:
bus-info: 0000:01:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Why is TSO turned off, anyway? I recall there used to be problems with TSO. Are there still? Are we covering up some other problem by turning it on?

 

--Scot

 

--

---
Scot Colburn

Network Engineer NSF NCAR/UCAR/NETS/FRGP

303-497-2452

--

---
Scot Colburn

Network Engineer NSF NCAR/UCAR/NETS/FRGP

303-497-2452