perfSONAR User Q&A and Other Discussion

[Doug Wussler <>]

I posted about this a couple weeks ago.  My system throws the same error because the

card does not support a value of 0 for the rx-usecs parameter.  Since a value of 0 means

to use the value of rx-frames, I looked up that value and then set rx-usecs to that value.

so on my system, the solution is to modify the script in two places as shown between the

asterisks.

 

    if [ $DISABLE_INTERRUPT_COALESCING ]; then

        # test if this interface supports IC adjustments

        /sbin/ethtool -c $interface > /dev/null 2>&1

        RETVAL=$?

        if [ $RETVAL -eq 0 ]; then

            # apply multi-ethtool settings

            echo -n $"disabling interrupt coalescing for interface $interface: "

            ######################################################

            # 2017.09.29 - Doug Wussler - See comment below in IC_OFF_em subroutine

            if echo $interface | grep -q '^em[12]$'; then

                IC_OFF_em $interface

            else

                IC_OFF $interface

            fi

            ######################################################

            if [ $? -eq 0 ]; then

                success; echo

            else

                failure; echo; ret=1

            fi

        fi

    fi

 

###########################################################################

# 2017.09.29 - Doug Wussler - The Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe

# card does not support a value of 0 for the rx-usecs parameter,

# which says to use the value of rx-frames, so we must set it to 1.

IC_OFF_em() {

IC_RET_em=0

for i in "$1"; do

    if ! /usr/sbin/ethtool -c $i | grep -q '^rx-frames: 1$'; then

         /usr/sbin/ethtool -C $i rx-frames 1 || IC_RET_em=$?

    fi

 

    if ! /usr/sbin/ethtool -c $i | grep -q '^rx-usecs: 1$'; then

         /usr/sbin/ethtool -C $i rx-usecs 1  || IC_RET_em=$?

    fi

done

return $IC_RET_em

}

###########################################################################

 

 

Doug Wussler

Florida State University

 

From: <> on behalf of dan dev <>
Date: Saturday, January 13, 2018 at 3:12 AM
To: Valentin Vidic <>
Cc: "" <>
Subject: Re: Re: [perfsonar-user] Installation errors in perfSONAR-toolkit

 

Hi Valentin,

 

Here is the output for the command

 

# SYSTEMCTL_SKIP_REDIRECT=1 bash -x /etc/init.d/perfsonar-configure_nic_parameters start

 

 

+ . /lib/lsb/init-functions

+++ run-parts --lsbsysinit --list /lib/lsb/init-functions.d

++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d 2>/dev/null)'

++ '[' -r /lib/lsb/init-functions.d/01-upstart-lsb ']'

++ . /lib/lsb/init-functions.d/01-upstart-lsb

+++ unset UPSTART_SESSION

+++ _RC_SCRIPT=/etc/init.d/perfsonar-configure_nic_parameters

+++ '[' -r /etc/init//etc/init.d/perfsonar-configure_nic_parameters.conf ']'

+++ _UPSTART_JOB=perfsonar-configure_nic_parameters

+++ '[' -r /etc/init/perfsonar-configure_nic_parameters.conf ']'

++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d 2>/dev/null)'

++ '[' -r /lib/lsb/init-functions.d/20-left-info-blocks ']'

++ . /lib/lsb/init-functions.d/20-left-info-blocks

++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d 2>/dev/null)'

++ '[' -r /lib/lsb/init-functions.d/40-systemd ']'

++ . /lib/lsb/init-functions.d/40-systemd

+++ _use_systemctl=0

+++ '[' -d /run/systemd/system ']'

+++ prog=perfsonar-configure_nic_parameters

+++ service=perfsonar-configure_nic_parameters.service

++++ systemctl -p LoadState show perfsonar-configure_nic_parameters.service

+++ state=LoadState=loaded

+++ '[' LoadState=loaded = LoadState=masked ']'

+++ '[' 18266 -ne 1 ']'

+++ '[' -z '' ']'

+++ '[' -z 1 ']'

+++ export _SYSTEMCTL_SKIP_REDIRECT=true

+++ _SYSTEMCTL_SKIP_REDIRECT=true

+++ '[' 0 = 1 ']'

++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d 2>/dev/null)'

++ '[' -r /lib/lsb/init-functions.d/50-ubuntu-logging ']'

++ . /lib/lsb/init-functions.d/50-ubuntu-logging

+++ LOG_DAEMON_MSG=

++ for hook in '$(run-parts --lsbsysinit --list /lib/lsb/init-functions.d 2>/dev/null)'

++ '[' -r /lib/lsb/init-functions.d/99-plymouth ']'

++ . /lib/lsb/init-functions.d/99-plymouth

+++ plymouth --ping

+++ return

++ FANCYTTY=

++ '[' -e /etc/lsb-base-logging.sh ']'

++ true

+ DISABLE_TCP_OFFLOAD=1

+ DISABLE_INTERRUPT_COALESCING=1

+ TXQUEUELEN=10000

++ /sbin/ifquery --all --list

++ grep -v -w lo

+ interfaces='eth0

eth1

xenbr0

xenbr1

br-ex'

++ /sbin/ifquery --allow hotplug --list

+ interfaces='eth0

eth1

xenbr0

xenbr1

br-ex '

+ case "$1" in

+ start

+ ret=0

+ for interface in '$interfaces'

+ /sbin/ip link show eth0

+ '[' 0 -ne 0 ']'

+ '[' 1 ']'

+ /sbin/ethtool -c eth0

+ RETVAL=0

+ '[' 0 -eq 0 ']'

+ log_action_begin_msg 'Disabling interrupt coalescing for interface eth0'

+ log_daemon_msg 'Disabling interrupt coalescing for interface eth0...'

+ '[' -z 'Disabling interrupt coalescing for interface eth0...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z ']'

+ FANCYTTY=1

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Disabling interrupt coalescing for interface eth0...       '

 * Disabling interrupt coalescing for interface eth0...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + IC_OFF eth0

+ IC_RET=0

+ for i in '"$1"'

+ /sbin/ethtool -c eth0

+ grep -q '^rx-frames: 1$'

+ /sbin/ethtool -C eth0 rx-frames 1

Cannot set device coalesce parameters: Unknown error 524

+ IC_RET=81

+ /sbin/ethtool -c eth0

+ grep -q '^rx-usecs: 0$'

+ return 81

+ '[' 81 -eq 0 ']'

+ log_action_end_msg 1

+ log_end_msg 1

+ '[' -z 1 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 1 -eq 0 ']'

+ printf '['

[+ /usr/bin/tput setaf 1

+ printf fail

fail+ /usr/bin/tput op

+ echo ']'

]

+ return 1

+ true

+ ret=1

+ '[' 1 ']'

+ log_action_begin_msg 'Disabling TCP offload for interface eth0'

+ log_daemon_msg 'Disabling TCP offload for interface eth0...'

+ '[' -z 'Disabling TCP offload for interface eth0...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Disabling TCP offload for interface eth0...       '

 * Disabling TCP offload for interface eth0...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TSO_OFF eth0

+ /sbin/ethtool -K eth0 tso off

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ '[' 10000 -ne 0 ']'

+ log_action_begin_msg 'Setting txqueuelen for eth0 to 10000'

+ log_daemon_msg 'Setting txqueuelen for eth0 to 10000...'

+ '[' -z 'Setting txqueuelen for eth0 to 10000...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Setting txqueuelen for eth0 to 10000...       '

 * Setting txqueuelen for eth0 to 10000...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TXQUEUELEN_SET eth0 10000

+ /sbin/ip link set eth0 txqueuelen 10000

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ for interface in '$interfaces'

+ /sbin/ip link show eth1

+ '[' 0 -ne 0 ']'

+ '[' 1 ']'

+ /sbin/ethtool -c eth1

+ RETVAL=0

+ '[' 0 -eq 0 ']'

+ log_action_begin_msg 'Disabling interrupt coalescing for interface eth1'

+ log_daemon_msg 'Disabling interrupt coalescing for interface eth1...'

+ '[' -z 'Disabling interrupt coalescing for interface eth1...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Disabling interrupt coalescing for interface eth1...       '

 * Disabling interrupt coalescing for interface eth1...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + IC_OFF eth1

+ IC_RET=0

+ for i in '"$1"'

+ /sbin/ethtool -c eth1

+ grep -q '^rx-frames: 1$'

+ /sbin/ethtool -C eth1 rx-frames 1

Cannot set device coalesce parameters: Unknown error 524

+ IC_RET=81

+ /sbin/ethtool -c eth1

+ grep -q '^rx-usecs: 0$'

+ return 81

+ '[' 81 -eq 0 ']'

+ log_action_end_msg 1

+ log_end_msg 1

+ '[' -z 1 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 1 -eq 0 ']'

+ printf '['

[+ /usr/bin/tput setaf 1

+ printf fail

fail+ /usr/bin/tput op

+ echo ']'

]

+ return 1

+ true

+ ret=1

+ '[' 1 ']'

+ log_action_begin_msg 'Disabling TCP offload for interface eth1'

+ log_daemon_msg 'Disabling TCP offload for interface eth1...'

+ '[' -z 'Disabling TCP offload for interface eth1...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Disabling TCP offload for interface eth1...       '

 * Disabling TCP offload for interface eth1...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TSO_OFF eth1

+ /sbin/ethtool -K eth1 tso off

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ '[' 10000 -ne 0 ']'

+ log_action_begin_msg 'Setting txqueuelen for eth1 to 10000'

+ log_daemon_msg 'Setting txqueuelen for eth1 to 10000...'

+ '[' -z 'Setting txqueuelen for eth1 to 10000...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Setting txqueuelen for eth1 to 10000...       '

 * Setting txqueuelen for eth1 to 10000...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TXQUEUELEN_SET eth1 10000

+ /sbin/ip link set eth1 txqueuelen 10000

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ for interface in '$interfaces'

+ /sbin/ip link show xenbr0

+ '[' 1 -ne 0 ']'

+ continue

+ for interface in '$interfaces'

+ /sbin/ip link show xenbr1

+ '[' 1 -ne 0 ']'

+ continue

+ for interface in '$interfaces'

+ /sbin/ip link show br-ex

+ '[' 0 -ne 0 ']'

+ '[' 1 ']'

+ /sbin/ethtool -c br-ex

+ RETVAL=82

+ '[' 82 -eq 0 ']'

+ '[' 1 ']'

+ log_action_begin_msg 'Disabling TCP offload for interface br-ex'

+ log_daemon_msg 'Disabling TCP offload for interface br-ex...'

+ '[' -z 'Disabling TCP offload for interface br-ex...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Disabling TCP offload for interface br-ex...       '

 * Disabling TCP offload for interface br-ex...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TSO_OFF br-ex

+ /sbin/ethtool -K br-ex tso off

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ '[' 10000 -ne 0 ']'

+ log_action_begin_msg 'Setting txqueuelen for br-ex to 10000'

+ log_daemon_msg 'Setting txqueuelen for br-ex to 10000...'

+ '[' -z 'Setting txqueuelen for br-ex to 10000...' ']'

+ log_use_fancy_output

+ TPUT=/usr/bin/tput

+ EXPR=/usr/bin/expr

+ '[' -t 1 ']'

+ '[' xxterm-256color '!=' x ']'

+ '[' xxterm-256color '!=' xdumb ']'

+ '[' -x /usr/bin/tput ']'

+ '[' -x /usr/bin/expr ']'

+ /usr/bin/tput hpa 60

+ /usr/bin/tput setaf 1

+ '[' -z 1 ']'

+ true

+ case "$FANCYTTY" in

+ true

+ /usr/bin/tput xenl

++ /usr/bin/tput cols

+ COLS=80

+ '[' 80 ']'

+ '[' 80 -gt 6 ']'

++ /usr/bin/expr 80 - 7

+ COL=73

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf ' * Setting txqueuelen for br-ex to 10000...       '

 * Setting txqueuelen for br-ex to 10000...       ++ /usr/bin/expr 80 - 1

+ /usr/bin/tput hpa 79

                                                                               + printf ' '

 + TXQUEUELEN_SET br-ex 10000

+ /sbin/ip link set br-ex txqueuelen 10000

+ '[' 0 -eq 0 ']'

+ log_action_end_msg 0

+ log_end_msg 0

+ '[' -z 0 ']'

+ '[' 73 ']'

+ '[' -x /usr/bin/tput ']'

+ log_use_plymouth

+ '[' n = y ']'

+ plymouth --ping

+ printf '\r'

+ /usr/bin/tput hpa 73

                                                                         + '[' 0 -eq 0 ']'

+ echo '[ OK ]'

[ OK ]

+ return 0

+ return 1

+ RETVAL=1

+ exit 1

 

 

On Fri, Jan 12, 2018 at 10:46 PM, Valentin Vidic <> wrote:

On Fri, Jan 12, 2018 at 10:03:30PM +0900, dan dev wrote:
>  # SYSTEMCTL_SKIP_REDIRECT=1 bash -x
> /etc/init.d/perfsonar-configure_nic_parameters
> start

Sorry, for Ubuntu this should be:

  # _SYSTEMCTL_SKIP_REDIRECT=1 bash -x /etc/init.d/perfsonar-configure_nic_parameters start

--
Valentin Vidic
Computer Systems Engineer - Expert
Department of Computer Infrastructure and Services
Croatian Academic and Research Network - CARNET
Josipa Marohnica 5, HR-10000 Zagreb, Croatia
tel: +385 1 6661 714, fax. +385 1 6661 635
gsm: +385 91 2480 919
www.CARNet.hr