perfSONAR User Q&A and Other Discussion

[Roderick Mooi <>]

Thanks Andy

I must admit that the upgrade process gets better and better with each version. The bundle options may be a good idea for us and we’ll definitely investigate.

Best regards,

Roderick

On 30 Sep 2015, at 2:39 PM, Andrew Lake <> wrote:

Hi Roderick,

It sounds like maybe you should look at one of our other bundles. The Toolkit installation expects a high-level control over the system, especially over things like sysctl. This is not a new development, this has how the toolkit has always behaved. What is new is that we actually give you other installation options if this is not right for your needs. The main motivation for the new bundles in 3.5 is for those that want more control. You may be interested in something like the perfSONAR-Core bundle as described here: http://docs.perfsonar.net/install_options.html. We have also split out things like the parts of the toolkit that controlsysctl, ntp, firewalls and regular service restarts so you can add the pieces you want on top of something like perfSONAR-Core and leave out the ones you want to manage yourself. Otherwise you either need to conform to the toolkit way of setting up things like sysctl and sudo users or need to manually (or through your own configuration management software) update things like /root/.bashrc and sysctl after each update. 

Thanks,

Andy

On Wed, Sep 30, 2015 at 5:09 AM, Roderick Mooi <> wrote:

Here is an extract from one of our sysctl.conf files showing the changes made by the update process at the bottom overriding our previously configured values :(

:

...

# Controls the maximum number of shared memory segments, in pages

kernel.shmall = 4294967296

#net.core.rmem_max = 33554432

#net.core.wmem_max = 33554432

#net.ipv4.tcp_rmem = 4096 87380 16777216

#net.ipv4.tcp_wmem = 4096 87380 16777216

# allow testing with buffers up to 128MB

net.core.rmem_max = 134217728

net.core.wmem_max = 134217728

# increase Linux autotuning TCP buffer limit to 64MB

net.ipv4.tcp_rmem = 4096 87380 67108864

net.ipv4.tcp_wmem = 4096 65536 67108864

net.core.netdev_max_backlog = 250000

net.ipv4.tcp_no_metrics_save = 1

net.ipv4.tcp_congestion_control = htcp

net.ipv4.tcp_mtu_probing = 1

####################################

# Default perfSONAR sysctl settings

####################################

# net.core.rmem_max = 67108864

# net.core.wmem_max = 67108864

# net.ipv4.tcp_rmem = 4096 87380 33554432

# net.ipv4.tcp_wmem = 4096 87380 33554432

# net.core.netdev_max_backlog = 30000

# net.ipv4.tcp_no_metrics_save = 1

# net.ipv4.tcp_congestion_control = htcp

# net.ipv4.conf.all.arp_ignore=1

# net.ipv4.conf.all.arp_announce=2

# net.ipv4.conf.default.arp_filter=1

# net.ipv4.conf.all.arp_filter=1

####################################

net.core.rmem_max = 67108864

net.core.wmem_max = 67108864

net.ipv4.tcp_rmem = 4096 87380 33554432

net.ipv4.tcp_wmem = 4096 87380 33554432

net.core.netdev_max_backlog = 30000

net.ipv4.tcp_no_metrics_save = 1

net.ipv4.tcp_congestion_control = htcp

net.ipv4.conf.all.arp_ignore=1

net.ipv4.conf.all.arp_announce=2

net.ipv4.conf.default.arp_filter=1

net.ipv4.conf.all.arp_filter=1

…

For some of our long links this caused a drop in throughput in our tests due to the fact that larger window sizes are required…

Regards,

Roderick

On 30 Sep 2015, at 11:04 AM, Roderick Mooi <> wrote:

Hi Andy and dev team

The update also seems to have altered our sysctl.conf files. We had optimised these for our setup - why change them without notification?

1. Can you possibly not do this automatically in future?

2. Is there (or can you make available) a list of all possible changes that an update to 3.5 (and 3.5.0.1) may make to a previous installation?

Thanks,

Roderick

On 28 Sep 2015, at 6:38 PM, Andrew Lake <> wrote:

Dear perfSONAR Community,

The final version of perfSONAR 3.5 is now available! Existing users may run "yum update" to get this new release. Alternatively, if you have automatic updates enabled, you should get the update within 24 hours. It is recommended, though not required, that you reboot the host after the update to ensure you are running the latest packages and kernel. For new Toolkit installs, you may find our ISOs at the following link:

http://downloads.perfsonar.net/toolkit/

For more information on install options, updating and using the software please see http://docs.perfsonar.net.

About 3.5:

Version 3.5 is intended to be an important first step toward increasing the usability, automation and ease of large-scale deployments for existing perfSONAR users. Read more here: https://www.internet2.edu/news/detail/8866/ Some specific changes in this release include the following:

- A brand new web interface that addresses many of the UI concerns voiced in the community over the past few years. You will be presented with this interface the next time you visit the toolkit after the update is applied. See the documentation for some screenshots of the new interface: http://docs.perfsonar.net/install_config_first_time.html#accessing-the-web-interface. For this release you will also be able to visit the previous interface by navigating to http://hostaddress/toolkit-old in your web browser.

- Support for alternative installation methods to the Toolkit ISO on both CentOS and Debian/Ubuntu. This allows for more flexibility in deployment, especially for large deployments and/or those running lower cost hardware. See http://docs.perfsonar.net/install_options.html in our documentation for more details.

- Enhanced central management and “auto-configuration” features to aid in the deployment of a large number of hosts. See our documentation at http://docs.perfsonar.net/#managing-multiple-perfsonar-hostsfor more details.

In addition to above there are of course the usual round of bug fixes and performance improvements that go hand-in-hand with a release. Thank you to all in the community who helped us to identify and solve those over the past few months. Full release notes can be found at http://www.perfsonar.net/release-notes/version-3-5/

Also, please take this chance to review our hardware advice page: http://www.perfsonar.net/deploy/hardware-selection/hardware-advice/. Version 3.5 should NOT introduce any significant hardware resource requirements over version 3.4, but as test meshes continue to grow many of the requirements such as memory, are becoming “stricter” minimum requirements than in the past so this is a good opportunity to verify your setup. In particular, note that we recommend a minimum of 4GB of RAM for a perfSONAR Toolkit host.

As always, please let the perfSONAR team know if you have any questions and we look forward to your feedback.

Thank you,

The perfSONAR Development Team

--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.

This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.

Please consider the environment before printing this email.

--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.

This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.

Please consider the environment before printing this email.

--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.

This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.

Please consider the environment before printing this email.

--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.

This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean.

Please consider the environment before printing this email.