perfSONAR User Q&A and Other Discussion

["Uhl, George D. (GSFC-423.0)[Arctic Slope Technical Services, Inc.]" <>]

Hi Mark,

 

The python assertion error that’s causing make to fail is happening on line 56 of limitprocessor_identifier_ipreversedns_test.py:

 

        self.assertEqual(

            ident.evaluate({ "requester": "8.8.8.8" }),

            True)

 

This looks like one of several reverse DNS tests that will fail every time on my system since I am only permitted to query internal DNS servers.  Can you recommend a work around please?

 

Thanks,

George

 

From: <> on behalf of <Uhl>, "Inc.]" <>
Reply-To: "George.D.Uhl" <>
Date: Monday, April 5, 2021 at 3:40 PM
To: Mark Feit <>, "" <>, "" <>
Subject: Re: [perfsonar-user] [EXTERNAL] Re: Adding homegrown tools to perfsonar

 

Hi Mark,

 

I guess it pays to read the liner notes…

 

I got further this time but the make blew up on python-pscheduler.  Was this a mixup between 4.3.0 source tree vs 4.3.2 checkout.

 

Here’s the output from the checkout for 4.3.2

 

# git checkout 4.3.2

Note: checking out '4.3.2'.

 

You are in 'detached HEAD' state. You can look around, make experimental

changes and commit them, and you can discard any commits you make in this

state without impacting any branches by performing another checkout.

 

If you want to create a new branch to retain commits you create, you may

do so (now or later) by using -b with the checkout command again. Example:

 

  git checkout -b new_branch_name

 

HEAD is now at dc55bca... Merge branch '4.3.0'

[root@psdev1 pscheduler]# make

((( \

(./scripts/build-all; echo $? >&3) \

| tee build.log >&4) 3>&1) \

| (read XS; exit $XS) \

) 4>&1

 

 

Here’s the mak output attempt to build pscheduler

 

#

# BUILDING python-pscheduler

#

 

make[1]: Entering directory `/local/uhl/pscheduler/python-pscheduler'

rm -rf pscheduler-4.3.0.tar.gz pscheduler-4.3.0 pscheduler-*.tar.gz ./rpmbuild ./BUILD-ROOT build.log *.rpm *.src.rpm

find . -name '*~' | xargs rm -rf

find . -depth -name Makefile \

    -exec /bin/sh -c \

    '[ "{}" != "./Makefile" ] && make -C `dirname {}` clean' \;

make[2]: Entering directory `/local/uhl/pscheduler/python-pscheduler/pscheduler'

make -C tests clean

make[3]: Entering directory `/local/uhl/pscheduler/python-pscheduler/pscheduler/tests'

rm -f *.pyc *~

make[3]: Leaving directory `/local/uhl/pscheduler/python-pscheduler/pscheduler/tests'

find . -name "*.pyc" | xargs rm -f

make[2]: Leaving directory `/local/uhl/pscheduler/python-pscheduler/pscheduler'

make[2]: Entering directory `/local/uhl/pscheduler/python-pscheduler/pscheduler/tests'

rm -f *.pyc *~

make[2]: Leaving directory `/local/uhl/pscheduler/python-pscheduler/pscheduler/tests'

cp -r pscheduler pscheduler-4.3.0

tar czf pscheduler-4.3.0.tar.gz pscheduler-4.3.0

rm -rf pscheduler-4.3.0

rm -rf rpmbuild

mkdir -p ./rpmbuild/BUILD ./rpmbuild/RPMS ./rpmbuild/SOURCES ./rpmbuild/SPECS ./rpmbuild/SRPMS 

cp python-pscheduler.spec ./rpmbuild/SPECS

cp pscheduler-4.3.0.tar.gz ./rpmbuild/SOURCES

mkdir -p BUILD-ROOT

set -o pipefail \

                && HOME=/local/uhl/pscheduler/python-pscheduler \

                   rpmbuild-with-deps -ba \

              --buildroot /local/uhl/pscheduler/python-pscheduler/BUILD-ROOT \

              python-pscheduler.spec 2>&1 \

       | tee build.log

Loaded plugins: fastestmirror

Loading mirror speeds from cached hostfile

 * base: mirror.cs.vt.edu

 * epel: mirror.math.princeton.edu

 * extras: centos.vwtonline.net

 * perfSONAR: mirror.net.cen.ct.gov

 * updates: mirror.cogentco.com

Resolving Dependencies

--> Running transaction check

---> Package numactl.x86_64 0:2.0.12-5.el7 will be installed

---> Package python3-psycopg2.x86_64 0:2.7.7-2.el7 will be installed

---> Package python36-coverage.x86_64 0:4.0.3-5.el7 will be installed

---> Package python36-dateutil.noarch 1:2.4.2-5.el7 will be installed

---> Package python36-dns.noarch 0:1.16.0-1.el7 will be installed

--> Processing Dependency: python36-crypto for package: python36-dns-1.16.0-1.el7.noarch

---> Package python36-netaddr.noarch 0:0.7.18-3.el7 will be installed

---> Package python36-nose.noarch 0:1.3.7-4.el7 will be installed

---> Package python36-pycurl.x86_64 0:7.43.0-7.el7 will be installed

---> Package python36-pytz.noarch 0:2017.2-3.el7 will be installed

--> Running transaction check

---> Package python36-crypto.x86_64 0:2.6.1-16.el7 will be installed

--> Processing Dependency: libtomcrypt.so.0()(64bit) for package: python36-crypto-2.6.1-16.el7.x86_64

--> Running transaction check

---> Package libtomcrypt.x86_64 0:1.17-26.el7 will be installed

--> Processing Dependency: libtommath >= 0.42.0 for package: libtomcrypt-1.17-26.el7.x86_64

--> Processing Dependency: libtommath.so.0()(64bit) for package: libtomcrypt-1.17-26.el7.x86_64

--> Running transaction check

---> Package libtommath.x86_64 0:0.42.0-6.el7 will be installed

--> Finished Dependency Resolution

 

Dependencies Resolved

 

================================================================================

 Package                 Arch         Version                Repository    Size

================================================================================

Installing:

 numactl                 x86_64       2.0.12-5.el7           base          66 k

 python3-psycopg2        x86_64       2.7.7-2.el7            epel         157 k

 python36-coverage       x86_64       4.0.3-5.el7            epel         273 k

 python36-dateutil       noarch       1:2.4.2-5.el7          epel          81 k

 python36-dns            noarch       1.16.0-1.el7           epel         248 k

 python36-netaddr        noarch       0.7.18-3.el7           epel         1.3 M

 python36-nose           noarch       1.3.7-4.el7            epel         301 k

 python36-pycurl         x86_64       7.43.0-7.el7           epel         223 k

 python36-pytz           noarch       2017.2-3.el7           epel          45 k

Installing for dependencies:

 libtomcrypt             x86_64       1.17-26.el7            extras       224 k

 libtommath              x86_64       0.42.0-6.el7           extras        36 k

 python36-crypto         x86_64       2.6.1-16.el7           epel         487 k

 

Transaction Summary

================================================================================

Install  9 Packages (+3 Dependent packages)

 

Total download size: 3.4 M

Installed size: 15 M

Downloading packages:

--------------------------------------------------------------------------------

Total                                              4.7 MB/s | 3.4 MB  00:00     

Running transaction check

Running transaction test

Transaction test succeeded

Running transaction

Warning: RPMDB altered outside of yum.

  Installing : libtommath-0.42.0-6.el7.x86_64                              1/12 

  Installing : libtomcrypt-1.17-26.el7.x86_64                              2/12 

  Installing : python36-crypto-2.6.1-16.el7.x86_64                         3/12 

  Installing : python36-dns-1.16.0-1.el7.noarch                            4/12 

  Installing : python36-pycurl-7.43.0-7.el7.x86_64                         5/12 

  Installing : numactl-2.0.12-5.el7.x86_64                                 6/12 

  Installing : python36-netaddr-0.7.18-3.el7.noarch                        7/12 

  Installing : python36-coverage-4.0.3-5.el7.x86_64                        8/12 

  Installing : python36-nose-1.3.7-4.el7.noarch                            9/12 

  Installing : 1:python36-dateutil-2.4.2-5.el7.noarch                     10/12 

  Installing : python3-psycopg2-2.7.7-2.el7.x86_64                        11/12 

  Installing : python36-pytz-2017.2-3.el7.noarch                          12/12 

  Verifying  : python36-pytz-2017.2-3.el7.noarch                           1/12 

  Verifying  : python3-psycopg2-2.7.7-2.el7.x86_64                         2/12 

  Verifying  : python36-dns-1.16.0-1.el7.noarch                            3/12 

  Verifying  : python36-crypto-2.6.1-16.el7.x86_64                         4/12 

  Verifying  : 1:python36-dateutil-2.4.2-5.el7.noarch                      5/12 

  Verifying  : python36-nose-1.3.7-4.el7.noarch                            6/12 

  Verifying  : libtommath-0.42.0-6.el7.x86_64                              7/12 

  Verifying  : python36-coverage-4.0.3-5.el7.x86_64                        8/12 

  Verifying  : python36-netaddr-0.7.18-3.el7.noarch                        9/12 

  Verifying  : numactl-2.0.12-5.el7.x86_64                                10/12 

  Verifying  : libtomcrypt-1.17-26.el7.x86_64                             11/12 

  Verifying  : python36-pycurl-7.43.0-7.el7.x86_64                        12/12 

 

Installed:

  numactl.x86_64 0:2.0.12-5.el7          python3-psycopg2.x86_64 0:2.7.7-2.el7 

  python36-coverage.x86_64 0:4.0.3-5.el7 python36-dateutil.noarch 1:2.4.2-5.el7

  python36-dns.noarch 0:1.16.0-1.el7     python36-netaddr.noarch 0:0.7.18-3.el7

  python36-nose.noarch 0:1.3.7-4.el7     python36-pycurl.x86_64 0:7.43.0-7.el7 

  python36-pytz.noarch 0:2017.2-3.el7   

 

Dependency Installed:

  libtomcrypt.x86_64 0:1.17-26.el7          libtommath.x86_64 0:0.42.0-6.el7    

  python36-crypto.x86_64 0:2.6.1-16.el7    

 

Complete!

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.ibXX5C

+ umask 022

+ cd /local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD

+ cd /local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD

+ rm -rf pscheduler-4.3.0

+ /usr/bin/tar -xf -

+ /usr/bin/gzip -dc /local/uhl/pscheduler/python-pscheduler/rpmbuild/SOURCES/pscheduler-4.3.0.tar.gz

+ STATUS=0

+ '[' 0 -ne 0 ']'

+ cd pscheduler-4.3.0

+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .

+ exit 0

Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.R6SToY

+ umask 022

+ cd /local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD

+ cd pscheduler-4.3.0

+ make CLASSES=/usr/libexec/pscheduler/classes

make[2]: Entering directory `/local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD/pscheduler-4.3.0'

find pscheduler -type f -name "*.py" \

| xargs sed -i -e 's|__CLASSES__|/usr/libexec/pscheduler/classes|'

python3 -m nose

...................................F........................................................

======================================================================

FAIL: Limit Processor / Identifier IP Reverse DNS / Identifier

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD/pscheduler-4.3.0/tests/limitprocessor_identifier_ipreversedns_test.py", line 56, in test_identifier

    True)

AssertionError: False != True

 

----------------------------------------------------------------------

Ran 92 tests in 6.187s

 

FAILED (failures=1)

make[2]: *** [test] Error 1

make[2]: Leaving directory `/local/uhl/pscheduler/python-pscheduler/rpmbuild/BUILD/pscheduler-4.3.0'

error: Bad exit status from /var/tmp/rpm-tmp.R6SToY (%build)

 

 

RPM build errors:

    Bad exit status from /var/tmp/rpm-tmp.R6SToY (%build)

make[1]: *** [build] Error 1

make[1]: Leaving directory `/local/uhl/pscheduler/python-pscheduler'

make: *** [packages] Error 2

[root@psdev1 pscheduler]# 

 

 

From: <> on behalf of Mark Feit <>
Reply-To: Mark Feit <>
Date: Monday, April 5, 2021 at 12:03 PM
To: "George.D.Uhl" <>, "" <>, "" <>
Subject: Re: [perfsonar-user] [EXTERNAL] Re: Adding homegrown tools to perfsonar

 

Uhl, George D. (GSFC-423.0)[Arctic Slope Technical Services, Inc.] writes:

 

I’ve took this little project off the shelf and tried to build pscheduler in a vagrant VM.  I started the make but  it bombed while building jq.

…

make[1]: Entering directory `/home/uhl/pscheduler/jq'

 

This is a known problem, and there’s a note about it in the README.md that accompanies the Vagrantfile:

 

NOTE: On some systems, pScheduler fails to build in the shared

file system where the user's home directory resides but builds fine on

the guest's "native" file space.  This is likely a problem with

VirtualBox rather than Vagrant or this configuration and will be

investigated further.  Until this is resolved, the recommended

workaround is to do development done in `/local/<username>`.

 

It’s been a thorn in my side for quite some time.  :-)

 

--Mark