perfSONAR User Q&A and Other Discussion

["Garnizov, Ivan (RRZE)" <>]

Hi Orion,

 

I guess your issues have been cleared by now. The problem that you observe is that scheduled tasks already on the pipeline do not get updated with the new token. It is only the new tasks, that are being negotiated get the new token. Hence you see reports for new failures for archiving for some time after the update.

 

Please report back, if these still persist.

 

Regards,

Ivan Garnizov

 

GEANT SA1T2: pS deployments GN Operations

GEANT SA2T3: pS development team

GEANT SA3T5: eduPERT team

 

 

-----Original Message-----
From: [mailto:] On Behalf Of Orion Poplawski
Sent: Freitag, 30. Juni 2017 00:53
To: Andrew Lake;
Subject: Re: [perfsonar-user] Trouble after upgrade

 

There were 2 sets of 3 <measurement_archive> blocks - I suspect the migrate script simply added the old ones to it.

 

Needed to use --force, not the UTF-8 character in your email :)

 

Old keys were at the end, new ones at the top so it took me a bit to notice them.

 

Needed to reboot it seemed to get the new keys to apply.  Now though I seem to have perhaps a timezone issue:

 

Jun 28 15:48:11 perfsonar journal: runner INFO     73788: Run succeeded.

Jun 28 15:48:11 perfsonar journal: runner ERROR    73788: Failed to store

local result: Cannot set state on future runs. 2017-06-29 03:48:10+00 /

2017-06-28 21:48:11+00

 

those have gone away, but I'm back to the invalid token errors, though not on every run:

 

Jun 29 16:10:53 perfsonar journal: runner INFO     74533: Running

https://perfsonar.cora.nwra.com/pscheduler/tasks/98a0b8e9-8f92-44e6-8558-9a7578ab8738/runs/600e3ac7-10d1-4070-8347-0f773f28b62c

Jun 29 16:10:53 perfsonar journal: runner INFO     74533: With traceroute:

trace --dest yellowstone.ucar.edu --ip-version 4 --source 192.168.183.92

Jun 29 16:10:58 perfsonar journal: runner INFO     74533: Run succeeded.

Jun 29 16:13:54 perfsonar journal: archiver WARNING  71856: Failed to archive

https://perfsonar.cora.nwra.com/pscheduler/tasks/7f5b7af4-bb2a-4218-8a73-d1d4ab802c4f/runs/9e28d0ae-7e1f-4595-bdd9-d19c0d4fece4

to esmond: Archiver permanently abandoned registering test after 3 attempt(s):

401: Invalid token.

Jun 29 16:13:54 perfsonar journal: archiver WARNING  71856: Gave up archiving

https://perfsonar.cora.nwra.com/pscheduler/tasks/7f5b7af4-bb2a-4218-8a73-d1d4ab802c4f/runs/9e28d0ae-7e1f-4595-bdd9-d19c0d4fece4

to esmond

Jun 29 16:14:23 perfsonar journal: runner INFO     74531: Running

https://perfsonar.cora.nwra.com/pscheduler/tasks/d76421d9-9c55-404f-bf7f-31521e74f4da/runs/f31dc856-f076-425e-b044-da01a0b919bf

Jun 29 16:14:23 perfsonar journal: runner INFO     74531: With ping: rtt

--count 10 --dest yellowstone.ucar.edu --interval PT1S --ip-version 4 --source

192.168.183.92 --length 1000 --ttl 255

Jun 29 16:14:32 perfsonar journal: runner INFO     74531: Run succeeded.

Jun 29 16:15:05 perfsonar journal: runner INFO     74530: Running

https://perfsonar.cora.nwra.com/pscheduler/tasks/fe35a293-ee23-4ebc-879b-0efa5127d37e/runs/5bb29ea9-414c-4287-90aa-3069f997c5bc

Jun 29 16:15:05 perfsonar journal: runner INFO     74530: With ping: rtt

--count 10 --dest 140.172.13.100 --interval PT1S --ip-version 4 --source

192.168.183.92 --length 1000 --ttl 255

Jun 29 16:15:15 perfsonar journal: runner INFO     74530: Run succeeded.

 

Nothing is showing up on the web interface.

 

I did use the -data option, but it doesn't look like any data got migrated.

 

Thanks.

 

On 06/28/2017 01:02 PM, Andrew Lake wrote:

> Hi,

>

> Looks like its complaining that its using the wrong API token to store

> results. You should be able to get it back in sync by doing the following:

>

> 1. Remove any <measurement_archive> blocks in

> /etc/perfsonar/meshconfig-agent-tasks.conf. There should be 3 blocks

> and removing everything between

> <measurement_archive>...</measurement_archive>

> including the measurement_archive tags themselves.

>

> 2. Run "/usr/lib/perfsonar/scripts/system_environment/configure_esmond —force”

> to recreate the blocks with the correct key

>

> It will take some time for it to recreate the tasks with the correct

> key so you might still see the errors for awhile afterwards. You will

> have a good idea it worked if

> /etc/perfsonar/meshconfig-agent-tasks.conf has new

> <measurement_archive> blocks with a different “password” field after running the command above.

>

>  Out of curiosity, did you use the option to migrate your database

> data or did you just have the script migrate your configuration files?

>

> Thanks,

> Andy

>

>

>

> On June 28, 2017 at 11:15:16 AM, Orion Poplawski (

> <>) wrote:

>

>> I just updated my perfsonar node from EL6 to EL7 and used the migrate scripts.

>> However, pscheduler.log has a number of errors and no results of

>> tests appear on the web console:

>> 

>> Jun 28 09:04:46 perfsonar journal: archiver WARNING 70564: Failed to

>> archive

>> https://perfsonar.cora.nwra.com/pscheduler/tasks/2668e69d-b807-4017-8

>> be1-e5ae6ee4c508/runs/7b435ded-9cee-4388-9f36-d2863770410c

>> 

>> to esmond: 401: Invalid token.

>> Jun 28 09:04:55 perfsonar journal: archiver WARNING 70562: Failed to

>> archive

>> https://perfsonar.cora.nwra.com/pscheduler/tasks/d6d4b928-be85-4c0d-b

>> 9e5-f8b9e18f5708/runs/ce760599-8105-4aa7-9e37-924c409f8558

>> 

>> to esmond: Archiver permanently abandoned registering test after 3 attempt(s):

>> 401: Invalid token.

>> Jun 28 09:04:55 perfsonar journal: archiver WARNING 70562: Gave up

>> archiving

>> https://perfsonar.cora.nwra.com/pscheduler/tasks/d6d4b928-be85-4c0d-b

>> 9e5-f8b9e18f5708/runs/ce760599-8105-4aa7-9e37-924c409f8558

>> 

>> to esmond

>> Jun 28 09:05:47 perfsonar journal: archiver WARNING 70564: Failed to

>> archive

>> https://perfsonar.cora.nwra.com/pscheduler/tasks/2668e69d-b807-4017-8

>> be1-e5ae6ee4c508/runs/7b435ded-9cee-4388-9f36-d2863770410c

>> 

>> to esmond: 401: Invalid token.

>> 

>> Haven't been able to figure out how to fix this.

>> 

>> Thanks,

>> Orion

>> 

>> --

>> Orion Poplawski

>> Technical Manager 720-772-5637

>> NWRA, Boulder/CoRA Office FAX: 303-415-9702

>> 3380 Mitchell Lane <> Boulder, CO

>> 80301 http://www.nwra.com

 

 

--

Orion Poplawski

Technical Manager                          720-772-5637

NWRA, Boulder/CoRA Office             FAX: 303-415-9702

3380 Mitchell Lane                      

Boulder, CO 80301                   http://www.nwra.com