perfSONAR User Q&A and Other Discussion

["Whitworth, Luke" <>]

Potentially but of course that would incur additional cost, I’m currently looking at the network booting options too.  There’s a few ways to achieve this I would guess but the ideal for me is to have the SD card handling the OS etc. with all the heavy stuff off on remote / alternate storage.

 

Luke

 

From: Onno Zweers <>
Sent: 09 September 2020 15:17
To: Whitworth, Luke <>
Subject: Re: [perfsonar-user] SD card lifespan

 

Hi Luke,

 

You can boot Raspberry Pi 4 from a USB attached SSD. Isn’t that a better option?

 

Cheers,

Onno

 

Van: <> namens "Whitworth, Luke" <>
Beantwoorden - Aan: "Whitworth, Luke" <>
Datum: woensdag 9 september 2020 om 15:06
Aan: "" <>
Onderwerp: RE: [perfsonar-user] SD card lifespan

 

Okay so think I’ve found my own answer here:  https://github.com/perfsonar/pscheduler/blob/2209888989487c29e2bfee88b900bd8250af032e/pscheduler-server/pscheduler-server/database/external_program.sql

 

-- TODO: Figure out what do do when DB is off-node. Probably need an

-- external process like the task runner.

 

Back to the drawing board I go…….wonder if I can keep the postgres db in ram and just recreate it fresh on a boot………….

 

From: Whitworth, Luke
Sent: 09 September 2020 11:01
To:
Subject: SD card lifespan

 

Morning all,

 

So we’re using a load of Pi 4’s for testpoints and early results were favourable.  However, a few weeks in and we’re starting to see SD cards dying (3 and counting out of the 8 we deployed).  We’d already done a level of work to get stuff onto tmpfs to eliminate unnecessary writes to the sd cards:

 

administrator@pi:~ $ mount | grep tmpfs

devtmpfs on /dev type devtmpfs (rw,relatime,size=823904k,nr_inodes=103729,mode=755)

tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime)

tmpfs on /run type tmpfs (rw,nosuid,noatime,size=524288k,mode=755)

tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)

tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)

tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,relatime,size=262144k)

tmpfs on /var/log type tmpfs (rw,nosuid,noatime,size=262144k,mode=755)

tmpfs on /var/tmp type tmpfs (rw,nosuid,noatime,size=102400k)

tmpfs on /var/spool/mqueue type tmpfs (rw,nosuid,noatime,size=30720k,mode=700,gid=12)

tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,size=191200k,mode=700,uid=1001,gid=1001)

 

Seems though that postgres, specifically the pscheduler db, are still generating continual writes so are therefore our current suspect.  I’ve been looking at having just a simple central postgres server to point the Pi’s to, but am hitting issues in so much as if I specify a different DB name during the initial install it still seems to create a pscheduler database that’s got the schema in it, and an empty db with my chosen name.  Got a bit further by renaming the pscheduler db to my chosen name and editing the DSN file @ /etc/pscheduler/database/database-dsn to point to the remote host, but now running:

 

pscheduler log

 

I see

 

Sep  9 10:57:52 pi safe_run/ticker ERROR    Traceback (most recent call last):

Sep  9 10:57:52 pi safe_run/ticker ERROR      File "/usr/lib/python2.7/dist-packages/pscheduler/saferun.py", line 73, in safe_run

Sep  9 10:57:52 pi safe_run/ticker ERROR        function()

Sep  9 10:57:52 pi safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 170, in <lambda>

Sep  9 10:57:52 pi safe_run/ticker ERROR        pscheduler.safe_run(lambda: main_program())

Sep  9 10:57:52 pi safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 132, in main_program

Sep  9 10:57:52 pi safe_run/ticker ERROR        cursor.execute("SELECT cold_boot()")

Sep  9 10:57:52 pi safe_run/ticker ERROR    InternalError: Unable to list installed tests: OSError: [Errno 2] No such file or directory: 'pscheduler'

Sep  9 10:57:52 pi safe_run/ticker ERROR    Traceback (most recent call last):

Sep  9 10:57:52 pi safe_run/ticker ERROR      File "/usr/lib/python2.7/site-packages/pscheduler/program.py", line 276, in run_program

Sep  9 10:57:52 pi safe_run/ticker ERROR        process = __get_process(argv, new_env, attempts)

Sep  9 10:57:52 pi safe_run/ticker ERROR      File "/usr/lib/python2.7/site-packages/pscheduler/program.py", line 268, in __get_process

Sep  9 10:57:52 pi safe_run/ticker ERROR        raise ex

Sep  9 10:57:52 pi safe_run/ticker ERROR    OSError: [Errno 2] No such file or directory: 'pscheduler'

Sep  9 10:57:52 pi safe_run/ticker ERROR    CONTEXT:  SQL statement "SELECT test_boot()"

Sep  9 10:57:52 pi safe_run/ticker ERROR    PL/pgSQL function warm_boot() line 4 at PERFORM

Sep  9 10:57:52 pi safe_run/ticker ERROR    SQL statement "SELECT warm_boot()"

Sep  9 10:57:52 pi safe_run/ticker ERROR    PL/pgSQL function cold_boot() line 4 at PERFORM

Sep  9 10:57:52 pi safe_run/ticker ERROR    Waiting 15.5 seconds before restarting

Sep  9 10:58:08 pi safe_run/ticker ERROR    Restarting: ['/usr/lib/pscheduler/daemons/ticker', '--daemon', '--pid-file', '/var/run/pscheduler-server/pscheduler-ticker.pid', '--dsn', '@/etc/pscheduler/database/database-dsn']

 

Just wondering if what I’m trying to achieve is even possible, or if I just need to look at a way to tune the write level of postgres maybe…….

 

Any help or tips gratefully received!

 

Luke

 

Luke Whitworth
Network Specialist
Information Services
Building 63 (IT) G46, Cranfield University, Cranfield, Bedfordshire MK43 0AL
E:
T: +44 (0) 1234 75 4007
W: www.cranfield.ac.uk

 

This email and any attachments to it may be confidential and are intended only for the named addressee. If you are not the named addressee, please accept our apology, notify the sender immediately and then delete the email. We request that you do not disclose, use, copy or distribute any information within it.

 

Any opinions expressed are not necessarily the corporate view of Cranfield University. This email is not intended to be contractually binding unless specifically stated and the sender is an authorised University signatory.

 

Whilst we have taken steps to ensure that this email and all attachments are free from any virus, we advise that, in keeping with good computing practice, the recipient should ensure they are actually virus free.