perfSONAR User Q&A and Other Discussion

[Hyojoon Kim <>]

Hi, 

I found some answers. For PostgreSQL, the link (http://docs.perfsonar.net/multi_ma_backups.html) actually does state: 

"This will delete any existing data and replace it with the backup” 

For Cassandra and nodetool, I think it does overwrite existing data as well. 
https://specs.openstack.org/openstack/trove-specs/specs/liberty/cassandra-backup-restore.html

"A snapshot can be restored by moving all *.db files from a snapshot directory to the respective keyspace overwriting any existing files.”

Good to know! :-) 

Thanks,

Joon

On Jan 22, 2016, at 3:10 PM, Brian Tierney <> wrote:


Ah, I'm not sure either.

Does anyone know the answer to that?

On Fri, Jan 22, 2016 at 11:46 AM, Hyojoon Kim <> wrote:
Yeah, I already read this. However, it does not have anything about what command might overwrite data, etc, so I just wanted to ask around.  

I assume doing copy of “/var/lib/cassandra/“ to a MA machine will overwrite any existing data in that MA, but the website does not state any danger of that. 

I’m not sure if taking a snapshot from a old machine and restoring it on a new MA will overwrite data on the new MA or not. I’m asking because the new MA already has some data, apparently. 

Thanks,
Joon 

On Jan 22, 2016, at 2:17 PM, Brian Tierney <> wrote:


I think all you need to know for this is here:

http://docs.perfsonar.net/multi_ma_backups.html?highlight=migration


On Fri, Jan 22, 2016 at 11:12 AM, Hyojoon Kim <> wrote:
Hi all,

We are going to build a new MA (measurement archive) machine (call this MA-2), which is going to take over the functionality of our old MA machine (call this MA-1). I want to make sure this process goes smoothly, and more importantly, without losing historical data.

To preserve the historical data we have collected so far on MA-1, I am thinking to follow these steps:

0. Keep MA-1 running as usual.
1. On MA-2, stop Cassandra and PostgreSQL service for now. (service cassandra stop, service pgsql stop)
2. Configure all perfSONAR nodes to send measurement data to *both* MA-1 and MA-2.
3. Configure MA-2 so that it's ready to receive measurement data from all perfSONAR nodes. However, don’t start Cassandra and PostgreSQL service yet.
4. On MA-1, stop Cassandra and PostgreSQL service, then copy over “/var/lib/cassandra” and “/var/lib/pgsql” to MA-2, same directory location.
5. When copying is done, start Cassandra and PostgreSQL service on MA-2.
6. Configure all perfSONAR nodes to stop sending data to MA-1.
7. Get rid of MA-1.

Will this generally work? I just want to make sure some step doesn’t *overwrite* historical data with new data on MA-2.

I am also assuming that the time gap when perfSONAR nodes cannot send data to either MA-1 and MA-2 (between step 4 and 5 above) is okay as far as it’s not too long, because that failed-to-send-data will be backlogged in each node for at least some time, without losing that data. Well, some loss of measurement data during the migration is relatively tolerable, compared to the loss of historical data.

Any feedback would be helpful!

Thanks,
Joon









-- 
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net

-- 
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net