perfSONAR User Q&A and Other Discussion

[Andrew Lake <>]

Hi,

It’s a known bug in the cassandra RPM on CentOS 7. Cassandra creates /var/run/cassandra on install, but that directory is cleaned-out on reboot by the OS. Really the cassandra init script should be creating that directory, not the RPM. We don't actually control that RPM, but we might have to roll our own or tweak some boot scripts if they don’t fix that. 

Thanks,

Andy

 

On November 16, 2016 at 11:45:59 AM, Michael Petry () wrote:

Only one process running when I looked, but this was straight off the ISO behavior.   That was my concern.  There may be a synchronization issue hidden in the startup scripts/configs.

The cassandra log shows:

/usr/sbin/cassandra: line 155: /var/run/cassandra/cassandra.pid: No such file or directory

and there is no /var/run/cassandra

Looks like something in the setup on the ISO needs tweaking.

Mike

 

On Wed, Nov 16, 2016 at 11:16 AM, Dan Doyle <> wrote:

Hi Mike,

Do you already have an instance of cassandra running? I have seen similar cases where you "start" cassandra on a machine where it's still running, particularly where the pidfile gets out of sync and so the init scripts think things are okay.

You may want to look at something like `ps aux | grep cassandra` and kill any existing processes that may be running, that start again via the init script.

Dan Doyle
GlobalNOC Software Developer
1-812-856-3892

On Nov 16, 2016, at 11:11 AM, Michael Petry <> wrote:

cassandra  leaves a fatal error in its logs.  Is this expected behavior?

 /var/log/cassandra/cassandra.log

Btw: I've tried using the 

INFO 11:00:15,857 Cassandra version: 2.0.17

 INFO 11:00:15,857 Thrift API version: 19.39.0

 INFO 11:00:15,861 CQL supported versions: 2.0.0,3.1.7 (default: 3.1.7)

 INFO 11:00:15,874 Loading persisted ring state

 INFO 11:00:15,890 Starting up server gossip

 INFO 11:00:15,893 Enqueuing flush of Memtable-local@1959711969(375/3750 serialized/live bytes, 13 ops)

 INFO 11:00:15,904 Writing Memtable-local@1959711969(375/3750 serialized/live bytes, 13 ops)

 INFO 11:00:15,931 Completed flushing /var/lib/cassandra/data/system/local/system-local-jb-24-Data.db (313 bytes) for commitlog position ReplayPosition(segmentId=1479312014388, position=179710)

 INFO 11:00:15,932 Loading settings from file:/etc/cassandra/default.conf/cassandra.yaml

ERROR 11:00:16,101 Fatal configuration error

org.apache.cassandra.exceptions.ConfigurationException: localhost/127.0.0.1:7000 is in use by another process.  Change listen_address:storage_port in cassandra.yaml to values that do not conflict with other services

at org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:492)

at org.apache.cassandra.net.MessagingService.listen(MessagingService.java:438)

at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:692)

at org.apache.cassandra.service.StorageService.initServer(StorageService.java:622)

at org.apache.cassandra.service.StorageService.initServer(StorageService.java:514)

at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:424)

at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:554)

at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:643)

localhost/127.0.0.1:7000 is in use by another process.  Change listen_address:storage_port in cassandra.yaml to values that do not conflict with other services

Fatal configuration error; unable to start server.  See log for stacktrace.

 INFO 11:00:16,121 Announcing shutdown

ERROR 11:00:16,122 Exception in thread Thread[StorageServiceShutdownHook,5,main]

java.lang.IllegalArgumentException: Invalid token for Murmur3Partitioner. Got true but expected a long value (unsigned 8 bytes integer).

at org.apache.cassandra.dht.Murmur3Partitioner$1.fromString(Murmur3Partitioner.java:190)

at org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1470)

at org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1532)

at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1354)

at org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1171)

at org.apache.cassandra.gms.Gossiper.addLocalApplicationStateInternal(Gossiper.java:1401)

at org.apache.cassandra.gms.Gossiper.addLocalApplicationStates(Gossiper.java:1416)

at org.apache.cassandra.gms.Gossiper.addLocalApplicationState(Gossiper.java:1406)

at org.apache.cassandra.gms.Gossiper.stop(Gossiper.java:1432)

at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:583)

at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)

at java.lang.Thread.run(Thread.java:745)

[petry@oit-82-33 cassandra]$ 

Mike