perfSONAR User Q&A and Other Discussion

[Murilo Vetter <>]

Hi list,

We tryed to change what was suggested but not worked either.

Looking in the internet, in http://blog.tomas.cat/en/2012/10/31/basic-apache-cassandra-installation-and-some-fine-tune-improvements/ (item 6.4).

We changed the file /etc/security/limits.conf and inserted the following lines:

---

* soft memlock 4194304
* hard memlock 4194304

---

It seams it is working now.

Regards,

--

Murilo Vetter
Analista REMEP-FLN/PoP-SC
PoP-SC - Ponto de Presença da RNP em Santa Catarina
REMEP-FLN - Rede Metropolitana de Educação e Pesquisa da Região de Florianópolis
RNP - Rede Nacional de Ensino e Pesquisa
UFSC - Universidade Federal de Santa Catarina
SETIC - Superintendência de Governança Eletrônica e Tecnologia da Informação e Comunicação
Fundações de Apoio FEESC e FEPESE
--/--

www.pop-sc.rnp.br
remep.pop-sc.rnp.br
CHAMADOS:
+55(48)3721-3000
INOC: 11242*100

---

De: "Andrew Lake" <>
Para: "Szymon Trocha" <>,
Cc: "Murilo Vetter" <>
Enviadas: Sexta-feira, 29 de setembro de 2017 11:09:14
Assunto: Re: [perfsonar-user] ESMOND Tuning

Hi,

Did adjust some settings in /etc/cassandra/conf/cassandra-env.sh from the default? By default MAX_HEAP_SIZE will only use 1/4 of the system memory. If your host has 8GB of memory what you shared seems to indicate it is willing to use 100% of the memory which would cause exactly the behavior you are seeing. If if have lines similar to the following in  /etc/cassandra/conf/cassandra-env.sh, you should comment them out by adding a # to the beginning of the line and restart Cassandra:

MAX_HEAP_SIZE=“8G"

HEAP_NEWSIZE="2048M"

That should restore the defaults and get you to a more sensible memory usage.

Thanks,

Andy

On September 29, 2017 at 10:00:48 AM, Szymon Trocha () wrote:

W dniu 29.09.2017 o 14:31, Murilo Vetter pisze:

Hi list,

Here we are running ESMOND in our production environment and we notice that it is crashing the service a lot.

It starts once every hour. The host in that time was with 4GB of memory.

After trying to tuning Cassandra and increasing the memory of the host to 8GB, it is crashing once a day.

The size of the base is using ~100GB.

What is the recommendation size of memory to use in production environment?

In this host does not exist swap, is it necessary?

Some data below of the host:

[cqlsh 4.1.1 | Cassandra 2.0.17 | CQL spec 3.1.1 | Thrift protocol 19.39.0] ----- running with super-csv-2.4.0.jar

Java(TM) SE Runtime Environment (build 1.7.0_80-b15)

MAX_HEAP_SIZE="8G"
HEAP_NEWSIZE="2048M"
key_cache_size_in_mb: 512
file_cache_size_in_mb: 512
memtable_total_space_in_mb: 2048
commitlog_total_space_in_mb: 4096
in_memory_compaction_limit_in_mb: 256
DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap

In the initialization of Cassandra it was appearing:

CLibrary.java (line 63) JNA not found. Native methods will be disabled.

We tryed to update to jna 3.5.1 and jna 4.5, no one works with Cassandra, it appears the follow error:

java.lang.NoClassDefFoundError: org/supercsv/prefs/CsvPreference$Builder

We tryed also to use disk_access_mode: standard, but it doesn't work too.

Does anyone have a clue to help in this problem? Any proper configuration of Cassandra in this scenario?

Hi Murilo,

I'm just curious - did you maybe run some kind of SNMP monitoring for this host? To see historical mem and cpu behaviour

Regards,

--  
Szymon Trocha

Poznań Supercomputing & Networking Center
Tel. +48 618582022 ::: http://noc.pcss.pl