Skip to Content.
Sympa Menu

grouper-dev - [grouper-dev] Real time loader, messaging and RabbitMQ

Subject: Grouper Developers Forum

List archive

[grouper-dev] Real time loader, messaging and RabbitMQ


Chronological Thread 
  • From: "Gettes, Michael" <>
  • To: "" <>
  • Subject: [grouper-dev] Real time loader, messaging and RabbitMQ
  • Date: Mon, 22 Apr 2019 00:27:54 +0000

Hi all,

I am trying to make use of the real time loader again. Last time I tried it
there was a limitation to using the table only on the same DB as the subject
source so I could not proceed. Shilen fixed that. At that time (last
September, I believe) I was able to have the rabbitmq listener populate a
table (just not the right table because of the limitation just noted). Now,
I am not. My config is below...

Loader log says the listener runs without errors - zero counts and nothing is
inserted into loaderRealTime1. Logging - nothing logged at DEBUG.
And I waited longer than 5 minutes after start-up. I have
otherJob.RealTime1 disabled so I would see if something got inserted and
there are 2 JSON messages sitting in the queue. I’m not seeing any errors
from rabbit server.
I’m wondering if the changes made to rabbit recently by Vivek caused the
listener to not work properly?

grouper-loader.properties

otherJob.RealTime1.class =
edu.internet2.middleware.grouper.app.loader.GrouperLoaderIncrementalJob
otherJob.RealTime1.quartzCron = 6/10 * * * * ?
otherJob.RealTime1.databaseName = grouper
otherJob.RealTime1.tableName = loaderRealTime1
otherJob.RealTime1.fullSyncThreshold=100

messaging.listener.rabbitListener.class =
edu.internet2.middleware.grouper.app.loader.GrouperLoaderIncrementalMessagingListener
messaging.listener.rabbitListener.quartzCron = 4/10 * * * * ?
messaging.listener.rabbitListener.messagingSystemName = rabbitmqSystem
messaging.listener.rabbitListener.queueName = RealTime1
messaging.listener.rabbitListener.queueOrTopicName = RealTime1
messaging.listener.rabbitListener.messageQueueType = queue
messaging.listener.rabbitListener.exchangeType = direct
messaging.listener.rabbitListener.numberOfTriesPerIteration = 3
messaging.listener.rabbitListener.pollingTimeoutSeconds = 18
messaging.listener.rabbitListener.sleepSecondsInBetweenIterations = 0
messaging.listener.rabbitListener.maxMessagesToReceiveAtOnce = 20
messaging.listener.rabbitListener.maxOuterLoops = 50
messaging.listener.rabbitListener.incrementalLoaderJobName = RealTime1

I initially only had queueName so I tossed in messageQueueOrTopicName,
messageQueueType and exchangeType above.

rabbitmqSystem is the very same rabbitmqSystem I am using with sending and it
works great with:

changeLog.consumer.rabbit.quartzCron = 2/10 * * * * ?

changeLog.consumer.rabbit.class =
edu.internet2.middleware.grouper.changeLog.esb.consumer.EsbConsumer
changeLog.consumer.rabbit.publisher.class =
edu.internet2.middleware.grouper.changeLog.esb.consumer.EsbMessagingPublisher
changeLog.consumer.rabbit.publisher.messagingSystemName = rabbitmqSystem
changeLog.consumer.rabbit.publisher.queueOrTopicName = amq.topic
changeLog.consumer.rabbit.publisher.messageQueueType = topic
changeLog.consumer.rabbit.publisher.exchangeType = topic
changeLog.consumer.rabbit.regexRoutingKeyReplacementDefinition =
${groupName.replaceAll(":", ".”)}

grouper.client
grouper.messaging.system.rabbitmqSystem.name = rabbitmqSystem
grouper.messaging.system.rabbitmqSystem.class =
edu.internet2.middleware.grouperMessagingRabbitmq.GrouperMessagingRabbitmqSystem
grouper.messaging.system.rabbitmqSystem.host =
az1-rabbitmq-XXXXX.server.ufl.edu
grouper.messaging.system.rabbitmqSystem.virtualhost = /
grouper.messaging.system.rabbitmqSystem.port = 5672
grouper.messaging.system.rabbitmqSystem.defaultPageSize = 800
grouper.messaging.system.rabbitmqSystem.maxPageSize = 10000
grouper.messaging.system.rabbitmqSystem.username = grouper
grouper.messaging.system.rabbitmqSystem.password = XXXXXXXXXXXXXX

Log4j
log4j.logger.edu.internet2.middleware.grouper.app.loader.GrouperLoaderIncrementalMessagingListener
= DEBUG




  • [grouper-dev] Real time loader, messaging and RabbitMQ, Gettes, Michael, 04/22/2019

Archive powered by MHonArc 2.6.19.

Top of Page