Grouper Users - Open Discussion List

[Chris Hyzer <>]

Why do you not just have loader jobs with a quartz-cron schedule to run these 
hourly?  Sorry, I know you explained it, but can you expand?  I think quartz 
will not run the same job twice if already running.  Also, for the 
dependencies, if the students job isn’t done when the instructors job runs, 
it shouldn’t really matter since it just adds groups to other groups and will 
catch up if there is a race condition in the next hour...  once the jobs are 
setup they should run quickly should generally will be done by the time the 
next one is scheduled.

Thanks,
Chris

-----Original Message-----
From: Scott Koranda 
[mailto:]
 
Sent: Tuesday, July 15, 2014 4:41 PM
To: Chris Hyzer
Cc: grouper-users
Subject: Re: [grouper-users] database loader job with START_TO_START_INTERVAL

Hi,

I have decided to take this outside of the loader and instead use the
'monit' utility to check once every hour if a GSH script is running.
The GSH script runs the 3 loader jobs in series that together
provision the groups we need provisioned (students, instructors, then
'all', much like the Penn example in the documentation).

monit will check every hour if the GSH script is running and if not it
will start it again.

Rather than creating, running, and then destroying the loader jobs on
the fly I prefer to use loaderRunOneJob() for jobs that are
permanently saved in Grouper. The issue then is to make sure the
loader never actually runs the jobs.

I have tried to do that by setting the grouperLoaderQuartzCron for the
jobs to "0 0 0 * * ? 2099" so that, theoretically, they should only
run in the year 2099 (by which time I hope to be retired and no longer
responsible for any group provisioning).

Is there any reason to expect that Grouper and the loader will not
respect that cron configuration?

Thanks,

Scott

On Wed, Jul 9, 2014 at 2:08 PM, Scott Koranda 
<>
 wrote:
> Understood.
>
> This issue is that we want the loader jobs to run as often as possible
> to help beat down provisioning latency. But at certain times of the
> year (when semesters turn over) the amount of work, and hence the
> amount of time it takes for the loader job to run, spikes. The danger
> then is that two instances run at the same time.
>
> I do not see how to reconcile those requirements/needs with CRON so I
> was testing the START_TO_START_INTERVAL functionality, but I did not
> expect random start times.
>
> I appreciate any recommendations you can make.
>
> Thanks,
>
> Scott
>
> On Wed, Jul 9, 2014 at 2:03 PM, Chris Hyzer 
> <>
>  wrote:
>> Hmmmm, yes, you would :)
>>
>> For a while now I have always used cron scheduling.  Pick a random time 
>> for it to run and run it every hour at that time (so they don’t all run at 
>> the same time).  Then you can have a better idea about this :)
>>
>> Thanks,
>> Chris
>>
>>
>> -----Original Message-----
>> From: Scott Koranda 
>> [mailto:]
>> Sent: Wednesday, July 09, 2014 3:01 PM
>> To: Chris Hyzer
>> Cc: grouper-users
>> Subject: Re: [grouper-users] database loader job with 
>> START_TO_START_INTERVAL
>>
>> Thanks.
>>
>> If I kick one off manually do I have to worry about the possibility of
>> two running simultaneously, or will having one instance running
>> prevent the other one from running?
>>
>> Thanks,
>>
>> Scott
>>
>> On Wed, Jul 9, 2014 at 1:58 PM, Chris Hyzer 
>> <>
>>  wrote:
>>> It doesn’t necessarily start when the loader starts:
>>>
>>>       //start time is the interval seconds / 5, rand
>>>       int startSeconds = (int)(Math.random() * intervalSeconds);
>>>       Date startTime = new Date(System.currentTimeMillis() + 
>>> (startSeconds*1000));
>>>
>>> Don’t want all START_TO_STARTs to start when the loader starts or you 
>>> could have performance problems :)
>>>
>>> If you want to kick one off manually, you can do that from GSH:
>>>
>>> https://spaces.internet2.edu/display/Grouper/GrouperShell+%28gsh%29#GrouperShell%28gsh%29-Loader
>>>
>>> Thanks,
>>> Chris
>>>
>>> -----Original Message-----
>>> From: 
>>> 
>>>  
>>> [mailto:]
>>>  On Behalf Of Scott Koranda
>>> Sent: Wednesday, July 09, 2014 1:46 PM
>>> To: grouper-users
>>> Subject: [grouper-users] database loader job with START_TO_START_INTERVAL
>>>
>>> Hi,
>>>
>>> I created a loader job that uses START_TO_START_INTERVAL with an
>>> interval of 3600 seconds.
>>>
>>> My understanding is that when I restart the loader process the loader
>>> job should start immediately, and then run again one hour after it
>>> completes.
>>>
>>> Is that correct?
>>>
>>> I do not see any evidence in grouper_error.log (at the INFO level)
>>> that the loader started the job after it was restarted. Should I?
>>>
>>> Thanks,
>>>
>>> Scott