Subject: Grouper Developers Forum
- From: John Tran <>
- To: Chris Hyzer <>
- Cc: "" <>
- Subject: Re: [grouper-dev] Grouper Processing Speed
- Date: Mon, 31 Aug 2015 14:03:22 +0000
- Accept-language: en-US
- Authentication-results: spf=none (sender IP is ) ; isc.upenn.edu; dkim=none (message not signed) header.d=none;isc.upenn.edu; dmarc=none action=none header.from=wisc.edu;
- Spamdiagnosticmetadata: NSPM
- Spamdiagnosticoutput: 1:23
Thanks for all your time. I’ll do some more digging into the date and a
couple other things that don’t seem to be working correctly on our end.
You’ve been of great help.
IS Technical Services Senior
Division of Information Technology
University of Wisconsin - Madison
> On Aug 27, 2015, at 11:29 AM, Chris Hyzer
> Oracle should have time in the DATE column.
> To see which jobs are running, can you query for rows where there is no
> end_date, but yes start date? Something like that?
> Something like this?
> select * from grouper_loader_log where ended_time is null and started_time
> > sysdate - 2
> For loader jobs there should be insert/update/delete, for other jobs maybe
> Pending jobs are in the same table with no ended_time
> -----Original Message-----
> From: John Tran
> Sent: Thursday, August 27, 2015 10:27 AM
> To: Chris Hyzer
> Subject: Re: [grouper-dev] Grouper Processing Speed
> Thanks again for the reply. I missed your email again, but I do believe I
> found the issue with my mail client now. Apologies once again. I’ve
> included some answers as well as followup questions below.
>> On Aug 24, 2015, at 8:30 AM, Chris Hyzer
>>> In the grouper_loader_log, there is started_time and ended_time, but
>>> those are dates without timestamps.
>> Are you in mysql (or something else non-oracle). You could try changing
>> the column type from DATE to DATETIME and see if that helps (do that in
>> test first) :)
> We are using Oracle. I’ll talk to our admin and try that DATETIME switch!
>>> I see total_count which I assume
>>> is the total number of changes in that job.
>> Total is the total number of members for a group or list of groups. The
>> insert/update/delete shows how many have been changed, though there isn’t
>> something to say how many will be changed.
> The insert, update, and delete fields are always zero. A majority of the
> time total_count is at zero as well. I assume it’s an issue with our
> configuration somewhere.
>>> However, I don’t see a
>>> field that would indicate how many have been processed and/or how many
>>> are left to process in that job.
>> If you look at a job in process, it should update that record in the table
>> with progress. You are looking at jobs which are status SUCCESS which
>> means done
> The main reason for this inquiry is to see if we’re able to see how large
> the current and pending jobs are and to see how far along we are. This
> will allow us to give our customers time estimates when there is a
> processing delay. Are you aware of any available data to determine the
> number of records to be processed for a particular job? It seems the
> answer is no as you stated above that there isn’t something to say how many
> will be changed.
>>> If I’m correct to assume that grouper_change_log_entry_temp are the jobs
>>> queue, is there a way to see how large these jobs are?
>> No, that is just the change log temp to change log job, copies records to
>> a different table and orders them and assigns ID's and maybe other things
> Is there a table that contains the pending jobs?
>> -----Original Message-----
>> From: John Tran
>> Sent: Monday, August 24, 2015 9:18 AM
>> To: Chris Hyzer
>> Cc: John Tran;
>> Subject: Re: [grouper-dev] Grouper Processing Speed
>> Apologies, I just came across your response this morning. I’m not quite
>> sure how I missed it.
>> In the grouper_loader_log, there is started_time and ended_time, but those
>> are dates without timestamps. I see total_count which I assume is the
>> total number of changes in that job. However, I don’t see a field that
>> would indicate how many have been processed and/or how many are left to
>> process in that job.
>> If I’m correct to assume that grouper_change_log_entry_temp are the jobs
>> in queue, is there a way to see how large these jobs are?
>> John Tran
>> IS Technical Services Senior
>> Division of Information Technology
>> University of Wisconsin - Madison
>>> On Jul 30, 2015, at 10:51 AM, Chris Hyzer
>>> Does the grouper_loader_log have what you need for those jobs?
>>> -----Original Message-----
>>> On Behalf Of
>>> Sent: Wednesday, July 29, 2015 4:37 PM
>>> Subject: [grouper-dev] Grouper Processing Speed
>>> Hello All,
>>> We are in need of a solution that will give us pertinent processing
>>> information during CHANGE_LOG_changeLogTempToChangeLog jobs, specifically
>>> with a large amount of changes. Currently, I can see when the job has
>>> started, but haven't been able to figure out much more around processing.
>>> How can we see how many changes are in the current running job? Is there
>>> way to view the progress?
>>> On a related note, it would be nice to see at what rate changes are
>>> processing. We'd like to get a baseline, then monitor whether processing
>>> slower than normal.
>>> Thanks for your time,
>>> John Tran
>>> Department of Information Services
- Re: [grouper-dev] Grouper Processing Speed, John Tran, 08/24/2015
Archive powered by MHonArc 2.6.16.