Grouper Developers Forum

[Chris Hyzer <>]

Can you put a handful of tests in there so we can see the breakdown between 
startup time, and test time.  

Thanks,
Chris

-----Original Message-----
From: 

 
[mailto:]
 On Behalf Of Tom Zeller
Sent: Friday, September 30, 2011 3:57 PM
To: Shilen Patel
Cc: Grouper Dev
Subject: Re: [grouper-dev] help with writing changeLog tests ?

Right.

The following is a test I just cobbled together. The changelog entries
are from grouper initialization.

 postgres : Time: 70.81
 hsql: Time: 4.415

So postgres needed over a minute and hsql a few seconds.


package edu.internet2.middleware.grouper.pit;

import junit.textui.TestRunner;

import org.apache.commons.lang.time.StopWatch;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity;
import edu.internet2.middleware.grouper.helper.GrouperTest;

public class TomTest extends GrouperTest {

  protected static final Logger LOG = LoggerFactory.getLogger(TomTest.class);

  public void testOne() {
    LOG.info("starting");
    StopWatch sw = new StopWatch();
    sw.start();
    ChangeLogTempToEntity.convertRecords();
    sw.stop();
    LOG.info(sw.toString());
  }
}

On Fri, Sep 30, 2011 at 1:52 PM, Shilen Patel 
<>
 wrote:
> The time it takes to run that depends on how many changes you have in your
> temp change log among other things.  Is there a particular junit test that
> is having this slow response time when using postgres?  If so, I can try
> to reproduce it and take a look.
>
> Thanks!
>
> -- Shilen
>
> On 9/30/11 2:22 PM, "Tom Zeller" 
> <>
>  wrote:
>
>>Thanks.
>>
>>I found that in GrouperLoaderType, in the CHANGE_LOG enum, the following
>>
>> 826 ChangeLogTempToEntity.convertRecords(hib3GrouploaderLog);
>>
>>seems slow (10s per test) using postgres, it is faster with hsql.
>>
>>I guess it is the PIT saveOrUpdate() stuff.
>>
>>This may be of concern, perhaps the postgres ddl needs tweaking, I
>>have no idea how :-)
>>
>>On Fri, Sep 30, 2011 at 11:06 AM, Chris Hyzer 
>><>
>>wrote:
>>> I think you can make it think it is actually running to be a more
>>>realistic test... i.e. configure it for tests in the grouper-loader
>>>testConfig in a startUp() [and remove in tearDown()]:
>>>
>>>
>>>GrouperLoaderConfig.testConfig.put("changeLog.consumer.ldappcng.class",
>>>LdappcngConsumer.class.getName());
>>>
>>>GrouperLoaderConfig.testConfig.put("changeLog.consumer.ldappcng.quartzCro
>>>n", "0 0 8 * * ?");
>>>
>>> Then run it to register the consumer and clear out the change log:
>>>
>>>    GrouperLoader.runOnceByJobName(grouperSession,
>>>GrouperLoaderType.GROUPER_CHANGE_LOG_TEMP_TO_CHANGE_LOG);
>>>    GrouperLoader.runOnceByJobName(grouperSession,
>>>GrouperLoaderType.GROUPER_CHANGE_LOG_CONSUMER_PREFIX + "ldappcng");
>>>
>>> Do stuff:
>>>
>>> Call those again and it will process whatever you did
>>>
>>>    GrouperLoader.runOnceByJobName(grouperSession,
>>>GrouperLoaderType.GROUPER_CHANGE_LOG_TEMP_TO_CHANGE_LOG);
>>>    GrouperLoader.runOnceByJobName(grouperSession,
>>>GrouperLoaderType.GROUPER_CHANGE_LOG_CONSUMER_PREFIX + "ldappcng");
>>>
>>> Ok?
>>>
>>> Thanks,
>>> Chris
>>>
>>> -----Original Message-----
>>> From: 
>>> 
>>>  
>>> [mailto:]
>>>  On Behalf Of Tom
>>>Zeller
>>> Sent: Friday, September 30, 2011 10:31 AM
>>> To: Grouper Dev
>>> Subject: [grouper-dev] help with writing changeLog tests ?
>>>
>>> I am starting to write tests for the ldappcng changelog consumer. Is
>>> this the correct way to write changelog tests ?
>>>
>>> public void testSomething() {
>>>
>>>  ldappcngConsumer.runChangeLog();
>>>
>>>  ChangeLogTempToEntity.convertRecords();
>>>
>>>  // clear changelog
>>>  HibernateSession.byHqlStatic().createQuery("delete from
>>> ChangeLogEntryTemp").executeUpdate();
>>>  HibernateSession.byHqlStatic().createQuery("delete from
>>> ChangeLogEntryEntity").executeUpdate();
>>>
>>>  // do something e.g. group.addMember();
>>>
>>>  ChangeLogTempToEntity.convertRecords();
>>>
>>>  ldappcngConsumer.runChangeLog();
>>>
>>>  // verify test
>>> }
>>>
>>> And here is the runChangeLog() method :
>>>
>>> public void runChangeLog() {
>>>    LOG.info("runChangeLog");
>>>    Hib3GrouperLoaderLog hib3GrouploaderLog = new Hib3GrouperLoaderLog();
>>>    hib3GrouploaderLog.setHost(GrouperUtil.hostname());
>>>    hib3GrouploaderLog.setJobName(CHANGELOG_JOB_NAME);
>>>    hib3GrouploaderLog.setStatus(GrouperLoaderStatus.RUNNING.name());
>>>    hib3GrouploaderLog.store();
>>>
>>>    try {
>>>      ChangeLogHelper.processRecords(CHANGELOG_JOB_NAME,
>>> hib3GrouploaderLog, this);
>>>      hib3GrouploaderLog.setStatus(GrouperLoaderStatus.SUCCESS.name());
>>>    } catch (Exception e) {
>>>      LOG.error("Error processing records", e);
>>>      hib3GrouploaderLog.setStatus(GrouperLoaderStatus.ERROR.name());
>>>    }
>>>    hib3GrouploaderLog.store();
>>>  }
>>>
>
>