Grouper Developers Forum

[Jim Fox <>]

I promised to run a trial upgrade from grouper 2.1.2 to 2.2.1 and
keep track of times.

The database is postgres on linux.

  #groups about 100K
  #member about 1M
  #audits about 25M

prod DB was V26

----------------------------------------------

1) copy prod DB to test DB
   [ Not really a grouper conversion step. ]

   $ (ssh iamdb11 '/usr/local/pgversion/bin/pg_dump -U postgres gws2'; echo 
'\q') | psql -U postgres gwstest1
   -- took 75 min


2) Analyze tables

3) Make the build script
   $ time bin/gsh.sh -registry -check
   -- took 65 min

4) Run the script
   $ time bin/gsh.sh -registry -runsqlfile 
.../grouperDdl_20141209_18_54_48_695.sql
   -- took 639 min

5) Run the misc updater
   $ time bin/gsh.sh misc/postGrouper2_2Upgrade.gsh

   part of this ended with an error
   
------------------------------------------------------------------------------------
   Searching for attribute assignments
   Found 433566 attribute assignments
   11:04:42: Processed 346 of 433566 (0%) of current phase.
   ...
   18:42:11: Processed 361156 of 433566 (83%) of current phase.  Estimated 
completion time: 20:14.
   18:42:41: Processed 361553 of 433566 (83%) of current phase.  Estimated 
completion time: 20:14.
   // Error: unable to evaluate command: Sourced file: inline evaluation of: 
``new MigrateLegacyAttributes().saveUpdates(true).fullMigration();'' : Method 
Invocation fullMigration
   // See error log for full stacktrace
   // caused by: 
edu.internet2.middleware.grouper.exception.AttributeDefNameAddException:
   // Cannot saveOrUpdate attribute assign: Exception in save: 
edu.internet2.middleware.grouper.attr.assign.AttributeAssign, 
edu.internet2.middleware.grouper.hibernate.ByObject@40471668,
   Problem in HibernateSession: HibernateSession (55f2ef75): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (61641f27),
   Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.attr.assign.AttributeAssign, ByObjectStatic, 
query: ', cacheable: null, cacheRegion: null, entityName: null, tx type: null,
   Problem in HibernateSession: HibernateSession (14df6357): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (61641f27),
   Problem in HibernateSession: HibernateSession (54fb7283): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (61641f27)
   // caused by: java.lang.NullPointerException:
   // Exception in save: 
edu.internet2.middleware.grouper.attr.assign.AttributeAssign, 
edu.internet2.middleware.grouper.hibernate.ByObject@40471668,
   Problem in HibernateSession: HibernateSession (55f2ef75): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (61641f27),
   Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.attr.assign.AttributeAssign, ByObjectStatic, 
query: ', cacheable: null, cacheRegion: null, entityName: null, tx type: null
   
------------------------------------------------------------------------------------

   -- that pass took 594 min

6) Reran the same misc job
   $ time bin/gsh.sh misc/postGrouper2_2Upgrade.gsh

   Found 148951 group type assignments
   Note: I don't use the PIT tables and did not run part 7/7 of this script

   completed this time without error - and faster as it had less to do
   -- took 190 min

7) Run the second misc script
   $ time bin/gsh.sh misc/postGrouper2_2Upgrade.gsh
   --  took 17 sec


Done.

Total time (not including the initial DB copy):   about 25 hours

Jim