grouper-users - [grouper-users] RE: Grouper Loader - Create reference groups from SQL rows?
Subject: Grouper Users - Open Discussion List
List archive
- From: "Hyzer, Chris" <>
- To: Carl Waldbieser <>, grouper-users <>
- Cc: Michael Gettes <>, John Schrader <>
- Subject: [grouper-users] RE: Grouper Loader - Create reference groups from SQL rows?
- Date: Wed, 25 Jul 2018 12:56:59 +0000
- Accept-language: en-US
- Authentication-results: spf=none (sender IP is ) ;
- Ironport-phdr: 9a23:dhyT5h8sHw/RD/9uRHKM819IXTAuvvDOBiVQ1KB21e0cTK2v8tzYMVDF4r011RmVBduds6oMotGVmpioYXYH75eFvSJKW713fDhBt/8rmRc9CtWOE0zxIa2iRSU7GMNfSA0tpCnjYgBaF8nkelLdvGC54yIMFRXjLwp1Ifn+FpLPg8it2O2+55zebx9UiDahfLh/MAi4oQLNu8cMnIBsMLwxyhzHontJf+RZ22ZlLk+Nkhj/+8m94odt/zxftPw9+cFAV776f7kjQrxDEDsmKWE169b1uhTFUACC+2ETUmQSkhpPHgjF8BT3VYr/vyfmquZw3jSRMNboRr4oRzut86ZrSAfpiCgZMT457HrXgdF0gK5CvR6tuwBzz4vSbYqINvRxY7ndcMsaS2RfQ8hRSyJPDICyb4QNDuoOIelWoIbmqlsSthSyGRWgCP/zxjJKgHL9wK000/4mEQHDxAEsEdAAsXTVrdrrLqceSf2+wbfIzTXZbvNWwjnx45XPfxAkvf6MXbJxcMvPxkk1DA7Fj06QppD/MzOIzuQCrW6b7+x6We2xlmEnthh8rz6yzckvkonEnpwZxkzG+Cljz4s4JMe0RFN6bNOqCpddtyCXO5NrTs4sX21koiI3xqActZKlYSQHzI4rywPdZvGGa4SI7AzsWeWNLTp9gX9oe6+whxi9/EWlyeDxWNS730xPoypLjNXBuX8A2hnP5cWEUPRx4EGs0iuV2Q/J8OFLO0U0mLLbK5E/xr4wkYIev1zfEyHxhEn6kraae1sm9eS18ujnZa7pqYGGO49zlwH+Lr8hmsuiAeQ+LwcCRXCb+f671L3/40L2XKlKjvwxkqnfqpzaItkbprK9Aw9S1YYj6AyzACuh0NQdhXUHLVRFdwybj4XxJV3CPPL1Ae2wjlmtijtn2vHLM7P7DpjNL3XPiLLhcqx8605Yxgoz19df55dMB7EEPP3zWk/wtd3GAR8jKAG0wv3nB8ln2oMDQWKAHKmZMKXIvVCW++0vPvGAZJUJtzblN/gl+/nugGc2mV8HeqmpwIMXZ26iHvh/PkWZemTjgs0aEWoRugo+TffqiEGZUTJNfXqyXqQ85i0lB4K8C4fMWJytjKKb0CilA5JWezMONlfZOn7yc827UvMAYiOfOcJ72mgbXLytQY4s3jmqvQHz2rd7MueS9yEF49ar28Jy+vXej1Qv7jFuFIzJ3HuKUnl5hCYVXDIsx4h+p1Bw0FGOzfI+jvBFQ499/fRMB00aJIzR1agyINDoWxmLNoOMQ1a3UNi8KTAqRZQs29IIZQBwF8j03UOL5DajH7JAz+/DP5cz6K+JmiGpf5wnmX/byKksiUUnScJTNGqgw7Ry7BXXG5WXzxeCj6j/c6Ma0WaN72qFwWeU9GBgGA9rGeSgPzgEY1fO69Hw50fMVbirXLEsMhpTxNSqK7BBLMDxgFNAAvrvJYeWbg==
- Spamdiagnosticmetadata: NSPM
- Spamdiagnosticoutput: 1:99
Depends how you set it up.
If you have a SQL like string configured, it will get deleted.
If you have a group query, it will lose all memberships I believe.
Thanks
Chris
-----Original Message-----
From:
[mailto:]
On Behalf Of Carl Waldbieser
Sent: Tuesday, July 24, 2018 4:00 PM
To: grouper-users
<>
Cc: Michael Gettes
<>;
John Schrader
<>
Subject: Re: [grouper-users] Grouper Loader - Create reference groups from
SQL rows?
I finally managed to get this working, so now I have a follow up question.
What happens if my data changes such that one of the groups that has been
dynamically created goes away in the data?
I'm guessing that the group remains in Grouper but doesn't have any members
in it after the next time the loader runs? Is that correct?
Thanks,
Carl Waldbieser
ITS Identity Management
Lafayette College
----- Original Message -----
From: "John Schrader"
<>
To: "waldbiec"
<>
Cc: "Michael Gettes"
<>,
"grouper-users"
<>
Sent: Friday, 13 July, 2018 15:58:42
Subject: Re: [grouper-users] Grouper Loader - Create reference groups from
SQL rows?
fwiw ND has a structure:
etc:loader:banner:{a_group_that_represents_the_SQL_GROUP_LIST}
The loader attributes are set on this group.
We also use the grouperLoaderGroupQuery attribute (a distinct list of
groups to load)
# This loader job will create and load orgCode reference groups from
blablabla.
grouperSession = GrouperSession.startRootSession();
addGroup("etc:loader:banner", "orgCodes", "orgCodes");
cloaderGroup = GroupFinder.findByName(grouperSession,
"etc:loader:banner:orgCodes");
groupAddType(cloaderGroup.name, "grouperLoader");
setGroupAttr(cloaderGroup.name, "grouperLoaderDbName", "banner");
setGroupAttr(cloaderGroup.name, "grouperLoaderGroupQuery", "SELECT DISTINCT
lower('nd:basis:banner:employee:orgcode:' || ORGN_LEVEL_8) AS GROUP_NAME,
ORGN_LEVEL_8_TITLE AS GROUP_DESCRIPTION FROM bla bla bla WHERE BLA");
setGroupAttr(cloaderGroup.name, "grouperLoaderQuery", "SELECT
lower('nd:basis:banner:employee:orgcode:' || ORGN_LEVEL_8) AS GROUP_NAME,
NDID AS SUBJECT_IDENTIFIER, 'nd' AS SUBJECT_SOURCE_ID FROM bla bla bla
WHERE BLA");
setGroupAttr(cloaderGroup.name, "grouperLoaderType", "SQL_GROUP_LIST");
setGroupAttr(cloaderGroup.name, "grouperLoaderQuartzCron", "02 45 11,22 * *
?");
setGroupAttr(cloaderGroup.name, "grouperLoaderScheduleType", "CRON");
loaderRunOneJob(cloaderGroup);
Happy Friday !
-John
On Fri, Jul 13, 2018 at 3:51 PM Carl Waldbieser
<>
wrote:
> Michael,
>
> Thanks-- I think I get it, except-- what do you assign the LoaderType
> properties on? The stem? Some random group?
>
> Thanks,
> Carl Waldbieser
> ITS Identity Management
> Lafayette College
>
> ----- Original Message -----
> From: "Michael Gettes"
> <>
> To: "waldbiec"
> <>
> Cc: "grouper-users"
> <>
> Sent: Friday, 13 July, 2018 15:36:34
> Subject: Re: [grouper-users] Grouper Loader - Create reference groups from
> SQL rows?
>
> Change your SQL to do the following:
>
> SELECT SUBJECT_ID, ‘Stem1:Stem2:’||grpname as GROUP_NAME
> From tables
> Where conditions;
>
> And configure this as Loadertype SQL_GROUP_LIST and you’re done.
>
> Later you can do things like SELECT SUBJECT_ID,
> ‘basis:byDept:’||DeptID||’:Staff:’StaffLevel as GROUP_NAME and now you are
> creating multi-level groups. Be sure to set things such as 'Groups like
> sql part’ and 'Group query (metadata on groups)’ as needed.
>
> A single SQL query with a loader job can generate 10s of thousands of
> groups. Did this at PSU and managed 90,000 course groups in 1 hour daily
> run.
>
> Happy to discuss by phone if you wish.
>
> /mrg
>
> On Jul 13, 2018, at 3:08 PM, Carl Waldbieser
> <
> <mailto:>>
> wrote:
>
>
> I have rows coming from a SQL connection like this:
>
> SUBJECT_ID GROUP_NAME
> ========== ==========
> subject-A group-1
> subject-B group-1
> subject-C group-2
> subject-D group-1
> ...
>
> I'd like the loader to create "group-1" and "group-2" and "group-N" in a
> particular stem and populate the memberships, but I'm unsure if (1) the
> loader can do that, and (2) how to configure the loader to do that if it is
> possible.
>
> Any pointers would be great!
>
> Thanks,
> Carl Waldbieser
> ITS Identity Management
> Lafayette College
>
--
John Schrader
Identity and Access Management
Office of Information Technologies
University of Notre Dame
EVERYTHING SHOULD BE MADE AS SIMPLE AS POSSIBLE, BUT NOT ANY SIMPLER
—ALBERT EINSTEIN
- [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Carl Waldbieser, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Gettes, Michael, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Carl Waldbieser, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, John Schrader, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Carl Waldbieser, 07/24/2018
- [grouper-users] RE: Grouper Loader - Create reference groups from SQL rows?, Hyzer, Chris, 07/25/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Carl Waldbieser, 07/24/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Gettes, Michael, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, John Schrader, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Carl Waldbieser, 07/13/2018
- Re: [grouper-users] Grouper Loader - Create reference groups from SQL rows?, Gettes, Michael, 07/13/2018
Archive powered by MHonArc 2.6.19.