I finished coding the subject resolution batching performance improvement… Penn has a subject picker which finds subjects who are employees, and it would take more than 20 seconds to search which caused a timeout. Now it takes a few seconds,
and the number of queries and amount of data is bounded (not by N
J ).
https://bugs.internet2.edu/jira/browse/GRP-713
Subject batching to not require N queries has been added to the following places:
UI subject picker when subject is in a group
UI get members
UI get members lite
WS get members
WS get privileges
WS get memberships
https://bugs.internet2.edu/jira/browse/GRP-712
There should be batching (for jdbc and jdbc2 source adapters) so that multiple subjects can be resolved at once. Note: for source adapters which do not implement a batched method to retrieve multiple subjects, they will be retrieved as
they were before one at a time.
If you use the jdbc2 source adapter, you dont have to configure anything differently, it will just work.
If you use the jdbc source adapter, compare the new sources.example.xml, you can set these properties
<!-- if you are going to use the inclause attribute
on the search to make the queries batchable when searching
by id or identifier -->
<init-param>
<param-name>useInClauseForIdAndIdentifier</param-name>
<param-value>true</param-value>
</init-param>
<!-- comma separate the identifiers for this row, this is for the findByIdentifiers if using an in clause -->
<init-param>
<param-name>identifierAttributes</param-name>
<param-value>LOGINID</param-value>
</init-param>
Then you can change your queries for id or identifier to have an inclause variable, and assign that variable for the inclause part:
<search>
<searchType>searchSubject</searchType>
<param>
<param-name>sql</param-name>
<param-value>
select
s.subjectid as id, s.name as name,
(select sa2.value from subjectattribute sa2 where name='name' and sa2.SUBJECTID = s.subjectid) as lfname,
(select sa3.value from subjectattribute sa3 where name='loginid' and sa3.SUBJECTID = s.subjectid) as loginid,
(select sa4.value from subjectattribute sa4 where name='description' and sa4.SUBJECTID = s.subjectid) as description,
(select sa5.value from subjectattribute sa5 where name='email' and sa5.SUBJECTID = s.subjectid) as email
from
subject s
where
{inclause}
</param-value>
</param>
<param>
<param-name>inclause</param-name>
<param-value>
s.subjectid = ?
</param-value>
</param>
</search>
<search>
<searchType>searchSubjectByIdentifier</searchType>
<param>
<param-name>sql</param-name>
<param-value>
select
s.subjectid as id, s.name as name,
(select sa2.value from subjectattribute sa2 where name='name' and sa2.SUBJECTID = s.subjectid) as lfname,
(select sa3.value from subjectattribute sa3 where name='loginid' and sa3.SUBJECTID = s.subjectid) as loginid,
(select sa4.value from subjectattribute sa4 where name='description' and sa4.SUBJECTID = s.subjectid) as description,
(select sa5.value from subjectattribute sa5 where name='email' and sa5.SUBJECTID = s.subjectid) as email
from
subject s, subjectattribute a
where
a.name='loginid' and s.subjectid = a.subjectid and {inclause}
</param-value>
</param>
<param>
<param-name>inclause</param-name>
<param-value>
a.value = ?
</param-value>
</param>
</search>
There are now these SubjectFinder methods:
SubjectFinder.findByIds(Collection<String>)
SubjectFinder.findByIds(Collection<String>, String)
SubjectFinder.findByIdentifiers(Collection<String>)
SubjectFinder.findByIdentifiers(Collection<String>, String)
SubjectFinder.findByIdsOrIdentifiers(Collection<String>)
SubjectFinder.findByIdsOrIdentifiers(Collection<String>, String)
There is this method in member to resolve subjects in a collection of members at once:
Member.resolveSubjects(Collection<Member>, boolean)
This method will resolve subjects in membership array results:
Membership.resolveSubjects(Collection<Object[]>)
This method will resolve subjects in privilege objects:
PrivilegeHelper.resolveSubjects(Collection<GrouperPrivilege>, boolean)
Thanks,
Chris
