Skip to Content.
Sympa Menu

grouper-users - [grouper-users] export of groups with all member (count ~33k members) data fails under some conditions ( smaller groups work under all conditions)

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] export of groups with all member (count ~33k members) data fails under some conditions ( smaller groups work under all conditions)


Chronological Thread 
  • From: "Black, Carey M." <>
  • To: "" <>
  • Subject: [grouper-users] export of groups with all member (count ~33k members) data fails under some conditions ( smaller groups work under all conditions)
  • Date: Fri, 9 Feb 2018 20:16:08 +0000
  • Accept-language: en-US
  • Authentication-results: spf=pass (sender IP is 128.146.163.18) smtp.mailfrom=osu.edu; internet2.edu; dkim=none (message not signed) header.d=none;internet2.edu; dmarc=pass action=none header.from=osu.edu;
  • Ironport-phdr: 9a23:xPbFbRWjXWTnVbH20ZAoJY8IW33V8LGtZVwlr6E/grcLSJyIuqrYbRGCt8tkgFKBZ4jH8fUM07OQ7/i5HzRYqb+681k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i764jEdAAjwOhRoLerpBIHSk9631+ev8JHPfglEnjWwba9vIBmssQndqtQdjJd/JKo21hbHuGZDdf5MxWNvK1KTnhL86dm18ZV+7SleuO8v+tBZX6nicKs2UbJXDDI9M2Ao/8LrrgXMTRGO5nQHTGoblAdDDhXf4xH7WpfxtTb6tvZ41SKHM8D6Uaw4VDK/5KptVRTmijoINyQh/W/XlMJ+kb5brhyiqRx+34Hab46aOeFifqPEed4WWXZNUtpPWyFHH4iyb5EPD+0EPetAoYXwul8OogGiCgmvHO/k1yFFiWXs3aIk0+UqDAbI0BYuE9kTt3nbt9H1O70OXuC21qXE0CvPYv1N1Df684jIbwotru+RUrJtaMfcz1QkGQ3CjlWVs4PlPjWV2/wKs2iA8upsT/yghHM6qwx3vzOhxd8sh5HXio0P1lzI6CB0zJsoKdGlTUN7b9GpHIdMuyyfOYt5XMwvT31ttSs/yLAKp5C7cDAXxJg5xBPTceGLfoeJ7x75UOucLi10iXBmdb6niRa970utxvH5W8Sx0FtFsjZKktfJu3wT2BHe69WIR/5y80qh2zuC2QXe5f1fLk8vj6XWLYMqzKQqmZoJq0vDGzf7mEXog6+ScUUp4vCm5vjgbLn6u5OQLpJ6hBnjPqg3ncywGvo3PhIJX2iG5eSzz7rj/VD/QLpXlPE2irPZsJfGJcsFuqG5Hw5V0oEl6xqlCDemzcgYnX0ALFJCexKLlZTmO1bLIPzgDPe/hUqjkCtzyvzYJLLtHojBI33enLv8YLpw7kBcxQUvwdxD4p9UBK8OLf3vVkL0sdHVDAE1MwKsz+biEtp914ceWWyVAq+eNaPfqUSH5uY1LOaQZ48ZoivxJ+Yr6vP1in85gkIdcbO30pQKdXC0Bu5mLFmBYXrwntcBFn8HvgUkTOztlV2CSSBcZ2yrU6Ih/TE0FpimDZzYS4CpgbyBxzu7HoZIamxcC1CMF2voeJueW/cKdi2SPtFtniYaWre8Vo9ynS2p4UX10b16NufOvzADuIj4/Nlz++DJkxwur3p5A9nXmzWCVWZphm4SAiIt0bpkiU171lqZ16Vk2bpVGcEFtN1TVQJvf7TYxud5T5jZUxjMb5+sDhzuFt+iCDoyCItrmPcJeFs7Ftm/2EOQlxG2CqMYwuTYTKc/9bjRiiD8
  • Spamdiagnosticmetadata: NSPM
  • Spamdiagnosticoutput: 1:99

All,

I am still kicking the UI tires and I found something curious.

Open a group
"More actions" (button/menu)
--> "Export Members"
If I export a group it fails ... sometimes.

Using a group with about 310 members. (If it matters: All direct subject
members, no groups as members)
If I export the group with "only member IDs" then it works.
I get a CSV with these columns:
sourceId,entityId

If I export the group with "All member data" then it works.
I get a CSV with these columns:

sourceId,screenLabel,entityId,name,description,success,errorMessage

Using a group with about 764 members. (If it matters: All direct subject
members, no groups as members)
If I export the group with "only member IDs" then it works.
If I export the group with "All member data" then it works.



Using a group with about 12.5k members. (If it matters: All direct subject
members, no groups as members)
or
Using a group with about 33k members. (If it matters: All direct subject
members, no groups as members)

If I export the group with "only member IDs" then it works. ( after
about 7 seconds)

If I export the group with " All member data" then it fails. (after
about 60 seconds [ measured several times ])
The UI session ends on a URL like this:

https://....osu.edu/.../grouperUi/app/UiV2GroupImport.groupExportSubmit/groupId%3decfbfc3dd4b848209e52d2c600740414/all/groupExportAll_Group.csv
With this page:
"
Service Unavailable
The server is temporarily unable to service your request due to
maintenance downtime or capacity problems. Please try again later.
"

Tomcat did not fail. ( I can go back to the root of the site ( or use
the back button) and my session is still good, and everything works.)
I find nothing in the UI Error.log. (and the debug log is
empty)





So it seems (on the surface) like a scaling issue with the UI export
function. But it could be much more complicated, or simpler, than that too. (
Shrug )

Any ideas what to look at to fix this?
Log level suggestions?
Any config suggestions? ( a timeout setting somewhere? )
Is the Subject API being called with the "All member data" option or
are more grouper RDMBS data columns being returned?


Also:
Any ideas about how to HIDE this option?
Is there a Grouper UI ACL to restrict access to export members?
Is there a Grouper UI ACL to restrict access to *import members*?
Is there a Grouper UI ACL to restrict access to *Remove all members*?
I could hack at the JSP to remove it, but it might be nice
for some users and dangerous in the hand of others. :) [ So... ACL's to
grant/revoke would be nice....]
Maybe "Admin" grants these? ( It is hard to test with Shib in
front of the UI. I kind of need to "step down" to a lesser access to test

--
Carey Matthew




Archive powered by MHonArc 2.6.19.

Top of Page