Skip to Content.
Sympa Menu

grouper-users - Loading Module Information into Grouper using Talend

Subject: Grouper Users - Open Discussion List

List archive

Loading Module Information into Grouper using Talend

Chronological Thread 
  • From:
  • To:
  • Subject: Loading Module Information into Grouper using Talend
  • Date: Mon, 19 Oct 2009 08:28:33 -0400 (EDT)

Hi Everyone,

Inspired by Chris’ email regarding the management of class lists using the
grouper loader I thought I would share what we have been doing at Newcastle
University to provision students module enrolments into Grouper.

Module information, including course details, and student enrolments for
Newcastle University are held in a database which the Faculty of Medical
Sciences (FMSC) holds the responsibility for. We would normally take
advantage of the grouper loader, however due to factors including database
permissions and some required restructuring of the data we have not been able
to this in this scenario. We find the loader to be a very useful feature, and
as Grouper has access to the corporate data for the University’s departmental
structure, we are able to use the Grouper Loader to create our Organisational
structure, which we find very beneficial to provide an up to date
representation of the organisation. This would also be our desired route for
the provisioning of module data into grouper, and as the IDMAPs project
continues to develop, we hope to be able implement this in the future.

As a consequence of not being able to use the loader, we needed a method of
extracting the appropriate information from the database and then feed this
into Grouper, the method to do this has been created using the Open Source
data integration suite, Talend. As part of a JISC funded project, IDMAPS, we
have recently started to using Talend to try and address the problems posed
by data flows which are inconsistent, badly documented and hard to maintain.

With Talend, we have been able to produce a job which allows us to extract
the data from the FMSC’s database, manipulate the data into the required
format, and feed this data into Grouper using the Grouper Shell, also
allowing us to keep an audit trail of any changes made to modules and module
enrolments. To demonstrate how we have achieved this we have created some
videos which show the process of extracting the data from the source location
and feeding this into Grouper, the videos can be found at This is the first time we have
created videos for this purpose, and we have encountered some problems with
the videos being temperamental at times, we have tried to overcome this by
splitting the video into a number of parts.

Talend alone is a very useful tool for the creation and maintainability of
the numerous data feeds throughout an organisation, in the short time of
using it we have already seen how it can improve the current flows of data
throughout the University. It is provides a quick and efficient way of
producing data feeds to meet a systems required data rules, and allows you to
easily augment data into one overall data flow. This was recently required to
provision role based access control from Grouper into the Scientia Syllabus
Plus Room Booking service, for this we needed to bring together data from
Grouper and various SAP data sources. We are currently in the process of
producing a video to demonstrate how we do this, and will share this in the
near future.

I hope the videos are of some interest,


Richard James
ISS Middleware Team
Newcastle University

Archive powered by MHonArc 2.6.16.

Top of Page