Grouper Users - Open Discussion List

[Richard James <>]

Hi Chris,

We are running this on a Unix machine. Yep that is the main problem we have 
had, trying to identify at what stage the characters aren't being interpreted 
correctly.

Currently our character variables show up as follows;

mysql> SHOW VARIABLES LIKE 'char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

It may be the case that these need to be utf8, although when querying the 
database via the command line or PhpMyAdmin the accented characters are 
displayed correctly. I will try to change these on our test box to see if it 
makes any difference. (The local variables shown on PhpMyAdmin are set to 
UTF-8). 

It's the subject where we are coming across the problem, we load subject 
details into a subject table in the grouper database, when loading this data 
we have tried loading it using ISO-8859-1 and UTF-8 but this doesn't seem to 
change anything. The table has been setup so that it is using the 
utf8_unicode_ci collation. I was wondering if it might be when the JDBC 
connector for the subject sources connects to the database that it isn't 
using the correct char encoding? Just a guess not sure about the internals.

This might well be a separate issue from Grouper, I will keep on having a 
look into it and feedback any possible solutions I come across.

Cheers,

Richard






Regards

Richard James
Infrastructure Systems Administrator
ISS Systems Architecture
Newcastle University


>-----Original Message-----
>From: Chris Hyzer 
>[mailto:]
>Sent: 28 March 2011 17:33
>To: Richard James; 
>
>Subject: RE: Foreign characters not displaying correctly in user
>interfaces
>
>Are you running the server on unix or windows?  Ive seen difficulties in
>windows if it is setup a certain way to support these chars.
>Unfortunately there are a lot of factors here: database OS, app server
>OS, database charset, app server OS, browser, etc
>
>
>Does this show utf-8 on all lines that it should?  (mine doesn't, does
>yours?)
>
>mysql> SHOW VARIABLES LIKE 'char%';
>+--------------------------+----------------------------+
>| Variable_name            | Value                      |
>+--------------------------+----------------------------+
>| character_set_client     | utf8                       |
>| character_set_connection | utf8                       |
>| character_set_database   | latin1                     |
>| character_set_filesystem | binary                     |
>| character_set_results    | utf8                       |
>| character_set_server     | latin1                     |
>| character_set_system     | utf8                       |
>| character_sets_dir       | /usr/share/mysql/charsets/ |
>+--------------------------+----------------------------+
>8 rows in set (0.00 sec)
>
>
>SHOW VARIABLES LIKE 'char%';
>
>
>
>The grouperdemo server is on unix, and I didn't do anything special do
>it, and it seems fine.  I created a group with a special char, and
>attached are the screenshots for the UI and UI-lite with a windows
>browser (unix server/db mysql)...
>
>Thanks,
>Chris
>
>
>
>
>-----Original Message-----
>From: 
>
> [
>]
> On Behalf Of Richard James
>Sent: Monday, March 28, 2011 12:04 PM
>To: 
>
>Subject: [grouper-users] Foreign characters not displaying correctly in
>user interfaces
>
>Hi,
>
>We've recently come across a problem with foreign characters (most
>notably accented characters) not displaying correctly in the main
>Grouper UI and the Lite UI. It replaces these characters with a question
>mark as shown in the attached screenshot. I saw a post in the mailing
>list from February last year
>(https://lists.internet2.edu/sympa/arc/grouper-users/2010-
>02/msg00012.html) which discussed possible solutions to rectify the
>problem, unfortunately we have not been able to come up with a solution.
>I have tried amending the following files to use UTF-8, unfortunately
>although the changes are reflected in the source code and in the browser
>meta data, the character is still displayed incorrectly.
>
>edu.internet2.middleware.grouper.ui.util.HttpContentType
>grouper.html
>head.jsp
>template.jsp
>
>The characters are displayed correctly in the database (mysql), the
>collation for the column is set to utf8_unicode_ci. I've tried
>explicitly setting a LANG environment variable to en_GB.UTF-8 on the
>server, and also in the tomcat start-up script in case there was a
>conflict at the end, but again to no avail.
>
>A few little tests I tried to test behaviour was to, via the main UI,
>change a name of group to include an accented character, i.e. tást. This
>displays correctly in the UI, but if you look in the database it is
>represented by tást.
>
>I was wondering if anyone else has come across this issue? And if so if
>can you suggest how we can overcome this problem.
>
>Unfortunately within the subject search in the lite UI, the accented
>character does also cause an xml load error (screenshot attached). This
>happens when searching for a user where the name contains an foreign
>character.
>
>Thanks
>
>Richie
>
>
>Richard James
>Infrastructure Systems Administrator
>ISS Systems Architecture
>Newcastle University
>