Grouper Users - Open Discussion List

[Chris Hyzer <>]

Im not convinced it’s the DB or the mysql command line client / OS of command line client… maybe in the DB it is ok… hmmm

 

chris

 

From: Richard James [mailto:]
Sent: Tuesday, March 29, 2011 9:41 AM
To: Chris Hyzer;
Subject: RE: Foreign characters not displaying correctly in user interfaces

 

That is the same behaviour that I saw when I created a group called ‘tást’ through the UI, it converts the foreign character into another character representation in the database, but the UI can still understand it.

 

mysql> select name from grouper_groups where name like 'Pencil2:%';

+---------------+

| name          |

+---------------+

| Pencil2:tást |

+---------------+

1 row in set (0.00 sec)

 

When we load users into our subject table the accented character is displayed correctly within the database (command line/phpmyadmin) but then this changes to the question mark in the UI’s. Have you got any users/subjects within your grouper installation who’s name includes a character such as this?

 

Also I’m using Firefox, and the default character encoding is set to UTF-8, this displays the question mark. If I change this to a western setting (ISO-8859-1 or ISO-8859-15) the character is displayed correctly. I’ll keep looking...

 

 

Regards

 

Richard James

Infrastructure Systems Administrator

ISS Systems Architecture

Newcastle University

 

 

>-----Original Message-----

>From: Chris Hyzer [mailto:]

>Sent: 29 March 2011 14:11

>To: Richard James;

>Subject: RE: Foreign characters not displaying correctly in user

>interfaces

> 

>Fyi when I query my DB command line or in sqlyog, I don't see what I

>expect, but Im not sure why, Grouper shows the right thing on the

>screen...

> 

>mysql> select name from grouper_groups where name like 'test:%';

>+----------------+

>| name           |

>+----------------+

>| test:tást     |

>| test:testGroup |

>+----------------+

>2 rows in set (0.00 sec)

> 

>-----Original Message-----

>From: Richard James [mailto:]

>Sent: Tuesday, March 29, 2011 6:53 AM

>To: Chris Hyzer;

>Subject: RE: Foreign characters not displaying correctly in user

>interfaces

> 

>Hi Chris,

> 

>We are running this on a Unix machine. Yep that is the main problem we

>have had, trying to identify at what stage the characters aren't being

>interpreted correctly.

> 

>Currently our character variables show up as follows;

> 

>mysql> SHOW VARIABLES LIKE 'char%';

>+--------------------------+----------------------------+

>| Variable_name            | Value                      |

>+--------------------------+----------------------------+

>| character_set_client     | latin1                     |

>| character_set_connection | latin1                     |

>| character_set_database   | latin1                     |

>| character_set_filesystem | binary                     |

>| character_set_results    | latin1                     |

>| character_set_server     | latin1                     |

>| character_set_system     | utf8                       |

>| character_sets_dir       | /usr/share/mysql/charsets/ |

>+--------------------------+----------------------------+

>8 rows in set (0.00 sec)

> 

>It may be the case that these need to be utf8, although when querying

>the database via the command line or PhpMyAdmin the accented characters

>are displayed correctly. I will try to change these on our test box to

>see if it makes any difference. (The local variables shown on PhpMyAdmin

>are set to UTF-8).

> 

>It's the subject where we are coming across the problem, we load subject

>details into a subject table in the grouper database, when loading this

>data we have tried loading it using ISO-8859-1 and UTF-8 but this

>doesn't seem to change anything. The table has been setup so that it is

>using the utf8_unicode_ci collation. I was wondering if it might be when

>the JDBC connector for the subject sources connects to the database that

>it isn't using the correct char encoding? Just a guess not sure about

>the internals.

> 

>This might well be a separate issue from Grouper, I will keep on having

>a look into it and feedback any possible solutions I come across.

> 

>Cheers,

> 

>Richard

> 

> 

> 

> 

> 

> 

>Regards

> 

>Richard James

>Infrastructure Systems Administrator

>ISS Systems Architecture

>Newcastle University

> 

> 

>>-----Original Message-----

>>From: Chris Hyzer [mailto:]

>>Sent: 28 March 2011 17:33

>>To: Richard James;

>>Subject: RE: Foreign characters not displaying correctly in user

>>interfaces

>> 

>>Are you running the server on unix or windows?  Ive seen difficulties

>in

>>windows if it is setup a certain way to support these chars.

>>Unfortunately there are a lot of factors here: database OS, app server

>>OS, database charset, app server OS, browser, etc

>> 

>> 

>>Does this show utf-8 on all lines that it should?  (mine doesn't, does

>>yours?)

>> 

>>mysql> SHOW VARIABLES LIKE 'char%';

>>+--------------------------+----------------------------+

>>| Variable_name            | Value                      |

>>+--------------------------+----------------------------+

>>| character_set_client     | utf8                       |

>>| character_set_connection | utf8                       |

>>| character_set_database   | latin1                     |

>>| character_set_filesystem | binary                     |

>>| character_set_results    | utf8                       |

>>| character_set_server     | latin1                     |

>>| character_set_system     | utf8                       |

>>| character_sets_dir       | /usr/share/mysql/charsets/ |

>>+--------------------------+----------------------------+

>>8 rows in set (0.00 sec)

>> 

>> 

>>SHOW VARIABLES LIKE 'char%';

>> 

>> 

>> 

>>The grouperdemo server is on unix, and I didn't do anything special do

>>it, and it seems fine.  I created a group with a special char, and

>>attached are the screenshots for the UI and UI-lite with a windows

>>browser (unix server/db mysql)...

>> 

>>Thanks,

>>Chris

>> 

>> 

>> 

>> 

>>-----Original Message-----

>>From: [mailto:grouper-users-

>>] On Behalf Of Richard James

>>Sent: Monday, March 28, 2011 12:04 PM

>>To:

>>Subject: [grouper-users] Foreign characters not displaying correctly in

>>user interfaces

>> 

>>Hi,

>> 

>>We've recently come across a problem with foreign characters (most

>>notably accented characters) not displaying correctly in the main

>>Grouper UI and the Lite UI. It replaces these characters with a

>question

>>mark as shown in the attached screenshot. I saw a post in the mailing

>>list from February last year

>>(https://lists.internet2.edu/sympa/arc/grouper-users/2010-

>>02/msg00012.html) which discussed possible solutions to rectify the

>>problem, unfortunately we have not been able to come up with a

>solution.

>>I have tried amending the following files to use UTF-8, unfortunately

>>although the changes are reflected in the source code and in the

>browser

>>meta data, the character is still displayed incorrectly.

>> 

>>edu.internet2.middleware.grouper.ui.util.HttpContentType

>>grouper.html

>>head.jsp

>>template.jsp

>> 

>>The characters are displayed correctly in the database (mysql), the

>>collation for the column is set to utf8_unicode_ci. I've tried

>>explicitly setting a LANG environment variable to en_GB.UTF-8 on the

>>server, and also in the tomcat start-up script in case there was a

>>conflict at the end, but again to no avail.

>> 

>>A few little tests I tried to test behaviour was to, via the main UI,

>>change a name of group to include an accented character, i.e. tást.

>This

>>displays correctly in the UI, but if you look in the database it is

>>represented by tást.

>> 

>>I was wondering if anyone else has come across this issue? And if so if

>>can you suggest how we can overcome this problem.

>> 

>>Unfortunately within the subject search in the lite UI, the accented

>>character does also cause an xml load error (screenshot attached). This

>>happens when searching for a user where the name contains an foreign

>>character.

>> 

>>Thanks

>> 

>>Richie

>> 

>> 

>>Richard James

>>Infrastructure Systems Administrator

>>ISS Systems Architecture

>>Newcastle University

>>