Grouper Users - Open Discussion List

[Yoann Delattre <>]

Yes, it works.
Because i think in server.xml tomcat file, you don't have URIEncoding="UTF-8".
But with this option it's the breadcrumb now who doesn't work...

If you click on "accentó"



You have this error :




I'l try to resume :-D

You set URIEncoding="UTF-8" in server.xml tomcat file. The breadcumbs works but not auto-completion with ComboBox.
In this case, with the getParameter method, you have a double encoding.

You don't set URIEncoding="UTF-8". Auto-completion with ComboBox works but not the breadcumb.
Without URIEncoding parameter tomcat default encoding is ISO-8859-1. So it should works, but the Content Type of the http request (when you click on accentó) is : application/x-www-form-urlencoded; charset=UTF-8. It's UTF-8 but tomcat will treat that like ISO-8859-1. Conclusion : it doesn't work.

I don't know if it's really what happened. But it should be something like that.

The best way (at least, for us), i think, it's to assume that everything is in UTF-8 and configure convertInputToUtf8 to false with URIEncoding="UTF-8" in server.xml.
Don't know if it's the best solution but, for us, it seems to be a logical solution.

Yoann Delattre	✆	03 20 95 69 61	✉
Équipe SIAD (Systèmes d'Information et Aide à la Décision) DSI de l'académie de Lille (Direction des Systèmes d'Information) 110 avenue Gaston Berger - 59000 Lille

Le 21/08/2014 00:32, Chris Hyzer a écrit :

Im trying to reproduce this on the demo server and I cannot.  Can you reproduce on the demo server?  Please let me know the exact steps.

 

Folder with accent:

 

https://grouperdemo.internet2.edu/grouper_v2_2/grouperUi/app/UiV2Main.index?operation=UiV2Stem.viewStem&stemId=ecc7d19e9b874f73b10748e73e0cff3e

 

Here is a search for an accented char:

 

 

 

 

Here is an accented folder found in the combobox:

 

 

 

 

From: Yoann Delattre []
Sent: Wednesday, August 20, 2014 11:49 AM
To: Chris Hyzer;
Subject: Re: [grouper-users] Accented characters and ComboBox in New UI

 

Hi,

i agree, i use a PostgreSQL database, there is no problem doing a basic search with accented character.
It's only when i was using ComboBox. I had no result at all. Because GET variable "name" was badly encoded using getParameters method.



In DojoComboLogic.java, with this debug code :

        if (StringUtils.isBlank(query)) {
                  LOG.debug(request.getCharacterEncoding());
                        LOG.debug(request.getParameter("name"));
          isLookup = false;

        query = StringUtils.trimToEmpty(request.getParameter("name"));
        }

i had something like this in my log :

2014-08-20 09:24:42,671: [http-8080-4] DEBUG DojoComboLogic.logic(109) -  - UTF-8
2014-08-20 09:24:42,671: [http-8080-4] DEBUG DojoComboLogic.logic(112) -  - Recherche avec :éété*

With the normal search, like this :



There was no problem :). It was only when i was using ComboBox and the auto-completion...

I don't know if it's a default option but in server.xml from tomcat6, there is this parameter who already encoded in UTF-8 all GET/POST variables :
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               URIEncoding="UTF-8"
               redirectPort="8443" />

so in addition with getParameters method, i think the result is a double encoding. And for me, that's why the GET variable "name" was badly encoded.

I actually play with Grouper 2.2 and with convertInputToUtf8= false, i have no problem... I check frequently our database if the data is still in UTF-8 and no problem...

I don't know if my explication is easy to understand :-\

Yoann Delattre	✆	03 20 95 69 61	✉
Équipe SIAD (Systèmes d'Information et Aide à la Décision) DSI de l'académie de Lille (Direction des Systèmes d'Information) 110 avenue Gaston Berger - 59000 Lille

Le 20/08/2014 16:41, Chris Hyzer a écrit :

When I use 2.2 without any changes, I created a folder with an accented character, I am using mysql, I did a search, and it sent the accented char to the DB (from p6spy).  Mysql by default will treat that accented char as an “e” I guess…  so you get more results than you think you should, but it seems to be working, right?  If you don’t convert input to utf8, will creating objects as utf8 still work?  I think there was something that wasn’t working that it helped with J

 

 

2014/08/20 10:35:46:840, 7ms, statement: ByHqlStatic.java.listSet() line 421, Hib3StemDAO.java.getAllStemsSecureHelper() line 1533, Hib3StemDAO.java.getAllStemsSecure() line 2132, StemFinder.java.findStems() line 628, UiV2Stem.java.search() line 1484, DojoComboLogic.java.logic() line 133, UiV2Stem.java.createGroupParentFolderFilter() line 1462, GrouperUtil.java.invokeMethod() line 3951, GrouperUtil.java.callMethod() line 3902, GrouperUiRestServlet.java.doGet() line 277, GrouperUiFilter.java.doFilter() line 985

   select distinct stem0_.id as id32_, stem0_.hibernate_version_number as hibernate2_32_, stem0_.parent_stem as parent3_32_, stem0_.name as name32_, stem0_.display_name as display5_32_, stem0_.creator_id as creator6_32_, stem0_.create_time as create7_32_, stem0_.modifier_id as modifier8_32_, stem0_.modify_time as modify9_32_, stem0_.display_extension as display10_32_, stem0_.extension as extension32_, stem0_.description as descrip12_32_, stem0_.last_membership_change as last13_32_, stem0_.alternate_name as alternate14_32_, stem0_.context_id as context15_32_, stem0_.id_index as id16_32_ from grouper_stems stem0_ where lower(stem0_.name) like '%ést%' or lower(stem0_.display_name) like '%ést%' or lower(stem0_.description) like '%ést%' or stem0_.id='ést' order by stem0_.display_name asc limit 200

2014/08/20 10:35:46:870, resultset: ALIASES: 0: id32_, 1: hibernate2_32_, 2: parent3_32_, 3: name32_, 4: display5_32_, 5: creator6_32_, 6: create7_32_, 7: modifier8_32_, 8: modify9_32_, 9: display10_32_, 10: extension32_, 11: descrip12_32_, 12: last13_32_, 13: alternate14_32_, 14: context15_32_, 15: id16_32_

2014/08/20 10:35:46:871, resultset: 0: 183821004210420fade310db3a72923f, 1: 0, 2: 267b5c51292a46c1bb1a2022a7b2fc68, 3: someStem, 4: someStem, 5: dc7cb1680c3c48bab2499cd3169f6418, 6: 1408003384426, 7: null, 8: 0, 9: someStem, 10: someStem, 11: null, 12: null, 13: null, 14: d1afb6813bde439a8a1ce05ce22d8e57, 15: 10160

2014/08/20 10:35:46:876, resultset: 0: f19aaeb3580f4aa499d67567b9da5947, 1: 0, 2: 267b5c51292a46c1bb1a2022a7b2fc68, 3: someStem3, 4: someStem3, 5: dc7cb1680c3c48bab2499cd3169f6418, 6: 1408004060711, 7: null, 8: 0, 9: someStem3, 10: someStem3, 11: null, 12: null, 13: null, 14: 92ece379ee5a484cbf1d234425c35a18, 15: 10171

2014/08/20 10:35:46:879, resultset: 0: 7e2672121fe0496ea27b0cf6d679b7c0, 1: 0, 2: 267b5c51292a46c1bb1a2022a7b2fc68, 3: test, 4: test, 5: dc7cb1680c3c48bab2499cd3169f6418, 6: 1400681869188, 7: null, 8: 0, 9: test, 10: test, 11: null, 12: null, 13: null, 14: 10f9212216b94b2abbaba6565e92caed, 15: 10031

 

From: [] On Behalf Of Yoann Delattre
Sent: Wednesday, August 20, 2014 9:26 AM
To:
Subject: [grouper-users] Accented characters and ComboBox in New UI

 

Hi everyone !

Actually, i'm working to get our grouper 2.2 ready for production (thanks again for your work on this version !).
Don't know if it's usefull but i had a problem with ComboBox in New UI. When i was trying to search a group/folder who contain an accented character, i had no result...

It was an encoding problem despiste the fact that UTF-8 is use everywhere...

The problem was the getParameter method (use in DojoComboLogic.java) in GrouperRequestWrapper.java, especially those lines (408 - 415) :

      if (param != null && StringUtils.equals("GET", this.getMethod()) && GrouperUiConfig.retrieveConfig().propertyValueBoolean("convertInputToUtf8", true)) {
        try {
          byte[] bytes = param.getBytes("ISO-8859-1");
          param = new String(bytes, "UTF-8");
        } catch (Exception e) {
          throw new RuntimeException(e);
        }
      }

So i change convertInputToUtf8 to false in grouper-ui.properties and everything works now ! :-)

But if everything is in UTF-8, i don't know if this part of code is still useful ???

Maybe some people will have the same problem...That's why i share the solution :)

I hope this will help.

Yoann.

--

Yoann Delattre	✆	03 20 95 69 61	✉
Équipe SIAD (Systèmes d'Information et Aide à la Décision) DSI de l'académie de Lille (Direction des Systèmes d'Information) 110 avenue Gaston Berger - 59000 Lille