Grouper Users - Open Discussion List

[Chris Hyzer <>]

Right.  I wonder if we should be using chars in the DB instead of bytes.  I opened a jira to explore this:

 

https://bugs.internet2.edu/jira/browse/GRP-916

 

At Penn, using Oracle, we use chars instead of bytes (not on Grouper but for other projects), so based on that experience (works fine) it seems like something we can do in Grouper unless someone has reasons why this is not a good idea…

 

Thanks,

Chris

 

From: Gagné Sébastien [mailto:]
Sent: Monday, June 17, 2013 9:03 AM
To: Chris Hyzer; Nicolas Forney;
Subject: RE: [grouper-users] TR : [Grouper API] problem with special characters in displayName

 

The source of the problem is that the grouper DB field is 255 BYTES, not 255 CHARS, accent characters use 2 Bytes,  hence the larger value (some other like the apostrophe take 3)

 

 

De : [] De la part de Chris Hyzer
Envoyé : 14 juin 2013 10:33
À : Nicolas Forney;
Objet : RE: [grouper-users] TR : [Grouper API] problem with special characters in displayName

 

Grouper does not convert any characters, but the problem is the database requires multiple chars to store non empty chars.  So, the algorithm I use which is safe, and any non-ascii char we count as 3 J  So you can truncate that way…  Here is the method I use when doing the calculation:

 

  /**

   * find the length of ascii chars (non ascii are counted as three)

   * @param input is the string to operate on

   * @param requiredLength length we need the string to be

   * @return the length of ascii chars

   */

  public static String truncateAscii(String input, int requiredLength) {

    if (input == null) {

      return input;

    }

    //see what real length is

    int utfLength = input.length();

   

    //see if not worth checking

    if (utfLength * 2 < requiredLength) {

      return input;

    }

   

    //count how many non asciis

    int asciiLength = 0;

    for (int i=0;i<utfLength;i++) {

     

      asciiLength++;

     

      //keep count of non ascii chars

      if (!isAscii(input.charAt(i))) {

        asciiLength+=(nonAsciiCharLength()-1);

      }

     

      //see if we are over

      if (asciiLength > requiredLength) {

        //do not include the current char

        return input.substring(0,i);

      }

    }

    //must have fit

    return input;

  }

 

 

From: [] On Behalf Of Nicolas Forney
Sent: Friday, June 14, 2013 6:44 AM
To:
Subject: [grouper-users] TR : [Grouper API] problem with special characters in displayName

 

Hello,

 

I use the Grouper client API from a java application and I have encountered a problem when trying to create a stem with a large displayName containing multiple special chars (à, è, ô, ü, /, etc.).

 

Before creating the stem I truncate the displayName variable to 255 and then I set the stem's displayExtension and Extension.

When the webservice request is executed, the webservice return an Interal server error (500) which indicates the displayName is to long.

 

As the displayName has been truncated before executing the webservice the problem probably come from the special characters conversion which is done by the webservice (a special chat may be represented with more than one char). So I'd like to know how the special characters are converted (with which format?). If I know the format I should be able to know exactly how many characters must be truncated before calling the webservice.

 

The stacktrace shows the following error (raised by the Oracle Dirver) :

Caused by: java.sql.SQLException: ORA-12899: value too large for column "GROUPER_213"."GROUPER_STEMS"."DISPLAY_NAME" (actual: 266, maximum: 255)

 

 

I joined the full stacktrace to this mail.

 

 

Thank you in advance for your help.