Re: Encoding issues

From: Andrew Dunbar (hippietrail@yahoo.com)
Date: Sat Nov 02 2002 - 22:32:15 EST

  • Next message: Andrew Dunbar: "Re: Encoding issues"

     --- Jordi Mas <jmas@softcatala.org> wrote:
    > En/na Christian Biesinger ha escrit:
    > > Hi,
    > > so you may remember that some time ago, I checked
    > > in a patch to change the encoding that
    > > AP_DiskStringSet uses to whatever
    > > XAP_App::getDefaultEncoding uses (or something
    > > like that, can't remember what exactly I did :) ).
    > >
    > > Anyway, it looks like this broke non-US-ASCII
    > > characters in the statusbar, because of this piece
    > > of code in ap_StatusBar.cpp, line 493, in
    > > AP_StatusBar::setStatusMessage(const char * pBuf,
    > > int redraw)
    > > UT_UCS4_strcpy_char(bufUCS,pBuf);
    > >
    > > That function just uses the encoding that the
    > > default constructor of mbtowc thinks is good as
    > > the source encoding. That seems to be ISO-8859-1
    > > for me.
    > > However, due to the patch I mentioned above, that
    > > string is already in UTF-8. This means that the
    > > statusbar will not display special characters
    > > (like, but not limited to, german umlauts)
    > > correctly. Instead, it will show characters
    > > looking like undecoded UTF-8 (like Ì)
    > >
    > > So... the question is:
    > > What's the best way for fixing this?
    > > Should UT_UCS4_strcpy_char take an additional
    > > (maybe optional) argument, specifying the charset
    > > to convert from? AP_StatusBar would pass the
    result
    > > of XAP_App::getDefaultEncoding to it, and this
    > > would work...
    >
    > Well,
    >
    > I did very recently the win32 status bar native
    > implementation. Under win32 this is displayed
    > correctly because I added this piece of code on
    > ap_Win32StatusBar.cpp to convert from UTF8 to the
    > native encoding:
    >
    > ;-------------------------
    > char *pText = UT_convert (utf8.utf8_str(),
    > utf8.byteLength(),
    > "utf-8",
    >
    >
    XAP_EncodingManager::get_instance()->getNative8BitEncodingName(),
    > &uRead, &uWrite);

    Hi Jordi. This code is better than some but it only
    works on all Windows builds at the moment because we
    don't yet support Windows Unicode builds (which we
    badly need for many reasons). There is in fact no
    guarantee that the Windows GUI will be using an 8-bit
    encoding, and many newly supported languages on Win32
    *only* work in 16-bit Unicode!

    What you should have is something like this:

    XAP_EncodingManager::get_instance()-getDefaultSystemEncodingName(),
                             &uRead, &uWrite);

    Sorry I don't have the code in front of me so that may
    be slightly wrong. More and more I think we should
    really convert to this though:

    XAP_EncodingManager::get_instance()-getDefaultGUIEncodingName(),
                             &uRead, &uWrite);

    I also doubt that some of the functions we use these
    days are able to handle both 8 and 16-bit based
    strings. This is one of the things which I think was
    lost in my ancient unicode string megapatch...

    > ;-------------------------
    >
    > I saw that in Unix there was no conversion and the
    > string was sent in UTF8 format directly to the GTK
    > widget. If GTK does not process UTF8 directly, then
    > the UTF8->Native conversion may be can be done at XP
    > level.

    This is fine now but a null-conversion in XP space is
    better since it avoids special cases. The old GTK
    wouldn't have worked with this.

    > Well, I hope this brings some light.

    It does indeed (:

    Andrew.

    > Jordi,
    > --
    >
    > Jordi Mas
    > http://www.softcatala.org
    >
    >
    >

    =====
    http://linguaphile.sourceforge.net/cgi-bin/translator.pl http://www.abisource.com

    __________________________________________________
    Do You Yahoo!?
    Everything you'll ever need on one web page
    from News and Sport to Email and Music Charts
    http://uk.my.yahoo.com



    This archive was generated by hypermail 2.1.4 : Sat Nov 02 2002 - 22:40:51 EST