UTF-8 Issues in Sample

[trevelyan]

Hi everyone,

I'm new to QT and am running into problems displaying UTF8 data. If anyone with more experience with the library has any thoughts I'd be very grateful to hear them. The code base in question is the "previewer" program in the /examples/webkit/previewer subdirectory. I am trying to get a QString containing Chinese characters to show up in the "HTML Preview" box.

I've spent about three to four hours trying to debug the problem. I can confirm that everything from fonts to system-variables are set up properly. I can also confirm that the QString contains data which was successfully read in from a std::string containing UTF-8 data. I know this because the text is not garbled when printed to the command-line terminal. Nor is it garbled when it is displayed in the "HTML Editing" box. It is only getting garbled when it is displayed in the "HTML Preview" box.

The code that handles HTML processing seems capable of displaying Chinese content though. When I request a random webpage with Chinese characters, I can see that Chinese characters fetched from remote sites work perfectly fine. The following code works fine, for instance:

// webView->setHtml(text); // text is a QString containing Chinese characters
webView->load(tr("http://www.popupchinese.com"));

The problem seems to be with passing a QString through the webView->setHTML function. As a newcomer to QT, I'm somewhat baffled by this, and would love to hear any thoughts or suggestions someone with more experience might have at this point. I'm not sure if this is a bug with the software, or if there is something related to the way the Webkit works that I'm simply missing. The online documentation has not been much help, although I've mostly been googling things and working through the class reference, so suggestions on additional resources would be much appreciated.

[jacek]

Can you correctly display that web page using a normal browser?

[trevelyan]

Can you correctly display that web page using a normal browser?

Yes. And webpages loaded into the HTML Preview field using the webView->load() function display UTF-8 characters perfectly as well. The problem only seems to affect content that is passed in as a QString. I've spent the last half hour trying to get around this with setContent() and ByteArrays, and can report that that approach fails as well. The content appears to be simply getting treated as Latin1 regardless, or defaulting to a font that does not support UTF-8 glyphs (is there a way to set that separately?).

The relevant code from the tutorial is this:

// We get the UTF-8 Data
QString text = plainTextEdit->toPlainText();

/* These lines don't change things either way - they're already set in main.cpp */
QTextCodec *utfCodec = QTextCodec::codecForName("UTF-8"); //creating new utf-8 codec
QTextCodec::setCodecForLocale(utfCodec); // setting the utf-8 codec for the tr() tags
QTextCodec::setCodecForTr(utfCodec); // setting the utf-8 codec for the tr() tags
QTextCodec::setCodecForCStrings(utfCodec); // setting the utf-8 codec for the tr() tags


// We can test the QString and confirm the data is UTF-8 as follows:
std::string stdstring = text.toUtf8().constData();
std::cout << "Now: " << stdstring.c_str() << std::endl;

// But this still results in garbled text
webView->setHtml(text);

The plainTextEdit section (HTML Editing) is displaying UTF-8 without a problem.

[trevelyan]

I'd like to check the explicit encoding of the webView object after passing along the String. Something like:

QWebSettings *ws = webView->settings();

Any chance someone here knows how to do it?

[trevelyan]

Update - might be useful for others even if no-one else can really help with this....

Adding the following javascript to the string resulted in an alert with the number "1":

<script>
ourTest="我";
alert(ourTest.length);
</script>

Since Chinese characters a multibyte, this suggests the problem is the default font being used with QTWebkit, which is apparently not defauting to the rest of the system. Will rummage around and see if I can find the way to fix that, and post if I do.

[trevelyan]

Yes... the problems were fonts. Adding an explicit style to the string being passed in solved the problem. WebKit doesn't default to QT system defaults apparently. Easy enough to deal with once you know how.

[gosku]

Hello,

I'm desperate and after 2 weeks looking for I've come here. I think I have a similar problem with the setHtml function. However I'm programming with python and Plasma:


class crearWidgetApplet(plasmascript.Applet):
def __init__(self,parent,args=None):
plasmascript.Applet.__init__(self,parent)

def init(self):
self.setHasConfigurationInterface(False)
self.theme = Plasma.Svg(self)
self.theme.setImagePath("widgets/translucentbackground")
self.setBackgroundHints(Plasma.Applet.DefaultBackg round)
self.setAspectRatioMode(Plasma.IgnoreAspectRatio)
self.layout = QGraphicsLinearLayout(Qt.Horizontal, self.applet)
self.resize(697, 763)

url = "http://www.eltiempo.es/"
html = Browser().open(url).read()
webView = Plasma.WebView(self.applet)
webView.setHtml(html)

self.layout.addItem(webView)
self.setLayout(self.layout)

Please, I appreciate some type of help. The especial characters ('ñ' 'º' 'á' 'é' ª ...) don't show correctly when I use the setHml function. However if I write the setUrl function the webpage shows correctly. I have tried to save the html content in a file.html and I can visualize well with a browser (firefox, chrome...). Furthermore, I tried the setUrl function with a file///directory/code.html argument and it works too.

I need the setHtml code because I have to modify the code before showing it.
I read about "Adding an explicit style to the string being passed in solved the problem" but I dont understand what I have to do (english isn't my first language).

Thank You for reading and please, heeeeelp!!