Results 1 to 2 of 2

Thread: cURL & QWebKit don't mix

  1. #1
    Join Date
    Jul 2011
    Posts
    2
    Qt products
    Qt4
    Platforms
    Windows

    Unhappy cURL & QWebKit don't mix

    Hello, I have a small problem with Qt and cURL: it seems as though the Qt Web parsing library only works with certain sites and its very frustrating because I can't figure out why some sites work and some don't. Here's my code (slightly altered to shorten):
    Qt Code:
    1. #include <iostream>
    2. #include <string>
    3. #include <QApplication>
    4. #include <QtWebkit>
    5. #include <QDomAttr>
    6. #define CURL_STATICLIB
    7. #include "curl/curl.h"
    8.  
    9. using namespace std;
    10. class Filter : public QString
    11. {
    12. public:
    13. QString content_;
    14. static size_t handle(char * data, size_t size, size_t nmemb, void * p);
    15. size_t handle_impl(char * data, size_t size, size_t nmemb);
    16. };
    17.  
    18. size_t Filter::handle(char * data, size_t size, size_t nmemb, void * p)
    19. {
    20. return static_cast<Filter*>(p)->handle_impl(data, size, nmemb);
    21. }
    22.  
    23. size_t Filter::handle_impl(char* data, size_t size, size_t nmemb)
    24. {
    25. content_.append(data);
    26. return size * nmemb;
    27. }
    28. int main(int argc, char *argv[])
    29. {
    30. QApplication a(argc, argv);
    31. CURL *curl;
    32. CURLcode res;
    33. QWebPage page;
    34. QWebFrame* frame = page.mainFrame();
    35. curl = curl_easy_init();
    36. if(curl) {
    37.  
    38. Filter f;
    39. curl_easy_setopt(curl, CURLOPT_URL, "http://www.teamliquid.net");
    40. curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &Filter::handle);
    41. curl_easy_setopt(curl, CURLOPT_WRITEDATA, &f);
    42. curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION,1);
    43. res = curl_easy_perform(curl);
    44. frame->setHtml(f.content_);
    45. QWebElement document = frame->documentElement();
    46. QWebElementCollection elements = frame->findAllElements("a");
    47. int i = elements.count();
    48.  
    49. curl_easy_cleanup(curl);
    50. }
    51. return 0;
    52. }
    To copy to clipboard, switch view to plain text mode 

    int i never has the correct amount of links. It always returns some random number in the millions.

    This only happens for certain sites like the one in the program, google and yahoo, and other big sites work fine, so I'm really baffled.

    EDIT: I wasn't sure if this is supposed to be in newbie or not, sorry in advanced if wrong section.

  2. #2
    Join Date
    Jul 2011
    Posts
    2
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: cURL & QWebKit don't mix

    Sorry for such a quick reply but I've discovered a few new things and have some things to clarify.
    1. the int i actually isn't returning random numbers in the millions: it only does that if I look at the value with a breakpoint in Visual Studio, it's real value is 0 when printed (which is still wrong).

    2. Sites that aren't being correctly parsed in Qt seem to have this at the start of their HTML doc:
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
    "http://www.w3.org/TR/html4/loose.dtd">
    sites with:
    <!DOCTYPE html>
    OR
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml">
    Seem to work 100% fine.

    I tried a work around by removing:
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
    "http://www.w3.org/TR/html4/loose.dtd">
    With Qtstring.remove, but it didn't work.

Similar Threads

  1. Curl in QT Creator
    By januszmk in forum Newbie
    Replies: 10
    Last Post: 26th August 2011, 00:14
  2. Bus error on QWebkit
    By geoffroylaca in forum Qt for Embedded and Mobile
    Replies: 3
    Last Post: 24th August 2011, 12:13
  3. cURL With Qt Creator [Windows and Linux]
    By Chiggins in forum Qt Tools
    Replies: 1
    Last Post: 24th June 2010, 09:29
  4. Hacking QWebkit
    By qtfeeder in forum Qt Programming
    Replies: 4
    Last Post: 8th July 2009, 18:54
  5. QString static callback function from CURL
    By tpf80 in forum Qt Programming
    Replies: 12
    Last Post: 16th May 2007, 21:47

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.