QNetworkAccessManager and http redirection

[grayfox]

Is it true that QNetworkAccessManager has no built-in support for http redirection?
I would like to download raw html from the Internet but it seems that I can't get to the final redirected webpage.

#ifndef PROGRAM_H
#define PROGRAM_H
 
#include <QtNetwork>
#include <QtCore>
 
class Program : public QObject
{
    Q_OBJECT
 
public:
    Program()
    {
        net = 0;
    }
    ~Program()
    {
        delete net;
    }
    void run()
    {
        net = new QNetworkAccessManager;
        connect(net, SIGNAL(finished(QNetworkReply*)),this, SLOT(replyFinished(QNetworkReply*)));
        net->get(QNetworkRequest(QUrl("http://www.yahoo.com")));
    }
 
private slots:
    void replyFinished(QNetworkReply* r){
        QString s = r->readAll();
        qDebug() << s;
    }
private :
    QNetworkAccessManager* net;
};
 
#endif // PROGRAM_H

#ifndef PROGRAM_H
#define PROGRAM_H

#include <QtNetwork>
#include <QtCore>

class Program : public QObject
{
    Q_OBJECT

public:
    Program()
    {
        net = 0;
    }
    ~Program()
    {
        delete net;
    }
    void run()
    {
        net = new QNetworkAccessManager;
        connect(net, SIGNAL(finished(QNetworkReply*)),this, SLOT(replyFinished(QNetworkReply*)));
        net->get(QNetworkRequest(QUrl("http://www.yahoo.com")));
    }

private slots:
    void replyFinished(QNetworkReply* r){
        QString s = r->readAll();
        qDebug() << s;
    }
private :
    QNetworkAccessManager* net;
};

#endif // PROGRAM_H

To copy to clipboard, switch view to plain text mode 

When run it called, I got the following:

<!-- w84.fp.sk1.yahoo.com uncompressed/chunked Thu Jul  7 08:37:01 PDT 2011 -->

<!-- w84.fp.sk1.yahoo.com uncompressed/chunked Thu Jul  7 08:37:01 PDT 2011 -->

To copy to clipboard, switch view to plain text mode 

instead of the final redirected webpage.

[squidge]

What you are getting is the last line only. I get much more:

--- snip! ---
if ( "undefined" != typeof(rt_LogTime) ) {
rt_LogTime(
"t1", rtJSLoad - rtJSStart, true);
rt_LogTime("t2", rtJSDone - rtJSLoad
, true);
if ("undefined" != typeof(rtAdStart) && "undefined" != typeof(rt
AdDone)) { rt_LogTime("t3", rtAdDone - rtAdStart, true); }
}
</script></body>

</html>
<!-- bid=701 -->
<!-- sid=2023538075 -->
2c

<!-- myproperty:myservice-us:0:Success -->

50
<!-- w25.fp.ch1.yahoo.com uncompressed/chunked Thu Jul 7 14:33:30 PDT 2011 -->

[ChrisW67]

What greyfox is getting is the body content of the initial server response. It is not following the redirect, which you will find in the Location: header of the response.

In the finished() signal you could check the QNetworkRequest::RedirectionTargetAttribute attribute and, if the return is valid, create a new request and get() it. You might want to test for redirect loops or endless redirects though.

[squidge]

Although I didn't follow any redirect requests, and to confirm it I also got the above by using telnet to connect to www.yahoo.com, port 80.

[ChrisW67]

I get a full response through telnet with a bare request also. However, using wget or QNetworkAccessManager the "Host:" header is sent and that triggers a different response from Yahoo.

chrisw@newton /tmp $ telnet www.yahoo.com 80
Trying 98.137.149.56...
Connected to any-fp.wa1.b.yahoo.com.
Escape character is '^]'.
GET / HTTP/1.0
Host: www.yahoo.com 
 
HTTP/1.0 302 Found
Date: Fri, 08 Jul 2011 07:41:09 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Cache-Control: private
Set-Cookie: IU=deleted; expires=Thu, 08-Jul-2010 07:41:08 GMT; path=/; domain=.yahoo.com
Set-Cookie: fpc=d=mE9W1x3vmwH5qQfykKO9i17Mi68As827SzEoRYZfeHl.7YKK16lgEMZn2AeBTCkH6rMVH2R7O3CIDu1vDGA9rMySC_7rhu_x1X_RXpUZsQFrhS_3gLr7VfouTwfBLAwisocjqWRT..pSmkbn_0UfXPfN1SxzCgHt9l5Sq8l4dyX7Cx21tKKImt_2DUe36ZK5cMj63JY-&v=2; expires=Sat, 07-Jul-2012 07:41:09 GMT; path=/; domain=www.yahoo.com
Location: http://au.yahoo.com/?p=us
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Age: 0
Server: YTS/1.20.0
 
<!-- w27.fp.sp2.yahoo.com uncompressed/chunked Fri Jul  8 00:41:09 PDT 2011 -->
Connection closed by foreign host.

chrisw@newton /tmp $ telnet www.yahoo.com 80
Trying 98.137.149.56...
Connected to any-fp.wa1.b.yahoo.com.
Escape character is '^]'.
GET / HTTP/1.0
Host: www.yahoo.com 

HTTP/1.0 302 Found
Date: Fri, 08 Jul 2011 07:41:09 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Cache-Control: private
Set-Cookie: IU=deleted; expires=Thu, 08-Jul-2010 07:41:08 GMT; path=/; domain=.yahoo.com
Set-Cookie: fpc=d=mE9W1x3vmwH5qQfykKO9i17Mi68As827SzEoRYZfeHl.7YKK16lgEMZn2AeBTCkH6rMVH2R7O3CIDu1vDGA9rMySC_7rhu_x1X_RXpUZsQFrhS_3gLr7VfouTwfBLAwisocjqWRT..pSmkbn_0UfXPfN1SxzCgHt9l5Sq8l4dyX7Cx21tKKImt_2DUe36ZK5cMj63JY-&v=2; expires=Sat, 07-Jul-2012 07:41:09 GMT; path=/; domain=www.yahoo.com
Location: http://au.yahoo.com/?p=us
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Age: 0
Server: YTS/1.20.0

<!-- w27.fp.sp2.yahoo.com uncompressed/chunked Fri Jul  8 00:41:09 PDT 2011 -->
Connection closed by foreign host.

To copy to clipboard, switch view to plain text mode 

This is a redirect, in my case to the Australian subsidiary. The only content in the response is the one line the OP is seeing.

[squidge]

Indeed. I forgot that QNetworkAccessManager sent the 'host' header.