Ajax Unknown error

[brcontainer]

[edit]
In QtWebKit often occurs error "Unknown error" in Ajax requests of type POST
This problem does not occur in conventional browsers (like Google Chrome and Firefox).

It would be possible to create a "DEBUG" all requests that occur within the "QWebView" to analyze the cause of the problem?

bq. Note: There is always occurring, but still quite frequently

Ignore is part:
Is it a bug in QT?
What would be the best way around the problem?

I believe that it is not something simple to discover, would analyze the requests of QtWebKit (using QtWebKit itself) made ​​by Ajax (not the console QWebInspector, I've tried this and did not help me)?

[ChrisW67]

It would be possible to create a "DEBUG" all requests that occur within the "QWebView" to analyze the cause of the problem?

webView->page()->settings()->setAttribute(QWebSettings::DeveloperExtrasEnabled, true);

webView->page()->settings()->setAttribute(QWebSettings::DeveloperExtrasEnabled, true);

To copy to clipboard, switch view to plain text mode 

In your web view window Right-click and select Inspect and you get access to a wealth of tools to debug your script including inspecting all the network exchanges and any console log output your script makes. I don't how this could not help debug your problem.

You can always watch the entire network exchange using Wireshark or Fiddler

[brcontainer]

as I said earlier, QWebInspector not work, just returns "QWebInspector" has no details.

not the console QWebInspector, I've tried this and did not help me

Thanks for trying, but is there any way to do this than with QWebInspector?

[edit]
Fiddler2 capture ajax GoogleChrome but fails to capture the QWebView.

[edit]
Fiddler2 uses proxy (local proxy), so the error does not occur, because practically the page has already been generated.

[ChrisW67]

What do you mean "just returns "QWebInspector"? Invoking the web inspector opens a separate window containing debugging tools. Since you are adamant that the internal tool "does not work" then you have limited yourself to external tools.

Fiddler will not "capture the QWebView" if the QWebView is not configured to use the system proxy (or Fiddler direct). If your software behaves differently talking through a non-caching proxy then that sounds like another problem to solve.

[brcontainer]

Thanks for the quick reply.

Fiddler2 uses proxy (local proxy), so the error does not occur, because practically the page has already been generated (by localhost).

When I said QWebInspector, I was referring to the same console (I was writing or revised what I typed was so confused, sorry).
In the tab "network", the only message that returns me is: "Unknown error", as I already said.

What I would like to know:

I'm thinking of creating a local server ("proxy") inside the "project" and set the application to use the proxy, would it be viable (when used proxy (fiddler) stopped the error occurs)?

Or have another way to generate the pages, rebuild the requests and send the QtWebKit?

[ChrisW67]

Fiddler2 uses proxy (local proxy), so the error does not occur, because practically the page has already been generated (by localhost).

Huh? Your Ajax request passes through the proxy to the upstream host, and the response comes back from the upstream host via the proxy. The proxy host does not create anything new in the payload and, in the case of Fiddler, it also does not change or cache anything.

Both Fiddler and the Network panel in the web inspector show you the content of your request (headers, data payload) and the content of the response (headers and data payload). I can only assume that the "Unknown error" string you refer to is the payload of the response. That comes from the remote host, it's not generated Qt, Qt WebKit or Fiddler. It is most likely the result of sending a malformed request and this would generally be visible as difference between a working request and a failing one. We have no way to know.

[brcontainer]

I do not know if I did not understand you, or you do not understand me (I refer to my last post).

Yes I believe it is something malformed, but I believe it has to do with the QtWebKit (or something parecidog), which sends data "truncated" (or something similar) to the server. When using proxy "Fiddler2", who made the request to the server is was "Fiddler2" and not the "QWebView" with this error "xxxx" stopped happening.

Therefore I believe that the best way to solve the problem is to create a "thread" in my project that is a local server, the local server is a proxy and I would set up so that the application runs only with the local proxy (Thread).

What I wonder, is another way (besides the "local proxy") requests to recreate the QWebView (QtWebKit)?

For example: is how to handle the "QNetworkRequest" of "QtWebKit"?

Thanks.

[ChrisW67]

QtWebKit uses an instance of QNetworkAccessManager you can replace with a customised one. see the QWebPage docs. No threads required. Threads rarely simplify a problem anyway.

The best way to solve the problem is to stop guessing. Look at the raw request that is actually sent. Look at the logs on the server. Look at the raw headers ( not the payload ) of the response. Compare to a good one. Check your Javascript for common errors. If you build and send a request in C++ make sure you are doing it correctly with the server-expected Content-Type and encoding. Make sure objects relevant to send requests are staying in-scope long enough to send the request. Ask questions accompanied with relevant information about behaviours you do not understand. Stop assuming QtWebKit is making some magical change to the request just to spite you. Ultimately if XmlHttpRequest or the Javascript engine was fundamentally broken then there would be a lot more people than just you with issues.

[brcontainer]

QtWebKit uses an instance of QNetworkAccessManager you can replace with a customised one.

Sorry, I'm a beginner, do not know how to replace the instance, could help me?

------------------

even so I used to generate a Wire LOG apparently everything is normal.

The error occurred in the request line: 560

POST /testXhr.php?teste=13 HTTP/1.1
Accept: */*
Referer: http://localhost/testXhr.php
Origin: http://localhost
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows;) AppleWebKit/537.21 (KHTML, like Gecko) Safari/537.21
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 5
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: pt-BR,en,*
Host: localhost
 
ok=13

POST /testXhr.php?teste=13 HTTP/1.1
Accept: */*
Referer: http://localhost/testXhr.php
Origin: http://localhost
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows;) AppleWebKit/537.21 (KHTML, like Gecko) Safari/537.21
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 5
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: pt-BR,en,*
Host: localhost

ok=13

To copy to clipboard, switch view to plain text mode 

Download the complete LOG:
http://www.mediafire.com/download/b9.../log-full.pcap

PrintScreen:
xhr-erro-on-qt.jpg

Javascript Code:

window.delta=0;
window.running = true;
function run(){
	++window.delta;
	$.ajax({
		'type':'POST',
		'url':"testXhr.php?teste="+(window.delta),
		'data':'ok='+(window.delta),
		'success':function(data){ },
		'error':function(a,b,c){
			console.log([a,b,c]);
			window.running = false;
		},
		'complete':function(a,b){
			console.log(b);
			setTimeout(function(){
				if(window.running){
					run();
				}
			},500);
		}
	});
}
window.onload=run;

window.delta=0;
window.running = true;
function run(){
	++window.delta;
	$.ajax({
		'type':'POST',
		'url':"testXhr.php?teste="+(window.delta),
		'data':'ok='+(window.delta),
		'success':function(data){ },
		'error':function(a,b,c){
			console.log([a,b,c]);
			window.running = false;
		},
		'complete':function(a,b){
			console.log(b);
			setTimeout(function(){
				if(window.running){
					run();
				}
			},500);
		}
	});
}
window.onload=run;

To copy to clipboard, switch view to plain text mode 

Please help me with this.

[ChrisW67]

My copy of Wireshark 1.8.8 and tcpdump claim your 24620 byte pcap file is corrupt. I see five exchanges, the last of which is a POST that I don't see a response for.

[brcontainer]

Log WireShark:
http://www.mediafire.com/download/fz...-wireshark.zip

I tried to analyze the LOG,
I realized that the request failed ("unknown error")

Request failed:

POST /testXhr.php?teste=3 HTTP/1.1

POST /testXhr.php?teste=3 HTTP/1.1

To copy to clipboard, switch view to plain text mode 

In wireshark shows that the data arrived normally see in the picture, the contents usually arrived:
print-wire.jpg

Note: I used the website for a client, by any chance you want to test the operation of the script, I'm currently without website for online tests.

[edit]
I know you said you stay in this guessing no use,
but I suppose the problem is with some "timeout" (or similar) within the framework of the "QT" making requests.


Help me please.

[ChrisW67]

You seem to have a very unreliable connection if the TCP retransmissions are anything to go by.

The payload of the POST request is not correctly encoded. The spaces in the field names or values should be encoded either as '+' or as "%20". Other special characters, e.g. % or +, also need to be encoded.

Wireshark shows that what looks like a normal request "teste=3" was sent and the bytes acknowledged. I see no response to that request from upstream, so I gather "unknown error" is internally generated. The other requests are taking about 2.5 seconds to fill which is not long enough to trigger any default time out I am aware of. How long after sending packet 29 does the client report the error? Where is the error being reported, and where (your error function, complete function...) is it being generated?

[brcontainer]

The payload of the POST request is not correctly encoded. The spaces in the field names or values should be encoded either as '+' or as "%20". Other special characters, e.g. % or +, also need to be encoded.

But I did the same test without sending data (do not send anything in the querystring parameters and no data to POST) and the same error occurred, example:

$.ajax({
	'type':'POST',
	'url':"testXhr.php",
	'data':'',
	'success':function(data){ }
});

$.ajax({
	'type':'POST',
	'url':"testXhr.php",
	'data':'',
	'success':function(data){ }
});

To copy to clipboard, switch view to plain text mode 

It's not really got anything, now that I realized that this line was sending and not receiving:

[truncated] ok=3&data=jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljlj

[truncated] ok=3&data=jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljljlk jkljljlj

To copy to clipboard, switch view to plain text mode 

How long after sending "packet 29" does the client report the error?

I did a test to see the time difference:

success: 1858ms
success: 2055ms
success: 2357ms
error: 1163ms

success: 1858ms
success: 2055ms
success: 2357ms
error: 1163ms

To copy to clipboard, switch view to plain text mode 

[Edit]
I did a test with "GoogleChrome" apparently the "bug" (is not actually a bug) him too, but he try a new request, see there are two repeated requests and between them there is an "unknown":
Chrome.png

[brcontainer]

@ChrisW67 I did the following reimplementation, I'm having trouble getting this code to work, requisitions cause "500 internal server error", I wonder if you have any tips for this code.

QNetworkReply * myNAM::createRequest(Operation op, const QNetworkRequest &request, QIODevice *outgoingData){
    if(op==PostOperation) {
        qDebug() << "isPost";
        QNetworkAccessManager *n = new QNetworkAccessManager();
        QNetworkReply *p = n->post(request,outgoingData);
        //qDebug() << p->url();
        //qDebug() << "newRequest";
        p->waitForReadyRead(8000);
        qDebug() << "readyRequest";
        if(p->isReadable()){
            qDebug() << "readyRequest is ready";
           return QNetworkAccessManager::createRequest(op, p->request(), outgoingData);
        }
        p->abort();
    }
    return QNetworkAccessManager::createRequest(op, request, outgoingData);
}

QNetworkReply * myNAM::createRequest(Operation op, const QNetworkRequest &request, QIODevice *outgoingData){
    if(op==PostOperation) {
        qDebug() << "isPost";
        QNetworkAccessManager *n = new QNetworkAccessManager();
        QNetworkReply *p = n->post(request,outgoingData);
        //qDebug() << p->url();
        //qDebug() << "newRequest";
        p->waitForReadyRead(8000);
        qDebug() << "readyRequest";
        if(p->isReadable()){
            qDebug() << "readyRequest is ready";
           return QNetworkAccessManager::createRequest(op, p->request(), outgoingData);
        }
        p->abort();
    }
    return QNetworkAccessManager::createRequest(op, request, outgoingData);
}

To copy to clipboard, switch view to plain text mode 

[ChrisW67]

I am struggling to work out what you are trying to achieve.
You create a new instance of QNetworkAccessManager (4, a memory leak), post the original request (5, the new QNetworkReply is a memory leak), wait to see if a response is returned (8), discard the result, and return what should be the exact same thing (12) as it originally returned anyway (16).

What is it that you want from your QNetworkAccessManager subclass?

[brcontainer]

Look I really do not know how to do this, it was just a try, what I'm trying to achieve is:



In browsers Google Chrome, Firefox and Internet Explorer, when a failure occurs requisition they try to retrieve the request (try to make a new request) as shown in the picture below (LOG WireShark with Google Chrome).

See there are two requests:

POST /testXhr.php?test=43 HTTP/1.1

POST /testXhr.php?test=43 HTTP/1.1

To copy to clipboard, switch view to plain text mode 

I want to know how to QWebView work the same way that the normal browser (Google Chrome, Firefox and Internet Explorer).

Could you help me?

[ChrisW67]

Can you post a Wireshark log of whatever Chrome does. The second line of that trace makes no sense and implies the server is sending complete rubbish.

If you want your Javascript to retry a request it sent then generally you need to code it, something like:

$.ajax({
    url : 'someurl',
    type : 'POST',
    data :  ....,   
    tryCount : 0,
    retryLimit : 3,
    success : function(json) {
        //do something
    },
    error : function(xhr, textStatus, errorThrown ) {
        if (textStatus == 'Unknown error') {
            this.tryCount++;
            if (this.tryCount <= this.retryLimit) {
                //try again
                $.ajax(this);
                return;
            }            
            return;
        }
        if (xhr.status == 500) {
            //handle error
        } else {
            //handle error
        }
    }
});

$.ajax({
    url : 'someurl',
    type : 'POST',
    data :  ....,   
    tryCount : 0,
    retryLimit : 3,
    success : function(json) {
        //do something
    },
    error : function(xhr, textStatus, errorThrown ) {
        if (textStatus == 'Unknown error') {
            this.tryCount++;
            if (this.tryCount <= this.retryLimit) {
                //try again
                $.ajax(this);
                return;
            }            
            return;
        }
        if (xhr.status == 500) {
            //handle error
        } else {
            //handle error
        }
    }
});

To copy to clipboard, switch view to plain text mode 

[brcontainer]

This is not feasible, I am developing a web browser, in other words:
my application can access any site.

Need to reimplement the QtWebKit so that it works the same way as conventional web browsers.

I believe it is with QNetworkAccessManager::createRequest (or something) could help me with this function?
How should I redeploy it?

Help me please!

[ChrisW67]

In your example capture you show Chrome(?) responding to some unexpected response, i.e HTTP/DL 717 unknown line). Since I have no idea what that unexpected response is I cannot tell you how it might be available inside your code in order for your code to react to the situation. That is why I asked for a capture log corresponding to the situation in your post. It might be achieved by using the QWebPage extension mechanism or it might need to be lower down the stack.

Alternatively, if you can provide a complete, compilable example program that reproduces the failure with a public site...

[brcontainer]

I solved the problem:

QNetworkReply * myNAM::createRequest(Operation op, const QNetworkRequest &request, QIODevice *outgoingData){
    if(op==PostOperation) {
        QNetworkAccessManager *n = new QNetworkAccessManager();
        QString contentType = "";
 
        QNetworkRequest req(request.url());
 
        const QList<QByteArray>a = request.rawHeaderList();
        int i = 0; int j = a.length();
        for(;i<j;i++){
            qDebug() << a[i] << ": " << request.rawHeader(a[i]);
            req.setRawHeader(a[i],request.rawHeader(a[i]));
        }
 
        if(contentType!=""){
            req.setHeader(QNetworkRequest::ContentTypeHeader, contentType);
        }
 
        const QByteArray data = outgoingData->readAll();
        QNetworkReply *p = n->post(req,data);
 
        //wait request finish
        QEventLoop loop;
        connect(p, SIGNAL(finished()), &loop, SLOT(quit()));
        loop.exec();
 
        if(/*the attempt limit*/){
            if(p->errorString()=="Unknown error") {
                //tries to resend again when the error occurs "Unknown Error"
                return myNAM::createRequest(op, request, outgoingData);
            }
            //sends request ready.
            return QNetworkAccessManager::createRequest(op, p->request(), outgoingData);
        }
    }
 
    //send normal mode, GET, PUT and DELETE or chance the retry limit has finished
    return QNetworkAccessManager::createRequest(op, request, outgoingData);
}

QNetworkReply * myNAM::createRequest(Operation op, const QNetworkRequest &request, QIODevice *outgoingData){
    if(op==PostOperation) {
        QNetworkAccessManager *n = new QNetworkAccessManager();
        QString contentType = "";

        QNetworkRequest req(request.url());

        const QList<QByteArray>a = request.rawHeaderList();
        int i = 0; int j = a.length();
        for(;i<j;i++){
            qDebug() << a[i] << ": " << request.rawHeader(a[i]);
            req.setRawHeader(a[i],request.rawHeader(a[i]));
        }

        if(contentType!=""){
            req.setHeader(QNetworkRequest::ContentTypeHeader, contentType);
        }

        const QByteArray data = outgoingData->readAll();
        QNetworkReply *p = n->post(req,data);

        //wait request finish
        QEventLoop loop;
        connect(p, SIGNAL(finished()), &loop, SLOT(quit()));
        loop.exec();

        if(/*the attempt limit*/){
            if(p->errorString()=="Unknown error") {
                //tries to resend again when the error occurs "Unknown Error"
                return myNAM::createRequest(op, request, outgoingData);
            }
            //sends request ready.
            return QNetworkAccessManager::createRequest(op, p->request(), outgoingData);
        }
    }

    //send normal mode, GET, PUT and DELETE or chance the retry limit has finished
    return QNetworkAccessManager::createRequest(op, request, outgoingData);
}

To copy to clipboard, switch view to plain text mode 

The code worked perfectly, did not need the number of attempts, only to recreate the request stopped the error occurs, occurs randomly but I added a "IF" with the limited number of attempts.

But still would like to know if you have any tips to further improve my code?