Http Posting using QNetworkAccessManager

Hello
I'm facing another problem
it's about uploading file through HTTP posting
this is my code

Code:

---

from PyQt4 import QtCore, QtGui, QtNetwork
import sys
class Window(QtGui.QWidget):
    def __init__(self, parent=None):
        super(Window, self).__init__(parent)
 
        self.netaccess = QtNetwork.QNetworkAccessManager(self)
        self._uploaders = {}
        row = 1 #this number will change per upload in real program,this is just demo
        self.address = 'http://localhost/01/upload/demo.php'
        stream = QtCore.QFile('/home/patriot/icon.png')
        if stream.open(QtCore.QIODevice.ReadOnly):
            data = stream
 
            uploader = self._uploaders[row] = Uploader(row, self.netaccess)
 
            uploader.upload(data, self.address)
 
class Uploader(QtCore.QObject):
 
    def __init__(self, key, parent):
        QtCore.QObject.__init__(self, parent)
        self._key = key
        self._reply = None
 
 
    def upload(self, data, url):
        if self._reply is None:
 
            self._stream = data
 
            self._multiPart = QtNetwork.QHttpMultiPart(QtNetwork.QHttpMultiPart.FormDataType)
 
            fileName = QtCore.QFileInfo(self._stream.fileName()).fileName()
            key = 'file'
 
            imagePart = QtNetwork.QHttpPart()
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentDispositionHeader,
                     "form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentTypeHeader,
                     'image/png')
            imagePart.setBodyDevice(self._stream)
 
            self._multiPart.append(imagePart)
 
            request = QtNetwork.QNetworkRequest(QtCore.QUrl(url))
            request.setHeader(QtNetwork.QNetworkRequest.ContentTypeHeader,
  'multipart/form-data; boundary=%s' % self._multiPart.boundary())
            request.setRawHeader('User-Agent','Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36')
 
            self._reply = self.parent().post(request, self._multiPart)
            self._reply.uploadProgress.connect(self.handleUploadProgress)
            self._reply.error.connect(self.handleError)
            self._reply.finished.connect(self.handleFinished)
 
 
 
    def handleUploadProgress(self, sent, total):
        if sent >= total:
            # prevent duplicated uploads
            self._reply.close()
 
    def handleFinished(self):
        print('Content: ',self._reply.readAll())#no output here :(
        self._stream.close()
        self._multiPart.deleteLater()
        self._reply.deleteLater()
        self._reply = None
        app.quit()
 
    def handleError(self):
        print('Error String :',self._reply.errorString())
        print('Error number: ',self._reply.error())
 
 
app = QtGui.QApplication(sys.argv)
demo = Window()
sys.exit(app.exec_())

---

I use an answer in stackoverflow.com,the code needs to upload to a php script but I don't know why it's not working
anyone have any idea ?anything
thanks

edit: this is html form

Code:

---

<form action="demo.php" method="post"
enctype="multipart/form-data">
<label for="file">Filename:</label>
<input type="file" name="file" id="file"><br>
<input type="submit" name="submit" value="Submit">
</form>

---

• Install Wireshark (UNIX or Windows).
• Open the HTML form in your browser,
• Submit the form using the same data as your Python program (Make it a small file).
• Inspect what exactly is sent.
• Run your program.
• Inspect what exactly is sent.

The differences will be informative.

I'll look into this exactly
but my problem is I can't see anything from python(even a simple echo in php)
I think I can't even get into apache(for no reason)
I can upload with requests(a python library) with this headers,but in Qt I can't,nothing,something is not right I can't figure out what is that.hope members could help

If your Qt code cannot even fetch a page from Apache (a GET) then you have more fundamental issues. You should work these out first.

Read the docs for QNetworkReply::uploadProgress() again and ask yourself what happens if the total size cannot be determined.

BTW: I do not see any reason from the close() call at all.

I edited my first code,I was thinking because my code is long it's looks ugly,thanks for mention that

Quote:

---

Originally Posted by ChrisW67

If your Qt code cannot even fetch a page from Apache (a GET) then you have more fundamental issues. You should work these out first.

Read the docs for QNetworkReply::uploadProgress() again and ask yourself what happens if the total size cannot be determined.

BTW: I do not see any reason from the close() call at all.

---

you see,I need to upload through HTTP,at first when I was writing my first Qt program and did not know about HTTP stuff,I read data as a QByteArray and pass it to Uploader,after processing was done I could fetch the page and I could see contents,so uploadProgress is Ok,when I added QHttpMultiPart first of all I change data from QByteArray to QFile then pass it to Uploader and append data to multipart,at the end I posted QHttpMultiPart and request
All I'm saying is Since I added QHttpMultiPart I can't fetch any pages,do you still think it's because QNetworkReply::uploadProgress() ?
If I don't use close() or deleteLater() program will crash(seagment crash) or if you mean in UploadProgress it's because PyQt sometimes will post file twice(after uploading it start uploading again) I don't know it's my code or PyQt bug,but that solved my problem

Yes, I suspect it is. I suspect you are closing the reply at the first sign of progress. Print the value of total in handleUploadProgress.

I print both total and sent and at the end the value were the same(sent = total)
Is this mean that closing reply is ok ? if yes do you have any idea ?


Oh Chris,you are great,I did remove self._reply.close() in my real program and I could see a variable which I echo in PHP,you were right
but still,can't send any file

You don't have to close a QNetworkReply. It will emit its finished() signal when it is done. If you don't need to do anything else when it is done, just connect the signal to the reply's deleteLater() slot.
If you have other clean up to do, connect to a slot elsewhere.

Cheers,
_

Thanks for reply
Yeah I got that but my main problem right now is not that,It's uploading file or data to a PHP script

Can you see the data in the network transfer using Wireshark (or Fiddler)?

Yes I did
these are the result:
html form :
Attachment 9955

python program:
Attachment 9956

some headers are not sending while In my code I'm sending them,:confused:

can someone drive me to right direction please ?

Your multipart form data needs to include the other elements of the form. In this case the submit button.

Code:

---

imagePart = QtNetwork.QHttpPart()
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentDispositionHeader,
                     "form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentTypeHeader,
                     'image/jpeg')
            imagePart.setBodyDevice(self._stream)
 
            submitPart = QtNetwork.QHttpPart()
            submitPart.setHeader(QtNetwork.QNetworkRequest.ContentDispositionHeader,
                "form-data; name=\"submit\"")
            submitPart.setBody("submit")
 
            self._multiPart.append(imagePart)
            self._multiPart.append(submitPart)

---

I tried this but no differences.still not working
I don't know what to do anymore :(

Yes it is. Your code with only the web server and file path changed, the submit button encoded, and the close() call removed:

Code:

---

from PyQt4 import QtCore, QtGui, QtNetwork
import sys
class Window(QtGui.QWidget):
    def __init__(self, parent=None):
        super(Window, self).__init__(parent)
 
        self.netaccess = QtNetwork.QNetworkAccessManager(self)
        self._uploaders = {}
        row = 1 #this number will change per upload in real program,this is just demo
        self.address = 'http://dev/demo.php'
        stream = QtCore.QFile('icon.png')
        if stream.open(QtCore.QIODevice.ReadOnly):
            data = stream
 
            uploader = self._uploaders[row] = Uploader(row, self.netaccess)
 
            uploader.upload(data, self.address)
 
class Uploader(QtCore.QObject):
 
    def __init__(self, key, parent):
        QtCore.QObject.__init__(self, parent)
        self._key = key
        self._reply = None
 
 
    def upload(self, data, url):
        if self._reply is None:
 
            self._stream = data
 
            self._multiPart = QtNetwork.QHttpMultiPart(QtNetwork.QHttpMultiPart.FormDataType)
 
            fileName = QtCore.QFileInfo(self._stream.fileName()).fileName()
            key = 'file'
 
            imagePart = QtNetwork.QHttpPart()
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentDispositionHeader,
                     "form-data; name=\"%s\"; filename=\"%s\"" % (key, fileName))
            imagePart.setHeader(QtNetwork.QNetworkRequest.ContentTypeHeader,
                     'image/png')
            imagePart.setBodyDevice(self._stream)
 
            submitPart = QtNetwork.QHttpPart()
            submitPart.setHeader(QtNetwork.QNetworkRequest.ContentDispositionHeader,
            "form-data; name=\"submit\"")
            submitPart.setBody("submit")
 
            self._multiPart.append(imagePart)
            self._multiPart.append(submitPart)
 
            request = QtNetwork.QNetworkRequest(QtCore.QUrl(url))
            request.setHeader(QtNetwork.QNetworkRequest.ContentTypeHeader,
  'multipart/form-data; boundary=%s' % self._multiPart.boundary())
            request.setRawHeader('User-Agent','Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36')
 
            self._reply = self.parent().post(request, self._multiPart)
            self._reply.uploadProgress.connect(self.handleUploadProgress)
            self._reply.error.connect(self.handleError)
            self._reply.finished.connect(self.handleFinished)
 
 
 
    def handleUploadProgress(self, sent, total):
        print sent, total
 
    def handleFinished(self):
        print('Content: ',self._reply.readAll())#no output here :(
        self._stream.close()
        self._multiPart.deleteLater()
        self._reply.deleteLater()
        self._reply = None
        app.quit()
 
    def handleError(self):
        print('Error String :',self._reply.errorString())
        print('Error number: ',self._reply.error())
 
 
app = QtGui.QApplication(sys.argv)
demo = Window()
sys.exit(app.exec_())

---

Using this simple PHP file upload receiver:

Code:

---

$ cat demo.php
<html>
    <body>
        <form method="post" enctype="multipart/form-data">
            <label for="file">Filename:</label>
            <input type="file" name="file" id="file" />
            <br />
            <input type="submit" name="submit" value="Submit" />
        </form>
        <p>
            <?php
            if(isset($_POST['submit'])) {
                if ($_FILES["file"]["error"] > 0) {
                    echo "Error: " . $_FILES["file"]["error"] . "<br />";
                } else {
                    echo "Upload: " . $_FILES["file"]["name"] . "<br />";
                    echo "Type: " . $_FILES["file"]["type"] . "<br />";
                    echo "Size: " . ($_FILES["file"]["size"] / 1024) . " Kb<br />";
                    echo "Stored in: " . $_FILES["file"]["tmp_name"];
                }
            }
            ?>
        </p>
    </body>
</html>

---

Gives this output with an artificially large icon.png:

Code:

---

16384 530779
32768 530779
...
524288 530779
530779 530779
('Content: ', PyQt4.QtCore.QByteArray('<html>\n    <body>\n        <form method="post" enctype="multipart/form-data">\n            <label for="file">Filename:</label>\n            <input type="file" name="file" id="file" /> \n            <br />\n            <input type="submit" name="submit" value="Submit" />\n        </form>\n        <p>\n            Upload: icon.png<br />Type: image/png<br />Size: 517.9921875 Kb<br />Stored in: /tmp/phpKg8FG7        </p>\n    </body>\n</html>\n'))

---

Oh my God
I can't believe finally solution is here
I don't know how to say thank you,you helped me a lot,thanks Chris
it's working like a charm
but I also found the real issue in here
I tried your code in python3 for the first time and like always I didn't get any answer(no file uploaded) I become so upset since your code didn't working too !
anyway,I just thought maybe it's because PyQt4 have problem in python3,So I installed PyQt4 on python2 and Got exactly your result and file is uploading too
so it's obviously a PyQt4 bug in python3(the exact code is working on python2 but not in python3)
Do I need to report this ? how ?

I am running Python 2.7.5 and Python 3.2.5 with PyQt 4.10.2 on 64-bit Linux.

My Python 3.2 on Linux also fails to send the data. The multipart message never makes it onto the wire correctly. Look closely at the two requests in Wireshark. With Python 2.7 the request goes on the wire with this header:

Code:

---

MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "boundary_.oOo._Nzc0MjMxOTU4MTgyMzc5MTk0Mw==NzIxMTQzNjA4"

---

but with Python 3 the Boundary value gets mangled:

Code:

---

MIME Multipart Media Encapsulation, Type: multipart/form-data, Boundary: "b'boundary_.oOo._NjY2OTgyMTQ3MTg1NTk1MzgyNw==NDY5Njg1NTMw'"

---

resulting in a malformed payload (because the payload is encoded with boundary = "boundary_.oOo._NjY2OTgyMTQ3MTg1NTk1MzgyNw==NDY5Nj g1NTMw"). The PyQt4 under Python 3 is outputting the boundary as a bytes literal not a string. This could be a python wrapper type mapping issue.

You should address the issue to the PyQt mailing list or Riverbank Software directly if you have a commercial licence.

I reported bug
Thanks for help Chris