sending encrypted data via mail using smtp

[vermarajeev]

Hello,
I'm writing a mail utility program in my application. I make use of smtp example given in Qt3 (/examples/network/mail).

Description of my application:
The application is very similar to mspaint in windows. The only difference is my application creates data files while mspaint creates image files.

Problem:
The files created by my application can be a plain/encrypted one. I use AES::CBC( Advanced encryption standard in Cipher Block Chaining) algorithm to encryt the files.
I'm able to send both types of files( plain/encrypt ) via mail. The problem is when I send an encrypted file, the recipient is unable to decrypt the file.

Consider below scenario.
PersonA sends an encrypted file to PersonB
PersonB detaches the file
PersonB supplies detached file through decryption functionality (AES::CBC) to get back the plain text but fails.

I went through some of the specifications of SMTP and found that, smtp can only transfer data with ascii value less than 127. Also mentioned in the specification that we have to use base64 alogorithm to encode the text/binary data.
Basically the structure is something like below.

Sending email:
File --> (Encoding base64) --> encoded file

Receiving email:
encoded file --> (Decoding base64) --> File

The file can be text or binary.

Even though I'm using the same structure as discussed above, the recipient is unable to decrypt the file (the decryption of file fails). I wonder why? I have been trying to figure out what is wrong but was unsuccessful. Can somebody please help me?

Thanks in advance.

[marcel]

But there can be any number of reasons.
Have you tried to isolate the encoding (to base 64 and the reverse ) and test it locally?
Also have you tested the encryption and decryption separately?

Regards

[vermarajeev]

Also have you tested the encryption and decryption separately?

Yes the encryption and description works fine separately. On the local machine I can encrypt/decrypt, works fine.
The only problem comes when I send the encrypted file via mail and then try to decrypt the file.

Have you tried to isolate the encoding (to base 64 and the reverse ) and test it locally?

I didnt get you.

[marcel]

I went through some of the specifications of SMTP and found that, smtp can only transfer data with ascii value less than 127. Also mentioned in the specification that we have to use base64 alogorithm to encode the text/binary data.

Don't you do this? Encode all data(encrypted or not) prior to sending to base 64 and decode it upon receiving?

I was under the impression that you did this.

Regards

[vermarajeev]

Don't you do this? Encode all data(encrypted or not) prior to sending to base 64 and decode it upon receiving?

I was under the impression that you did this.

Regards

Encode all data by what?
below is the structure

Sending email:
File --> (Encoding base64) --> encoded file

Receiving email:
encoded file --> (Decoding base64) --> File

File can be plain/encoded
I use AES::CBC to encode the file initially, then encode it again using base64 algorithm then give it to SMTP. I use Transfer encoding scheme as base64 in smtp.
Please be more specific?

[marcel]

I use AES::CBC to encode the file initially, then encode it again using base64 algorithm then give it to SMTP. I use Transfer encoding scheme as base64 in smtp.

This is what I was asking.
If you encode the data to base 64 before sending it.
Also, I asked if you tested this base 64 encoding/decoding locally.
Maybe you don't decode from base64 ok.

I imagine you cannot post the code, but are you sure there aren't any differences between plain files? Not even a byte?
Do you handle well SMTP transparency and the end of data message indicator?
Because a plain file might work even if you miss a byte, but an encoded file won't.

Regards

[vermarajeev]

This is what I was asking.
If you encode the data to base 64 before sending it.
Also, I asked if you tested this base 64 encoding/decoding locally.
Maybe you don't decode from base64 ok.
Regards

The decoding from base64 is taken care by smtp once I use

Content-Transfer-Encoding: base64

after the DATA tag used for smtp.

I imagine you cannot post the code, but are you sure there aren't any differences between plain files? Not even a byte?
Do you handle well SMTP transparency and the end of data message indicator?
Because a plain file might work even if you miss a byte, but an encoded file won't.

I can here is the small sample

void SynsupMail::sendMessage()
{
	QString message;
	string msg;
 
	QFile f ( textFilename->text() ); //textFilename->text() contains file path. The file can be en
									  //encrypted using AES::CBC or a plain text
	if ( f.open (IO_ReadOnly) )
		{
			// file opened successfully
			QTextStream t ( &f );
			// read the contents of the file into message
			while( ! t.eof() )
				{
				message += t.readLine();
				message += "\n";
				}
			f.close();		
 
			//encode the data using base64 using crypto++ library for Qt3, in Qt4
            //there is a function tobase64().			
			StringSource (string(message.data()), true, new Base64Encoder(new StringSink(msg))) ;		
 
			Smtp *smtp = new Smtp( textFrom->text(), textTo->text(), textSubject->text(),
			QString(msg.data()), fileName, );		
		}
}

void SynsupMail::sendMessage()
{
	QString message;
	string msg;
	
	QFile f ( textFilename->text() ); //textFilename->text() contains file path. The file can be en
									  //encrypted using AES::CBC or a plain text
	if ( f.open (IO_ReadOnly) )
		{
			// file opened successfully
			QTextStream t ( &f );
			// read the contents of the file into message
			while( ! t.eof() )
				{
				message += t.readLine();
				message += "\n";
				}
			f.close();		
			
			//encode the data using base64 using crypto++ library for Qt3, in Qt4
            //there is a function tobase64().			
			StringSource (string(message.data()), true, new Base64Encoder(new StringSink(msg))) ;		
				
			Smtp *smtp = new Smtp( textFrom->text(), textTo->text(), textSubject->text(),
			QString(msg.data()), fileName, );		
		}
}

To copy to clipboard, switch view to plain text mode 

All the code in smtp.cpp is same as that of /examples/network/mail except this
else if in function void Smtp::readyRead(). I made this modification to
send the file as an attachment

else if ( state == Body && responseLine[0] == '3' )
	{
		 QString fileName("");
 		 QString version;
 		 version += QString("MIME-Version: 1.0\r\n");
 
 		 QString Header;
 		 Header += QString("Content-Type: multipart/mixed; ")  + 
		 QString("boundary=unique-boundary-1\r\n") ;	 		
 
 		 QString headerContent;
 		 headerContent += QString("--unique-boundary-1\r\n");
 		 headerContent += QString("Content-Type: application/octet-stream ") + QString("\r\n");
 
		 headerContent += QString("Content-Transfer-Encoding: base64\r\n");
 		 headerContent += QString("Content-Disposition: attachment;") + 
		 QString( "filename=" ) + _fileName ;
 
		 //t is QTextStream	
		 *t << version;		 
		 *t << address;
 		 *t << Header;
 		 *t << headerContent;		 		 
 
 		 *t << message; 
 		 QString closeTag;
 		 closeTag += QString("--unique-boundary-1--\r\n");
 
 		 *t << closeTag << ".\r\n";	
		 state = Quit;
	}

else if ( state == Body && responseLine[0] == '3' )
	{
		 QString fileName("");
 		 QString version;
 		 version += QString("MIME-Version: 1.0\r\n");
		 		 
 		 QString Header;
 		 Header += QString("Content-Type: multipart/mixed; ")  + 
		 QString("boundary=unique-boundary-1\r\n") ;	 		
		 		 		 		 
 		 QString headerContent;
 		 headerContent += QString("--unique-boundary-1\r\n");
 		 headerContent += QString("Content-Type: application/octet-stream ") + QString("\r\n");

		 headerContent += QString("Content-Transfer-Encoding: base64\r\n");
 		 headerContent += QString("Content-Disposition: attachment;") + 
		 QString( "filename=" ) + _fileName ;

		 //t is QTextStream	
		 *t << version;		 
		 *t << address;
 		 *t << Header;
 		 *t << headerContent;		 		 
		 		 
 		 *t << message; 
 		 QString closeTag;
 		 closeTag += QString("--unique-boundary-1--\r\n");

 		 *t << closeTag << ".\r\n";	
		 state = Quit;
	}

To copy to clipboard, switch view to plain text mode 

These are the things that happen,
Plain text file--->

plaintext file --> Encoding base64 --> Over N/w --> SMTP decodes base64 --> plaintext file to receiver

Encrypted file (encrypted with AES::CBC)--->

Encrypted file --> Encoding base64 --> Over N/w --> SMTP decodes base64 --> Encrypted file

M i doing anything wrong above ?

[marcel]

while( ! t.eof() )
{
  message += t.readLine();
  message += "\n";
}

while( ! t.eof() )
{
  message += t.readLine();
  message += "\n";
}

To copy to clipboard, switch view to plain text mode 

I see you add a newline character for each line read from the file.
Why is that? RFC 821 specifies maximum line lengths of 998 bytes.
Do you do this for both encrypted and plain files? Because it is not ok to modify an encrypted file like this. A plain file would work, though.

If you do this for encrypted files, then do you also do you do the reverse process on receiving?

Regards

[vermarajeev]

while( ! t.eof() )
{
  message += t.readLine();
  message += "\n";
}

while( ! t.eof() )
{
  message += t.readLine();
  message += "\n";
}

To copy to clipboard, switch view to plain text mode 

I see you add a newline character for each line read from the file.
Why is that?

A newline. So mean to say I shouldnt append the new line like above.
Ok I'll try removing the newline "\n" from message and see if it works.

If you do this for encrypted files, then do you also do you do the reverse process on receiving?

No I was not doing that.

Marcel one more doubt, I use

message = QString::fromLatin1( "\n\n" ) + body + "\n";		 
replace( message, QString::fromLatin1( "\n" ), QString::fromLatin1( "\r\n" ) );
replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" ) );

message = QString::fromLatin1( "\n\n" ) + body + "\n";		 
replace( message, QString::fromLatin1( "\n" ), QString::fromLatin1( "\r\n" ) );
replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" ) );

To copy to clipboard, switch view to plain text mode 

where message is the base64 encoded data. Do I need to use replace() here? I think replacing "\n" with "\r\n" might also modify the data. I use the same code as in smtp.cpp given in /examples/network/mail

[marcel]

Marcel one more doubt, I use

message = QString::fromLatin1( "\n\n" ) + body + "\n";         
replace( message, QString::fromLatin1( "\n" ), QString::fromLatin1( "\r\n" ) );
replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" ) );

message = QString::fromLatin1( "\n\n" ) + body + "\n";         
replace( message, QString::fromLatin1( "\n" ), QString::fromLatin1( "\r\n" ) );
replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" ) );

To copy to clipboard, switch view to plain text mode 

where message is the base64 encoded data. Do I need to use replace() here? I think replacing "\n" with "\r\n" might also modify the data. I use the same code as in smtp.cpp given in /examples/network/mail

It is not ok to alter the encrypted message in this way, unless you apply the reverse before decrypting it.

replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" )

replace( message, QString::fromLatin1( "\r\n.\r\n" ), QString::fromLatin1( "\r\n..\r\n" )

To copy to clipboard, switch view to plain text mode 

It is required.
This is actually the transparency part.
So you do this after encryption but prior to sending?
Then do the reverse after you receive it.

Regards

[vermarajeev]

So you do this after encryption but prior to sending?

Did you mean,

For plainText
plaintext-->base64-->replace() for transparency part -->send

For encrypted file with AES::CBC
encrypted file-->base64-->replace() for transparency part-->send

Then do the reverse after you receive it.

Ok for plain text file there in no problem in reversing the opr, but how do I reverse
for an encrypted file as the data is binary ?

[wysota]

I suggest you try to base64 encode a plain text file using Qt methods and send it over smtp to see if it is decoded correctly by your mail client. Then you can focus on encrypting the content.

[vermarajeev]

I suggest you try to base64 encode a plain text file using Qt methods and send it over smtp to see if it is decoded correctly by your mail client. Then you can focus on encrypting the content.

Yes, I have tried it and it works fine for plain text. The only problem is with encrypted file.