QMetaType dynamic object creation and initialization

[d_stranz]

I am implementing an XML-based format to save and restore QSettings - basically using QSettings::registerFormat() with my own ReadFunc() and WriteFunc() methods that use QXmlStreamReader and QXmlStreamWriter respectively.

I have writing working just fine. I also have *most* of the reading part working as well. When I write a QSettings value out, I store the string representation of the value along with the string representation of the value's metatype. For example, the QColor "red" gets written to XML as the element:

<Color value="#ff0000" type="QColor"/>

When I read it back in, I retrieve the type name and use QMetaType::nameToType() to get an integer QMetaType type id value. I can then use QMetaType::create() with this type value.

So, what I get is a void * pointer to some memory that actually contains a QColor instance. I don't actually know it is a QColor except indirectly through the type id.

I could conceivably implement a gigantic switch() statement that checked the type id against every QMetaType::Type constant, and cast the result of QMetaType::create() appropriately, but this seems like a stupid way to do it. I'd have to include the header files for almost every fundamental type in Qt.

It looks like the QMetaType::load() method might be a generic way that would work for this. If I used QMetaType::save() to write my values to strings that get saved to the XML file and QMetaType::load() to retrieve them from the strings that are read in, is that the way to do it?

[wysota]

I think using load() just converts one problem to another one as you'd have to store binary (written by QDataStream) representations of your types instead of the nice xml ones. In my opinion you either need a "gigantic switch" to save and load a given meta type id from/to a given XML node or you can use inheritance to expose an interface for reading and writing such nodes and implement that interface for any meta type id you want to handle (which is basically what QMetaType::load() uses as well).

[d_stranz]

Actually, after much head beating and hair pulling, I figured out how to do what I want:

- use Q_DECLARE_METATYPE() for each custom class
- implement the operator>>() and operator<<() QDataStream operators for the class
- use qRegisterStreamOperatorsForMetaType() before attempting to serialize the class

After doing this, you can then convert between QVariant and the custom class using QVariant::value<T>() and QVariant::fromValue<T>().

Most of my head-banging was due to a stupid coding error. Instead of writing stream >> variant, I had written variant << stream, which of course gives all sorts of nonsense compilation errors.

The essential code to serialize QSettings in XML boils down to this. I use the XML element <Settings> as the root document element for proper XML. Sorry for the formatting; cut and paste into CODE blocks does a terrible job when the copied code is indented with tabs.

// readSettings() and writeSettings() are static members of a non-instantiable  XMLSettings class, and are passed
// in the call QSettings::registerFormat()
 
bool XMLSettings::readSettings( QIODevice & device, QSettings::SettingsMap & map )
{
	bool	bResult = false;
 
	QStringList	path;
 
	 QXmlStreamReader	reader( &device );
	while ( !reader.atEnd() )
	{
		QXmlStreamReader::TokenType	token = reader.readNext();
		switch( token )
		 {
		case QXmlStreamReader::StartElement:
			if ( reader.name() != "Settings" )
			{
				QByteArray	bytes( reader.name().toLatin1() );
				path.push_back( QString( bytes ) );
				QXmlStreamAttributes	attributes = reader.attributes();
				 if ( !attributes.isEmpty() )
				{
					QStringRef	value = attributes.value( "value" );
					 if ( !value.isEmpty() )
					{
						 QByteArray	valueBytes = QByteArray::fromBase64( value.toLatin1() );
						QDataStream	stream( valueBytes );
						 QVariant	var;
						 stream >> var;
 
						 // Adds an entry of the form Group1/Subgroup1/Subsubgroup1 = value
						map[ path.join( '/') ] = var;
					}
				}
			}
			 break;
 
		case QXmlStreamReader::EndElement:
			if ( !path.isEmpty() )
				path.removeLast();
			break;
 
		default:
			break;
		 }
	}
 
	bResult = !map.isEmpty();
	return bResult;
}
 
struct TreeNode 
{
	TreeNode( const QString & key = QString(), QVariant & value = QVariant(), TreeNode * pParent = 0 )
		:	mKey( key )
		,	mValue( value )
		,	mpParent( pParent )
	{
	}
 
	~TreeNode()
	{
		QList<TreeNode *>::iterator	it = mChildren.begin();
		QList<TreeNode *>::iterator	eIt = mChildren.end();
		while( it != eIt )
		{
			delete *it++;
		}
	}
 
	TreeNode *	addChild( const QString & key, QVariant & value = QVariant() )
	{
		TreeNode * pChild = new TreeNode( key, value, this );
		mChildren.push_back( pChild );
 
		return pChild;
	}
 
	TreeNode *	findChild( const QString & key ) const
	{
		TreeNode *	pChild = 0;
		QList<TreeNode *>::const_iterator	it = mChildren.begin();
		QList<TreeNode *>::const_iterator	eIt = mChildren.end();
		while( it != eIt && !pChild )
		{
			TreeNode * pNode = *it++;
			if ( pNode->mKey == key )
				pChild = pNode;
		}
		return pChild;
	}
 
	TreeNode *			mpParent;
	QList< TreeNode * >	mChildren;
 
	QString	mKey;
	QVariant	mValue;
};
 
static void writeTree( QXmlStreamWriter & writer, TreeNode * pNode )
{
	if ( pNode )
	{
		writer.writeStartElement( pNode->mKey );
		if ( !pNode->mValue.isNull() )
		 {
			QByteArray	byteArray;
			QBuffer		buffer( &byteArray );
			 buffer.open( QIODevice::WriteOnly );
 
			QDataStream	valueStream( &buffer );
			valueStream << pNode->mValue;
			buffer.close();
 
			 QString	value( byteArray.toBase64() );
			writer.writeAttribute( "value", value );
			writer.writeAttribute( "type", pNode->mValue.typeName() );
		 }
 
		 QList<TreeNode *>::const_iterator	it = pNode->mChildren.begin();
		QList<TreeNode *>::const_iterator	eIt = pNode->mChildren.end();
		while( it != eIt )
		{
			writeTree( writer, *it++ );
		}
		writer.writeEndElement();
	}
}
 
bool CSAQtXMLSettings::writeSettings( QIODevice & device, const QSettings::SettingsMap & map )
{
	bool	bResult = false;
 
	TreeNode *	pRootNode = new TreeNode( "Settings" );
 
	QSettings::SettingsMap::const_iterator	it = map.begin();
	QSettings::SettingsMap::const_iterator	eIt = map.end();
	while ( it != eIt )
	{
		const QString &	key = it.key();
		QVariant				value = it.value();
 
		qDebug() << "Key: " << key;
 
		TreeNode *	pTarget = pRootNode;
		TreeNode *	pChild = 0;
		QString		lastKey;
 
		QStringList	path = key.split( '/' );
 
		QStringList::const_iterator	lIt = path.begin();
		QStringList::const_iterator	lEIt = path.end();
		while ( lIt != lEIt )
		{
			const QString &	subPath = *lIt++;
			pChild = pTarget->findChild( subPath );
			if ( !pChild  )
			{
				if ( lIt != lEIt )
					pTarget	= pTarget->addChild( subPath );
				else
					lastKey	= subPath;
			}
			else
			 {
				pTarget = pChild;
			}
		}
 
		pTarget->addChild( lastKey, value );
		it++;
	}
 
	if ( pRootNode->mChildren.size() )
	{
		bResult = true;
 
		QXmlStreamWriter	writer( &device );
		writer.setAutoFormatting( true );
		writer.writeStartDocument();
		writeTree( writer, pRootNode );
		writer.writeEndDocument();
	}
 
	delete pRootNode;
	return bResult;
}

// readSettings() and writeSettings() are static members of a non-instantiable  XMLSettings class, and are passed
// in the call QSettings::registerFormat()

bool XMLSettings::readSettings( QIODevice & device, QSettings::SettingsMap & map )
{
	bool	bResult = false;

	QStringList	path;

	 QXmlStreamReader	reader( &device );
	while ( !reader.atEnd() )
	{
		QXmlStreamReader::TokenType	token = reader.readNext();
		switch( token )
		 {
		case QXmlStreamReader::StartElement:
			if ( reader.name() != "Settings" )
			{
				QByteArray	bytes( reader.name().toLatin1() );
				path.push_back( QString( bytes ) );
				QXmlStreamAttributes	attributes = reader.attributes();
				 if ( !attributes.isEmpty() )
				{
					QStringRef	value = attributes.value( "value" );
					 if ( !value.isEmpty() )
					{
						 QByteArray	valueBytes = QByteArray::fromBase64( value.toLatin1() );
						QDataStream	stream( valueBytes );
						 QVariant	var;
						 stream >> var;
	
						 // Adds an entry of the form Group1/Subgroup1/Subsubgroup1 = value
						map[ path.join( '/') ] = var;
					}
				}
			}
			 break;

		case QXmlStreamReader::EndElement:
			if ( !path.isEmpty() )
				path.removeLast();
			break;

		default:
			break;
		 }
	}

	bResult = !map.isEmpty();
	return bResult;
}

struct TreeNode 
{
	TreeNode( const QString & key = QString(), QVariant & value = QVariant(), TreeNode * pParent = 0 )
		:	mKey( key )
		,	mValue( value )
		,	mpParent( pParent )
	{
	}

	~TreeNode()
	{
		QList<TreeNode *>::iterator	it = mChildren.begin();
		QList<TreeNode *>::iterator	eIt = mChildren.end();
		while( it != eIt )
		{
			delete *it++;
		}
	}

	TreeNode *	addChild( const QString & key, QVariant & value = QVariant() )
	{
		TreeNode * pChild = new TreeNode( key, value, this );
		mChildren.push_back( pChild );

		return pChild;
	}

	TreeNode *	findChild( const QString & key ) const
	{
		TreeNode *	pChild = 0;
		QList<TreeNode *>::const_iterator	it = mChildren.begin();
		QList<TreeNode *>::const_iterator	eIt = mChildren.end();
		while( it != eIt && !pChild )
		{
			TreeNode * pNode = *it++;
			if ( pNode->mKey == key )
				pChild = pNode;
		}
		return pChild;
	}

	TreeNode *			mpParent;
	QList< TreeNode * >	mChildren;

	QString	mKey;
	QVariant	mValue;
};

static void writeTree( QXmlStreamWriter & writer, TreeNode * pNode )
{
	if ( pNode )
	{
		writer.writeStartElement( pNode->mKey );
		if ( !pNode->mValue.isNull() )
		 {
			QByteArray	byteArray;
			QBuffer		buffer( &byteArray );
			 buffer.open( QIODevice::WriteOnly );

			QDataStream	valueStream( &buffer );
			valueStream << pNode->mValue;
			buffer.close();

			 QString	value( byteArray.toBase64() );
			writer.writeAttribute( "value", value );
			writer.writeAttribute( "type", pNode->mValue.typeName() );
		 }

		 QList<TreeNode *>::const_iterator	it = pNode->mChildren.begin();
		QList<TreeNode *>::const_iterator	eIt = pNode->mChildren.end();
		while( it != eIt )
		{
			writeTree( writer, *it++ );
		}
		writer.writeEndElement();
	}
}

bool CSAQtXMLSettings::writeSettings( QIODevice & device, const QSettings::SettingsMap & map )
{
	bool	bResult = false;

	TreeNode *	pRootNode = new TreeNode( "Settings" );

	QSettings::SettingsMap::const_iterator	it = map.begin();
	QSettings::SettingsMap::const_iterator	eIt = map.end();
	while ( it != eIt )
	{
		const QString &	key = it.key();
		QVariant				value = it.value();

		qDebug() << "Key: " << key;

		TreeNode *	pTarget = pRootNode;
		TreeNode *	pChild = 0;
		QString		lastKey;

		QStringList	path = key.split( '/' );

		QStringList::const_iterator	lIt = path.begin();
		QStringList::const_iterator	lEIt = path.end();
		while ( lIt != lEIt )
		{
			const QString &	subPath = *lIt++;
			pChild = pTarget->findChild( subPath );
			if ( !pChild  )
			{
				if ( lIt != lEIt )
					pTarget	= pTarget->addChild( subPath );
				else
					lastKey	= subPath;
			}
			else
			 {
				pTarget = pChild;
			}
		}

		pTarget->addChild( lastKey, value );
		it++;
	}

	if ( pRootNode->mChildren.size() )
	{
		bResult = true;

		QXmlStreamWriter	writer( &device );
		writer.setAutoFormatting( true );
		writer.writeStartDocument();
		writeTree( writer, pRootNode );
		writer.writeEndDocument();
	}

	delete pRootNode;
	return bResult;
}

To copy to clipboard, switch view to plain text mode 

QSettings stores key / value pairs in the QSettingsMap. The hierarchy of groups within groups is accomplished by concatenating group-level keys using a "/". When writing the QSettings out to XML, the concatenated key is split into the group level parts, and each group name becomes an XML element in the XML hierarchy. Likewise, when reading in the XML hierarchy, the QStringList "path" is used as a stack to assemble the compound keys for each QSettings value.

Note that the "type" attributes are unused when reading in the XML. Since the actual values are written out in base64, I use this for debugging purposes to ensure that the code is translating and creating the QVariant instances correctly when reading.

[anda_skoa]

Cool stuff!

One question though: any reason for converting the string to bytes just to convert back again?

QByteArray	bytes( reader.name().toLatin1() );
				path.push_back( QString( bytes ) );

QByteArray	bytes( reader.name().toLatin1() );
				path.push_back( QString( bytes ) );

To copy to clipboard, switch view to plain text mode 

And this

QList<TreeNode *>::iterator	it = mChildren.begin();
		QList<TreeNode *>::iterator	eIt = mChildren.end();
		while( it != eIt )
		{
			delete *it++;
		}

QList<TreeNode *>::iterator	it = mChildren.begin();
		QList<TreeNode *>::iterator	eIt = mChildren.end();
		while( it != eIt )
		{
			delete *it++;
		}

To copy to clipboard, switch view to plain text mode 

can probably be shortened like this

qDeleteAll(mChildren);

qDeleteAll(mChildren);

To copy to clipboard, switch view to plain text mode 

Cheers,
_

[d_stranz]

Cool stuff!

Thanks. Learned a lot about QVariant and QMetaType in the process. It is unfortunate that QSettings::SettingsMap is not accessible except in the read / write methods, because if it was then you could easily convert from one QSettings format to another simply by assigning the map. The only way to do it now is to iterate over QSettings::allKeys() and copy the keys and values from one QSettings instance to another.

One question though: any reason for converting the string to bytes just to convert back again?

No - that's probably junk code left over from when I was trying to get it all working. QXmlStreamReader returns QStringRef rather than QString for strings, so I should change this to:

path.push_back( reader.name().toString() );

path.push_back( reader.name().toString() );

To copy to clipboard, switch view to plain text mode 

qDeleteAll(mChildren);

Yeah, although I'm guessing that qDeleteAll() does the essentially same thing. I get shy about too much Qt-ness in my code; I generally stay away from Qt containers in favor of STL (for even more portability) and likewise prefer to use STL-style iteration over containers for the same reason. Not that this particular piece of code would likely ever be used outside of a Qt context, but I like to be habitual about my coding conventions.

[wysota]

AFAIK qDeleteAll works with non-qt containers as well:

template <typename ForwardIterator>
Q_OUTOFLINE_TEMPLATE void qDeleteAll(ForwardIterator begin, ForwardIterator end)
{
    while (begin != end) {
        delete *begin;
        ++begin;
    }
}

template <typename ForwardIterator>
Q_OUTOFLINE_TEMPLATE void qDeleteAll(ForwardIterator begin, ForwardIterator end)
{
    while (begin != end) {
        delete *begin;
        ++begin;
    }
}

To copy to clipboard, switch view to plain text mode