QtTest and asserts

[doberkofler]

I was wondering how to deal with asserts fired from a function that is tested in a QTest module. Imagine, you want to test a method bool xyz() that internally uses a precondition that might fire if something is inconsistent. QCOMPARE(xyz(), true) would never be executed because xyz() already exists with an assert.

Is this simply not supported in the Qt test environment, does it need to somehow managed manually (using some global semaphore for example) or did I just not get it?

[lyuts]

I think that a function whose assert fires is not working properly and is not ready for testing at all. Besides, there are not so many cases when you should keep asserts in your production code.

[doberkofler]

I get the point but:
- I would actually also want to test if the pre-conditions, post-conditions, invariants and asserts are working as supposed in my test
- I'm very much convinced (and 20 years of C++ experience never proved me wrong) that assertions are even more important in production code then in debug version

[lyuts]

Could you post some sample code of such function, demonstrating what you are trying to do?

[doberkofler]

The following example should show what I'm talking about. my_assert would be my own assert macro that usually logs the error and reports it to the user before aborting the application.

I was now wondering, how to test a function that is supposed to fail (by asserting or raising an exception) in QtTest?

#include <QtTest/QtTest>
 
static float myFunction(float value)
{
	my_assert(value > 0.0); // check precondition
	return 2 / value;
}
 
class tst_QDate : public QObject {
	Q_OBJECT
public:
	tst_myClass() {}
	virtual ~tst_myClass() {}
public slots:
	void init() {}
	void cleanup() {}
private slots:
	void myClass();
};
 
void tst_QDate::myClass()
{
    QCOMPARE(float(1), myFunction(2.0));
 
    // Next QCOMPARE should fail (e.g. raise an exception)
    QCOMPARE(float(1), myFunction(0.0));
}
 
QTEST_APPLESS_MAIN(tst_myClass)
 
#include "moc_tst_myClass.cxx"

#include <QtTest/QtTest>

static float myFunction(float value)
{
	my_assert(value > 0.0); // check precondition
	return 2 / value;
}

class tst_QDate : public QObject {
	Q_OBJECT
public:
	tst_myClass() {}
	virtual ~tst_myClass() {}
public slots:
	void init() {}
	void cleanup() {}
private slots:
	void myClass();
};

void tst_QDate::myClass()
{
    QCOMPARE(float(1), myFunction(2.0));

    // Next QCOMPARE should fail (e.g. raise an exception)
    QCOMPARE(float(1), myFunction(0.0));
}

QTEST_APPLESS_MAIN(tst_myClass)

#include "moc_tst_myClass.cxx"

To copy to clipboard, switch view to plain text mode 

[lyuts]

How is it better than:

static float myFunction(float value)
{
    if (value <= 0.0)  // check precondition
    {
        return eNAN;
    }
    return 2 / value;
}

static float myFunction(float value)
{
    if (value <= 0.0)  // check precondition
    {
        return eNAN;
    }
    return 2 / value;
}

To copy to clipboard, switch view to plain text mode 

?

I realize that after your function is completely polished you can remove your assert and doing "if" will not take time. But, I think that your function should be error-resistable, since it gets value inputted by someone. Your function should take care of exceptional/errorneous situation (in your case, it is division by zero).
Lets pretend that your app reads numbers from some file and does divisions for them. This file is created by somebody who put 0 in that file.
Way 1: Your application is crashed by your assert only because someone was not careful enough and put 0 in that file.
Way 2: On processing that division by 0, your application returns NAN value and reports that wrong arguments have been specified.

What would you choose? I think that the 2nd way is more correct.

[doberkofler]

We seem to be moving a little away from my actual question but I'm very interested in this more generic topic as well. My answer to your question would (as always) be: IT DEPENDS!
It depends on what the contract between invoking function and the invoked one specifies and this can be to either not to care at all (most common), return a status and leave it to the caller to decide (your suggestion) or impose a strict rule of parameter passing using a precondition (this is what I wanted to do).

A more realistic (longer) version of myClass::myFunction would typically look like this:

float myClass::myFunction(float value)
{
   /* test consistency of class */
   my_class_invariant(...);
 
   /*test consistency of parameter */
   my_precondition(value > 0, "you are using an invalid value=("+toString(value)+")");
 
   /* usually the algorithms are "little" more complex */
   float temp = 10.0 / value;
 
   /* test consistency of intermediate result */
   my_assert(temp > 0.0 && ...);
 
   /* test consistency of result */
   my_postcondition(fuzzy_compare(temp * value, 10.0);
 
   return temp;
}

float myClass::myFunction(float value)
{
   /* test consistency of class */
   my_class_invariant(...);

   /*test consistency of parameter */
   my_precondition(value > 0, "you are using an invalid value=("+toString(value)+")");

   /* usually the algorithms are "little" more complex */
   float temp = 10.0 / value;

   /* test consistency of intermediate result */
   my_assert(temp > 0.0 && ...);

   /* test consistency of result */
   my_postcondition(fuzzy_compare(temp * value, 10.0);

   return temp;
}

To copy to clipboard, switch view to plain text mode 

[wysota]

- I'm very much convinced (and 20 years of C++ experience never proved me wrong) that assertions are even more important in production code then in debug version

If an assert triggers during tests it means it will fire in the production code. All asserts should be removed prior to testing (which is easy - just build your app in release mode). If you still want to test the conditions, substitute asserts with regular checks and exit the function gracefully.

If you insist on asserts not killing your application, redefine the assert/Q_ASSERT macro to do something else than abort the process.

[doberkofler]

If an assert triggers during tests it means it will fire in the production code. All asserts should be removed prior to testing (which is easy - just build your app in release mode). If you still want to test the conditions, substitute asserts with regular checks and exit the function gracefully.

As I explained in my reply to lyuts, I very strongly believe that there is no rule or best practice on how to deal with assertions. It really depends and I have made some very positive experience over a long period of time in keeping all asserts in the production code.

If you insist on asserts not killing your application, redefine the assert/Q_ASSERT macro to do something else than abort the process.

Thank you for making this clear. I just wanted to make sure, that there is no explicit functionality in QtTest that would intercept an exception instead of doing this myself.

[wysota]

As I explained in my reply to lyuts, I very strongly believe that there is no rule or best practice on how to deal with assertions.

Assert, by definition, kills your app. If you want an assert that will not kill your app but instead will return an error, it will not be an assert anymore.

Thank you for making this clear. I just wanted to make sure, that there is no explicit functionality in QtTest that would intercept an exception instead of doing this myself.

Asserts don't throw exceptions. They call abort(). So there is nothing to "intercept". Even as the manual for abort says - "abort() does not return". If you want, just throw an exception instead of asserting and you'll be able to catch that in QtTestLib.

[doberkofler]

Assert, by definition, kills your app. If you want an assert that will not kill your app but instead will return an error, it will not be an assert anymore.

I agree! My "assert" would log an error, show the error and then abort.

Asserts don't throw exceptions. They call abort(). So there is nothing to "intercept". Even as the manual for abort says - "abort() does not return". If you want, just throw an exception instead of asserting and you'll be able to catch that in QtTestLib.

Sorry for the misunderstanding, but I used the term exception in a generally and did not specifically refer to the C++ exception handling. More precisely put, I was unsure if QtTest might intercept the assert.

Intercepting my own assert handling is actually what I have done now and it seems to give me expected results. QtTets can actually catch C++ exception thrown by the test methods.

[lyuts]

The only place where I use asserts is in my internal classes. By saying internal, I mean those that do not provide public interface and is used only by me in my modules. Otherwise, I try to do regular checks.