QLabel calculates wrong size in a layout

Put a QLabel in a vertical layout. Put a vertical spacer under it. Set the font of the label as Arial. Set the point size as 50.
When you setText() the label gets smaller height, when you clear() the label gets bigger height.
While setting font as Tahoma won't cause this problem. I think the calculation of size works differently when there is (no) text in the label. But some fonts have same result while some don't.

How to make the auto size always have the same calculation?