Problem solved.
It was wrong that I scaled the painter to (1/m11,1/m22)
Using :

Qt Code:
  1. painter->save();
  2. painter->translate(pos);
  3. m11=painter->matrix().m11();m22=painter->matrix().m22();
  4. painter->scale(1.0,-yAxisSign()*1.0);
To copy to clipboard, switch view to plain text mode 
the problem is gone.
Though I have the question , why the painter->font() didn't change its fontMetrics() and why painter->drawText() was returning the wrong boundingRect when I DID change the scale.