setMask (smooth)

[pakulo]

How i can set the mask look like the right picture (frmae is smooth)?
http://file.oboz.ua/files/vf46476dce...0751322586.jpg

My result in the left.

[wysota]

The mask is a bitmap (opaque/transparent) so if you need to include antialiasing, the mask should return opaque (1) for parts that are to be antialiased.

[spud]

It depends on what you want to do. If you want to display a QPixmap in your application you should use a transparent png or use setAlphaCannel() instead of setMask() and the background will be propagated.
On the other hand if you want to set the shape of a window(like the shaped clock example) you're out of luck. There is no easy platform independent way to do this. See this thread.