Not really. I don't see any way in which you could remove some size options for a specific font from the QFontDatabase.
Why don't you implement your own font dialog and populate it with the fonts you need. In this case it will also be possible to restrict the size options the user has.
Regards
Bookmarks