Did you try using setSceneRect?
Make sure you don't center it in the view when you want to move it around.