There's no doubt that QGraphicsView/Scene/Item is the preferred way to do this. You'll basically have to reimplement everything those classes already do, or a significant portion of it.

Take an hour to read through the documentation for those classes, and to look at the demos that are availalble. They're very simple to use, and will save you a great deal of time.