Results 1 to 6 of 6

Thread: Floating point imprecision

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Join Date
    May 2006
    Posts
    58
    Thanks
    12
    Thanked 1 Time in 1 Post
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Floating point imprecision

    OK, I have been programming in C++ for over 10 years now and I've never run into this problem...I've always known that floating point operations are somewhat "fuzzy", but I always assumed it would be down to a resolution that I never encounter...well, in a point of sale app I wrote using QT, I am finding that for purchases totaling over $13,000 I am getting imprecision down to 0.1 (I'm running into 5 cents off)

    At first I thought I was crazy, but after doing tons and tons of printfs, it seems that something flaky is going on. I have a function called RoundMoney that does a floor and a couple of divisions to force the float value to two decimal places. What's odd is that when I echo the value with qDebug:

    qDebug() << fValue

    I get the result I'd expect (I think it's 13125.2) ...but when I use

    printf("%.5f\n", fValue);

    I am getting the values that are off by a nickel (13125.240 or something close...I don't have my debug output in front of me) Note that my arithmetic comparisons (greater than, less than) reflect the value reported by printf and NOT the qDebug output. If anything, the qDebug output is "wrong" because it doesn't agree with the actual if( x < y ) comparisons I'm making. Basically my code looks roughly like this:

    if( fOwed > fPaid ) {
    printf("%.3f owed, %.3f paid\n", fOwed, fPaid);
    qDebug << fOwed << " owed " << fPaid << " paid";
    }

    And my output (roughly) is:
    13125.240 owed, 13125.200 paid
    13125.2 owed, 13125.2 paid

    How does QT "know" to truncate that .04??

    I am looking for one of two possible solutions:

    1) Do whatever the hell it is that QT is doing internally to "know" that I earlier truncated my value and ignore the remaining "imprecise" floating point values

    2) Use something besides floating point values and arithmetic. Trolltech seems to have thought of everything with QT, is there some sort of high level "decimal" or "money" class to deal with this?
    Last edited by hardgeus; 3rd August 2006 at 02:58.

Similar Threads

  1. Replies: 3
    Last Post: 20th April 2006, 11:21

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Qt is a trademark of The Qt Company.