Results 1 to 8 of 8

Thread: Wrong in qt calculation?

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #6
    Join Date
    Apr 2010
    Posts
    769
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11
    Thanks
    1
    Thanked 94 Times in 86 Posts

    Default Re: Wrong in qt calculation?

    C++ always performs computations at the full precision allowed by the data type. If doubles aren't working, you can try to convert the problem into an integer expression, or analyze the order of operations to try and minimize effects caused by limited precision, such as adding a very small number to a relatively very large number. Again, wysota's suggestion of using fixed point point math is also viable, although it may not be necessary.

    If you could post the inputs, intermediate results and outputs of your functions with two calls, and the expected results and discrepancies, we might be able to lend more assistance.

    ------------------------

    Something else to consider: what, exactly, is "wrong" with the results? It's entirely possible that Excel is doing some rounding itself when displaying results; C++ may also not be displaying the full precision of the numbers used, depending on how you're outputting them.
    Last edited by SixDegrees; 8th April 2011 at 13:34.

Similar Threads

  1. Exponential calculation
    By bollibompa in forum Newbie
    Replies: 1
    Last Post: 4th April 2011, 23:11
  2. QTime to a format which I can use in a calculation
    By Gavin Harper in forum Qt Programming
    Replies: 4
    Last Post: 14th March 2011, 23:15
  3. Monetary calculation
    By estanisgeyer in forum General Programming
    Replies: 1
    Last Post: 15th February 2008, 12:35
  4. Compiler's calculation on memory/address
    By vermarajeev in forum General Discussion
    Replies: 3
    Last Post: 10th October 2007, 11:25
  5. what to do during long calculation?
    By Weilor in forum Qt Programming
    Replies: 4
    Last Post: 14th September 2007, 12:11

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Qt is a trademark of The Qt Company.