Results 1 to 6 of 6

Thread: large data files in UNIX time. Possible to plot them in human-readable time??

  1. #1
    Join Date
    Oct 2010
    Posts
    58
    Thanks
    26
    Qt products
    Qt4
    Platforms
    Windows

    Default large data files in UNIX time. Possible to plot them in human-readable time??

    My computer:
    Visual Studio 2008, qt 4.7, qwt 6.0

    Hi,

    I am making a qwt plot from a very large file that has time and prices. The times in the file are all in Unix time as doubles. The plot comes out great with time on the x-axis and prices on the y-axis. The only problem is that the times are all in Unix and not very useful to look at. Does anyone know if there is a way that I can convert the times on the plot into human-readable dates? The kinds of files that I want to plot come from online databases and are all in Unix time.

    The only thing I can think of is loading up all the time data from the files and using mktime to make them all into readable dates, but I don't really know how I'd do that. It seems like a lot of work (especially because the files have between 2-4 million time values) Also, I'm not sure if changing these files would make my code break (like the numOfPts when I load the data)...

    here is some of my code:


    Qt Code:
    1. newPlot::newPlot()
    2. {
    3. QwtScaleEngine::Floating;
    4. QwtPlotCurve *Curve = new QwtPlotCurve("plotty");
    5. Curve->setPen(QPen(Qt::red));
    6. Curve->setStyle(QwtPlotCurve::Steps);
    7. Curve->attach(this);
    8.  
    9. ////chose which file to load and load the data
    10. char * FileName = "C:/data/....price_and_time";
    11. FILE * fp = fopen ( FileName, "rb");
    12. if(fp == NULL) EXCEPTION( "no file" );
    13.  
    14. fseek(fp, 0, SEEK_END);
    15. int numOfPts = ftell( fp )/ (2 * sizeof (double));
    16.  
    17. double *time = new double[numOfPts];
    18. double *value = new double[numOfPts];
    19. double *tempTime = time;
    20. double *tempValue = value;
    21.  
    22. fseek(fp, 0, 0);
    23. for( int i=0; i<numOfPts; i++){
    24. int j = fread(tempTime, sizeof(double), 1, fp);
    25. int k = fread(tempValue, sizeof(double), 1, fp);
    26. tempTime++;
    27. tempValue++;
    28. }
    29. Curve->setRawSamples(time, value, numOfPts);
    30. }
    31.  
    32. if (fp!=NULL)
    33. fclose(fp);
    34. }
    To copy to clipboard, switch view to plain text mode 

    I am putting the plot into a gui so that I can add features later:

    Qt Code:
    1. class MyWidget : public QWidget{
    2. public:
    3. MyWidget(QWidget *parent = 0);
    4. };
    5.  
    6. MyWidget::MyWidget(QWidget *parent):QWidget(parent)
    7. {
    8. setFixedSize(1500,750);
    9. newPlot *myPlot = new newPlot;
    10. QVBoxLayout *layout = new QVBoxLayout;
    11.  
    12. layout->addWidget(myPlot);
    13. setLayout(layout);
    14. }
    15.  
    16.  
    17. int main(int argc, char * argv[])
    18. {
    19. QApplication app(argc, argv);
    20. MyWidget widget;
    21. widget.show();
    22. return app.exec();
    23.  
    24. }
    To copy to clipboard, switch view to plain text mode 



    Any ideas?

    Thanks for your help!

  2. #2
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: large data files in UNIX time. Possible to plot them in human-readable time??

    QDateTime::fromTime_t() and QDateTime::toString()
    Assuming your double time values are seconds and fractions of seconds from 1 Jan 1970 you will lose the fractional seconds.

    You probably don't want to convert 4 million entries when you only need a few axis labels to cover the period.
    Last edited by ChrisW67; 9th November 2010 at 02:58.

  3. The following user says thank you to ChrisW67 for this useful post:

    kja (9th November 2010)

  4. #3
    Join Date
    Oct 2010
    Posts
    58
    Thanks
    26
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: large data files in UNIX time. Possible to plot them in human-readable time??

    Thanks for the reply, I will check out QDateTime.

    You probably don't want to convert 4 million entries when you only need a few axis labels to cover the period.
    Actually that was another question I was thinking about. I am new to qt and c++ and I don't know how I would go about writing a program to only plot some of the data in a way that makes a useful plot. Could I make it plot only every certain number of pixels? I would also want more data points to be printed when I zoomed in, like having it replot with more data points, but I don't know where to start. Any Idea?

    Thanks a lot!

  5. #4
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: large data files in UNIX time. Possible to plot them in human-readable time??

    If you already have Qwt plotting your entire data set then just tell it to plot a smaller time or price range.

  6. #5
    Join Date
    Oct 2010
    Posts
    58
    Thanks
    26
    Qt products
    Qt4
    Platforms
    Windows

    Default Re: large data files in UNIX time. Possible to plot them in human-readable time??

    Ok so now that I can convert the doubles to strings how can I get qwtplot to plot it? Or if I don't need to plot it how can I get the x-axis to be labeled with my dates as strings?

    I was plotting before with SetRawSamples that takes double *'s

    Any ideas?

  7. #6
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: large data files in UNIX time. Possible to plot them in human-readable time??

    I guess you would use QwtScaleDraw and QwtScaleDiv classes. You should probably post this question in the Qwt sub-forum.

  8. The following user says thank you to ChrisW67 for this useful post:

    kja (20th November 2010)

Similar Threads

  1. Replies: 6
    Last Post: 18th August 2010, 13:52
  2. QFileSystemWatcher with a Qwt Real-time plot
    By gen_mass in forum Qt Programming
    Replies: 1
    Last Post: 25th June 2010, 22:28
  3. Best way in Qt to plot curve per real-time reading?
    By Sheng in forum Qt Programming
    Replies: 1
    Last Post: 10th February 2009, 23:33
  4. How to convert unix time to QDateTime
    By lni in forum Qt Programming
    Replies: 3
    Last Post: 14th November 2007, 01:25
  5. fast writing of large amounts of data to files
    By TheKedge in forum Qt Programming
    Replies: 1
    Last Post: 13th February 2007, 17:33

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.