QVector crashes when array size is big

[Sheng]

#include<QApplication>
#include<iostream>
#include<QString>
#include<QVector>
 
struct myStruc
{
 QString name;
 int array[3000*2000];
};
 
int main(int argc,char* argv[])
{
  QApplication app(argc,argv);
  QVector<myStruc> myVector;
  myStruc currentStruc;
  for (int i=0;i<5;i++)
   {
     currentStruc.name+="a";
     currentStruc.array[0]=i;
     myVector.push_back(currentStruc);
   }
  for(int i=0;i<5;i++)
   {
     std::cout<<myVector[i].name.toStdString()<<"   "<<myVector[i].array[0]<<std::endl;
   }
 myVector.clear();
 return app.exec();
}

#include<QApplication>
#include<iostream>
#include<QString>
#include<QVector>

struct myStruc
{
 QString name;
 int array[3000*2000];
};

int main(int argc,char* argv[])
{
  QApplication app(argc,argv);
  QVector<myStruc> myVector;
  myStruc currentStruc;
  for (int i=0;i<5;i++)
   {
     currentStruc.name+="a";
     currentStruc.array[0]=i;
     myVector.push_back(currentStruc);
   }
  for(int i=0;i<5;i++)
   {
     std::cout<<myVector[i].name.toStdString()<<"   "<<myVector[i].array[0]<<std::endl;
   }
 myVector.clear();
 return app.exec();
}

To copy to clipboard, switch view to plain text mode 

Compile fine, got "segmentation fault" error when run it.
It works OK if I decrease array size to 3000, any solutions?
Thanks in advance.

[abernat]

Have you stepped through it in a debugger or added qDebug() output to determine where you're getting the seg fault?

[Sheng]

Have you stepped through it in a debugger or added qDebug() output to determine where you're getting the seg fault?

The segmentation fault comes when push_back function is called.

[talk2amulya]

well, the issue is that QVector and other containers allocate CONTIGUOUS memory..and ur every array takes 24+MB of memory plus QString which is also dynamic..so there are definitely chances that when u try to push back one more..it being another contiguous location enters into another segment..thus a segmentation fault..a solution to this would be PROBABLY create a container that doesn't allocate contiguous memory but spreaded..but still its risky cuz amount being allocated is huge...maybe some guru can give a better insight on this

[talk2amulya]

what happens when u try other container classes like QList?

[Sheng]

what happens when u try other container classes like QList?

Same thing happens (QVector,QList,QLinkedList, std::vector, std::list). Did some search and found this looks like a notorious problem of STL container. I guess Qt Container might use STL during implementation since STL Containers crash as well.
Anyway, I will write a simple list container tomorrow.

[wysota]

6*3000*2000*sizeof(int) = 36M*sizeof(int) = 144 (or 288) MB just for creating the array. Aren't you running out of memory? If not, are you sure C++ copies arrays? I doubt that so copying the struct might lose its integrity.

[lni]

The crash is not because it is QVector. You program will crash without QVector in pure C++..

Don't put such a huge array in stack. You should use memory in the heap. But it is going to swap if you don't have enough memory...

Do this:

struct myStruc
{
 QString name;
 int *array;
 
  myStruc() : array( new int[ 3000*2000] ) {
  }
 
  // or with size argument in constructor
  myStruct( size_t sz ) : array( new int[ sz ] ) {
  }
 
  ~myStruc() {
     delete [] array;
  }
 
};

struct myStruc
{
 QString name;
 int *array;

  myStruc() : array( new int[ 3000*2000] ) {
  }

  // or with size argument in constructor
  myStruct( size_t sz ) : array( new int[ sz ] ) {
  }

  ~myStruc() {
     delete [] array;
  }

};

To copy to clipboard, switch view to plain text mode 

[wysota]

Actually QVector allocates its data on the heap even if the vector object itself is on the stack.

[lni]

Actually QVector allocates its data on the heap even if the vector object itself is on the stack.

QVector is fine. He has a huge array in the stack, exceeding stack limit.

[Sheng]

QVector is fine. He has a huge array in the stack, exceeding stack limit.

Read my post #19, use heap instead of stack, container crashes.

[lni]

Read my post #19, use heap instead of stack, container crashes.

How do you use heap? Try this:

#include <stdio.h>
#include <unistd.h>
 
struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};
 
 
void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}
 
int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);
 
    myStruc currentStruc;
    foo(currentStruc);
}

#include <stdio.h>
#include <unistd.h>

struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};


void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}

int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);

    myStruc currentStruc;
    foo(currentStruc);
}

To copy to clipboard, switch view to plain text mode 

[Sheng]

How do you use heap? Try this:

#include <stdio.h>
#include <unistd.h>
 
struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};
 
 
void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}
 
int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);
 
    myStruc currentStruc;
    foo(currentStruc);
}

#include <stdio.h>
#include <unistd.h>

struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};


void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}

int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);

    myStruc currentStruc;
    foo(currentStruc);
}

To copy to clipboard, switch view to plain text mode 

It also crashes when you push back structure into vector.
Here is the code:

#include <stdio.h>
#include <unistd.h>
#include <QVector>
 
struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};
 
 
void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}
 
int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);
 
    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

#include <stdio.h>
#include <unistd.h>
#include <QVector>

struct myStruc {
    int *array;
    myStruc() {
      array = new int[ 3000*2000 ];
    }
    ~myStruc() {
      delete [] array;
    }
};


void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}

int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);

    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

To copy to clipboard, switch view to plain text mode 

[lni]

It crashes because you don't have copy constructor. The QVector needs a copy constructor and it uses a default shadow copy constructor. Your "array" pointer is deleted twice when it goes out of scope...

Run the following program and see how many times the constructor is called and how many times the destructor is called.

#include <stdio.h>
#include <unistd.h>
#include <QVector>
#include <QDebug>
 
struct myStruc {
    int *array;
    myStruc() {
      qDebug() << "myStruc";
      array = new int[ 3000*2000 ];
    }
 
    ~myStruc() {
      qDebug() << "~myStruc";
      delete [] array;
    }
};
 
 
void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}
 
int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);
 
    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

#include <stdio.h>
#include <unistd.h>
#include <QVector>
#include <QDebug>

struct myStruc {
    int *array;
    myStruc() {
      qDebug() << "myStruc";
      array = new int[ 3000*2000 ];
    }

    ~myStruc() {
      qDebug() << "~myStruc";
      delete [] array;
    }
};


void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}

int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);

    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

To copy to clipboard, switch view to plain text mode 

[Sheng]

OK, thanks, you are right, both of programs below works fine.

1.

#include <stdio.h>
#include <unistd.h>
#include <QVector>
#include <QDebug>
 
struct myStruc {
    int *array;
    myStruc() {
      qDebug() << "myStruc";
      array = new int[ 3000*2000 ];
    }
 
    myStruc(const myStruc& struc1) {
      qDebug() << "copyStruc";
      array = new int[3000*2000];
      for(int i=0;i<3000*2000;++i)
        array[i]=struc1.array[i];
      }
 
    ~myStruc() {
      qDebug() << "~myStruc";
      delete [] array;
    }
};
 
 
void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}
 
int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);
 
    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

#include <stdio.h>
#include <unistd.h>
#include <QVector>
#include <QDebug>

struct myStruc {
    int *array;
    myStruc() {
      qDebug() << "myStruc";
      array = new int[ 3000*2000 ];
    }

    myStruc(const myStruc& struc1) {
      qDebug() << "copyStruc";
      array = new int[3000*2000];
      for(int i=0;i<3000*2000;++i)
        array[i]=struc1.array[i];
      }

    ~myStruc() {
      qDebug() << "~myStruc";
      delete [] array;
    }
};


void foo(const myStruc &v)
{
    printf("%d\n", v.array[1]);
}

int main(int , char* [])
{
    printf("hello\n");
    fflush(stdout);
    sleep(1);

    myStruc currentStruc;
    foo(currentStruc);
   QVector<myStruc> myVector;
   myVector.push_back(currentStruc);
}

To copy to clipboard, switch view to plain text mode 

2.

int main(int,char* [])
{
 QVector<int> array(3000*2000);
 QVector<QVector<int> > myVector;
 myVector.push_back(array);
}

int main(int,char* [])
{
 QVector<int> array(3000*2000);
 QVector<QVector<int> > myVector;
 myVector.push_back(array);
}

To copy to clipboard, switch view to plain text mode 

[lni]

You 2nd example is a no-op for the myStruct. You define a "myStruct" but it is not used in your main example code. QVector<int> array(3000*2000) is allocated in heap by QVector.

More over, even if you have enough stack size, the code is dangerous because your program could run in other machine which doesn't have enough stack size, it would be a nightmare to debug because it runs in your development machine, and crash in the client machines.

Avoid excessive use of stack memory, certainly not 2000x3000 size of int, which is 6,000,000 int array!

[Sheng]

6*3000*2000*sizeof(int) = 36M*sizeof(int) = 144 (or 288) MB just for creating the array. Aren't you running out of memory? If not, are you sure C++ copies arrays? I doubt that so copying the struct might lose its integrity.

the Container will crash even I only push_back one object.

[drhex]

QVector probably allocates memory for several objects at a time, to avoid having to reallocate at every push_back(). How much RAM is there in your testbox, Sheng?

Does it work if you supply an initial vector size, e.g:

QVector<myStruc> myVector(5);

QVector<myStruc> myVector(5);

To copy to clipboard, switch view to plain text mode 

[Sheng]

QVector probably allocates memory for several objects at a time, to avoid having to reallocate at every push_back(). How much RAM is there in your testbox, Sheng?

at least 1G.
It does not work either with reserve or resize.

[Sheng]

Just wrote and tested my own linked list, it works fine.