Sizeof wrong with structures ?????

[tonnot]

struct AA {
    float a;
    float b;
    unsigned char metadata;
};

struct AA {
    float a;
    float b;
    unsigned char metadata;
};

To copy to clipboard, switch view to plain text mode 

The size of this are 4+4+1 , isn't it ?

AA *aa = new AA;
sizeof *aa = 12 ?????

struct AA {
    float a;
    float b;
    float c;
    float d;
    float e;
    short int c1;
    short int c2;  
};

struct AA {
    float a;
    float b;
    float c;
    float d;
    float e;
    short int c1;
    short int c2;  
};

To copy to clipboard, switch view to plain text mode 

5*4 +4+4 = 28
sizeof *aa = 24 ?????

And I could show more errors .
What is happen ?

I have read something related to pragma option
Y

our compiler may have a "#pragma option" feature that you can use to
wrap your structure definition, to enforce the byte alignment you want.
You have to refer to your documentation for details.

How to do this witn Qt + mingw ?


Thanks

[stampede]

sizeof is ok, it's about aligning the data structure members for efficient access, read this article for more details: Data structure alignment
Your second example is ok (both are), sizeof(short) = 2.
To encourage you for further reading, consider this example:

struct AA {
    float a;
    unsigned char metadata;
    float a2;
    unsigned char metadata2;
};
 
 
struct AA2 {
    float a;
    float a2;
    unsigned char metadata2;
    unsigned char metadata;
};
 
int main(){
std::cout << "AA is " << sizeof(AA) << " , AA2 is " << sizeof(AA2) << "\n";
return 0;
}

struct AA {
    float a;
    unsigned char metadata;
    float a2;
    unsigned char metadata2;
};


struct AA2 {
    float a;
    float a2;
    unsigned char metadata2;
    unsigned char metadata;
};

int main(){
std::cout << "AA is " << sizeof(AA) << " , AA2 is " << sizeof(AA2) << "\n";
return 0;
}

To copy to clipboard, switch view to plain text mode 

You may think that output will be:

AA is 10, AA2 is 10

Why not, both structures have the same members, only the order is different, it should not matter (and it gives 10 when you simply add the sizes). But the order is important, here is output on my machine:

AA is 16, AA2 is 12

What happened ? Read the linked article, the answer is there.

[wysota]

To add to that, since the referred article doesn't mention it, for GCC there is a "packed" attribute you can set on a data structure.

e.g.

struct foo
{
    char a;
    int x[2] __attribute__ ((packed));
};

struct foo
{
    char a;
    int x[2] __attribute__ ((packed));
};

To copy to clipboard, switch view to plain text mode 

[tonnot]

I've read the article. Thanks stampede.
This is the most important

Padding is only inserted when a structure member is followed by a member with a larger alignment requirement or at the end of the structure

....

One use for such "packed" structures is to conserve memory. For example, a structure containing a single byte and a four-byte integer would require three additional bytes of padding. A large array of such structures would use 37.5% less memory if they are packed, although accessing each structure might take longer. This compromise may be considered a form of space-time tradeoff.

I have a new question. My data structures are used to store CAD information, in example hundreds of thousands of 3D points (float x,float y, float, charmetadata, int other, short other1,etc) .
What is the best ?
1.- Review my structures to order them to find the small size. (even so I'm going to have more memory occupied but....)
2.- Using a pack ? This are going to do my program slower ?

Thanks

---

Added after 6 minutes:

Can you give me a last help ...
I have this two functions to save - load data to a stream
the_stream is a stringstream I create so:

char mybuffer [265536];
  the_stream.rdbuf()->pubsetbuf(mybuffer,265536);

char mybuffer [265536];
  the_stream.rdbuf()->pubsetbuf(mybuffer,265536);

To copy to clipboard, switch view to plain text mode 

template <class TT_stream>
void W_stream::put (TT_stream &value) {
                 const char * buffer ;
                 buffer = reinterpret_cast<const char*> (&value);
                 the_stream.write(buffer,sizeof value);
        }
 
template <class TTget_stream>
void W_stream::get(TTget_stream & value) {
    char * buffer ;
    buffer = reinterpret_cast< char*> (&value);
    the_stream.read(buffer, sizeof value);  
}

template <class TT_stream>
void W_stream::put (TT_stream &value) {
                 const char * buffer ;
                 buffer = reinterpret_cast<const char*> (&value);
                 the_stream.write(buffer,sizeof value);
        }

template <class TTget_stream>
void W_stream::get(TTget_stream & value) {
    char * buffer ;
    buffer = reinterpret_cast< char*> (&value);
    the_stream.read(buffer, sizeof value);  
}

To copy to clipboard, switch view to plain text mode 

What is your opinion, are going to run ok ? (thinking at read -store structures )
Thanks

[DanH]

sizeof is defined (often literally) to be the stride of an array containing elements of the specified type.

[stampede]

Using a pack ? This are going to do my program slower ?

It may slow down your program, but it depends on compiler and optimization options during compilation, I think if you pass -O2 or -O3, the end performance will be pretty much the same for packed or regular structures.

What is your opinion, are going to run ok ?

Depends on how you use it. It can work ok for saving one structure to single stream and read it later. If you want to create files that are meant to be saved on one machine and loaded on different platform, you can be surprised that sizeof(your struct) returns different values, and the data saved on one machine is not loaded correctly on another. This depends on the type of data kept in structures, for example, long int can be 32 bit or 64 bit on different platforms (reference: GNU C Manual: Primitive Types).
Moreover, if you want to store structures of different types in one stringstream for later use, I think it could be good to define some kind of "header" for the file (I mean a structure with informations about what is saved in the file). But I don't know how this is used so it's hard to say more.

[wysota]

I think the speed might depend on the architecture as well. Some processors may not support reading non-aligned data directly in which case it might require a couple additional instructions.

As for the other thing, I'm personally against dumping structures directly to storage because if you read it back on a machine with different endianness than the one the data was stored on, you'll get garbage.

[tonnot]

Thanks.
But the problem of types and endianness are going to be the same if I want to load - save a simple variable , isn't it ?

If you want to load or save a long int to a binary file what would you do ? (Suposing my progr are going to be used on different platforms)

And I dont find references to compile options for Mingw. Can you tell me a usefull help-doc ?

Thanks. Really thank you very much.

[wysota]

But the problem of types and endianness are going to be the same if I want to load - save a simple variable , isn't it ?

It depends on the variable. But in general dumping the variable without any control to disk is not the best thing to do. You should decide whether you store data as big or little endian and perform the conversion before storing and after restoring the data.

If you want to load or save a long int to a binary file what would you do ? (Suposing my progr are going to be used on different platforms)

I would store it in a well defined format -- for example as a 32 bit unsigned big endian natural binary encoded value.

And I dont find references to compile options for Mingw. Can you tell me a usefull help-doc ?

MinGW is GCC. Most of what applies to GCC, applies to MinGW as well.

[squidge]

If I was storing lots of small amounts of data in a file (such as thousands of positions), then I would state the endian format in the file format specification and ensure that format regardless of platform. I would also never use sizeof() as I have been bitten before by that returning different values on different compilers/platforms. I ensure that each object can serialise/deserialise itself from any storage (either disk or memory)

If the data format lent itself to it, or I thought I could be extending the format in the future, then I wouldn't use a binary format at all. The nightmare in supporting lots of different versions isn't worth the compactness of the file and processing speed advantages. So I'd use something like XML (possibly compressed if they turn out large and I knew I had plenty of ram to work with)

[tonnot]

I have found information about the theme.
Intel has little endian always ?
It depends on the operating system ? In that case Windows has one and mac, unix, linux the other?
( I think I know the answer, " it depends ....".)

So, the solution are to set endianness or use types that are going to use always the same endianness.
I can't find usefull information about 'Can I and how set endianness for my prog" (using c++ -Qt + mingw), or 'using safe endianness types '
I have found this link, but I dont understand it .... http://www.realtime.bc.ca/articles/endian-safe

Any help doc about it would be appreciated
Thanks

[squidge]

Its the processor which specifies endianness as its the processor which ends up storing the values (upto 64-bits) in memory.

However, you shouldn't need to worry about this, you should be storing variables in a known endian regardless of processor your application is working on.

Eg. instead of:

int a;
 
write(&a, sizeof(a));

int a;

write(&a, sizeof(a));

To copy to clipboard, switch view to plain text mode 

you would use something like:

typedef unsigned int ui32;
typedef unsigned char uc8;
 
ui32 a;
ui8 c;
 
c = a >> 24; write (&c, 1);
c = a >> 16; write (&c, 1);
c = a >> 8; write (&c, 1);
c = a;
write (&a, 1);

typedef unsigned int ui32;
typedef unsigned char uc8;

ui32 a;
ui8 c;

c = a >> 24; write (&c, 1);
c = a >> 16; write (&c, 1);
c = a >> 8; write (&c, 1);
c = a;
write (&a, 1);

To copy to clipboard, switch view to plain text mode 

then you know you've just written out as MSB (big endian) and guarantee that it will always be MSB regardless. It also guarantees that you'll also always write out 32 bits, regardless if the host OS has a 16-bit int (of course there maybe other consequences of this if you actually use all 32-bits)