Storing lots of data.

[Granum]

Hey everyone,

I'm currently designing a software, which is trying to organise your life.

You are able to create rich text notes, voice memos, sticky notes, to-do lists etc. One can write his/hers whole lecture notes as a text note so I expect lots of data.

Now to store each note in memory seems like a bad idea from performance point of view, so I thought it would be better if I could store the each data in a file, and every time I want to display it to the user, I could fetch it using its path, and read it on the spot. And if any changes are made, I overwrite the given file.

Now the issue is, I'm not entirely sure how to store that data. I'm thinking about making files that have two sections, "#SETTINGS ... TITLE;some_title; DATE_CREATED;some_date... #END_OF_SETTINGS /n #DATA ... text ... #END_OF_DATA" sort of thing. For the voice memo the data would be empty, but in #SETTINGS section would be a path to .mp3 file which can be loaded to the software.

I also thought about storing all this data in XML format, but then I'm stuck thinking there are so many ways to go about it, and I can't figure out the best one. So I'm wondering if anyone of you guys could suggest a good way of structuring this sort of file storing.

Of course a path builder class will be implemented that will maintain a coherent file structure with directories and sub-directories.

Any input is appreciated,
Thank you!

[high_flyer]

I am not trying to get the wind out of your sales, but this is quite a feat you took on your self.
This is quite a big project that touches on quite a wide range of problems.
From years of experience here is some advice:
Don't worry (yet) about such details as the internal structure of your data files.
This is implementation detail, and can be differed at this point in time.
Take one basic feature and first design it.
When I say design I mean think about what type of work it should do, which input it needs, which output, what type of work it should do.
I would suggest you start with the back end part - all the code that is doing work, not the UI.
Break this work in to responsibility areas, and encapsulated them in classes and methods - without implementing them yet, simply write the interfaces.
Then, start implementing and testing.
Once you finished that little part, take the next.
You will see how with each step of the way much of what you think today about how your application should look like will change.
Feel free to ask here any specific questions about problems you may encounter.

[jefftee]

I personally would prefer to store all of the data in a database rather than have individual files for each item. The primary reason would be the ability to search for content. If you store all of the content in individual files, you'll either have to build/maintain an index to provide reasonable search capabilities or open each item and search contents dynamically when the user wants to find a piece of data.