|
|
||
|
This article describes a persistency solution for simple projects.
See this video for a short presentation:
For many small projects dependency on a separate database package might be unnecessary and out of proportions as well. If the program can fit all its data in memory (whatever it is on a target machine - which is a wide spectrum), then simple persistency does not require a separate database server.
One possible solution is for the program to store all data modification events in a file, just like in regular log, ordered by time. Not only such a stream of events can be useful as a log for later inspection, but it can be used to reconstruct all data structures at the next program run.
There is no restriction on what kind of modification records can be stored - anything that allows the program to repeat all necessary operations later on will work. For example, the program can store records as text lines describing what has happened with a given data structure:
add apple ... add banana ... modify apple ... remove banana ... ...
The advantage of such simple text format is that it can be inspected by hand or processed with standard text-processing tools. Any other format is of course possible, including binary, and it is up to the program to define the necessary syntax and parameters of such records.
When the program starts, it should read its data file and reconstruct data structures based on these records, in the order they are read from data file.
It is possible that with time the data file will accumulate records that are redundant, for example because they will refer to objects that were created and then deleted, or to data values that were modified multiple times, when only the last used value is needed. The program could detect such cases and compact its data file by recreating it from scratch, based on the current state of its data structures.
The demonstration program is a "phonebook" example, written in C++. A separate class store encapsulates all file operations and has the following outline (shortened):
class store
{
public:
result open(const std::string & file_name, access_mode mode);
result replay();
result append_record(const std::string & cmd, const std::vector<std::string> & words);
// ...
private:
virtual result replay_add(const std::vector<std::string> & words);
virtual result replay_remove(const std::vector<std::string> & words);
virtual result replay_modify(const std::vector<std::string> & words);
// ...
};
The most important functions of this class are:
open() - opens a data file for reading or writingreplay() - reads and parses the data file and detects add, remove and modify commands with their parameters; the parameters can be simple words or arbitrary strings, which are prefixed by length to allow unambiguous parsingappend_record() - allows the program to create new entries in the filereplay_add(), replay_remove(), replay_modify() - virtual functions that can be overriden by the program, they are called for each recognized entry and allow the program to reconstruct its data structures when the stream is replayed
The phonebook program defines its own phonebook_db class that derives from store and that overrides the above virtual functions. As added functionality, it also counts the entries while they are parsed and if the amount of "fluff" reaches some level, the data file is archived and compacted.
The file streamdb-1.0.zip contains the phonebook example in C++ with reusable store class and a simple data file to play with.