Resource files management IV

Previous posts:
Resource files management. Part I
Resource files management. Part II
Resource files management. Part III

Okay, so far we've been writing helper classes and now we're getting to the main stuff: ResourceFile class.
ResourceFile is pretty simple, let's take a look over its interface:

class ResourceFile
{
protected:
    ResFileHeader mFileHeader;       // Resource file header structure
    std::vector<Lump*> mLumpList;    // List holding every lump
                                     // of the resource file

    std::string mFileName;           // Local storage of the
                                     // resource file's name
    OpenMode mFileMode;              // Current file access mode
    std::fstream mFileStream;        // Primary stream for reading data
    std::fstream mSaveStream;        // Helper stream for saving data

    bool LoadLumpList();             // Loads each lump (info) from file
    bool SaveLumpList();             // Saves each lump (info) to file
    void UnloadLumpList();           // Clears mLumpList
    bool SaveLumps();                // Saves each lump (data) to file

public:
    ResourceFile() {}
    ResourceFile(const std::string& FileName, OpenMode FileMode)
        { this->Open(FileName, FileMode); }
    ~ResourceFile();

    // Open existing resource file and read data or create new file
    bool Open (const std::string& FileName, OpenMode FileMode);

    // Save data to resource file
    bool Save (const std::string& AltFileName = std::string());

    // Close resource file and clear data from memory
    void Close();

    // Add new lump to resource file from external file
    bool CreateLumpFromFile (const std::string& Name,
                             const std::string& FileName);

    // Add new lump to resource file using data from memory
    bool CreateLump (const std::string& Name, void* Data = NULL);

    // Delete selected lump from resource file
    bool DeleteLump (const std::string& Name);

    // Load data for selected lump
    bool LoadLump   (const std::string& Name);

    // Unload data for selected lump
    bool UnloadLump (const std::string& Name);

    // Accessor functions
    std::vector<Lump*>* GetLumpList();
    Lump* GetLump(const std::string& Name);
};


That's it - simple resource file class. Now you can take it and use in your projects, though you may want to tweak it more - add compression, encryption, internal directory hierarchies and so on, there is much to do more.
But now I'll just stop on this simpliest implementation of resource file. There is several things I would like to mention.
First of all as you may notice resource file reads or writes lumps in two separate passes. Let's look at opening resource file logic:
At the beginning ResourceFile checks if we want to open existing file or create a new file. If we are creating new file, then it just opens a new file for writing, but if we want to open an existing file, then the ResourceFile class opens that file and starts reading useful information from it.
First it reads mFileHeader, from which it gets information about the file, more specifically how much lumps are stored in file and where is the lump information list. Having this information the ResourceFile class starts creating N lumps according to information stored in ResFileHeader and reads information of each lump from lump information list in file.
After performing these operations we are having something like this, assuming there are two lumps in a file:

ResourceFile -
    mLumpList[0] -
        mLumpInfo -
            Size     = 215
            Position = 25
            Name     = "Grass Texture"
        mData = NULL
    mLumpList[1] -
        mLumpInfo -
            Size     = 198
            Position = 240
            Name     = "Sand Texture"
        mData = NULL


As you can see, after having this information we can dynamically load the data for any lump we want in our lump list, we just specify the name of the texture, ResourceFile finds it in a file according to the stored information and loads its data to the memory. This is how LoadLump works.

Now the second thing is about saving files. The saving procedure is implemented this way:

1. Write mFileHeader
2. Write all lumps data
3. Get current position in file and update mFileHeader.LumpListOffset
4. Write all lumps information
5. Return to the beginning of the file and rewrite mFileHeader


Here you can see that we need to write resource file header twice. Well that is because I'm pretty lazy now for writing better methods. The reason we need to write file header twice is because ResourceFile doesn't knows where is the lump info list is located in file before it actually writes it. Therefore when we get to the position where the lump info list will be written, we update the mFileHeader with this information and proceed. After all information is written we get back to the beginning of the file and rewrite the header. Of course this can be battled if you write updating of LumpListOffset each time you are adding or removing lump to mLumpList, i.e. add lump's size to the LumpListOffset if you're adding a lump or subtract the size if you're removing it. Also, if you're creating a new file you'll need to set the initial size of LumpListOffset to be equal to size of the ResFileHeader, because the LumpListOffset is equal to ResFileHeader size + sum of each lump size. This is not needed when you're reading existing file as initial value of LumpListOffset is loaded from file.

Now I must admit that the previous saving sequence is not real. The full sequence looks like:

1. Create temporary file
2. Write mFileHeader
3. Write all lumps data:

    for each lump
    if lump's data loaded in memory
    3.1 Write data from memory to file
    else
    3.2 Copy data from existing file to temporary file

4. Get current position in file and update mFileHeader.LumpListOffset
5. Write all lumps information
6. Return to the beginning of the file and rewrite mFileHeader
7. Close both files
8. Remove original file if we're overwriting it or remove alternative-name file (Save As)
9. Rename the temporary file to original or alternate name


As you can see, there is much going at the background. Though, this is not the best solution (imagine doubling 8 GB resource file), but it is easier to implement.

Phew, that was long one.
What to do next? You can go further and create GUI editor of resource files. Well, my friend made one already, it is Qt based.

Source included:
ResourceFile.h
ResourceFile.cpp

Please, feel free to contact me if you're having problems compiling this source.

0 comments: (+add yours?)

Post a Comment