Legend:
Library
Module
Module type
Parameter
Class
Class type
Abstraction that governs the lifetime of the various files that are part of a pack store (except the branch store).
The file manager (FM) handles the files one by one and makes explicit all the interactions between them (except the index which is handled from a high level API). This allows to gain confidence on SWMR consistency and crash consistency.
Two types of guarantees
Irmin_pack_unix supports the SWMR access scheme. It means that it is undefined for the files to be opened twice in RW mode by 2 FMs. It also means that it is undefined for a FM in RW mode to be used simultaneously from 2 different fibers.
Irmin_pack_unix aims to be (crash) consistent (in the ACID sense). In case of a system failure (e.g. power outage), the files should be left in a consistent state for later uses.
Ensuring SWMR consistency is easier than ensuring crash consistency because the of the OS's shared page cache.
Files mutations
Here are all the moments where the files managed may be mutated:
1. During create_rw.
2. During open_rw if a major version upgrade is necessary.
3. During the flush routines in file_manager.
4. During a GC, from the worker.
5. At the end of a GC, from the RW fiber, in swap.
6. During integrity check routines.
1. 2. and 6. don't support SWMR and leave the store in an undefined state in case of crash.
4. operates on files private to the worker. It doesn't necessitate to worry about crash concistency and SWMR.
Create a rw instance of t by opening existing files.
If the pack store has already been garbage collected, opening with a non-minimal indexing strategy will return an error.
If no_migrate = false in the config, the store will undergo a major version upgrade if necessary.
Note on SWMR consistency: It is undefined for a reader to attempt an opening during an open_rw, because of major version upgrades.
Note on crash consistency: If open_rw crashes during a major version upgrade, the storage is left in an undefined state. Otherwise the storage is unaffected.
Note on errors: If open_rw returns an error during a major version upgrade, the storage is left in an undefined state. Otherwise the storage is unaffected. Anyhow, some file descriptors might not be closed.
Create a ro instance of t by opening existing files.
Note on SWMR consistency: open_ro is supposed to work whichever the state of the pack store and the writer, with 2 exceptions: 1. the files must exist and create_rw must be over and 2. during a major version upgrade of the files (which occurs during a open_rw).
Note on crash consistency: Crashes during open_ro cause no issues because it doesn't mutate the storage.
Note on errors: The storage is never mutated. Some file descriptors might not be closed.
This call fails if the append buffers are not in a flushed stated. This situation will most likely never occur because the append buffers will contain data only during the scope of a batch function.
fsync executes an fsync for all files of the file manager.
Note: This function exists primarily for operations like snapshot imports. If fsync is enabled for the store (see Irmin_pack.Config.use_fsync), calls to flush will also call fsync and therefore there is little need to call this function directly.
Swaps to using files from the GC generation. The values suffix_start_offset, chunk_start_idx, chunk_num, and suffix_dead_bytes are used to properly load and read the suffix after a GC. The value volume is used to reload the lower if it was modified by the GC. The control file is also updated on disk.
create_one_commit_store t conf generation new_store_root key is called when creating a new store at new_store_root from the existing one, containing only one commit, specified by the key. Ths new store will use configuration options from conf and set to generation.