# package octez-libs

`TEZOS_CONTEXT`

is the module type implemented by all storage implementations. This is the module type that the *shell* expects for its operation. As such, it should be a strict superset of the interface exposed to the protocol (see module type `S`

above and `Tezos_protocol_environment.Environment_context_intf.S`

).

The main purpose of this module type is to keep the on-disk and in-memory implementations in sync.

### Generic interface

`include VIEW with type key = string list and type value = bytes`

### Getters

`mem t k`

is an Lwt promise that resolves to `true`

iff `k`

is bound to a value in `t`

.

`find t k`

is an Lwt promise that resolves to `Some v`

if `k`

is bound to the value `v`

in `t`

and `None`

otherwise.

`list t key`

is the list of files and sub-nodes stored under `k`

in `t`

. The result order is not specified but is stable.

`offset`

and `length`

are used for pagination.

`length t key`

is an Lwt promise that resolves to the number of files and sub-nodes stored under `k`

in `t`

.

It is equivalent to `let+ l = list t k in List.length l`

but has a constant-time complexity.

### Setters

`add t k v`

is an Lwt promise that resolves to `c`

such that:

`k`

is bound to`v`

in`c`

;- and
`c`

is similar to`t`

otherwise.

If `k`

was already bound in `t`

to a value that is physically equal to `v`

, the result of the function is a promise that resolves to `t`

. Otherwise, the previous binding of `k`

in `t`

disappears.

`remove t k v`

is an Lwt promise that resolves to `c`

such that:

`k`

is unbound in`c`

;- and
`c`

is similar to`t`

otherwise.

### Folding

```
val fold :
?depth:depth ->
t ->
key ->
order:[ `Sorted | `Undefined ] ->
init:'a ->
f:(key -> tree -> 'a -> 'a Lwt.t) ->
'a Lwt.t
```

`fold ?depth t root ~order ~init ~f`

recursively folds over the trees and values of `t`

. The `f`

callbacks are called with a key relative to `root`

. `f`

is never called with an empty key for values; i.e., folding over a value is a no-op.

The depth is 0-indexed. If `depth`

is set (by default it is not), then `f`

is only called when the conditions described by the parameter is true:

`Eq d`

folds over nodes and values of depth exactly`d`

.`Lt d`

folds over nodes and values of depth strictly less than`d`

.`Le d`

folds over nodes and values of depth less than or equal to`d`

.`Gt d`

folds over nodes and values of depth strictly more than`d`

.`Ge d`

folds over nodes and values of depth more than or equal to`d`

.

If `order`

is ``Sorted`

(the default), the elements are traversed in lexicographic order of their keys. For large nodes, it is memory-consuming, use ``Undefined`

for a more memory efficient `fold`

.

### Configuration

The type of references to tree objects annotated with the type of that object (either a value or a node). Used to build a shallow tree with `Tree.shallow`

`module Tree : sig ... end`

```
type ('proof, 'result) producer :=
index ->
kinded_key ->
(tree -> (tree * 'result) Lwt.t) ->
('proof * 'result) Lwt.t
```

`produce r h f`

runs `f`

on top of a real store `r`

, producing a proof and a result using the initial root hash `h`

.

The trees produced during `f`

's computation will carry the full history of reads. This history will be reset when `f`

is complete so subtrees escaping the scope of `f`

will not cause memory leaks.

Calling `produce_proof`

recursively has an undefined behaviour.

```
type ('proof, 'result) verifier :=
'proof ->
(tree -> (tree * 'result) Lwt.t) ->
(tree * 'result,
[ `Proof_mismatch of string
| `Stream_too_long of string
| `Stream_too_short of string ])
result
Lwt.t
```

`verify p f`

runs `f`

in checking mode. `f`

is a function that takes a tree as input and returns a new version of the tree and a result. `p`

is a proof, that is a minimal representation of the tree that contains what `f`

should be expecting.

Therefore, contrary to trees found in a storage, the contents of the trees passed to `f`

may not be available. For this reason, looking up a value at some `path`

can now produce three distinct outcomes:

- A value
`v`

is present in the proof`p`

and returned :`find tree path`

is a promise returning`Some v`

; `path`

is known to have no value in`tree`

:`find tree path`

is a promise returning`None`

; and`path`

is known to have a value in`tree`

but`p`

does not provide it because`f`

should not need it:`verify`

returns an error classifying`path`

as an invalid path (see below).

The same semantics apply to all operations on the tree `t`

passed to `f`

and on all operations on the trees built from `f`

.

The generated tree is the tree after `f`

has completed. That tree is disconnected from any storage (i.e. `index`

). It is possible to run operations on it as long as they don't require loading shallowed subtrees.

The result is `Error (`Msg _)`

if the proof is rejected:

- For tree proofs: when
`p.before`

is different from the hash of`p.state`

; - For tree and stream proofs: when
`p.after`

is different from the hash of`f p.state`

; - For tree proofs: when
`f p.state`

tries to access invalid paths in`p.state`

; - For stream proofs: when the proof is not consumed in the exact same order it was produced;
- For stream proofs: when the proof is too short or not empty once
`f`

is done.

`type tree_proof := Proof.tree Proof.t`

The type for tree proofs.

Guarantee that the given computation performs exactly the same state operations as the generating computation, *in some order*.

`val produce_tree_proof : (tree_proof, 'a) producer`

`produce_tree_proof`

is the producer of tree proofs.

`val verify_tree_proof : (tree_proof, 'a) verifier`

`verify_tree_proof`

is the verifier of tree proofs.

`type stream_proof := Proof.stream Proof.t`

The type for stream proofs.

Guarantee that the given computation performs exactly the same state operations as the generating computation, in the exact same order.

`val produce_stream_proof : (stream_proof, 'a) producer`

`produce_stream_proof`

is the producer of stream proofs.

`val verify_stream_proof : (stream_proof, 'a) verifier`

`verify_stream`

is the verifier of stream proofs.

`type context = t`

`memory_context_tree`

is a forward declaration of the type of an in-memory Irmin tree. This type variable is to be substituted by a concrete type wherever the `TEZOS_CONTEXT`

signature is used.

```
val init :
?patch_context:(context -> context Tezos_base.TzPervasives.tzresult Lwt.t) ->
?readonly:bool ->
?index_log_size:int ->
string ->
index Lwt.t
```

Open or initialize a versioned store at a given path.

The indexing_strategy, which determines whether newly-exported objects by this store handle should also be added to the store's index, is set to ``Minimal`

by default.

```
val compute_testchain_chain_id :
Tezos_base.TzPervasives.Block_hash.t ->
Tezos_base.TzPervasives.Chain_id.t
```

```
val compute_testchain_genesis :
Tezos_base.TzPervasives.Block_hash.t ->
Tezos_base.TzPervasives.Block_hash.t
```

Build an empty context from an index. The resulting context should not be committed.

`val is_empty : t -> bool`

Returns `true`

if the context is empty.

```
val commit_genesis :
index ->
chain_id:Tezos_base.TzPervasives.Chain_id.t ->
time:Tezos_base.TzPervasives.Time.Protocol.t ->
protocol:Tezos_base.TzPervasives.Protocol_hash.t ->
Tezos_base.TzPervasives.Context_hash.t Tezos_base.TzPervasives.tzresult Lwt.t
```

```
val commit_test_chain_genesis :
context ->
Tezos_base.TzPervasives.Block_header.t ->
Tezos_base.TzPervasives.Block_header.t Lwt.t
```

`val to_memory_tree : t -> string list -> memory_context_tree option Lwt.t`

Extract a subtree from the `Tezos_context.Context.t`

argument and returns it as a `Tezos_context_memory.Context.tree`

(note the the type change!). *

```
val merkle_tree :
t ->
Proof_types.merkle_leaf_kind ->
key ->
Proof_types.merkle_tree Lwt.t
```

`merkle_tree t leaf_kind key`

returns a Merkle proof for `key`

(i.e. whose hashes reach `key`

). If `leaf_kind`

is `Block_services.Hole`

, the value at `key`

is a hash. If `leaf_kind`

is `Block_services.Raw_context`

, the value at `key`

is a `Block_services.raw_context`

. Values higher in the returned tree are hashes of the siblings on the path to reach `key`

.

```
val merkle_tree_v2 :
t ->
Proof_types.merkle_leaf_kind ->
key ->
Proof.tree Proof.t Lwt.t
```

`merkle_tree_v2 t leaf_kind key`

returns an Irmin Merkle proof for `key`

(i.e. a proof that *something* is in the context at `key`

). The proof is supposed to be produced by Irmin's `produce_proof`

, and consumed by Irmin's `verify_proof`

. The value embedded in the proof depends on `leaf_kind`

. If `leaf_kind`

is `Block_services.Raw_context`

, the embeded value is the complete subtree in the context at `key`

. If `leaf_kind`

is `Block_services.Hole`

, the embedded value is the hash of the subtree described above.

### Accessing and Updating Versions

`val exists : index -> Tezos_base.TzPervasives.Context_hash.t -> bool Lwt.t`

```
val checkout :
index ->
Tezos_base.TzPervasives.Context_hash.t ->
context option Lwt.t
```

```
val checkout_exn :
index ->
Tezos_base.TzPervasives.Context_hash.t ->
context Lwt.t
```

```
val hash :
time:Tezos_base.TzPervasives.Time.Protocol.t ->
?message:string ->
t ->
Tezos_base.TzPervasives.Context_hash.t
```

```
val commit :
time:Tezos_base.TzPervasives.Time.Protocol.t ->
?message:string ->
context ->
Tezos_base.TzPervasives.Context_hash.t Lwt.t
```

`val gc : index -> Tezos_base.TzPervasives.Context_hash.t -> unit Lwt.t`

`gc index commit_hash`

removes from disk all the data older than the `commit_hash`

. Operations needing to checkout commits greater or equal to `commit_hash`

will continue to work. Calling `checkout h'`

on GC-ed commits will return `None`

.

From the irmin point of view, a successful gc call on a `commit_hash`

will result in a new prefix file containing that `commit_hash`

as a root commit. This prefix file is considered as standalone as all the data referenced by that commit is contained in that file.

`wait_gc_completion index`

will return a blocking thread if an Irmin GC is currently ongoing.

**Warning**: this currently only applies to GC started in the Irmin instance referenced as `index`

; other opened instances will always return instantly.

Sync the context with disk. Only useful for read-only instances. Does not fail when the context is not in read-only mode.

`val is_gc_allowed : index -> bool`

`is_gc_allowed index`

returns whether or not it is possible to run a GC on the given context tree. If false is returned, it means that the context was run, at least once, with the indexing strategy mode "always", which is not suitable for GC.

`split index`

creates a new suffix file, also called "chunk", into the irmin's file hierarchy.

To be optimal, the split function is expected to be called directly after committing, to the context, a commit (of hash `context_hash`

) that will be a future candidate of a GC target. Thus, the commit `commit_hash`

is the last commit stored on a given chunk. The GC called on that `commit_hash`

will be able to extract that `commit_hash`

into a new prefix file, and then, drop the whole chunk.

If the last commit of a chunk appears not to be the candidate of a future GC, it may result in keeping chunks containing partially needed data. This is not an issue, but it should be avoided to prevent storing unnecessary data and thus, to minimize the disk footprint.

```
val export_snapshot :
index ->
Tezos_base.TzPervasives.Context_hash.t ->
path:string ->
unit Lwt.t
```

`export_snapshot index context_hash ~path`

exports the context corresponding to `context_hash`

, if found in `index`

, into the given folder path. As the export uses the GC's behaviour to extract a single commit into a standalone fresh store, it is not possible to export a snapshot while a GC is running. This call will hang until the GC has finished.

Note: there is no associated `import_snapshot`

function as the import consist in copying the exported store.

```
val set_head :
index ->
Tezos_crypto.Hashed.Chain_id.t ->
Tezos_base.TzPervasives.Context_hash.t ->
unit Lwt.t
```

`val set_master : index -> Tezos_base.TzPervasives.Context_hash.t -> unit Lwt.t`

### Hash version

```
val get_hash_version :
context ->
Tezos_base.TzPervasives.Context_hash.Version.t
```

Get the hash version used for the context

```
val set_hash_version :
context ->
Tezos_base.TzPervasives.Context_hash.Version.t ->
context Tezos_base.TzPervasives.tzresult Lwt.t
```

Set the hash version used for the context. It may recalculate the hashes of the whole context, which can be a long process. Returns an `Error`

if the hash version is unsupported.

### Predefined Fields

`val get_protocol : context -> Tezos_base.TzPervasives.Protocol_hash.t Lwt.t`

```
val add_protocol :
context ->
Tezos_base.TzPervasives.Protocol_hash.t ->
context Lwt.t
```

```
val get_test_chain :
context ->
Tezos_base.TzPervasives.Test_chain_status.t Lwt.t
```

```
val add_test_chain :
context ->
Tezos_base.TzPervasives.Test_chain_status.t ->
context Lwt.t
```

```
val fork_test_chain :
context ->
protocol:Tezos_base.TzPervasives.Protocol_hash.t ->
expiration:Tezos_base.TzPervasives.Time.Protocol.t ->
context Lwt.t
```

```
val clear_test_chain :
index ->
Tezos_base.TzPervasives.Chain_id.t ->
unit Lwt.t
```

```
val find_predecessor_block_metadata_hash :
context ->
Tezos_base.TzPervasives.Block_metadata_hash.t option Lwt.t
```

```
val add_predecessor_block_metadata_hash :
context ->
Tezos_base.TzPervasives.Block_metadata_hash.t ->
context Lwt.t
```

```
val find_predecessor_ops_metadata_hash :
context ->
Tezos_base.TzPervasives.Operation_metadata_list_list_hash.t option Lwt.t
```

```
val add_predecessor_ops_metadata_hash :
context ->
Tezos_base.TzPervasives.Operation_metadata_list_list_hash.t ->
context Lwt.t
```