package gapi-ocaml

You can search for identifiers within the package.

in-package search v0.2.0

On This Page

General
Allocation and memory-mapping
Interpreting memory as values
I/O using memory as buffers
Buffer pools

package gapi-ocaml

gapi-ocaml
gapi-ocaml.netstring-local
gapi-ocaml.netsys-local
- Netexn
- Netlog
  - Debug
- Netsys
  - Debug
- Netsys_conf
- Netsys_crypto_types
  - DIGESTS
  - FILE_TLS_ENDPOINT
    
    TLS
    
    Exc
  - PUBKEY_CRYPTO
  - SYMMETRIC_CRYPTO
  - TLS_CONFIG
    
    TLS
    
    Exc
  - TLS_ENDPOINT
    
    TLS
    
    Exc
  - TLS_EXCEPTIONS
  - TLS_PROVIDER
    
    Exc
- Netsys_impl_util
- Netsys_mem
- Netsys_oothr
  - condition
  - mtprovider
  - mutex
  - thread
- Netsys_posix
  - Debug
  - post_fork_handler
- Netsys_signal
  - Debug
- Netsys_tls
  - Debug
- Netsys_tmp
- Netsys_types
  - mstring
- Netsys_win32
  - Debug

Legend:
Library
Module
Module type
Parameter
Class
Class type

Bigarrays as memory buffers

type memory = Netsys_types.memory

We consider 1-dimensional bigarrays of chars as memory buffers. They have the useful property that the garbage collector cannot relocate them, i.e. the address is fixed. Also, one can mmap a file, and connect the bigarray with shared memory.

General

val blit_memory_to_bytes : memory -> int -> Bytes.t -> int -> int -> unit

blit_memory_to_bytes src srcoff dst dstoff len copies len characters from buffer src, starting at character number srcoff, to string dst, starting at character number dstoff

Raise Invalid_argument if srcoff and len do not designate a valid subbuffer of src, or if dstoff and len do not designate a valid substring of dst.

val blit_memory_to_string : memory -> int -> Bytes.t -> int -> int -> unit

deprecated
arg

val blit_memory_to_bytes_unsafe : 
  memory ->
  int ->
  Bytes.t ->
  int ->
  int ->
  unit

Unsafe version

val blit_memory_to_string_unsafe : 
  memory ->
  int ->
  Bytes.t ->
  int ->
  int ->
  unit

deprecated
arg

val blit_bytes_to_memory : Bytes.t -> int -> memory -> int -> int -> unit

blit_bytes_to_memory src srcoff dst dstoff len copies len characters from string src, starting at character number srcoff, to buffer dst, starting at character number dstoff

Raise Invalid_argument if srcoff and len do not designate a valid substring of src, or if dstoff and len do not designate a valid subbuffer of dst.

val blit_bytes_to_memory_unsafe : 
  Bytes.t ->
  int ->
  memory ->
  int ->
  int ->
  unit

Unsafe version

val blit_string_to_memory : string -> int -> memory -> int -> int -> unit

blit_string_to_memory src srcoff dst dstoff len: A version for immutable strings

val blit_string_to_memory_unsafe : 
  string ->
  int ->
  memory ->
  int ->
  int ->
  unit

Unsafe version

val memory_of_bytes : Bytes.t -> memory

Return a new bigarray as a copy of the string

val memory_of_string : string -> memory

Return a new bigarray as a copy of the string

val bytes_of_memory : memory -> Bytes.t

Return a new string as a copy of the bigarray

val string_of_memory : memory -> string

Return a new string as a copy of the bigarray

val memory_address : memory -> nativeint

Returns the start address of the buffer

val memory_of_bigarray : ('a, 'b, 'c) Bigarray.Genarray.t -> memory

val memory_of_bigarray_1 : ('a, 'b, 'c) Bigarray.Array1.t -> memory

val memory_of_bigarray_2 : ('a, 'b, 'c) Bigarray.Array2.t -> memory

val memory_of_bigarray_3 : ('a, 'b, 'c) Bigarray.Array3.t -> memory

These functions return an arbitrary bigarray as memory.

Due to a bug in the OCaml runtime, this is for now limited to arrays of up to 2G size (in bytes). (The bug exists at least up to OCaml-3.12.1.)

Allocation and memory-mapping

val getpagesize : unit -> int

Returns the size of a page as reported by sysconf.

On many systems, a page has 4096 bytes, but this cannot be relied upon.

This function is only available if the system has sysconf.

val pagesize : int

The best guess at the page size

val alloc_memory_pages : ?addr:nativeint -> ?exec:bool -> int -> memory

Allocates memory in units of pages. The memory buffer will start on a page boundary.

The passed int is the requested number of bytes. The size of the buffer is rounded up so a whole number of pages is allocated.

Optionally, one can request a certain address addr (which must be a multiple of the page size). There is, however, no guarantee that this wish can be fulfilled. In any way, one should check with memory_address what the start address really is.

If exec, the memory region is marked as executable.

This function is only available if the system has sysconf, mmap, and allows to allocate anonymous memory with mmap (outside POSIX but common).

val alloc_aligned_memory : int -> int -> memory

alloc_aligned_memory alignment size: Allocates a buffer of size whose start address is a multiple of alignment. The alignment must be a power of two, and at least Sys.word_size/8.

Aligned memory can be useful for ensuring that the whole memory block is in the same cache line. A cache line typically has 64 or 128 bytes - but this is very platform-specific. (Linux: look at /proc/cpuinfo.)

This function is only available if the system has posix_memalign.

val memory_map_file : 
  Unix.file_descr ->
  ?pos:int64 ->
  ?addr:nativeint ->
  bool ->
  int ->
  memory

memory_map_file fd shared size: Maps size bytes of the file fd into memory, and returns the memory buffer like Bigarray.Array1.map_file. pos and shared have the same meaning as there. In addr one can suggest a start address. There is, however, no guarantee that this wish can be fulfilled.

val memory_unmap_file : memory -> unit

Unmaps the file. The memory block must have been allocated with memory_map_file or with Bigarray.Array1.map_file.

Note that the data pointer of the bigarray is set to NULL, and that any further access of the array will trigger a segmentation violation! The intention of this function is to control when the file mapping is removed. Normally, this is done first when the GC finalizer is run.

It is required that there are no subarrays at the time of calling this function. (If so, the function does nothing.)

val zero_pages : memory -> int -> int -> unit

zero_pages m pos len: If possible, the memory pages in the range pos to pos+len-1 of m are allocated again, so that they replace the previous pages.

It is required that the start address of the range is a multiple of the page size, and the len is a multiple of the page size. Fails with Invalid_argument if the requirements are not met, or the function is otherwise unavailable.

Calling zero_pages is sometimes an optimization when old memory pages can be dropped, and when the alternative of overwriting these pages would imply a copy-on-write operation.

val grab : nativeint -> int -> memory

grab addr len: Interprets the address range from addr to addr+len-1 as memory bigarray.

This function does not allocate! It assumes that the given address range points to valid memory.

Interpreting memory as values

val as_value : memory -> int -> 'a

as_value mem offset: Returns a pointer to mem+offset. There must be a valid boxed value at this address (i.e. at the word preceding mem+offset there must be a valid block header, followed by a valid value of the right type). However, this is not checked:

This is an unsafe function that may crash the program if used in the wrong way!

It is possible that the memory block is deallocated while the returned value still exists. Any attempt to access the value will result into undefined behavior (anything from funny results to crashes may happen).

Some Ocaml primitives might not work on the returned values (polymorphic equality, marshalling, hashing) unless Netsys_mem.value_area is called for the memory block.

val as_obj : memory -> int -> Obj.t

Same as as_value but returns the value as Obj.t

val obj_address : Obj.t -> nativeint

val hdr_address : Obj.t -> nativeint

These two functions return the address of the Obj.t and the address of the header of the Obj.t, respectively.

Note that this can only be relied upon if the input object cannot be moved around by the garbage collector!

val cmp_bytes : Bytes.t -> Bytes.t -> int

Compares two strings like String.compare. This also works when the strings reside outside the O'Caml heap, e.g. in a memory block.

val cmp_string : string -> string -> int

A version for immutable strings

exception Out_of_space

val init_header : memory -> int -> int -> int -> unit

init_header mem offset tag size: Initializes the word at mem+offset as an Ocaml value header with the given tag and the given size (in words). The GC color is always set to "white".

val init_string : memory -> int -> int -> int * int

let voffset, bytelen = init_string mem offset len: Initializes the memory at offset and following bytes as Ocaml string with length len. Returns in voffset the offset where the value starts (i.e. offset plus one word), and in bytelen the number of bytes used in mem.

offset must be a multiple of the word size in bytes.

The string can be accessed with

let s = (as_value mem voffset : string)

The function is useful for initializing shared memory as string so that several processes can directly access the string.

The string has the GC color White.

Raises Out_of_space if the memory block is too small.

val init_string_bytelen : int -> int

Returns bytelen if init_string was called with the passed len.

val init_array : memory -> int -> int -> int * int

let voffset, bytelen = init_array mem offset size: Initializes the memory at offset and following bytes as Ocaml array with size elements. Returns in voffset the offset where the value starts (i.e. offset plus one word), and in bytelen the number of bytes used in mem.

The array cannot be used as float array.

offset must be a multiple of the word size in bytes.

The array can be accessed with

let a = (as_value mem voffset : _ array)

The elements of the array have a value but it might not be valid for the element type of the array. Because of this, it is unwise to access the elements before setting them for the first time.

The array has the GC color White.

Raises Out_of_space if the memory block is too small.

val init_float_array : memory -> int -> int -> int * int

Same for arrays of floats

val init_array_bytelen : int -> int

Returns bytelen if init_array was called with the passed size.

val init_float_array_bytelen : int -> int

Same for arrays of floats

type custom_ops = nativeint

type init_value_flag =

| Copy_bigarray
| Copy_custom_int
| Copy_atom
| Copy_simulate
| Copy_conditionally
| Keep_atom

type color =

| White
| Gray
| Blue
| Black
(*
GC colors
*)

I/O using `memory` as buffers

val mem_read : Unix.file_descr -> memory -> int -> int -> int

A version of Unix.read that uses a memory buffer. Some OS allow faster I/O when memory is page-aligned (see alloc_memory_pages). Also, a copy in the stub function can be avoided. Both effects can result in a considerable speedup.

val mem_write : Unix.file_descr -> memory -> int -> int -> int

A version of Unix.single_write that uses a memory buffer.

val mem_recv : 
  Unix.file_descr ->
  memory ->
  int ->
  int ->
  Unix.msg_flag list ->
  int

val mem_send : 
  Unix.file_descr ->
  memory ->
  int ->
  int ->
  Unix.msg_flag list ->
  int

Versions of Unix.recv, and Unix.send using memory buffers.

Buffer pools

type memory_pool

A pool of memory blocks that are all the same size and page-aligned (if the OS supports this). The pool tries to bundle memory allocations so that not for every block a system call is required. This reduces the number of system calls, and the number of entries in the process page table. Also, unused blocks are automatically returned to the pool.

val create_pool : int -> memory_pool

Create a new pool. The argument is the size of the memory blocks (must be a multiple of the page size)

val pool_alloc_memory : memory_pool -> memory

let m = pool_alloc_memory p: Gets a memory block m from the pool p. If required, new blocks are automatically allocated and added to the pool. This function is thread-safe.

The memory block is automatically garbage-collected.

val pool_alloc_memory2 : memory_pool -> memory * (unit -> unit)

let m, free = pool_alloc_memory2 p: Gets a memory block m from the pool p like pool_alloc_memory. This function also returns the function free marking the block as free again. The block can then be immediately recycled for another use.

If free is not called, the block m is first recycled when it is not referenced any more (like in pool_alloc_memory).

val pool_reclaim : memory_pool -> unit

Reclaim as much memory as possible

val pool_block_size : memory_pool -> int

Returns the size of the memory blocks in bytes

val default_block_size : int

The default block size, normally 64 K (or better, 16 times the page size)

val default_pool : memory_pool

The default pool with the default block size. This pool is used by Ocamlnet itself as much as possible

val small_block_size : int

The block size of small_pool, normally 4K (or better, the page size)

val small_pool : memory_pool

Another standard pool where the blocks are smaller than in default_pool.

val pool_report : memory_pool -> string

Returns a report describing the memory allocation in the pool

package gapi-ocaml

General

Allocation and memory-mapping

Interpreting memory as values

I/O using memory as buffers

Buffer pools

I/O using `memory` as buffers