package rpc_parallel

  1. Overview
  2. Docs
package rpc_parallel
Legend:
Library
Module
Module type
Parameter
Class
Class type

A type-safe parallel library built on top of Async_rpc.

module Worker = Rpc_parallel.Make (T : Worker_spec)

The Worker module can be used to spawn new workers, either locally or remotely, and run functions on these workers. T specifies which functions can be run on a Worker.t as well as the implementations for these functions. In addition, T specifies worker states and connection states. See README for more details

module Fd_redirection : sig ... end
module How_to_run : sig ... end
module Map_reduce : sig ... end

A parallel map/reduce library. See examples/add_numbers.ml and examples/number_stats.ml for examples.

module Prog_and_args : sig ... end
module Remote_executable : sig ... end
module Rpc_settings : sig ... end
module Utils : sig ... end
module Managed : sig ... end
include module type of struct include Parallel end
module Function = Parallel.Function
module Backend_and_settings = Parallel.Backend_and_settings
module type Backend = Parallel.Backend
module type Worker = Parallel.Worker
module type Functions = Parallel.Functions
module type Creator = Parallel.Creator
module type Worker_spec = Parallel.Worker_spec
module Make = Parallel.Make

module Worker = Make(T)

val start_app : ?rpc_max_message_size:int -> ?rpc_buffer_age_limit:Async.Writer.buffer_age_limit -> ?rpc_handshake_timeout:Core.Time_float.Span.t -> ?rpc_heartbeat_config:Async.Rpc.Connection.Heartbeat_config.t -> ?when_parsing_succeeds:(unit -> unit) -> ?complete_subcommands: (path:string list -> part:string -> string list list -> string list option) -> ?add_validate_parsing_flag:bool -> Backend_and_settings.t -> Async.Command.t -> unit

start_app command should be called from the top-level in order to start the parallel application. This function will parse certain environment variables and determine whether to start as a master or a worker.

rpc_max_message_size, rpc_handshake_timeout, rpc_heartbeat_config can be used to alter the rpc defaults. These rpc settings will be used for all connections. This can be useful if you have long async jobs.

when_parsing_succeeds and complete_subcommands will be passed to Command.run in the master process.

module State = Parallel.State

Use State.get to query whether the current process has been initialized as an rpc parallel master (start_app or init_master_exn has been called). We return a State.t rather than a bool so that you can require evidence at the type level. If you want to certify, as a precondition, for some function that start_app was used, require a State.t as an argument. If you don't need the State.t anymore, just pattern match on it.

module For_testing = Parallel.For_testing
module Expert = Parallel.Expert

If you want more direct control over your executable, you can use the Expert module instead of start_app. If you use Expert, you are responsible for starting the master and worker rpc servers. worker_command_args will be the arguments sent to each spawned worker. Running your executable with these args must follow a code path that calls worker_init_before_async_exn and then start_worker_server_exn. An easy way to do this is to use worker_command.

module Parallel : sig ... end
OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct