bistro 0.4.0 · OCaml Package

A library to build scientific workflows.

This module introduces a type 'a worfklow that describe a set of inter-dependent actions that will eventually generate a target (file or directory). The 'a type variable represents the format of the file or the layout of the directory. Actions may be either command lines to be executed, or OCaml expressions to be evaluated.

To build workflows, use the EDSL module, that provide a set of combinators to write shell scripts easily. For instance, the following function shows how to create a gzipped file using the output of another workflow:

let gzip (x : 'a workflow) : 'a gz workflow =
  workflow ~descr:"unix.gzip" [
    cmd "gzip" [ string "-c" ; dep x ; string ">" dest ]
  ]

Note that a workflow is just a recipe to build some result. Building the workflow won't actually generate anything. In order to run the workflow, you have to run it using an execution engine like the one provided by bistro.engine.

module Path : sig ... end

Helper functions to represent paths as string lists. For absolute paths, the first element of the list is "/".

type id = string

type dep = [

| `Task of id
| `Select of id * Path.t
| `Input of Path.t

]

type env =
  < dep : dep -> string
  ; np : int
  ; mem : int
  ; tmp : string
  ; dest : string >

type docker_image = private {

dck_account : string;
dck_name : string;
dck_tag : string option;
dck_registry : string option;

}

Name and version of an external dependency for a workflow

module Command : sig ... end

type u = private

| Input of string * Path.t
| Select of string * u * Path.t
(*
invariant: u cannot be a Select
*)
| Step of step

Workflow untyped representation

and step = private {

id : string;
descr : string;
deps : u list;
action : action;
np : int;
(*
Required number of processors
*)
mem : int;
(*
Required memory in MB
*)
version : int option;
(*
Version number of the wrapper
*)

}

and action =

| Exec of dep Command.t
| Par_exec of {
1. dir : u;
2. cmd : u -> u Command.t;
}
| Eval of {
1. id : string;
2. f : env -> unit;
}

module U : sig ... end

type (+'a, +'b) selector = private

| Selector of Path.t

Describes the (relative) path from a 'a directory workflow target to some 'b workflow target. This is useful to construct new workflows by selecting a file or a subdirectory in the result of a directory workflow.

module Workflow : sig ... end

The type representing a set of actions (shell scripts or evaluations of OCaml expressions) to build a target of type 'a. The type 'a is a phantom type, which can be used to enforce static invariants.

module Template : sig ... end

Represents a text with special symbols

type 'a workflow = 'a Workflow.t

type any_workflow =

| Any_workflow : _ Workflow.t -> any_workflow

class type 'a directory = object ... end

Conventional type to represent directory targets

module EDSL : sig ... end

This module provides combinators to define new workflows that execute shell commands.

module Std : sig ... end

Standard definitions

package bistro