package alcotest
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=6b3b638fc7c6f4c52617b0261dc9f726ce21bb0c485e05bab6fe41a57697fc6b
md5=2db36741c413ab93391ecc1f983aa804
Description
Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.
Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.
Published: 17 Jan 2019
README
Alcotest is a lightweight and colourful test framework.
Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE
module type, a check
function to assert test predicates and a run
function to perform a list of unit -> unit
test callbacks.
Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.
Examples
A simple example:
(* Build with `ocamlbuild -pkg alcotest simple.byte` *)
(* A module with functions to test *)
module To_test = struct
let capit letter = Char.uppercase letter
let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end
(* The tests *)
let capit () =
Alcotest.(check char) "same chars" 'A' (To_test.capit 'a')
let plus () =
Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])
let test_set = [
"Capitalize" , `Quick, capit;
"Add entries", `Slow , plus ;
]
(* Run it *)
let () =
Alcotest.run "My first test" [
"test_set", test_set;
]
The result is a self-contained binary which displays the test results. Use ./simple.byte --help
to see the runtime options.
$ ./simple.native
[OK] test_set 0 Capitalize.
[OK] test_set 1 Add entries.
Test Successful in 0.001s. 2 tests run.
You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):
$ ./simple.native test test_set
Testing My first test.
[OK] test_set 0 Capitalize.
[OK] test_set 1 Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP] test_set 0 Capitalize.
[OK] test_set 1 Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.
Note that you cannot filter by test case name (i.e. Capitalize
or Add entries
), you have to use the test case number instead.
See the examples folder for more examples.
Quick and Slow tests
In general you should use `Quick
tests: tests that are ran on any invocations of the test suite. You should only use `Slow
tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q
flag on the command line, e.g.:
$ ./test.exe -q # run only the quick tests
$ ./test.exe # run quick and slow tests
Passing custom options to the tests
In most cases, the base tests are unit -> unit
functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit
, where 'a
is the type of the extra parameter.
In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a
(note: do not use positional arguments, only optional arguments are supported).
For instance:
let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42
let int =
let doc = "What is your prefered number?" in
Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")
let () =
Alcotest.run_with_args "foo" int [
"all", ["nice", `Quick, test_nice]
]
Will generate test.exe
such that:
$ test.exe test
test.exe: required option -n is missing
$ test.exe test -n 42
Testing foo.
[OK] all 0 int.
Lwt
Alcotest provides an Alcotest_lwt
module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit
, you provide one with the type unit -> unit Lwt.t
and alcotest-lwt calls Lwt_main.run
for you.
However, there are a couple of extra features:
If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).
You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.
For instance:
let free () = print_endline "freeing all resources"; Lwt.return ()
let test_lwt switch () =
Lwt_switch.add_hook (Some switch) free;
Lwt.async (fun () -> failwith "All is broken");
Lwt_unix.sleep 10.
let () =
Alcotest.run "foo" [
"all", [
Alcotest_lwt.test_case "one" `Quick test_lwt
]
]
Will generate:
$ test.exe
Testing foo.
[ERROR] all 0 one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken
Screenshots
The following screenshots demonstrate the HTML testing output from the odoc project.
All tests passed | Some tests failed | Failed test with custom diffing |
---|---|---|
Comparison with other testing frameworks
The README is pretty clear about that:
Alcotest is the only testing framework using colors!
More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:
Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;
Alcotest uses combinators to define pretty-printers and comparators between the things to test.
Other nice tools doing different kind of testing also exist:
qcheck qcheck does random generation and property testing (e.g. Quick Check)
crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.
ppx_inline_tests
allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.
Dependencies (7)
Dev Dependencies
None
- ahrocksdb
-
albatross
>= "1.5.0"
-
alcotest-async
< "1.0.0"
-
alcotest-lwt
< "1.0.0"
- alg_structs_qcheck
- ambient-context
- ambient-context-eio
-
angstrom
>= "0.7.0"
-
ansi
>= "0.6.0"
-
anycache
>= "0.7.4"
- anycache-async
- anycache-lwt
-
archetype
>= "1.4.2"
- archi
- arp
- arp-mirage
- arrakis
- art
-
asak
>= "0.2"
-
asli
>= "0.2.0"
-
asn1-combinators
>= "0.2.2"
-
atd
>= "2.3.3"
-
atdgen
>= "2.10.0"
- atdpy
- atdts
- base32
-
base64
>= "2.1.2"
-
bechamel
>= "0.5.0"
- bigarray-overlap
-
bigstring
>= "0.3"
-
bigstring-unix
>= "0.3"
- bigstringaf
- bitlib
- blake2
- bloomf
-
bls12-381
< "0.4.1" | >= "3.0.0" & < "18.0"
- bls12-381-hash
-
bls12-381-js
>= "0.4.2"
-
bls12-381-js-gen
>= "0.4.2"
- bls12-381-legacy
- bls12-381-signature
- bls12-381-unix
- blurhash
-
builder-web
< "0.2.0"
- bulletml
- bytebuffer
- ca-certs
- ca-certs-nss
- cactus
- caldav
-
calendar
>= "3.0.0"
- callipyge
- camlix
- camlkit
- camlkit-base
-
capnp-rpc
< "0.6.0"
-
capnp-rpc-lwt
< "0.3"
- carray
- carton
- cborl
-
ccss
>= "1.6"
- cf-lwt
- chacha
- channel
- charrua-client
- charrua-client-lwt
-
charrua-client-mirage
< "0.11.0"
-
checkseum
>= "0.0.3"
- cid
- clarity-lang
- class_group_vdf
-
cohttp
>= "0.17.0"
- cohttp-curl-async
- cohttp-curl-lwt
-
cohttp-eio
>= "6.0.0~beta2"
- colombe
- color
- conan
- conan-cli
- conan-database
- conan-lwt
- conan-unix
-
conduit
= "3.0.0"
-
conex
< "0.10.0"
- conex-mirage-crypto
- conex-nocrypto
- cookie
-
cow
>= "2.2.0"
- css
- css-parser
-
cstruct
>= "3.3.0"
- cstruct-sexp
- ctypes-zarith
- cuid
- curly
- current_incr
- cwe_checker
-
data-encoding
< "1.0.0"
-
datakit
>= "0.12.0"
-
datakit-bridge-github
>= "0.12.0"
- datakit-ci
-
datakit-client-git
>= "0.12.0"
-
decompress
>= "0.8" & < "1.5.3"
- depyt
-
digestif
>= "0.7"
-
dispatch
>= "0.4.1"
- dkim
- dkim-bin
- dkim-mirage
-
dns
>= "4.0.0"
- dns-cli
-
dns-client
>= "4.6.0"
-
dns-forward
< "0.9.0"
- dns-forward-lwt-unix
- dns-resolver
- dns-server
- dns-tsig
- dnssd
- dnssec
-
docfd
>= "2.2.0"
-
dog
< "0.2.1"
- domain-name
-
dot-merlin-reader
>= "5.3~5.3preview"
- dream
- dream-pure
- duff
-
dune-release
>= "1.0.0"
-
duration
>= "0.1.1"
- emile
- encore
-
eqaf
>= "0.5"
- equinoxe
- equinoxe-cohttp
- equinoxe-hlc
- eris
- eris-lwt
- ezgzip
-
ezjsonm
>= "0.4.2" & < "1.3.0"
-
ezjsonm-lwt
< "1.3.0"
- FPauth
- FPauth-core
- FPauth-responses
- FPauth-strategies
-
faraday
!= "0.2.0"
- farfadet
-
fat-filesystem
>= "0.12.0"
- ff
- ff-pbt
- fiat-p256
- flex-array
- fsevents-lwt
-
functoria
>= "2.2.0"
-
functoria-runtime
>= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
- geojson
-
geoml
>= "0.1.1"
-
git
= "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
-
git-mirage
< "3.0.0"
- git-split
-
git-unix
>= "1.10.0" & != "2.1.0" & < "3.0.0"
- git_split
- gitlab-unix
- glicko2
-
gmap
>= "0.3.0"
- gobba
- gpt
- graphql
- graphql-async
-
graphql-cohttp
>= "0.13.0"
- graphql-lwt
- graphql_parser
-
graphql_ppx
>= "0.7.1"
- h1_parser
- h2
- hacl
- hacl_func
-
hacl_x25519
>= "0.2.0"
- highlexer
- hkdf
- hockmd
- html_of_jsx
- http
-
http-multipart-formdata
< "2.0.0"
-
httpaf
>= "0.2.0"
- httpun
- httpun-ws
- hvsock
- icalendar
-
imagelib
>= "20200929"
- index
-
inferno
>= "20220603"
- influxdb-async
- influxdb-lwt
-
inquire
< "0.2.0"
- interval-map
- iomux
-
irmin
< "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
-
irmin-bench
>= "2.7.0"
-
irmin-chunk
< "2.0.0" | >= "2.3.0"
- irmin-cli
- irmin-containers
-
irmin-fs
< "2.0.0" | >= "2.3.0"
-
irmin-git
< "2.0.0" | >= "2.3.0"
-
irmin-http
< "2.0.0"
-
irmin-mem
< "2.0.0"
-
irmin-pack
>= "2.4.0" & != "2.6.1"
- irmin-pack-tools
-
irmin-test
< "2.2.0"
- irmin-tezos
- irmin-tezos-utils
-
irmin-unix
>= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
- irmin-watcher
- jekyll-format
- jerboa
- jitsu
- jose
-
json-data-encoding
>= "0.9"
- json_decoder
- jsonxt
- junit_alcotest
- jwto
- kdf
-
ke
>= "0.2"
- kkmarkdown
- lambda-runtime
-
lambdapi
>= "2.0.0"
-
lambdoc
>= "1.0-beta4"
-
ledgerwallet-tezos
>= "0.2.1" & < "0.4.0"
-
lmdb
>= "1.0"
- logical
-
logtk
>= "1.5.1"
- lp
- lp-glpk
- lp-glpk-js
- lp-gurobi
- lru
- lt-code
- luv
-
mbr-format
>= "1.0.0"
-
mdx
>= "1.6.0"
- mec
-
mechaml
>= "1.0.0"
-
merge-queues
>= "0.2.0"
-
merge-ropes
>= "0.2.0"
-
merlin
>= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
-
merlin-lib
>= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
- metrics
-
minicaml
>= "0.3.1"
-
mirage
>= "4.0.0~beta1"
-
mirage-block-partition
< "0.2.0"
-
mirage-block-ramdisk
>= "0.3"
-
mirage-channel
>= "4.0.0"
- mirage-channel-lwt
- mirage-crypto-ec
-
mirage-flow
>= "1.0.2" & < "1.2.0"
- mirage-flow-unix
- mirage-fs-mem
-
mirage-fs-unix
>= "1.2.0"
-
mirage-kv
>= "2.0.0"
- mirage-kv-mem
- mirage-kv-unix
-
mirage-logs
>= "0.3.0"
- mirage-nat
-
mirage-net-unix
>= "2.3.0"
-
mirage-runtime
>= "4.0.0~beta1" & < "4.5.0"
- mirage-tc
- mjson
- mmdb
- mnd
- monocypher
- mrmime
- mrt-format
-
msgpck
>= "1.6"
- multibase
- multihash
- multihash-digestif
- multipart-form-data
- multipart_form
- multipart_form-eio
- multipart_form-lwt
- named-pipe
- nanoid
-
nbd
>= "4.0.3"
- nbd-tool
- nloge
- nocoiner
- non_empty_list
-
OCADml
>= "0.6.0"
- obatcher
-
ocaml-index
>= "1.1"
-
ocaml-r
>= "0.5.0"
-
ocaml-version
>= "3.1.0"
-
ocamlformat
>= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
-
ocamlformat-rpc
< "removed"
- ocamline
-
odoc
>= "1.4.0" & < "2.1.0"
- ohex
- oidc
- opam-0install
-
opam-0install-cudf
>= "0.5.0"
-
opam-file-format
>= "2.1.1"
-
opentelemetry
>= "0.6"
-
opentelemetry-client-cohttp-lwt
>= "0.6"
-
opentelemetry-client-ocurl
>= "0.6"
-
opentelemetry-cohttp-lwt
>= "0.6"
-
opentelemetry-lwt
>= "0.6"
-
opium
>= "0.15.0"
- opium-graphql
- opium-testing
- opium_kernel
- orewa
- ortac-core
- osx-acl
- osx-attr
- osx-cf
- osx-fsevents
- osx-membership
- osx-mount
- osx-xattr
- otoggl
-
owl
>= "0.6.0" & != "0.9.0" & != "1.0.0"
-
owl-base
< "0.5.0"
-
owl-ode
>= "0.1.0" & != "0.2.0"
- owl-symbolic
- passmaker
- patch
- pbkdf
-
pecu
>= "0.2"
- pf-qubes
-
pg_query
>= "0.9.6"
- phylogenetics
- piaf
- polyglot
- polynomial
-
ppx_blob
>= "0.3.0"
- ppx_deriving_cmdliner
- ppx_deriving_rpc
- ppx_deriving_yaml
-
ppx_graphql
>= "0.2.0"
- ppx_mysql
-
ppx_protocol_conv
>= "5.0.0"
-
ppx_protocol_conv_json
>= "5.0.0"
-
ppx_protocol_conv_jsonm
>= "5.0.0"
-
ppx_protocol_conv_msgpack
>= "5.0.0"
-
ppx_protocol_conv_xml_light
>= "5.0.0"
- ppx_protocol_conv_xmlm
-
ppx_protocol_conv_yaml
>= "5.0.0"
- ppx_subliner
- ppx_units
-
ppx_yojson
>= "1.1.0"
- pratter
- prc
- preface
- pretty_expressive
- prettym
- proc-smaps
-
producer
< "0.2.0"
-
prom
< "0.2"
-
prometheus
< "1.2"
- prometheus-app
- protocell
-
protocol-9p
>= "0.3" & < "0.11.0" | >= "0.11.2"
- protocol-9p-unix
- psq
-
qcheck
>= "0.18" & < "0.22"
-
qcheck-alcotest
< "0.22"
- quickjs
- radis
- randii
- reason-standard
- red-black-tree
-
reparse
>= "2.0.0" & < "3.0.0"
-
reparse-unix
< "2.1.0"
-
resp
< "0.10.0"
- resp-unix
-
rfc1951
< "1.0.0"
-
routes
< "2.0.0"
-
rpc
>= "5.9.0"
- rpclib
- rpclib-async
- rpclib-lwt
-
rpmfile
< "0.3.0"
- rpmfile-eio
- rpmfile-unix
- rubytt
-
SZXX
>= "4.0.0"
- salsa20
- salsa20-core
-
sanddb
>= "0.2"
-
scaml
>= "1.5.0"
- scrypt-kdf
-
secp256k1
>= "0.4.1"
- secp256k1-internal
-
semver
>= "0.2.1"
- sendmail
- sendmail-lwt
- sendmail-miou-unix
- sendmail-mirage
- sendmsg
- server-reason-react
- session-cookie
- session-cookie-async
- session-cookie-lwt
- sherlodoc
- slug
- sodium-fmt
-
spin
>= "0.6.0"
- squirrel
- ssh-agent
-
ssl
>= "0.6.0"
- stramon-lib
- styled-ppx
- syslog-rfc5424
-
tcpip
>= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
-
tdigest
< "2.1.0"
- term-indexing
-
terminal_size
>= "0.1.1"
- terminus
- terminus-cohttp
- terminus-hlc
- terml
- textrazor
-
tezos-base-test-helpers
< "13.0"
-
tezos-client-base
< "12.0"
-
tezos-crypto
< "8.0"
- tezos-lmdb
-
tezos-signer-backends
< "8.0"
-
tezos-stdlib
< "8.0"
-
tezos-test-helpers
< "11.0"
- tftp
- timedesc
- timere
-
tls
>= "0.12.0"
- toc
- topojson
- topojsone
- transept
- twostep
- type_eq
- type_id
- typebeat
-
typeid
>= "1.0.1"
-
tyre
>= "0.4"
-
tyxml
>= "4.0.0"
- tyxml-jsx
-
tyxml-ppx
>= "4.3.0"
- tyxml-syntax
- uecc
- ulid
- universal-portal
- unix-dirent
-
unix-errno
>= "0.3.0"
-
unix-fcntl
>= "0.3.0"
- unix-sys-resource
- unix-sys-stat
- unix-time
- unstrctrd
- user-agent-parser
- uspf
- uspf-lwt
- uspf-mirage
- uspf-unix
-
utop
>= "2.13.0"
- validate
- validator
- vercel
- vpnkit
- wcwidth
- websocketaf
-
x509
>= "0.7.0"
-
xapi-rrd
>= "1.8.2"
- xapi-stdext-date
- xapi-stdext-encodings
-
xapi-stdext-std
>= "4.16.0"
-
yaml
< "3.2.0"
- yaml-sexp
-
yocaml
< "2.0.0"
-
yocaml_syndication
= "2.0.0"
-
yocaml_yaml
< "2.0.0"
-
yojson
>= "1.6.0"
- yojson-five
-
yuscii
>= "0.2.0"
- zar
-
zed
>= "3.2.2"
-
zlist
< "0.4.0"
Conflicts
None