package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (41)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.5.tbz
sha256=6b3b638fc7c6f4c52617b0261dc9f726ce21bb0c485e05bab6fe41a57697fc6b
md5=2db36741c413ab93391ecc1f983aa804

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Jan 2019

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Screenshots

The following screenshots demonstrate the HTML testing output from the odoc project.

All tests passed Some tests failed Failed test with custom diffing
ok err diff

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. ocaml >= "4.02.3"
  7. dune >= "1.1.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. alg_structs_qcheck
  6. ambient-context
  7. ambient-context-eio
  8. angstrom >= "0.7.0"
  9. ansi >= "0.6.0"
  10. anycache >= "0.7.4"
  11. anycache-async
  12. anycache-lwt
  13. archetype >= "1.4.2"
  14. archi
  15. arp
  16. arp-mirage
  17. arrakis
  18. art
  19. asak >= "0.2"
  20. asli >= "0.2.0"
  21. asn1-combinators >= "0.2.2"
  22. atd >= "2.3.3"
  23. atdgen >= "2.10.0"
  24. atdpy
  25. atdts
  26. base32
  27. base64 >= "2.1.2"
  28. bechamel >= "0.5.0"
  29. bigarray-overlap
  30. bigstring >= "0.3"
  31. bigstring-unix >= "0.3"
  32. bigstringaf
  33. bitlib
  34. blake2
  35. bloomf
  36. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  37. bls12-381-hash
  38. bls12-381-js >= "0.4.2"
  39. bls12-381-js-gen >= "0.4.2"
  40. bls12-381-legacy
  41. bls12-381-signature
  42. bls12-381-unix
  43. blurhash
  44. builder-web < "0.2.0"
  45. bulletml
  46. bytebuffer
  47. ca-certs
  48. ca-certs-nss
  49. cactus
  50. caldav
  51. calendar >= "3.0.0"
  52. callipyge
  53. camlix
  54. camlkit
  55. camlkit-base
  56. capnp-rpc < "0.6.0"
  57. capnp-rpc-lwt < "0.3"
  58. carray
  59. carton
  60. cborl
  61. ccss >= "1.6"
  62. cf-lwt
  63. chacha
  64. channel
  65. charrua-client
  66. charrua-client-lwt
  67. charrua-client-mirage < "0.11.0"
  68. checkseum >= "0.0.3"
  69. cid
  70. clarity-lang
  71. class_group_vdf
  72. cohttp >= "0.17.0"
  73. cohttp-curl-async
  74. cohttp-curl-lwt
  75. cohttp-eio >= "6.0.0~beta2"
  76. colombe
  77. color
  78. conan
  79. conan-cli
  80. conan-database
  81. conan-lwt
  82. conan-unix
  83. conduit = "3.0.0"
  84. conex < "0.10.0"
  85. conex-mirage-crypto
  86. conex-nocrypto
  87. cookie
  88. cow >= "2.2.0"
  89. css
  90. css-parser
  91. cstruct >= "3.3.0"
  92. cstruct-sexp
  93. ctypes-zarith
  94. cuid
  95. curly
  96. current_incr
  97. cwe_checker
  98. data-encoding < "1.0.0"
  99. datakit >= "0.12.0"
  100. datakit-bridge-github >= "0.12.0"
  101. datakit-ci
  102. datakit-client-git >= "0.12.0"
  103. decompress >= "0.8" & < "1.5.3"
  104. depyt
  105. digestif >= "0.7"
  106. dispatch >= "0.4.1"
  107. dkim
  108. dkim-bin
  109. dkim-mirage
  110. dns >= "4.0.0"
  111. dns-cli
  112. dns-client >= "4.6.0"
  113. dns-forward < "0.9.0"
  114. dns-forward-lwt-unix
  115. dns-resolver
  116. dns-server
  117. dns-tsig
  118. dnssd
  119. dnssec
  120. docfd >= "2.2.0"
  121. dog < "0.2.1"
  122. domain-name
  123. dot-merlin-reader >= "5.3~5.3preview"
  124. dream
  125. dream-pure
  126. duff
  127. dune-release >= "1.0.0"
  128. duration >= "0.1.1"
  129. emile
  130. encore
  131. eqaf >= "0.5"
  132. equinoxe
  133. equinoxe-cohttp
  134. equinoxe-hlc
  135. eris
  136. eris-lwt
  137. ezgzip
  138. ezjsonm >= "0.4.2" & < "1.3.0"
  139. ezjsonm-lwt < "1.3.0"
  140. FPauth
  141. FPauth-core
  142. FPauth-responses
  143. FPauth-strategies
  144. faraday != "0.2.0"
  145. farfadet
  146. fat-filesystem >= "0.12.0"
  147. ff
  148. ff-pbt
  149. fiat-p256
  150. flex-array
  151. fsevents-lwt
  152. functoria >= "2.2.0"
  153. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  154. geojson
  155. geoml >= "0.1.1"
  156. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  157. git-mirage < "3.0.0"
  158. git-split
  159. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  160. git_split
  161. gitlab-unix
  162. glicko2
  163. gmap >= "0.3.0"
  164. gobba
  165. gpt
  166. graphql
  167. graphql-async
  168. graphql-cohttp >= "0.13.0"
  169. graphql-lwt
  170. graphql_parser
  171. graphql_ppx >= "0.7.1"
  172. h1_parser
  173. h2
  174. hacl
  175. hacl_func
  176. hacl_x25519 >= "0.2.0"
  177. highlexer
  178. hkdf
  179. hockmd
  180. html_of_jsx
  181. http
  182. http-multipart-formdata < "2.0.0"
  183. httpaf >= "0.2.0"
  184. httpun
  185. httpun-ws
  186. hvsock
  187. icalendar
  188. imagelib >= "20200929"
  189. index
  190. inferno >= "20220603"
  191. influxdb-async
  192. influxdb-lwt
  193. inquire < "0.2.0"
  194. interval-map
  195. iomux
  196. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  197. irmin-bench >= "2.7.0"
  198. irmin-chunk < "2.0.0" | >= "2.3.0"
  199. irmin-cli
  200. irmin-containers
  201. irmin-fs < "2.0.0" | >= "2.3.0"
  202. irmin-git < "2.0.0" | >= "2.3.0"
  203. irmin-http < "2.0.0"
  204. irmin-mem < "2.0.0"
  205. irmin-pack >= "2.4.0" & != "2.6.1"
  206. irmin-pack-tools
  207. irmin-test < "2.2.0"
  208. irmin-tezos
  209. irmin-tezos-utils
  210. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  211. irmin-watcher
  212. jekyll-format
  213. jerboa
  214. jitsu
  215. jose
  216. json-data-encoding >= "0.9"
  217. json_decoder
  218. jsonxt
  219. junit_alcotest
  220. jwto
  221. kdf
  222. ke >= "0.2"
  223. kkmarkdown
  224. lambda-runtime
  225. lambdapi >= "2.0.0"
  226. lambdoc >= "1.0-beta4"
  227. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  228. lmdb >= "1.0"
  229. logical
  230. logtk >= "1.5.1"
  231. lp
  232. lp-glpk
  233. lp-glpk-js
  234. lp-gurobi
  235. lru
  236. lt-code
  237. luv
  238. mbr-format >= "1.0.0"
  239. mdx >= "1.6.0"
  240. mec
  241. mechaml >= "1.0.0"
  242. merge-queues >= "0.2.0"
  243. merge-ropes >= "0.2.0"
  244. merlin >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  245. merlin-lib >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  246. metrics
  247. minicaml >= "0.3.1"
  248. mirage >= "4.0.0~beta1"
  249. mirage-block-partition < "0.2.0"
  250. mirage-block-ramdisk >= "0.3"
  251. mirage-channel >= "4.0.0"
  252. mirage-channel-lwt
  253. mirage-crypto-ec
  254. mirage-flow >= "1.0.2" & < "1.2.0"
  255. mirage-flow-unix
  256. mirage-fs-mem
  257. mirage-fs-unix >= "1.2.0"
  258. mirage-kv >= "2.0.0"
  259. mirage-kv-mem
  260. mirage-kv-unix
  261. mirage-logs >= "0.3.0"
  262. mirage-nat
  263. mirage-net-unix >= "2.3.0"
  264. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  265. mirage-tc
  266. mjson
  267. mmdb
  268. mnd
  269. monocypher
  270. mrmime
  271. mrt-format
  272. msgpck >= "1.6"
  273. multibase
  274. multihash
  275. multihash-digestif
  276. multipart-form-data
  277. multipart_form
  278. multipart_form-eio
  279. multipart_form-lwt
  280. named-pipe
  281. nanoid
  282. nbd >= "4.0.3"
  283. nbd-tool
  284. nloge
  285. nocoiner
  286. non_empty_list
  287. OCADml >= "0.6.0"
  288. obatcher
  289. ocaml-index >= "1.1"
  290. ocaml-r >= "0.5.0"
  291. ocaml-version >= "3.1.0"
  292. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  293. ocamlformat-rpc < "removed"
  294. ocamline
  295. odoc >= "1.4.0" & < "2.1.0"
  296. ohex
  297. oidc
  298. opam-0install
  299. opam-0install-cudf >= "0.5.0"
  300. opam-file-format >= "2.1.1"
  301. opentelemetry >= "0.6"
  302. opentelemetry-client-cohttp-lwt >= "0.6"
  303. opentelemetry-client-ocurl >= "0.6"
  304. opentelemetry-cohttp-lwt >= "0.6"
  305. opentelemetry-lwt >= "0.6"
  306. opium >= "0.15.0"
  307. opium-graphql
  308. opium-testing
  309. opium_kernel
  310. orewa
  311. ortac-core
  312. osx-acl
  313. osx-attr
  314. osx-cf
  315. osx-fsevents
  316. osx-membership
  317. osx-mount
  318. osx-xattr
  319. otoggl
  320. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  321. owl-base < "0.5.0"
  322. owl-ode >= "0.1.0" & != "0.2.0"
  323. owl-symbolic
  324. passmaker
  325. patch
  326. pbkdf
  327. pecu >= "0.2"
  328. pf-qubes
  329. pg_query >= "0.9.6"
  330. phylogenetics
  331. piaf
  332. polyglot
  333. polynomial
  334. ppx_blob >= "0.3.0"
  335. ppx_deriving_cmdliner
  336. ppx_deriving_rpc
  337. ppx_deriving_yaml
  338. ppx_graphql >= "0.2.0"
  339. ppx_mysql
  340. ppx_protocol_conv >= "5.0.0"
  341. ppx_protocol_conv_json >= "5.0.0"
  342. ppx_protocol_conv_jsonm >= "5.0.0"
  343. ppx_protocol_conv_msgpack >= "5.0.0"
  344. ppx_protocol_conv_xml_light >= "5.0.0"
  345. ppx_protocol_conv_xmlm
  346. ppx_protocol_conv_yaml >= "5.0.0"
  347. ppx_subliner
  348. ppx_units
  349. ppx_yojson >= "1.1.0"
  350. pratter
  351. prc
  352. preface
  353. pretty_expressive
  354. prettym
  355. proc-smaps
  356. producer < "0.2.0"
  357. prom < "0.2"
  358. prometheus < "1.2"
  359. prometheus-app
  360. protocell
  361. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  362. protocol-9p-unix
  363. psq
  364. qcheck >= "0.18" & < "0.22"
  365. qcheck-alcotest < "0.22"
  366. quickjs
  367. radis
  368. randii
  369. reason-standard
  370. red-black-tree
  371. reparse >= "2.0.0" & < "3.0.0"
  372. reparse-unix < "2.1.0"
  373. resp < "0.10.0"
  374. resp-unix
  375. rfc1951 < "1.0.0"
  376. routes < "2.0.0"
  377. rpc >= "5.9.0"
  378. rpclib
  379. rpclib-async
  380. rpclib-lwt
  381. rpmfile < "0.3.0"
  382. rpmfile-eio
  383. rpmfile-unix
  384. rubytt
  385. SZXX >= "4.0.0"
  386. salsa20
  387. salsa20-core
  388. sanddb >= "0.2"
  389. scaml >= "1.5.0"
  390. scrypt-kdf
  391. secp256k1 >= "0.4.1"
  392. secp256k1-internal
  393. semver >= "0.2.1"
  394. sendmail
  395. sendmail-lwt
  396. sendmail-miou-unix
  397. sendmail-mirage
  398. sendmsg
  399. server-reason-react
  400. session-cookie
  401. session-cookie-async
  402. session-cookie-lwt
  403. sherlodoc
  404. slug
  405. sodium-fmt
  406. spin >= "0.6.0"
  407. squirrel
  408. ssh-agent
  409. ssl >= "0.6.0"
  410. stramon-lib
  411. styled-ppx
  412. syslog-rfc5424
  413. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  414. tdigest < "2.1.0"
  415. term-indexing
  416. terminal_size >= "0.1.1"
  417. terminus
  418. terminus-cohttp
  419. terminus-hlc
  420. terml
  421. textrazor
  422. tezos-base-test-helpers < "13.0"
  423. tezos-client-base < "12.0"
  424. tezos-crypto < "8.0"
  425. tezos-lmdb
  426. tezos-signer-backends < "8.0"
  427. tezos-stdlib < "8.0"
  428. tezos-test-helpers < "11.0"
  429. tftp
  430. timedesc
  431. timere
  432. tls >= "0.12.0"
  433. toc
  434. topojson
  435. topojsone
  436. transept
  437. twostep
  438. type_eq
  439. type_id
  440. typebeat
  441. typeid >= "1.0.1"
  442. tyre >= "0.4"
  443. tyxml >= "4.0.0"
  444. tyxml-jsx
  445. tyxml-ppx >= "4.3.0"
  446. tyxml-syntax
  447. uecc
  448. ulid
  449. universal-portal
  450. unix-dirent
  451. unix-errno >= "0.3.0"
  452. unix-fcntl >= "0.3.0"
  453. unix-sys-resource
  454. unix-sys-stat
  455. unix-time
  456. unstrctrd
  457. user-agent-parser
  458. uspf
  459. uspf-lwt
  460. uspf-mirage
  461. uspf-unix
  462. utop >= "2.13.0"
  463. validate
  464. validator
  465. vercel
  466. vpnkit
  467. wcwidth
  468. websocketaf
  469. x509 >= "0.7.0"
  470. xapi-rrd >= "1.8.2"
  471. xapi-stdext-date
  472. xapi-stdext-encodings
  473. xapi-stdext-std >= "4.16.0"
  474. yaml < "3.2.0"
  475. yaml-sexp
  476. yocaml < "2.0.0"
  477. yocaml_syndication = "2.0.0"
  478. yocaml_yaml < "2.0.0"
  479. yojson >= "1.6.0"
  480. yojson-five
  481. yuscii >= "0.2.0"
  482. zar
  483. zed >= "3.2.2"
  484. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct