package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. ambient-context-lwt
  8. angstrom >= "0.7.0"
  9. ansi >= "0.6.0"
  10. anycache >= "0.7.4"
  11. anycache-async
  12. anycache-lwt
  13. archetype >= "1.4.2"
  14. archi
  15. arp
  16. arp-mirage
  17. arrakis
  18. art
  19. asak >= "0.2"
  20. asli >= "0.2.0"
  21. asn1-combinators >= "0.2.2"
  22. atd >= "2.3.3"
  23. atdgen >= "2.10.0"
  24. atdpy
  25. atdts
  26. base32
  27. base64 >= "2.1.2"
  28. bechamel >= "0.5.0"
  29. bigarray-overlap
  30. bigstring >= "0.3"
  31. bigstring-unix >= "0.3"
  32. bigstringaf
  33. bitlib
  34. blake2
  35. bloomf
  36. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  37. bls12-381-hash
  38. bls12-381-js >= "0.4.2"
  39. bls12-381-js-gen >= "0.4.2"
  40. bls12-381-legacy
  41. bls12-381-signature
  42. bls12-381-unix
  43. blurhash
  44. builder-web
  45. bulletml
  46. bytebuffer
  47. ca-certs
  48. ca-certs-nss
  49. cactus
  50. calendar >= "3.0.0"
  51. callipyge
  52. camlix
  53. capnp-rpc < "0.6.0"
  54. capnp-rpc-lwt < "0.3"
  55. carray
  56. carton
  57. cborl
  58. ccss >= "1.6"
  59. cf-lwt
  60. chacha
  61. channel
  62. charrua-client
  63. charrua-client-lwt
  64. charrua-client-mirage < "0.11.0"
  65. checkseum >= "0.0.3"
  66. cid
  67. clarity-lang
  68. class_group_vdf
  69. cohttp >= "0.17.0"
  70. cohttp-curl-async
  71. cohttp-curl-lwt
  72. cohttp-eio >= "6.0.0~beta2"
  73. colombe
  74. color
  75. conan
  76. conan-cli
  77. conan-database
  78. conan-lwt
  79. conan-unix
  80. conduit = "3.0.0"
  81. conex < "0.10.0"
  82. conex-mirage-crypto
  83. conex-nocrypto
  84. cookie
  85. cow >= "2.2.0"
  86. css
  87. css-parser
  88. cstruct >= "3.3.0"
  89. cstruct-sexp
  90. ctypes-zarith
  91. cuid
  92. curly
  93. current_incr
  94. cwe_checker
  95. data-encoding < "1.0.0"
  96. datakit >= "0.12.0"
  97. datakit-bridge-github >= "0.12.0"
  98. datakit-ci
  99. datakit-client-git >= "0.12.0"
  100. decompress >= "0.8" & < "1.5.3"
  101. depyt
  102. digestif >= "0.7"
  103. dispatch >= "0.4.1"
  104. dkim
  105. dkim-bin
  106. dkim-mirage
  107. dns >= "4.0.0"
  108. dns-cli
  109. dns-client >= "4.6.0"
  110. dns-forward < "0.9.0"
  111. dns-forward-lwt-unix
  112. dns-resolver
  113. dns-server
  114. dns-tsig
  115. dnssd
  116. dnssec
  117. docfd >= "2.2.0"
  118. dog < "0.2.1"
  119. domain-name
  120. dream
  121. dream-pure
  122. duff
  123. dune-release >= "1.0.0"
  124. duration >= "0.1.1"
  125. emile
  126. encore
  127. eqaf >= "0.5"
  128. equinoxe
  129. equinoxe-cohttp
  130. equinoxe-hlc
  131. eris
  132. eris-lwt
  133. ezgzip
  134. ezjsonm >= "0.4.2" & < "1.3.0"
  135. ezjsonm-lwt < "1.3.0"
  136. FPauth
  137. FPauth-core
  138. FPauth-responses
  139. FPauth-strategies
  140. faraday != "0.2.0"
  141. farfadet
  142. fat-filesystem >= "0.12.0"
  143. ff
  144. ff-pbt
  145. fiat-p256
  146. flex-array
  147. fsevents-lwt
  148. functoria >= "2.2.0"
  149. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  150. geojson
  151. geoml >= "0.1.1"
  152. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  153. git-mirage < "3.0.0"
  154. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  155. gitlab-unix
  156. glicko2
  157. gmap >= "0.3.0"
  158. gpt
  159. graphql
  160. graphql-async
  161. graphql-cohttp >= "0.13.0"
  162. graphql-lwt
  163. graphql_parser
  164. graphql_ppx >= "0.7.1"
  165. h1_parser
  166. h2
  167. hacl
  168. hacl_func
  169. hacl_x25519 >= "0.2.0"
  170. highlexer
  171. hkdf
  172. hockmd
  173. html_of_jsx
  174. http
  175. http-multipart-formdata < "2.0.0"
  176. httpaf >= "0.2.0"
  177. hvsock
  178. icalendar
  179. imagelib >= "20200929"
  180. index
  181. inferno >= "20220603"
  182. influxdb-async
  183. influxdb-lwt
  184. inquire < "0.2.0"
  185. interval-map
  186. iomux
  187. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  188. irmin-bench >= "2.7.0"
  189. irmin-chunk < "2.0.0" | >= "2.3.0"
  190. irmin-cli
  191. irmin-containers
  192. irmin-fs < "2.0.0" | >= "2.3.0"
  193. irmin-git < "2.0.0" | >= "2.3.0"
  194. irmin-http < "2.0.0"
  195. irmin-mem < "2.0.0"
  196. irmin-pack >= "2.4.0" & != "2.6.1"
  197. irmin-pack-tools
  198. irmin-test < "2.2.0"
  199. irmin-tezos
  200. irmin-tezos-utils
  201. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  202. irmin-watcher
  203. jekyll-format
  204. jerboa
  205. jitsu
  206. jose
  207. json-data-encoding >= "0.9"
  208. json_decoder
  209. jsonxt
  210. junit_alcotest
  211. jwto
  212. ke >= "0.2"
  213. kkmarkdown
  214. lambda-runtime
  215. lambdapi >= "2.0.0"
  216. lambdoc >= "1.0-beta4"
  217. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  218. lmdb >= "1.0"
  219. logical
  220. logtk >= "1.5.1"
  221. lp
  222. lp-glpk
  223. lp-glpk-js
  224. lp-gurobi
  225. lru
  226. lt-code
  227. luv
  228. mbr-format >= "1.0.0"
  229. mdx >= "1.6.0"
  230. mec
  231. mechaml >= "1.0.0"
  232. merge-queues >= "0.2.0"
  233. merge-ropes >= "0.2.0"
  234. metrics
  235. mirage >= "4.0.0~beta1"
  236. mirage-block-partition < "0.2.0"
  237. mirage-block-ramdisk >= "0.3"
  238. mirage-channel >= "4.0.0"
  239. mirage-channel-lwt
  240. mirage-crypto-ec
  241. mirage-flow >= "1.0.2" & < "1.2.0"
  242. mirage-flow-unix
  243. mirage-fs-mem
  244. mirage-fs-unix >= "1.2.0"
  245. mirage-kv >= "2.0.0"
  246. mirage-kv-mem
  247. mirage-kv-unix
  248. mirage-logs >= "0.3.0"
  249. mirage-nat
  250. mirage-net-unix >= "2.3.0"
  251. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  252. mirage-tc
  253. mjson
  254. mnd
  255. monocypher
  256. mrmime
  257. mrt-format
  258. multibase
  259. multihash
  260. multihash-digestif
  261. multipart-form-data
  262. multipart_form
  263. multipart_form-eio
  264. multipart_form-lwt
  265. named-pipe
  266. nanoid
  267. nbd >= "4.0.3"
  268. nbd-tool
  269. nloge
  270. nocoiner
  271. non_empty_list
  272. OCADml >= "0.6.0"
  273. ocaml-r >= "0.5.0"
  274. ocaml-version >= "3.1.0"
  275. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  276. ocamlformat-rpc < "removed"
  277. ocamline
  278. odoc >= "1.4.0" & < "2.1.0"
  279. ohex
  280. oidc
  281. opam-0install
  282. opam-file-format >= "2.1.1"
  283. opentelemetry >= "0.6"
  284. opentelemetry-client-cohttp-lwt >= "0.6"
  285. opentelemetry-client-ocurl >= "0.6"
  286. opentelemetry-cohttp-lwt >= "0.6"
  287. opentelemetry-lwt >= "0.6"
  288. opium >= "0.15.0"
  289. opium-graphql
  290. opium-testing
  291. opium_kernel
  292. orewa
  293. ortac-core
  294. osx-acl
  295. osx-attr
  296. osx-cf
  297. osx-fsevents
  298. osx-membership
  299. osx-mount
  300. osx-xattr
  301. otoggl
  302. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  303. owl-base < "0.5.0"
  304. owl-ode >= "0.1.0" & != "0.2.0"
  305. owl-symbolic
  306. passmaker
  307. patch
  308. pbkdf
  309. pecu >= "0.2"
  310. pf-qubes
  311. pg_query >= "0.9.6"
  312. phylogenetics
  313. piaf
  314. polyglot
  315. polynomial
  316. ppx_blob >= "0.3.0"
  317. ppx_deriving_cmdliner
  318. ppx_deriving_rpc
  319. ppx_deriving_yaml
  320. ppx_graphql >= "0.2.0"
  321. ppx_mysql
  322. ppx_parser
  323. ppx_protocol_conv >= "5.0.0"
  324. ppx_protocol_conv_json >= "5.0.0"
  325. ppx_protocol_conv_jsonm >= "5.0.0"
  326. ppx_protocol_conv_msgpack >= "5.0.0"
  327. ppx_protocol_conv_xml_light >= "5.0.0"
  328. ppx_protocol_conv_xmlm
  329. ppx_protocol_conv_yaml >= "5.0.0"
  330. ppx_subliner
  331. ppx_units
  332. ppx_yojson >= "1.1.0"
  333. pratter
  334. prc
  335. preface
  336. pretty_expressive
  337. prettym
  338. proc-smaps
  339. producer < "0.2.0"
  340. prometheus < "1.2"
  341. prometheus-app
  342. protocell
  343. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  344. protocol-9p-unix
  345. psq
  346. qcheck >= "0.18"
  347. qcheck-alcotest
  348. qcheck-core >= "0.18"
  349. quickjs
  350. radis
  351. randii
  352. reason-standard
  353. reparse >= "2.0.0" & < "3.0.0"
  354. reparse-unix < "2.1.0"
  355. resp < "0.10.0"
  356. resp-unix
  357. rfc1951 < "1.0.0"
  358. routes < "2.0.0"
  359. rpc >= "5.9.0"
  360. rpclib
  361. rpclib-async
  362. rpclib-lwt
  363. rubytt
  364. SZXX >= "4.0.0"
  365. salsa20
  366. salsa20-core
  367. sanddb >= "0.2"
  368. scaml >= "1.5.0"
  369. scrypt-kdf
  370. secp256k1 >= "0.4.1"
  371. secp256k1-internal
  372. semver >= "0.2.1"
  373. sendmail
  374. sendmail-lwt
  375. sendmsg
  376. server-reason-react
  377. session-cookie
  378. session-cookie-async
  379. session-cookie-lwt
  380. sherlodoc
  381. slug
  382. sodium-fmt
  383. spin >= "0.6.0"
  384. squirrel
  385. ssh-agent
  386. ssl >= "0.6.0"
  387. stramon-lib
  388. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "8.0.0"
  389. tdigest < "2.1.0"
  390. terminal_size >= "0.1.1"
  391. terminus
  392. terminus-cohttp
  393. terminus-hlc
  394. terml
  395. textrazor
  396. tezos-base-test-helpers < "13.0"
  397. tezos-client-base < "12.0"
  398. tezos-lmdb
  399. tezos-test-helpers < "11.0"
  400. tftp
  401. timedesc
  402. timere
  403. tls >= "0.12.0"
  404. toc
  405. topojson
  406. topojsone
  407. transept
  408. twostep
  409. type_eq
  410. type_id
  411. typebeat
  412. typeid >= "1.0.1"
  413. tyre >= "0.4"
  414. tyxml >= "4.0.0"
  415. tyxml-jsx
  416. tyxml-ppx >= "4.3.0"
  417. tyxml-syntax
  418. uecc
  419. ulid
  420. universal-portal
  421. unix-dirent
  422. unix-errno >= "0.3.0"
  423. unix-fcntl >= "0.3.0"
  424. unix-sys-resource
  425. unix-sys-stat
  426. unix-time
  427. unstrctrd
  428. user-agent-parser
  429. uspf
  430. uspf-lwt
  431. uspf-unix
  432. utop >= "2.13.0"
  433. validate
  434. validator
  435. vercel
  436. vpnkit
  437. websocketaf
  438. x509 >= "0.7.0"
  439. xapi-rrd >= "1.8.2"
  440. xapi-stdext-date
  441. xapi-stdext-encodings
  442. xapi-stdext-std >= "4.16.0"
  443. yaml < "3.2.0"
  444. yaml-sexp
  445. yocaml
  446. yocaml_yaml
  447. yuscii >= "0.2.0"
  448. zar
  449. zed >= "3.2.2"
  450. zlist < "0.4.0"

Conflicts

None