package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (41)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
sha256=8643b39e74317e2ec49be9a0986e73acd4fcd1277a6d59fb1423c3a6ebdfaf61
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. angstrom >= "0.7.0"
  8. ansi >= "0.6.0"
  9. anycache >= "0.7.4"
  10. anycache-async
  11. anycache-lwt
  12. archetype >= "1.4.2"
  13. archi
  14. arp
  15. arp-mirage
  16. arrakis
  17. art
  18. asak >= "0.2"
  19. asli >= "0.2.0"
  20. asn1-combinators >= "0.2.2"
  21. atd >= "2.3.3"
  22. atdgen >= "2.10.0"
  23. atdpy
  24. atdts
  25. base32
  26. base64 >= "2.1.2"
  27. bechamel >= "0.5.0"
  28. bigarray-overlap
  29. bigstring >= "0.3"
  30. bigstring-unix >= "0.3"
  31. bigstringaf
  32. bitlib
  33. blake2
  34. bloomf
  35. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  36. bls12-381-hash
  37. bls12-381-js >= "0.4.2"
  38. bls12-381-js-gen >= "0.4.2"
  39. bls12-381-legacy
  40. bls12-381-signature
  41. bls12-381-unix
  42. blurhash
  43. builder-web < "0.2.0"
  44. bulletml
  45. bytebuffer
  46. ca-certs
  47. ca-certs-nss
  48. cactus
  49. calendar >= "3.0.0"
  50. callipyge
  51. camlix
  52. camlkit
  53. camlkit-base
  54. capnp-rpc < "0.6.0"
  55. capnp-rpc-lwt < "0.3"
  56. carray
  57. carton
  58. cborl
  59. ccss >= "1.6"
  60. cf-lwt
  61. chacha
  62. channel
  63. charrua-client
  64. charrua-client-lwt
  65. charrua-client-mirage < "0.11.0"
  66. checked_oint < "0.1.1"
  67. checkseum >= "0.0.3"
  68. cid
  69. clarity-lang
  70. class_group_vdf
  71. cohttp >= "0.17.0"
  72. cohttp-curl-async
  73. cohttp-curl-lwt
  74. cohttp-eio >= "6.0.0~beta2"
  75. colombe
  76. color
  77. conan
  78. conan-cli
  79. conan-database
  80. conan-lwt
  81. conan-unix
  82. conduit = "3.0.0"
  83. conex < "0.10.0"
  84. conex-mirage-crypto
  85. conex-nocrypto
  86. cookie
  87. cow >= "2.2.0"
  88. css
  89. css-parser
  90. cstruct >= "3.3.0"
  91. cstruct-sexp
  92. ctypes-zarith
  93. cuid
  94. curly
  95. current_incr
  96. cwe_checker
  97. data-encoding < "1.0.0"
  98. datakit >= "0.12.0"
  99. datakit-bridge-github >= "0.12.0"
  100. datakit-ci
  101. datakit-client-git >= "0.12.0"
  102. decompress >= "0.8" & < "1.5.3"
  103. depyt
  104. digestif >= "0.7"
  105. dispatch >= "0.4.1"
  106. dkim
  107. dkim-bin
  108. dkim-mirage
  109. dns >= "4.0.0"
  110. dns-cli
  111. dns-client >= "4.6.0"
  112. dns-forward < "0.9.0"
  113. dns-forward-lwt-unix
  114. dns-resolver
  115. dns-server
  116. dns-tsig
  117. dnssd
  118. dnssec
  119. docfd >= "2.2.0"
  120. dog < "0.2.1"
  121. domain-name
  122. dream
  123. dream-pure
  124. duff
  125. dune-release >= "1.0.0"
  126. duration >= "0.1.1"
  127. emile
  128. encore
  129. eqaf >= "0.5"
  130. equinoxe
  131. equinoxe-cohttp
  132. equinoxe-hlc
  133. eris
  134. eris-lwt
  135. ezgzip
  136. ezjsonm >= "0.4.2" & < "1.3.0"
  137. ezjsonm-lwt < "1.3.0"
  138. FPauth
  139. FPauth-core
  140. FPauth-responses
  141. FPauth-strategies
  142. faraday != "0.2.0"
  143. farfadet
  144. fat-filesystem >= "0.12.0"
  145. ff
  146. ff-pbt
  147. fiat-p256
  148. flex-array
  149. fsevents-lwt
  150. functoria >= "2.2.0"
  151. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  152. geojson
  153. geoml >= "0.1.1"
  154. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  155. git-mirage < "3.0.0"
  156. git-split
  157. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  158. git_split
  159. gitlab-unix
  160. glicko2
  161. gmap >= "0.3.0"
  162. gpt
  163. graphql
  164. graphql-async
  165. graphql-cohttp >= "0.13.0"
  166. graphql-lwt
  167. graphql_parser
  168. graphql_ppx >= "0.7.1"
  169. h1_parser
  170. h2
  171. hacl
  172. hacl_func
  173. hacl_x25519 >= "0.2.0"
  174. highlexer
  175. hkdf
  176. hockmd
  177. html_of_jsx
  178. http
  179. http-multipart-formdata < "2.0.0"
  180. httpaf >= "0.2.0"
  181. httpun
  182. httpun-ws
  183. hvsock
  184. icalendar
  185. imagelib >= "20200929"
  186. index
  187. inferno >= "20220603"
  188. influxdb-async
  189. influxdb-lwt
  190. inquire < "0.2.0"
  191. interval-map
  192. iomux
  193. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  194. irmin-bench >= "2.7.0"
  195. irmin-chunk < "2.0.0" | >= "2.3.0"
  196. irmin-cli
  197. irmin-containers
  198. irmin-fs < "2.0.0" | >= "2.3.0"
  199. irmin-git < "2.0.0" | >= "2.3.0"
  200. irmin-http < "2.0.0"
  201. irmin-mem < "2.0.0"
  202. irmin-pack >= "2.4.0" & != "2.6.1"
  203. irmin-pack-tools
  204. irmin-test < "2.2.0"
  205. irmin-tezos
  206. irmin-tezos-utils
  207. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  208. irmin-watcher
  209. jekyll-format
  210. jerboa
  211. jitsu
  212. jose
  213. json-data-encoding >= "0.9"
  214. json_decoder
  215. jsonxt
  216. junit_alcotest
  217. jwto
  218. kdf
  219. ke >= "0.2"
  220. kkmarkdown
  221. lambda-runtime
  222. lambdapi >= "2.0.0"
  223. lambdoc >= "1.0-beta4"
  224. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  225. lmdb >= "1.0"
  226. logical
  227. logtk >= "1.5.1"
  228. lp
  229. lp-glpk
  230. lp-glpk-js
  231. lp-gurobi
  232. lru
  233. lt-code
  234. luv
  235. mbr-format >= "1.0.0"
  236. mdx >= "1.6.0"
  237. mec
  238. mechaml >= "1.0.0"
  239. merge-queues >= "0.2.0"
  240. merge-ropes >= "0.2.0"
  241. metrics
  242. mirage >= "4.0.0~beta1"
  243. mirage-block-partition < "0.2.0"
  244. mirage-block-ramdisk >= "0.3"
  245. mirage-channel >= "4.0.0"
  246. mirage-channel-lwt
  247. mirage-crypto-ec
  248. mirage-flow >= "1.0.2" & < "1.2.0"
  249. mirage-flow-unix
  250. mirage-fs-mem
  251. mirage-fs-unix >= "1.2.0"
  252. mirage-kv >= "2.0.0"
  253. mirage-kv-mem
  254. mirage-kv-unix
  255. mirage-logs >= "0.3.0"
  256. mirage-nat
  257. mirage-net-unix >= "2.3.0"
  258. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  259. mirage-tc
  260. mjson
  261. mnd
  262. monocypher
  263. mrmime
  264. mrt-format
  265. multibase
  266. multihash
  267. multihash-digestif
  268. multipart-form-data
  269. multipart_form
  270. multipart_form-eio
  271. multipart_form-lwt
  272. named-pipe
  273. nanoid
  274. nbd >= "4.0.3"
  275. nbd-tool
  276. nloge
  277. nocoiner
  278. non_empty_list
  279. OCADml >= "0.6.0"
  280. obatcher
  281. ocaml-r >= "0.5.0"
  282. ocaml-version >= "3.1.0"
  283. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  284. ocamlformat-rpc < "removed"
  285. ocamline
  286. odoc >= "1.4.0" & < "2.1.0"
  287. ohex
  288. oidc
  289. opam-0install
  290. opam-0install-cudf >= "0.5.0"
  291. opam-file-format >= "2.1.1"
  292. opentelemetry >= "0.6"
  293. opentelemetry-client-cohttp-lwt >= "0.6"
  294. opentelemetry-client-ocurl >= "0.6"
  295. opentelemetry-cohttp-lwt >= "0.6"
  296. opentelemetry-lwt >= "0.6"
  297. opium >= "0.15.0"
  298. opium-graphql
  299. opium-testing
  300. opium_kernel
  301. orewa
  302. ortac-core
  303. osx-acl
  304. osx-attr
  305. osx-cf
  306. osx-fsevents
  307. osx-membership
  308. osx-mount
  309. osx-xattr
  310. otoggl
  311. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  312. owl-base < "0.5.0"
  313. owl-ode >= "0.1.0" & != "0.2.0"
  314. owl-symbolic
  315. passmaker
  316. patch
  317. pbkdf
  318. pecu >= "0.2"
  319. pf-qubes
  320. pg_query >= "0.9.6"
  321. phylogenetics
  322. piaf
  323. polyglot
  324. polynomial
  325. ppx_blob >= "0.3.0"
  326. ppx_deriving_cmdliner
  327. ppx_deriving_rpc
  328. ppx_deriving_yaml
  329. ppx_graphql >= "0.2.0"
  330. ppx_mysql
  331. ppx_protocol_conv >= "5.0.0"
  332. ppx_protocol_conv_json >= "5.0.0"
  333. ppx_protocol_conv_jsonm >= "5.0.0"
  334. ppx_protocol_conv_msgpack >= "5.0.0"
  335. ppx_protocol_conv_xml_light >= "5.0.0"
  336. ppx_protocol_conv_xmlm
  337. ppx_protocol_conv_yaml >= "5.0.0"
  338. ppx_subliner
  339. ppx_units
  340. ppx_yojson >= "1.1.0"
  341. pratter
  342. prc
  343. preface
  344. pretty_expressive
  345. prettym
  346. proc-smaps
  347. producer < "0.2.0"
  348. prometheus < "1.2"
  349. prometheus-app
  350. protocell
  351. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  352. protocol-9p-unix
  353. psq
  354. qcheck >= "0.18" & < "0.22"
  355. qcheck-alcotest < "0.22"
  356. quickjs
  357. radis
  358. randii
  359. reason-standard
  360. red-black-tree
  361. reparse >= "2.0.0" & < "3.0.0"
  362. reparse-unix < "2.1.0"
  363. resp < "0.10.0"
  364. resp-unix
  365. rfc1951 < "1.0.0"
  366. routes < "2.0.0"
  367. rpc >= "5.9.0"
  368. rpclib
  369. rpclib-async
  370. rpclib-lwt
  371. rpmfile < "0.3.0"
  372. rpmfile-eio
  373. rpmfile-unix
  374. rubytt
  375. SZXX >= "4.0.0"
  376. salsa20
  377. salsa20-core
  378. sanddb >= "0.2"
  379. scaml >= "1.5.0"
  380. scrypt-kdf
  381. secp256k1 >= "0.4.1"
  382. secp256k1-internal
  383. semver >= "0.2.1"
  384. sendmail
  385. sendmail-lwt
  386. sendmsg
  387. server-reason-react
  388. session-cookie
  389. session-cookie-async
  390. session-cookie-lwt
  391. sherlodoc
  392. slug
  393. sodium-fmt
  394. spin >= "0.6.0"
  395. squirrel
  396. ssh-agent
  397. ssl >= "0.6.0"
  398. stramon-lib
  399. styled-ppx
  400. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  401. tdigest < "2.1.0"
  402. term-indexing
  403. terminal_size >= "0.1.1"
  404. terminus
  405. terminus-cohttp
  406. terminus-hlc
  407. terml
  408. textrazor
  409. tezos-base-test-helpers < "13.0"
  410. tezos-client-base < "12.0"
  411. tezos-lmdb
  412. tezos-test-helpers < "11.0"
  413. tftp
  414. timedesc
  415. timere
  416. tls >= "0.12.0"
  417. toc
  418. topojson
  419. topojsone
  420. transept
  421. twostep
  422. type_eq
  423. type_id
  424. typebeat
  425. typeid >= "1.0.1"
  426. tyre >= "0.4"
  427. tyxml >= "4.0.0"
  428. tyxml-jsx
  429. tyxml-ppx >= "4.3.0"
  430. tyxml-syntax
  431. uecc
  432. ulid
  433. universal-portal
  434. unix-dirent
  435. unix-errno >= "0.3.0"
  436. unix-fcntl >= "0.3.0"
  437. unix-sys-resource
  438. unix-sys-stat
  439. unix-time
  440. unstrctrd
  441. user-agent-parser
  442. uspf
  443. uspf-lwt
  444. uspf-unix
  445. utop >= "2.13.0"
  446. validate
  447. validator
  448. vercel
  449. vpnkit
  450. wcwidth
  451. websocketaf
  452. x509 >= "0.7.0"
  453. xapi-rrd >= "1.8.2"
  454. xapi-stdext-date
  455. xapi-stdext-encodings
  456. xapi-stdext-std >= "4.16.0"
  457. yaml < "3.2.0"
  458. yaml-sexp
  459. yocaml
  460. yocaml_yaml
  461. yuscii >= "0.2.0"
  462. zar
  463. zed >= "3.2.2"
  464. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct