View all changes

OPAM 1.2.0 public beta released

It has only been 18 months since the first release of OPAM, but it is already difficult to remember a time when we did OCaml development without it. OPAM has helped bring together much of the open-source code in the OCaml community under a single umbrella, making it easier to discover, depend on, and maintain OCaml applications and libraries. We have seen steady growth in the number of new packages, updates to existing code, and a diverse group of contributors.

OPAM has turned out to be more than just another package manager. It is also increasingly central to the demanding workflow of industrial OCaml development, since it supports multiple simultaneous (patched) compiler installations, sophisticated package version constraints that ensure statically-typed code can be recompiled without conflict, and a distributed workflow that integrates seamlessly with Git, Mercurial or Darcs version control. OPAM tracks multiple revisions of a single package, thereby letting packages rely on older interfaces if they need to for long-term support. It also supports multiple package repositories, letting users blend the global stable package set with their internal revisions, or building completely isolated package universes for closed-source products.

Since its initial release, we have been learning from the extensive feedback from our users about how they use these features as part of their day-to-day workflows. Larger projects like XenAPI, the Ocsigen web suite, and the Mirage OS publish OPAM remotes that build their particular software suites. Complex applications such as the Pfff static analysis tool and Hack language from Facebook, the Frenetic SDN language and the Arakoon distributed key store have all appeared alongside these libraries. Jane Street pushes regular releases of their production Core/Async suite every couple of weeks.

One pleasant side-effect of the growing package database has been the contribution of tools from the community that make the day-to-day use of OCaml easier. These include the utop interactive toplevel, the IOCaml browser notebook, and the Merlin IDE extension. While these tools are an essential first step, there's still some distance to go to make the OCaml development experience feel fully integrated and polished.

Today, we are kicking off the next phase of evolution of OPAM and starting the journey towards building an OCaml Platform that combines the OCaml compiler toolchain with a coherent workflow for build, documentation, testing and IDE integration. As always with OPAM, this effort has been a collaborative effort, coordinated by the OCaml Labs group in Cambridge and OCamlPro in France. The OCaml Platform builds heavily on OPAM, since it forms the substrate that pulls together the tools and facilitates a consistent development workflow. We've therefore created this blog on opam.ocaml.org to chart its progress, announce major milestones, and eventually become a community repository of all significant activity.

Major points:

  • OPAM 1.2 beta available: Firstly, we're announcing the availability of the OPAM 1.2 beta, which includes a number of new features, hundreds of bug fixes, and pretty new colours in the CLI. We really need your feedback to ensure a polished release, so please do read the release notes below.

  • In the coming weeks, we will provide an overview of what the OCaml Platform is (and is not), and describe an example workflow that the Platform can enable.

  • Feedback: If you have questions or comments as you read these posts, then please do join the platform@lists.ocaml.org and make them known to us.

Releasing the OPAM 1.2 beta4

We are proud to announce the latest beta of OPAM 1.2. It comes packed with new features, stability and usability improvements. Here the highlights.

Binary RPMs and DEBs!

We now have binary packages available for Fedora 19/20, CentOS 6/7, RHEL7, Debian Wheezy and Ubuntu! You can see the full set at the OpenSUSE Builder site and download instructions for your particular platform.

An OPAM binary installation doesn't need OCaml to be installed on the system, so you can initialize a fresh, modern version of OCaml on older systems without needing it to be packaged there. On CentOS 6 for example:

cd /etc/yum.repos.d/
wget http://download.opensuse.org/repositories/home:ocaml/CentOS_6/home:ocaml.repo
yum install opam
opam init --comp=4.01.0

Simpler user workflow

For this version, we focused on improving the user interface and workflow. OPAM is a complex piece of software that needs to handle complex development situations. This implies things might go wrong, which is precisely when good support and error messages are essential. OPAM 1.2 has much improved stability and error handling: fewer errors and more helpful messages plus better state backups when they happen.

In particular, a clear and meaningful explanation is extracted from the solver whenever you are attempting an impossible action (unavailable package, conflicts, etc.):

$ opam install mirage-www=0.3.0
The following dependencies couldn't be met:
  - mirage-www -> cstruct < 0.6.0
  - mirage-www -> mirage-fs >= 0.4.0 -> cstruct >= 0.6.0
Your request can't be satisfied:
  - Conflicting version constraints for cstruct

This sets OPAM ahead of many other package managers in terms of user-friendliness. Since this is made possible using the tools from irill (which are also used for Debian), we hope that this work will find its way into other package managers. The extra analyses in the package solver interface are used to improve the health of the central package repository, via the OPAM Weather service.

And in case stuff does go wrong, we added the opam upgrade --fixup command that will get you back to the closest clean state.

The command-line interface is also more detailed and convenient, polishing and documenting the rough areas. Just run opam <subcommand> --help to see the manual page for the below features.

  • More expressive queries based on dependencies.

    $ opam list --depends-on cow --rec
# Available packages recursively depending on cow.0.10.0 for 4.01.0:
cowabloga   0.0.7  Simple static blogging support.
iocaml      0.4.4  A webserver for iocaml-kernel and iocamljs-kernel.
mirage-www  1.2.0  Mirage website (written in Mirage)
opam2web    1.3.1 (pinned)  A tool to generate a website from an OPAM repository
opium       0.9.1  Sinatra like web toolkit based on Async + Cohttp
stone       0.3.2  Simple static website generator, useful for a portfolio or documentation pages

  • Check on existing opam files to base new packages from.

    $ opam show cow --raw
opam-version: "1"
name: "cow"
version: "0.10.0"
[...]

  • Clone the source code for any OPAM package to modify or browse the interfaces.

    $ opam source cow
Downloading archive of cow.0.10.0...
[...]
$ cd cow.0.10.0

We've also improved the general speed of the tool to cope with the much bigger size of the central repository, which will be of importance for people building on low-power ARM machines, and added a mechanism that will let you install newer releases of OPAM directly from OPAM if you choose so.

Yet more control for the packagers

Packaging new libraries has been made as straight-forward as possible. Here is a quick overview, you may also want to check the OPAM 1.2 pinning post.

opam pin add <name> <sourcedir>

will generate a new package on the fly by detecting the presence of an opam file within the source repository itself. We'll do a followup post next week with more details of this extended opam pin workflow.

The package description format has also been extended with some new fields:

  • bug-reports: and dev-repo: add useful URLs
  • install: allows build and install commands to be split,
  • flags: is an entry point for several extensions that can affect your package.

Packagers can limit dependencies in scope by adding one of the keywords build, test or doc in front of their constraints:

depends: [
  "ocamlfind" {build & >= 1.4.0}
  "ounit" {test}
]

Here you don't specifically require ocamlfind at runtime, so changing it won't trigger recompilation of your package. ounit is marked as only required for the package's build-test: target, i.e. when installing with opam install -t. This will reduce the amount of (re)compilation required in day-to-day use.

We've also made optional dependencies more consistent by removing version constraints from the depopts: field: their meaning was unclear and confusing. The conflicts field is used to indicate versions of the optional dependencies that are incompatible with your package to remove all ambiguity:

depopts: [ "async" {>= "109.15.00"} & "async_ssl" {>= "111.06.00"} ]

becomes:

depopts: [ "async" "async_ssl" ]
conflicts: [ "async" {< "109.15.00"}
             "async_ssl" {< "111.06.00"} ]

There is an upcoming features field that will give more flexibility in a clearer and consistent way for such complex cases.

Easier to package and install

Efforts were made on the build of OPAM itself as well to make it as easy as possible to compile, bootstrap or install. There is no more dependency on camlp4 (which has been moved out of the core distribution in OCaml 4.02.0), and the build process is more conventional (get the source, run ./configure, make lib-ext to get the few internal dependencies, make and make install). Packagers can use make cold to build OPAM with a locally compiled version of OCaml (useful for platforms where it isn't packaged), and also use make download-ext to store all the external archives within the source tree (for automated builds which forbid external net access).

The whole documentation has been rewritten as well, to be better focused and easier to browse. Please leave any feedback or changes on the documentation on the issue tracker.

Try it out !

The public beta of OPAM 1.2 is just out. You're welcome to give it a try and give us feedback before we roll out the release!

We'd be most interested on feedback on how easily you can work with the new pinning features, on how the new metadata works for you... and on any errors you may trigger that aren't followed by informative messages or clean behaviour.

If you are hosting a repository, the administration scripts may help you quickly update all your packages to benefit from the new features.