package ocaml-migrate-parsetree

  1. Overview
  2. Docs
Convert OCaml parsetrees between different versions

Install

Dune Dependency

github.com Readme Changelog LGPL-2.1-only WITH OCaml-LGPL-linking-exception License Edit opam file Versions (32)

Authors

  1. F
    Frédéric Bour <frederic.bour@lakaban.net>
  2. J
    Jérémie Dimino <jeremie@dimino.org>

Maintainers

  1. F
    frederic.bour@lakaban.net

Sources

ocaml-migrate-parsetree-v1.6.0.tbz
sha256=9b018e7d25114ce17fc0b82b7cd7c927b84ebb6b043aa987fa7731c2484de33f
sha512=e03a5fe44ecf43683c764a7285a65bfa80639c09badf422661723bc3483d6d799c47c1ead34c2caa289a37e1b4b46d809c8cc56537d5c76e6004849d2d8a305f

Description

Deprecated. Please, use Ppxlib instead. More info on https://ocaml.org/changelog/2023-10-23-omp-deprecation

Convert OCaml parsetrees between different versions

This library converts parsetrees, outcometree and ast mappers between different OCaml versions. High-level functions help making PPX rewriters independent of a compiler version.

Tags

syntax org:ocamllabs

Published: 13 Feb 2020

README

OCaml-migrate-parsetree

Convert OCaml parsetrees between different major versions

This library converts between parsetrees of different OCaml versions.

Supported versions are 4.02, 4.03, 4.04, 4.05, 4.06, 4.07, 4.08 and 4.09. For each version, there is a snapshot of the parsetree and conversion functions to the next and/or previous version.

Asts

module Ast_402, Ast_403, Ast_404, Ast_405, Ast_406, Ast_407, Ast_408, Ast_409 : sig

  (* These two modules didn't change between compiler versions.
     Just share the ones from compiler-libs. *)
  module Location = Location
  module Longident = Longident

  (* Version specific copy of AST *)
  module Asttypes
  module Parsetree
  module Outcometree

  (* Other modules that are useful for implementing PPX.

     Docstrings and Ast_mapper only contain general definitions
     In particular, the internal state used by compiler-libs has been
     removed.
     Also equalities are lost for abstract types (Docstring.docstring).  *)
  module Docstrings
  module Ast_helper
  module Ast_mapper

  (* Magic numbers used for marshalling *)
  module Config : sig
    val ast_impl_magic_number : string
    val ast_intf_magic_number : string
  end
end

These embed copies of AST definitions for each supported OCaml major version.

The AST matching the version of the OCaml toolchain will contain equalities relating the copy of types to the definitions from compiler-libs. For instance, when installed with OCaml 4.04.x, Ast_404.Parsetree looks like.

Migration modules

For each pair of versions $(n) and $(n+1), the two modules Migrate_parsetree_$(n)_$(n+1) and Migrate_parsetree_$(n+1)_$(n) convert the AST forward and backward.

The forward conversion is total while the backward conversion is partial: when a feature is not available in a previous version of the parsetree, a Migrate_parsetree_def.Migration_error exception is raised detailing the failure case.

Migrate_parsetree_versions abstract versions of the compiler. Each version is represented as a module with OCaml_version signature. Instances are named OCaml_402, OCaml_403, ... OCaml_current is an alias to the version of the current compiler. The Convert functor takes two versions of OCaml and produce conversion functions.

Finally, the Migrate_parsetree_ast_io provides an easy interface for marshalling/unmarshalling.

Migrate_parsetree.Driver

The Migrate_parsetree.Driver provides an API for ppx rewriters to register OCaml AST rewriters. Ppx rewriters using this API can be used as standalone rewriter executable or as part of a driver including several rewriters.

Using a single driver for several rewritings has the advantage that it is faster. Especially when using many ppx rewriters, it can speed up compilation a lot.

If using Dune, you can consult the dune manual to see how to define and use ppx rewriters. Dune automatically creates drivers based on ocaml-migrate-parsetree on demand.

The rest of this section describes how to do things manually or with ocamlbuild.

Building a custom driver using ocamlfind

To build a custom driver using ocamlfind, simply link all the ppx rewriter libraries together with the ocaml-migrate-parsetree.driver-main package at the end:

ocamlfind ocamlopt -predicates ppx_driver -o ppx -linkpkg \
  -package ppx_sexp_conv -package ppx_bin_prot \
  -package ocaml-migrate-parsetree.driver-main

Normally, ocaml-migrate-parsetree based rewriters should be build with the approriate -linkall option on individual libraries. If one is missing this option, the rewriter might not get linked in. If this is the case, a workaround is to pass -linkall when linking the custom driver.

The resulting ppx program can be used as follow:

  • ./ppx file.ml to print the transformed code

  • ocamlc -pp './ppx --as-pp' ... to use it as a pre-processor

  • ocamlc -ppx './ppx --as-ppx' ... to use it as a -ppx rewriter

Development

It started from the work of Alain Frisch in ppx_tools.

The library is distributed under LGPL 2.1 and is copyright INRIA.

Adding a new OCaml version

We use Cinaps to generate boilerplate. You can install it via opam: opam install cinaps.

Add the new version in src/cinaps_helpers supported_versions.

Copy the last src/ast_xxx.ml file to src/ast_<new_version>.ml, then go over the file and update each sub-module by replacing its signature and implementation with the code from the compiler. For the Config sub-module, update the two variables with the values in utils/config.mlp in the compiler source tree.

Once this is done, call tools/add_special_comments.native on the file.

Then diff the src/ast_xxx.ml and src/ast_<new_version>.ml and go over the diff to make sure the difference are relevant. The ast_... files require some adjustments which should pop up when you do this diff. Port the old adjustments to the new file as required.

Add migration functions:

  • Manually compile the asts (ocamlc -c src/ast_{NEW,OLD}.ml -I +compiler-libs -I _build/default/src/.migrate_parsetree.objs/byte/ -open Migrate_parsetree__)

  • Using tools/gencopy.exe (dune build tools/gencopy.exe), generate copy code to and from previous version (assuming it is 408):

_build/default/tools/gencopy.exe -I . -I src/ -I +compiler-libs -map Ast_409:Ast_408 Ast_409.Parsetree.{expression,expr,pattern,pat,core_type,typ,toplevel_phrase} Ast_409.Outcometree.{out_phrase,out_type_extension} > src/migrate_parsetree_409_408_migrate.ml
_build/default/tools/gencopy.exe -I . -I src/ -I +compiler-libs -map Ast_408:Ast_409 Ast_408.Parsetree.{expression,expr,pattern,pat,core_type,typ,toplevel_phrase} Ast_408.Outcometree.{out_phrase,out_type_extension} > src/migrate_parsetree_408_409_migrate.ml

  • Fix the generated code by implementing new cases

  • The migration functor expects specific names, look at Migrate_parsetree_versions interface.

TODO: specialize and improve gencopy for these cases

Add mapper lifting functions in the files migrate_parsetree_NEW_408.ml and migrate_parsetree_408_NEW.ml:

  • include the corresponding Migrate_parsetree_40x_40y_migrate module

  • define copy_mapper function, look at existing Migrate_parsetree_40x_40y for guidance.

At any time, you can expand boilerplate code by running make cinaps.

Update build system:

  • make sure make cinaps reaches a fixed point :)

  • make should succeed

Dependencies (4)

  1. ocaml >= "4.02.3" & < "4.11"
  2. dune >= "1.9.0"
  3. ppx_derivers
  4. result

Dev Dependencies

None

  1. async >= "v0.9.0" & < "v0.12.0"
  2. async_durable < "v0.12.0"
  3. async_extended >= "v0.9.0"
  4. async_extra >= "v0.9.0" & < "v0.12.0"
  5. async_find >= "v0.9.0" & < "v0.12.0"
  6. async_inotify >= "v0.9.0" & < "v0.12.0"
  7. async_interactive < "v0.12.0"
  8. async_js < "v0.12.0"
  9. async_kernel >= "v0.9.0" & < "v0.12.0"
  10. async_parallel >= "v0.9.0"
  11. async_rpc_kernel >= "v0.9.0" & < "v0.12.0"
  12. async_sendfile < "v0.12.0"
  13. async_shell >= "v0.9.0" & < "v0.12.0"
  14. async_smtp >= "v0.9.0" & < "v0.12.0"
  15. async_ssl >= "v0.9.0" & < "v0.12.0"
  16. async_unix >= "v0.9.0" & < "v0.12.0"
  17. bignum >= "v0.9.0" & < "v0.12.0"
  18. bin_prot >= "v0.9.0" & < "v0.12.0"
  19. bisect_ppx >= "1.3.0" & < "2.4.1"
  20. bitstring >= "3.0.0" & < "4.0.0"
  21. command_rpc < "v0.12.0"
  22. conduit-lwt-unix < "1.3.0"
  23. configurator < "v0.10.0"
  24. core >= "v0.9.0" & < "v0.12.0"
  25. core_bench >= "v0.9.0" & < "v0.12.0"
  26. core_extended >= "v0.9.0" & < "v0.12.0"
  27. core_kernel >= "v0.9.0" & < "v0.12.0"
  28. core_profiler >= "v0.9.0" & < "v0.12.0"
  29. cstruct = "2.4.1"
  30. csvfields < "v0.12.0"
  31. delimited_parsing < "v0.12.0"
  32. dockerfile >= "3.0.0" & < "6.0.0"
  33. dockerfile-cmd < "6.0.0"
  34. dockerfile-opam >= "4.0.0" & < "6.0.0"
  35. ecaml < "v0.12.0"
  36. elpi < "1.13.2"
  37. email_message >= "v0.9.0" & < "v0.12.0"
  38. expect_test_helpers < "v0.12.0"
  39. expect_test_helpers_kernel < "v0.12.0"
  40. fieldslib >= "v0.9.0" & < "v0.12.0"
  41. fstar >= "0.9.6.0" & < "2021.06.06"
  42. GT < "0.4.0"
  43. gdbprofiler >= "0.2" & < "0.4"
  44. gen_js_api >= "1.0.6" & < "1.0.8"
  45. graphql < "0.4.0"
  46. graphql_parser < "0.9.0"
  47. graphql_ppx < "1.2.2"
  48. incr_dom < "v0.12.0"
  49. incr_dom_widgets < "v0.12.0"
  50. incr_map < "v0.12.0"
  51. incr_select < "v0.12.0"
  52. incremental >= "v0.9.0" & < "v0.12.0"
  53. incremental_kernel >= "v0.9.0"
  54. ipaddr = "2.8.0"
  55. jane-street-tests
  56. jenga >= "v0.9.0"
  57. js_of_ocaml >= "3.0" & < "3.8.0"
  58. js_of_ocaml-compiler >= "3.5.0" & < "3.8.0"
  59. js_of_ocaml-ppx < "3.8.0"
  60. js_of_ocaml-ppx_deriving_json >= "3.5.0" & < "3.8.0"
  61. json-wheel_jane_street_overlay
  62. jupyter >= "2.0.0" & < "2.2.2"
  63. kubecaml
  64. lablqml = "0.5.2"
  65. landmarks = "1.3"
  66. levenshtein >= "1.1.3"
  67. line-up-words < "v0.12.0"
  68. lwt >= "3.1.0" & < "4.0.0"
  69. lwt_ppx < "2.0.2"
  70. mdx >= "1.2.0" & < "1.8.1"
  71. mecab
  72. memtrace_viewer < "v0.15.0"
  73. metapp < "0.2.0"
  74. metaquot < "0.2.0"
  75. mirage-profile = "0.8.2"
  76. mlpost >= "0.9"
  77. mlt_parser < "v0.12.0"
  78. multipart-form-data = "0.2.0"
  79. notty_async < "v0.12.0"
  80. nsq >= "0.2.4"
  81. obus >= "1.2.0" & < "1.2.3"
  82. ocaml-basics >= "0.5.0"
  83. ocaml-logicalform
  84. ocaml-migrate-parsetree-ocamlbuild
  85. ocaml-monadic >= "0.4.0" & < "0.5"
  86. ocaml_plugin >= "v0.9.0" & < "v0.12.0"
  87. ocamlformat < "0.15.0"
  88. odoc >= "2.0.0" & < "2.1.0"
  89. omonad >= "0.3.3"
  90. openai-gym
  91. otetris
  92. parsexp < "v0.11.0"
  93. parsexp_io < "v0.12.0"
  94. partition_map
  95. passmaker
  96. patdiff >= "v0.9.0" & < "v0.12.0"
  97. patience_diff >= "v0.9.0" & < "v0.12.0"
  98. pattern >= "0.2.0"
  99. pgocaml >= "3.1" & < "4.0"
  100. pgocaml_ppx < "4.3.0"
  101. pla >= "1.2" & < "2.0"
  102. posixat >= "v0.10.0" & < "v0.12.0"
  103. ppx_assert >= "v0.9.0" & < "v0.12.0"
  104. ppx_ast < "v0.11.0"
  105. ppx_base < "v0.12.0"
  106. ppx_bench >= "v0.9.0" & < "v0.12.0"
  107. ppx_bigarray >= "3.0.0"
  108. ppx_bin_prot >= "v0.9.0" & < "v0.12.0"
  109. ppx_bitstring >= "2.0.0" & < "4.0.0"
  110. ppx_blob >= "0.3.0" & < "0.7.1"
  111. ppx_compare >= "v0.9.0" & < "v0.12.0"
  112. ppx_compose < "0.2.1"
  113. ppx_conv_func >= "v0.9.0" & < "v0.12.0"
  114. ppx_cstruct >= "3.0.1" & < "6.0.0"
  115. ppx_cstubs < "0.4.1"
  116. ppx_csv_conv >= "v0.9.0" & < "v0.12.0"
  117. ppx_custom_printf >= "v0.9.0" & < "v0.12.0"
  118. ppx_defer >= "0.3.0" & < "0.5.0"
  119. ppx_deriving >= "4.2" & < "5.2"
  120. ppx_deriving_argparse
  121. ppx_distr_guards = "0.2"
  122. ppx_driver >= "v0.9.0" & < "v0.11.0"
  123. ppx_dryunit
  124. ppx_enumerate >= "v0.9.0" & < "v0.12.0"
  125. ppx_expect >= "v0.9.0" & < "v0.12.0"
  126. ppx_fail >= "v0.9.0" & < "v0.12.0"
  127. ppx_fast_pipe
  128. ppx_fields_conv >= "v0.9.0" & < "v0.12.0"
  129. ppx_gen_rec < "2.0.0"
  130. ppx_hash < "v0.12.0"
  131. ppx_here >= "v0.9.0" & < "v0.12.0"
  132. ppx_implicits >= "0.2.0"
  133. ppx_import >= "1.5-3-gbd627d5" & < "1.8.0"
  134. ppx_inline_test >= "v0.9.0" & < "v0.12.0"
  135. ppx_jane >= "v0.9.0" & < "v0.12.0"
  136. ppx_js_style < "v0.12.0"
  137. ppx_jsobject_conv >= "0.5.0" & < "0.9.0"
  138. ppx_let >= "v0.9.0" & < "v0.12.0"
  139. ppx_metaquot < "v0.11.0"
  140. ppx_monoid = "0.3.2"
  141. ppx_nanocaml
  142. ppx_optional < "v0.12.0"
  143. ppx_pipebang >= "v0.9.0" & < "v0.12.0"
  144. ppx_regexp < "0.5.0"
  145. ppx_relit >= "0.2.0"
  146. ppx_sexp_conv >= "v0.9.0" & < "v0.12.0"
  147. ppx_sexp_message >= "v0.9.0" & < "v0.12.0"
  148. ppx_sexp_value >= "v0.9.0" & < "v0.12.0"
  149. ppx_sqlexpr
  150. ppx_tools_versioned >= "5.2.2" & < "5.4.0"
  151. ppx_traverse < "v0.11.0"
  152. ppx_type_conv >= "v0.9.0" & < "v0.11.0"
  153. ppx_typerep_conv >= "v0.9.0" & < "v0.12.0"
  154. ppx_tyre
  155. ppx_variants_conv >= "v0.9.0" & < "v0.12.0"
  156. ppx_view
  157. ppx_xml_conv >= "v0.9.0" & < "v0.12.0"
  158. ppxfind >= "1.4"
  159. ppxlib < "0.16.0"
  160. ppxx >= "2.0.0" & < "2.5.0"
  161. prettiest
  162. protocol-9p >= "0.11.2" & < "1.0.0"
  163. protocol-9p-unix = "0.11.3" | = "0.12.1"
  164. protocol_version_header < "v0.12.0"
  165. pumping
  166. re2 >= "v0.9.0" & < "v0.12.0"
  167. reason >= "1.11.0" & < "3.6.2"
  168. reason-parser
  169. record_builder < "v0.12.0"
  170. relit-reason
  171. relit_helper >= "0.2.0"
  172. resource_cache < "v0.12.0"
  173. rpc_parallel >= "v0.9.0" & < "v0.12.0"
  174. scaml < "1.5.0"
  175. sedlex >= "1.99.4" & < "2.3"
  176. sequencer_table < "v0.12.0"
  177. sexp_pretty < "v0.12.0"
  178. shared-memory-ring = "3.0.1"
  179. splay_tree < "v0.12.0"
  180. splittable_random < "v0.12.0"
  181. spotlib = "4.0.3"
  182. ssh-agent < "0.2.0"
  183. sslconf
  184. string_dict < "v0.12.0"
  185. tcpip >= "3.4.1" & < "3.7.0"
  186. textutils >= "v0.9.0" & < "v0.12.0"
  187. textutils_kernel < "v0.12.0"
  188. tezos-benchmark
  189. toplevel_expect_test >= "v0.9.1" & < "v0.12.0"
  190. topological_sort < "v0.12.0"
  191. treeprint = "2.2.0"
  192. typerep_extended >= "v0.9.0"
  193. unmagic >= "1.0.3"
  194. uri >= "1.9.4" & < "2.0.0"
  195. variantslib >= "v0.9.0" & < "v0.12.0"
  196. virtual_dom < "v0.12.0"
  197. vmnet >= "1.3.0" & < "1.3.2"
  198. wcs-lib >= "2017-05-26.02"
  199. yaml < "1.0.0"
  200. yara
  201. zarith-ppx

Conflicts (1)

  1. base-effects
OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct