tonyfettes/tree_sitter

This is a MoonBit binding to the tree-sitter incremental parsing library.

Quickstart

1. Add this module as a dependency to your MoonBit project:

   moon update # Update the mooncakes.io package index
   moon add tonyfettes/tree_sitter

2. Import the tonyfettes/tree_sitter package in moon.pkg.json:

   {
     "import": [
       "tonyfettes/tree_sitter"
     ]
   }

3. Optionally you may want to install MoonBit bindings to the tree-sitter grammars you want to use. For example, to install the tree-sitter-moonbit grammar:

   moon add tonyfettes/tree_sitter_moonbit

   And then import the tonyfettes/tree_sitter_moonbit package in your moon.pkg.json file as well:

   {
     "import": [
       "tonyfettes/tree_sitter",
       "tonyfettes/tree_sitter_moonbit"
     ]
   }

4. Use the tonyfettes/tree_sitter API to parse your code:

   fn main {
     let moonbit = @tree_sitter_moonbit.language()
     let parser = @tree_sitter.Parser::new()
     parser.set_language(moonbit)
     let source_code =
       #|fn main {
       #|  println("Hello, World!")
       #|}
     let tree = parser.parse_string(None, source_code)
     let root_node = tree.root_node()
     println(root_node.string())
   }

API Walkthrough

The Tree-sitter User Guide is a good place to start if you're new to tree-sitter. Although the API is written in C, this MoonBit binding is just a thin wrapper around the C API, so the documentation should be mostly applicable.

This binding is at its very early stages, so many APIs are not yet implemented. If you find a missing API that you need, please open an issue or a pull request.

Apart from the standard tree-sitter API, this binding also provides abilities to parse the grammar.json and node-types.json files that are generated by the tree-sitter CLI. This is useful if you want to generate MoonBit type definitions for a tree-sitter grammar.

Development

Prerequisites

Other than MoonBit toolchain, this project requires Python (3.9 or later) to build. Please make sure your system has python available in the PATH.

Build

The build should work out of the box on Linux and macOS. Windows is not tested, and you are more than welcome to open an issue or a pull request if you find problems.

# Initialize the tree-sitter submodule
git submodule update --init --recursive
moon update
moon build --target native

We amalgamate the tree-sitter source code with the C stubs together to build a monolithic C file (src/tree-sitter-lib/lib.c) that can be compiled as a single object file. This step is done by a Python script at scripts/prepare.py. The script is set to run automatically when you run moon build --target native or moon check --target native. If you find the generated C file is out of date, you can run the script manually:

python scripts/prepare.py

Test

Before running tests, make sure you have completed the build step above.

To avoid introducing the dependency of the tree-sitter grammars, we put the majority of the tests inside the test/ directory. To run the tests, you can cd into the test/ directory and run moon test --target native:

cd test && moon test --target native

Note

You may find that the LSP is not working when you are editing files under the test/ directory. This is because test/ is a MoonBit module itself, and therefore the LSP process has to be spawned in the test/ directory to work properly. In most cases, this means you need to spin up a new editor instance inside the test/ directory.