Architecture

Introduction

auto_lsp is a generic library for creating Abstract Syntax Trees (AST) and Language Server Protocol (LSP) servers.

It leverages crates such as lsp_types, lsp_server, salsa, and texter, and generates the AST of a Tree-sitter language to simplify building LSP servers.

auto_lsp provides useful abstractions while remaining flexible. You can override the default database as well as all LSP request and notification handlers.

The library is inspired by language tools such as rust-analyzer and ruff, but with a Tree-sitter touch.

It was originally created as a way to quickly ship LSP servers without reinventing the wheel for each new language.

Crates

auto_lsp

This the main crate that reexports auto_lsp_core, auto_lsp_codegen, auto_lsp_default and auto_lsp_server.

auto_lsp_core

auto_lsp_core is the most important crate, it exports:

• ast Defines the AstNode trait and templates used by the codegen crate to build the AST.
• document The Document struct that stores the text and Tree-sitter tree of a file.
• parsers Contains the Parsers struct that stores a Tree-sitter parser and an AST parser function, configured via the configure_parsers! macro. This is used by the db to know how to parse a file.

Additional features:

• document_symbols_builder A DocumentSymbols builder
• semantic_tokens_builder A SemanticTokens builder
• regex A method that applies regex captures over the results of a Tree-sitter query and returns the captures.
• dispatch! and dispatch_once! Macros that make it more convenient to call a method on one or all nodes that match a given concrete type without having to write redundant downcasting code.

auto_lsp_codegen

auto_lsp_codegen contains the code generation logic. Unlike auto_lsp_core, codegen is not reexported by the main crate.

It just exposes a generate function that takes a node-types.json, a LanguageFn and returns a proc_macro2::TokenStream.

auto_lsp_default

auto_lsp_default contains the default database and server capabilities. It is reexported by the main crate when the default feature is enabled.

auto_lsp_server

auto_lsp_server contains the logic for starting an LSP server. It is reexported by the main crate when the lsp_server feature is enabled.

Examples

• ast-python

This example is the most complete one, it contains the generated AST from tree_sitter_python, LSP requests, a database and a custom parser.

• ast-html

This example is a bit more minimal, it only contains the generated AST from tree_sitter_html and a database.

• vscode-wasi

Runs the ast-python example in a vscode extension using the WASI SDK.

• vscode-native

Runs the ast-python example in a vscode extension using either Windows or Linux.

• native

Runs the ast-python example in a native binary with a client mock.

Testing

Most tests are located in the examples folder.

Alongside testing the behavior of the AST, database, and LSP server, we also test whether the generated ASTs are correct in the corpus folder using insta.

Workflows:

• codegen: Tests the auto_lsp_codegen crate.

• test-ast-native: Run main crate tests, ast-python and ast-html tests on Windows and Linux targets.

• test-ast-wasi-p1-threads: Same as above but for wasi-p1-threads target.

• lsp-server-native: Runs a real LSP server with a client mock.

Workflow

This is the current workflow used in internal projects when creating a new language support.

graph TB
    A[generate the ast]
    B[configure parsers]
    C[create a database] 
    D[create an LSP server]
    E[run the server]

    C1[test the ast generation - with expect_test or insta]
    E1[test the LSP server - by mocking requests/notifications]

    A ==> B ==> C ==> D ==> E
    subgraph Db
        C --> C1
    end
    subgraph LSP 
        E --> E1
    end

License

auto_lsp is licensed under the GPL-3.0 license.