Skip to content

README.md

Latest commit

 

History

History
154 lines (119 loc) · 3.69 KB

README.md

File metadata and controls

154 lines (119 loc) · 3.69 KB

WDL Parser Package

A Go package for parsing WDL (Workflow Description Language) files, analyzing dependencies, and creating self-contained bundles.

Features

  • Parse WDL 1.0/1.1 files into an Abstract Syntax Tree (AST)
  • Analyze dependencies - resolve direct and transitive imports
  • Create bundles - ZIP archives with all required WDL files
  • Detect circular dependencies
  • Support for all WDL constructs - workflows, tasks, structs, types

Installation

import "github.com/lmtani/pumbaa/pkg/wdl"

Usage

Parsing a WDL file

// Parse from file
doc, err := wdl.Parse("workflow.wdl")
if err != nil {
    log.Fatal(err)
}

fmt.Printf("WDL Version: %s\n", doc.Version)
fmt.Printf("Workflow: %s\n", doc.Workflow.Name)
fmt.Printf("Imports: %d\n", len(doc.Imports))
fmt.Printf("Tasks: %d\n", len(doc.Tasks))

// Parse from bytes
content := []byte(`version 1.0

workflow Hello {
    output {
        String msg = "Hello, World!"
    }
}
`)
doc, err := wdl.ParseBytes(content)

Analyzing Dependencies

// Analyze dependencies from file
graph, err := wdl.AnalyzeDependenciesFromFile("workflow.wdl")
if err != nil {
    log.Fatal(err)
}

// Get all dependencies (including transitive)
allDeps := graph.GetAllDependencies()
fmt.Printf("Total dependencies: %d\n", len(allDeps))

// Get direct dependencies only
directDeps := graph.GetDirectDependencies()
fmt.Printf("Direct dependencies: %d\n", len(directDeps))

// Print dependency graph
fmt.Println(graph.PrintGraph())

Creating a Bundle

// Create a bundle with all dependencies
err := wdl.CreateBundle("workflow.wdl", "bundle.zip")
if err != nil {
    log.Fatal(err)
}

// Create bundle with custom options
opts := wdl.BundleOptions{
    IncludeMetadata:            true,
    PreserveDirectoryStructure: true,
}
err = wdl.CreateBundleWithOptions("workflow.wdl", "bundle.zip", opts)

// Build bundle in memory
bundle, err := wdl.BuildBundle("workflow.wdl", wdl.DefaultBundleOptions())
if err != nil {
    log.Fatal(err)
}

// List files in bundle
for _, file := range bundle.ListFiles() {
    fmt.Println(file)
}

// Get file content
content, ok := bundle.GetFile("tasks/helper.wdl")
if ok {
    fmt.Println(string(content))
}

Extracting a Bundle

err := wdl.ExtractBundle("bundle.zip", "output_dir")
if err != nil {
    log.Fatal(err)
}

AST Structure

The package provides detailed AST structures for WDL elements:

  • Document - Root of the AST, contains version, imports, tasks, workflow
  • Workflow - Workflow definition with inputs, outputs, calls, scatters, conditionals
  • Task - Task definition with inputs, outputs, command, runtime
  • Import - Import statement with URI and aliases
  • Call - Call to a task or subworkflow
  • Scatter - Scatter block
  • Conditional - If block
  • Declaration - Variable declaration with type and optional expression
  • Type - WDL type (String, Int, File, Array, Map, Pair, etc.)

Supported WDL Features

  • WDL 1.0 and 1.1 syntax
  • Workflows and tasks
  • Structs
  • All primitive types (String, Int, Float, Boolean, File)
  • Compound types (Array, Map, Pair)
  • Optional types
  • Imports with aliases
  • Calls with inputs
  • Scatter blocks
  • Conditional (if) blocks
  • Runtime sections
  • Meta and parameter_meta sections

Dependencies

This package uses ANTLR4 runtime for parsing:

  • github.com/antlr4-go/antlr/v4

The parser is generated from the official OpenWDL grammar.

Regenerating the Parser

To regenerate the parser from the grammar files:

cd pkg/wdl/parser
java -jar antlr-4.13.1-complete.jar -Dlanguage=Go -package parser -visitor -listener WdlV1_1Lexer.g4 WdlV1_1Parser.g4