Batch Translation Process

This project demonstrates a batch processing implementation that enables LLMs to translate documents into multiple languages simultaneously. It's designed to efficiently handle the translation of markdown files while preserving formatting.

Features

• Translates markdown content into multiple languages in parallel
• Saves translated files to specified output directory

Getting Started

1. Install the required packages:

pip install -r requirements.txt

2. Set up your API key:

export ANTHROPIC_API_KEY="your-api-key-here"

3. Run the translation process:

python main.py

How It Works

The implementation uses a TranslateTextNode that processes batches of translation requests:

flowchart LR
    batch[TranslateTextNode]

Loading

The TranslateTextNode:

1. Prepares batches for multiple language translations
2. Executes translations in parallel using the model
3. Saves the translated content to individual files
4. Maintains the original markdown structure

This approach demonstrates how PocketFlow can efficiently process multiple related tasks in parallel.

Example Output

When you run the translation process, you'll see output similar to this:

Translated Chinese text
Translated Spanish text
Translated Japanese text
Translated German text
Translated Russian text
Translated Portuguese text
Translated French text
Translated Korean text
Saved translation to translations/README_CHINESE.md
Saved translation to translations/README_SPANISH.md
Saved translation to translations/README_JAPANESE.md
Saved translation to translations/README_GERMAN.md
Saved translation to translations/README_RUSSIAN.md
Saved translation to translations/README_PORTUGUESE.md
Saved translation to translations/README_FRENCH.md
Saved translation to translations/README_KOREAN.md

=== Translation Complete ===
Translations saved to: translations
============================

Files

• main.py: Implementation of the batch translation node
• utils.py: Simple wrapper for calling the Anthropic model
• requirements.txt: Project dependencies

The translations are saved to the translations directory, with each file named according to the target language.