Add dataset generate command to CLI

[devin-ai-integration]

Add dataset generate command to CLI

This PR adds a new dataset subcommand to the vlmrun CLI with a generate operation that calls the /v1/dataset/generate endpoint. The command supports generating datasets from YouTube playlists and videos with all the parameters supported by the API.

Changes

• Add new dataset subcommand with generate operation
• Support all arguments from DatasetGenerationRequest model:
  • domain (required)
  • urls (required, list of URLs)
  • url_type (required, 'yt_playlist' or 'yt_video')
  • dataset_name (required)
  • dataset_format (defaults to 'json')
  • max_frames_per_video (required)
  • max_samples (required, between 10 and 100,000)
• Add input validation for url_type and max_samples
• Implement friendly output formatting for dataset generation results

Usage Example

vlmrun dataset generate \
  --domain "example-domain" \
  --urls "https://youtube.com/playlist?list=..." \
  --url-type "yt_playlist" \
  --dataset-name "my-dataset" \
  --dataset-format "json" \
  --max-frames-per-video 30 \
  --max-samples 1000

Testing

The command implementation includes:

• Input validation for url_type (must be 'yt_playlist' or 'yt_video')
• Input validation for max_samples (must be between 10 and 100,000)
• Friendly error messages and output formatting
• Help text for all arguments

Link to Devin run: https://app.devin.ai/sessions/24a085b8c2a64b80ba60775b4ee5c28a

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
• Look at CI failures and help fix them

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[spillai]

Use client = ctx.obj?

[spillai]

This file is unnecessary

[devin-ai-integration]

I've addressed the review comments:

• Removed the unnecessary utils.py file
• Updated dataset.py to use direct ctx.obj instead of get_context_client
• Cleaned up references in init.py

All CI checks are passing. Please let me know if you'd like any further changes!