Skip to content

README.md

Latest commit

 

History

History
136 lines (100 loc) · 3.61 KB

README.md

File metadata and controls

136 lines (100 loc) · 3.61 KB

Sample 01: Quick Chat via OpenAI SDK

A simple chat example demonstrating how to use the OpenAI SDK with Microsoft Foundry Local for local AI inference.

Overview

This sample shows how to:

  • Use the OpenAI Python SDK with Foundry Local
  • Handle both Azure OpenAI and local Foundry configurations
  • Implement proper error handling and fallback strategies
  • Use the FoundryLocalManager for service management

Prerequisites

  • Foundry Local: Installed and available on PATH
  • Python: 3.8 or later
  • Model: A model loaded in Foundry Local (e.g., phi-4-mini)

Installation

  1. Set up Python environment:

    cd Module08
py -m venv .venv
.venv\Scripts\activate

  2. Install dependencies:

    pip install -r requirements.txt

  3. Start Foundry Local service and load a model:

    foundry model run phi-4-mini

Usage

Foundry Local (Default)

# Using FoundryLocalManager (recommended)
python samples\01\chat_quickstart.py "Explain what Foundry Local is"

# Using manual configuration
set BASE_URL=http://localhost:8000
set MODEL=phi-4-mini
set API_KEY=
python samples\01\chat_quickstart.py "Write a welcome message"

Azure OpenAI

set AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
set AZURE_OPENAI_API_KEY=your-api-key
set AZURE_OPENAI_API_VERSION=2024-08-01-preview
set MODEL=your-deployment-name
python samples\01\chat_quickstart.py "Hello from Azure OpenAI"

Code Features

FoundryLocalManager Integration

The sample uses the official Foundry Local SDK for proper service management:

from foundry_local import FoundryLocalManager
from openai import OpenAI

# Initialize Foundry Local
manager = FoundryLocalManager(alias)
model_info = manager.get_model_info(alias)

# Configure OpenAI client
client = OpenAI(
    base_url=manager.endpoint,
    api_key=manager.api_key
)

Error Handling

Robust error handling with fallback to manual configuration:

  • Automatic service discovery
  • Graceful degradation if SDK is unavailable
  • Clear error messages for troubleshooting

Environment Variables

Variable Description Default Required
MODEL Model alias or name phi-4-mini No
BASE_URL Foundry Local base URL http://localhost:8000 No
API_KEY API key (usually not needed for local) "" No
AZURE_OPENAI_ENDPOINT Azure OpenAI endpoint - For Azure
AZURE_OPENAI_API_KEY Azure OpenAI API key - For Azure
AZURE_OPENAI_API_VERSION Azure API version 2024-08-01-preview No

Troubleshooting

Common Issues

  1. "Could not use Foundry SDK" warning:

    • Install foundry-local-sdk: pip install foundry-local-sdk
    • Or set environment variables for manual configuration

  2. Connection refused:

    • Ensure Foundry Local is running: foundry service status
    • Check if a model is loaded: foundry service ps

  3. Model not found:

    • List available models: foundry model list
    • Load a model: foundry model run phi-4-mini

Verification

# Check Foundry Local status
foundry service status

# List loaded models
foundry service ps

# Test API endpoint
curl http://localhost:8000/v1/models

References