Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
added question on model versioning
  • Loading branch information
Jeet009 committed Nov 11, 2025
commit 77c00575374770b0eeb4aceaeb03cbf78d61797d
14 changes: 0 additions & 14 deletions questions/187_mlops-etl-pipeline/description.md

This file was deleted.

5 changes: 0 additions & 5 deletions questions/187_mlops-etl-pipeline/example.json

This file was deleted.

24 changes: 0 additions & 24 deletions questions/187_mlops-etl-pipeline/learn.md

This file was deleted.

12 changes: 0 additions & 12 deletions questions/187_mlops-etl-pipeline/meta.json

This file was deleted.

43 changes: 0 additions & 43 deletions questions/187_mlops-etl-pipeline/solution.py

This file was deleted.

9 changes: 0 additions & 9 deletions questions/187_mlops-etl-pipeline/starter_code.py

This file was deleted.

14 changes: 0 additions & 14 deletions questions/187_mlops-etl-pipeline/tests.json

This file was deleted.

63 changes: 63 additions & 0 deletions questions/188_model-versioning-system/description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Model Versioning System

## Problem Statement

Implement a simple model versioning system that tracks model versions, performance metrics, and deployment stages. The system should support basic versioning, model comparison, and simple promotion workflows.

## System Requirements

### Core Components:

1. **Model Registry** - Store and manage model versions
2. **Version Management** - Simple versioning (1.0, 1.1, 1.2, etc.)
3. **Performance Tracking** - Store accuracy and basic metrics
4. **Stage Management** - Track deployment stages (dev, staging, production)

### Key Features:

- **Simple Versioning**: Increment version numbers (1.0, 1.1, 1.2)
- **Performance Tracking**: Store accuracy and basic metrics
- **Stage Management**: Track which stage each version is in
- **Model Comparison**: Compare performance between versions

## Implementation Requirements

### ModelRegistry Class Methods:

1. `register_model(model_name, accuracy, stage="dev")` - Register new model version
2. `get_model(model_name, version=None)` - Get model by version (latest if None)
3. `promote_model(model_name, version, new_stage)` - Move model to new stage
4. `compare_models(model_name, version1, version2)` - Compare two versions
5. `list_models()` - List all registered models

### Model Data Structure:

```python
{
"model_name": str,
"version": str, # e.g., "1.0", "1.1", "1.2"
"accuracy": float,
"stage": str, # "dev", "staging", "production"
"created_at": str # ISO timestamp
}
```

### Validation Rules:

- Accuracy must be between 0.0 and 1.0
- Stage must be one of: "dev", "staging", "production"
- Version format: number.number (e.g., "1.0", "1.1")

## Expected Behavior

- First model registration starts at version "1.0"
- Subsequent registrations increment version ("1.1", "1.2", etc.)
- Models can be promoted between stages
- Performance comparison shows accuracy differences
- System handles basic error cases

## Constraints

- Use only standard Python libraries
- Keep it simple and focused on core functionality
- Handle basic edge cases (invalid accuracy, unknown versions)
5 changes: 5 additions & 0 deletions questions/188_model-versioning-system/example.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"input": "registry = ModelRegistry()\nresult = registry.register_model('sentiment_classifier', 0.95, 'dev')\nprint(result['version'])",
"output": "1.0",
"reasoning": "First registration of a model creates version 1.0 automatically. The system assigns the initial version and stores the accuracy and stage information."
}
126 changes: 126 additions & 0 deletions questions/188_model-versioning-system/learn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Model Versioning System - Learning Guide

## Overview

This problem introduces you to MLOps fundamentals through a simplified model versioning system. You'll learn core concepts of model lifecycle management, version tracking, and deployment coordination without overwhelming complexity.

## Key Concepts

### 1. Model Versioning
- **Simple Versioning**: Use incrementing version numbers (1.0, 1.1, 1.2)
- **Automatic Increment**: Each new registration increments the version
- **Version Tracking**: Keep history of all model versions

### 2. Model Registry Design
- **Centralized Storage**: Store all model information in one place
- **Basic Metadata**: Track essential information (name, version, accuracy, stage)
- **Simple Queries**: Easy retrieval and comparison of models

### 3. Deployment Stages
```
Development → Staging → Production
↓ ↓ ↓
Testing Validation Live Use
```

### 4. Core Data Structure
```python
{
"model_name": "sentiment_classifier",
"version": "1.0",
"accuracy": 0.95,
"stage": "dev",
"created_at": "2024-01-15T10:30:00Z"
}
```

## Implementation Strategies

### 1. Data Storage
```python
# Simple approach using nested dictionaries
self.models = {
"model_name": {
"1.0": {"accuracy": 0.95, "stage": "dev", "created_at": "..."},
"1.1": {"accuracy": 0.97, "stage": "dev", "created_at": "..."}
}
}
```

### 2. Version Management
- Start with version "1.0" for first registration
- Increment minor version for subsequent registrations
- Parse version strings to determine next version

### 3. Stage Promotion
- Validate that model exists before promotion
- Update stage information for specific version
- Return success/failure status

### 4. Model Comparison
- Retrieve both model versions
- Calculate accuracy difference
- Return comparison results

## Common Implementation Patterns

### 1. Version Increment Logic
```python
def _get_next_version(self, model_name):
if model_name not in self.models:
return "1.0"

versions = list(self.models[model_name].keys())
latest_version = max(versions, key=lambda v: float(v))
major, minor = map(int, latest_version.split('.'))
return f"{major}.{minor + 1}"
```

### 2. Validation
```python
def _validate_inputs(self, model_name, accuracy, stage):
if not model_name or not isinstance(model_name, str):
return False, "Invalid model name"
if not 0.0 <= accuracy <= 1.0:
return False, "Accuracy must be between 0.0 and 1.0"
if stage not in ["dev", "staging", "production"]:
return False, "Invalid stage"
return True, ""
```

## Key Learning Points

### 1. System Design Basics
- **Separation of Concerns**: Each method has a single responsibility
- **Data Integrity**: Validate inputs and handle edge cases
- **Error Handling**: Return meaningful results for success/failure

### 2. MLOps Concepts
- **Model Lifecycle**: Track models from development to production
- **Version Control**: Maintain history of model improvements
- **Stage Management**: Control deployment progression
- **Performance Tracking**: Monitor model quality over time

### 3. Real-world Relevance
This simplified system mirrors concepts used in:
- **MLflow Model Registry**: Industry-standard model management
- **AWS SageMaker**: Cloud-based ML model deployment
- **Azure ML**: Microsoft's ML platform
- **Kubeflow**: Kubernetes-based ML workflows

## Common Pitfalls to Avoid

1. **Version Parsing**: Handle version string format correctly
2. **Edge Cases**: Check for non-existent models/versions
3. **Data Validation**: Ensure accuracy is within valid range
4. **Stage Validation**: Only allow valid stage transitions
5. **Return Types**: Match expected return formats

## Testing Approach

- **Happy Path**: Test normal operations
- **Edge Cases**: Test with invalid inputs
- **Error Handling**: Verify proper error responses
- **State Management**: Ensure data consistency

This problem provides a solid foundation for understanding MLOps system design while remaining approachable for learning the core concepts.
17 changes: 17 additions & 0 deletions questions/188_model-versioning-system/meta.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"id": "188",
"title": "Model Versioning System with Registry and Metadata Management",
"difficulty": "hard",
"category": "MLOps",
"video": "",
"likes": "0",
"dislikes": "0",
"contributor": [
{
"profile_link": "https://github.com/Jeet009",
"name": "Jeet Mukherjee"
}
],
"tinygrad_difficulty": "hard",
"pytorch_difficulty": "hard"
}
Loading