Scotland Yard AI Agent

This is my first AI project that uses actual ML principles. It utilizes a FastAPI-based REST API service combined with a deep neural network model built with Keras/TensorFlow and trained using reinforcement learning. This service predicts a reward/penalty score for a state of a Scotland Yard game plying for Mr X. According to those predictions player can pick a move that leads to a state with the highest score.

Overview

This project provides a machine learning service that:

• Trains a deep neural network on CSV data
• Serves predictions through REST API endpoints
• Supports real-time model retraining
• Implements z-score normalization for input features
• Uses L2 regularization to prevent overfitting
• Uses reinforcement learning paradigm

Architecture

Neural Network Model

• Architecture: 4-layer fully connected network
  • Input layer: Configurable size (default 408 features)
  • Hidden layer 1: 256 units (ReLU activation)
  • Hidden layer 2: 128 units (ReLU activation)
  • Hidden layer 3: 64 units (ReLU activation)
  • Output layer: 1 unit (Linear activation for regression)
• Initialization: He Normal for weights, zeros for biases
• Regularization: L2 (λ=0.01)
• Optimizer: Adam
• Loss Function: Mean Squared Error (MSE)

Project Structure

.
├── NN_Setup.py       # Neural network architecture definition
├── NeuralNet.py      # Model training and prediction logic
├── Runner.py         # FastAPI application and endpoints
├── Helper.py         # Utility functions and configuration
├── Testing.py        # Testing utilities (commented out)
├── model.keras       # Saved model file (generated)
├── MusAndSigmas      # Normalization parameters (generated)
├── data.csv          # Training data (user-provided)
└── log              # Training history log (generated)

Installation

Prerequisites

• Python 3.8+
• pip package manager

Dependencies

pip install keras
pip install tensorflow
pip install numpy
pip install fastapi
pip install uvicorn
pip install pydantic

Configuration

Update the file paths in Helper.py and Runner.py to match your system:

# Helper.py
pathToData = "/path/to/your/data.csv"
pathToModel = "model.keras"
pathToParams = "MusAndSigmas"

# Runner.py
pathToModel = "/path/to/your/model.keras"

Data Format

The CSV data file should have the following structure:

• Each row represents one training example
• Columns 1 to N: Input features (state + action)
• Last column: Target value (label)
• No header row
• Comma-separated values

Example:

1.2,3.4,5.6,...,7.8,9.0
2.3,4.5,6.7,...,8.9,10.1

Usage

Starting the API Server

# Using FastAPI CLI
fastapi run Runner.py

# Or using Uvicorn directly
uvicorn Runner:app --reload

# Specify host and port
uvicorn Runner:app --host 0.0.0.0 --port 8000

The API will be available at http://localhost:8000

API Endpoints

1. Prediction Endpoint

GET /predict/{item_id}?value={comma_separated_values}

Make predictions using the trained model.

Parameters:

• item_id (int): Identifier for the request
• value (string): Comma-separated input values

Example:

curl "http://localhost:8000/predict/1?value=1,2,3,4,5,6,7,8,9,10,..."

Response:

{
  "item_id": 1,
  "result": 42
}

2. Training Endpoint

GET /train/{item_id}

Retrain the model with new data in data.csv.

Parameters:

• item_id (int): Identifier for the request

Example:

curl "http://localhost:8000/train/1?value=408,0,>Users>username>path>to>data.csv"

Response:

{
  "item_id": 1,
  "result": "1"
}

3. Test Endpoints

GET /test/{item_id}

POST /test/{item_id}

Test endpoints for development purposes.

API Documentation

Once the server is running, visit:

• Interactive API docs: http://localhost:8000/docs
• Alternative API docs: http://localhost:8000/redoc

Training Process

1. Data Loading: Reads CSV file with features and labels
2. Normalization: Applies z-score normalization to input features
3. Training Configuration:
   • Batch size: Total samples / 700
   • Epochs: 30
   • Optimizer: Adam with default parameters
4. Model Saving: Saves trained model and normalization parameters
5. Logging: Records training history and evaluation metrics

Model Evaluation

The system tracks the following metrics:

• Initial and final training loss
• Average prediction error
• Maximum prediction error
• Percentage of predictions with error < 5

Metrics are automatically logged to the log file after each training session.

Development

Creating a New Model

from NN_Setup import newModel

# Create a new model with custom input size
model = newModel(
    path_to_model="custom_model.keras",
    path_to_params="custom_params",
    input_vector_size=512
)

Training Programmatically

from NeuralNet import NeuralNet

# Initialize neural network for RL
nn = NeuralNet("model.keras")

# Train the model with RL data
nn.train_(
    lenSt=408,      # State vector length (environment state representation)
    lenAc=0,        # Action vector length (discrete/continuous actions)
    path="data.csv" # Path to training data (state-action-reward tuples)
)

Making Predictions

from NeuralNet import NeuralNet

nn = NeuralNet("model.keras")

# Prepare input vector (state + action)
state = [1, 2, 3, 4, 5, ...]  # Environment state
action = [0, 1, 0, ...]        # Action representation
input_vector = state + action  # Must match input_vector_size

# Get prediction (expected reward/value)
predicted_value = nn.predict(input_vector)
print(f"Predicted value/Q-value: {predicted_value}")

Troubleshooting

Common Issues

1. Import Errors: Ensure all dependencies are installed correctly
2. File Path Issues: Update absolute paths in configuration files
3. Data Format Errors: Verify CSV file format matches expected structure
4. Memory Issues: Reduce batch size if running out of memory
5. Port Already in Use: Change port using --port flag

Logging

Training logs are automatically saved to the log file with format:

init loss: X.XX -> fin loss: Y.YY -> (avg: Z.ZZ, max: W.WW, '<5' error: PP%)

License

This project is meant for educational purposes only.

Acknowledgments

Built with:

• Keras - High-level neural networks API
• TensorFlow - Machine learning framework
• FastAPI - Modern web API framework
• NumPy - Numerical computing library