RAG Chatbot with TypeScript & Intelligent Analysis

A sophisticated Retrieval Augmented Generation (RAG) chatbot built with TypeScript that can ingest documents, perform semantic search, and provide intelligent responses with quality analysis and auto-improvement capabilities.

🚀 Features

• Ollama Support: Run completely local with open-source models (no API keys needed!)
• Multi-Provider: Support for both Ollama (local) and OpenAI (API)
• Document Ingestion: Support for PDF, DOCX, and TXT files
• Vector Embeddings: Uses Ollama's nomic-embed-text or OpenAI's text-embedding models
• Semantic Search: In-memory vector store with cosine similarity
• Conversation Management: Maintains conversation context and history
• 🆕 Intelligent Analysis: Multi-layered quality assessment and auto-improvement
• 🆕 Quality Scoring: Comprehensive metrics for retrieval and answer quality
• 🆕 Hallucination Detection: Identifies and mitigates unsupported information
• 🆕 Auto-Improvement: Automatically enhances poor-quality responses
• 🆕 10-Node Workflow: Advanced intelligent analysis pipeline for superior response quality
• REST API: Full API for web applications
• CLI Interface: Interactive command-line interface
• TypeScript: Full type safety and modern JavaScript features

🧠 Intelligent 10-Node Analysis Workflow

This chatbot features an advanced 10-node intelligent analysis pipeline that ensures high-quality responses through systematic processing:

Workflow Nodes:

1. start - Bắt đầu luồng

   • Initializes the conversation flow

2. check_memory_or_context - Kiểm tra memory/context có giúp được không

   • Analyzes existing conversation history and context relevance

3. clarify_question - Làm rõ câu hỏi nếu mơ hồ

   • Identifies and resolves ambiguous queries

4. rephrase_or_simplify_query - Đơn giản hóa câu hỏi

   • Optimizes query structure for better retrieval

5. retrieve_documents - Truy xuất tài liệu (RAG)

   • Performs semantic search and document retrieval

6. choose_tool_or_direct_answer - Quyết định dùng công cụ hay không

   • Determines whether external tools or direct answering is needed

7. call_tool - Gọi external tool/API

   • Integrates with external APIs and tools when necessary

8. generate_answer - Tạo câu trả lời cuối cùng

   • Generates the final response using retrieved context

9. save_to_memory - Lưu lại hội thoại vào memory

   • Persists conversation state for future reference

10. end - Trả kết quả cuối cùng

    • Returns the final processed response to the user

This workflow ensures that every response goes through rigorous analysis, context checking, and quality validation before being delivered to the user.

📋 Prerequisites

• Node.js 18+
• Option 1 (Recommended): Ollama for local inference
• Option 2: OpenAI API key for cloud inference
• TypeScript knowledge

🛠️ Installation

1. Clone and setup the project:

   cd agent-chatbot-rag2
   npm install

2. Choose your provider:

   🦙 Option A: Local with Ollama (Recommended)

   # Install Ollama from https://ollama.ai
   
   # Pull required models
   ollama pull llama3.1              # Chat model (~4.7GB)
   ollama pull nomic-embed-text      # Embedding model (~274MB)
   
   # Configuration is already set in .env for Ollama

   🌐 Option B: Cloud with OpenAI

   # Edit .env file
   LLM_PROVIDER=openai
   OPENAI_API_KEY=your_openai_api_key_here

3. Build the project:

   npm run build

🚀 Quick Start

Option A: Interactive CLI

# Start the interactive CLI
npm run cli

# Available CLI commands:
# /upload <file_path>  - Upload and process a document
# /stats              - Show document statistics
# /clear-docs         - Clear all documents
# /new                - Start a new conversation
# /history            - Show conversation history
# /health             - Check system health
# /help               - Show help message
# /quit               - Exit

Option B: REST API Server

# Start the API server
npm run api

# Server will start on http://localhost:3000

Option C: Programmatic Usage

import { RAGChatbot } from './src/chatbot';

const chatbot = new RAGChatbot();

// Ingest a document
await chatbot.ingestDocument('./path/to/document.pdf');

// Chat with the bot
const response = await chatbot.chat('What is this document about?');
console.log(response.content);

📡 API Endpoints

When running the API server (npm run api), the following endpoints are available:

Upload Document

POST /upload
Content-Type: multipart/form-data

curl -X POST -F "document=@/path/to/file.pdf" http://localhost:3000/upload

Chat

POST /chat
Content-Type: application/json

{
  "query": "What is this document about?",
  "conversationId": "optional-conversation-id"
}

Get Conversation

GET /conversation/:id

Get All Conversations

GET /conversations

Health Check

GET /health

Document Statistics

GET /documents/stats

💡 Usage Examples

Example 1: Document Q&A

# 1. Start the CLI
npm run cli

# 2. Upload a document
/upload ./examples/sample-document.txt

# 3. Ask questions
What features does this chatbot have?
Can you explain the workflow?
What file formats are supported?

Example 2: API Usage

# 1. Start the API server
npm run api

# 2. Upload a document
curl -X POST -F "document=@./examples/sample-document.txt" http://localhost:3000/upload

# 3. Chat via API
curl -X POST -H "Content-Type: application/json" \
  -d '{"query": "What is this chatbot capable of?"}' \
  http://localhost:3000/chat

Example 3: Programmatic Integration

import { RAGChatbot } from './src/chatbot';
import path from 'path';

async function example() {
  const chatbot = new RAGChatbot();
  
  // Ingest multiple documents
  await chatbot.ingestDocument('./docs/manual.pdf');
  await chatbot.ingestDocument('./docs/faq.docx');
  
  // Start a conversation
  const response1 = await chatbot.chat('How do I get started?');
  console.log(response1.content);
  
  // Continue the conversation with context
  const response2 = await chatbot.chat(
    'Can you be more specific?', 
    response1.id // Same conversation
  );
  console.log(response2.content);
  
  // Check quality metrics
  console.log(`Quality Score: ${response2.metadata?.qualityScore}`);
  console.log(`Sources: ${response2.metadata?.sources}`);
}

🔧 Configuration

The chatbot can be configured via environment variables in .env:

# Provider Selection
LLM_PROVIDER=ollama  # or 'openai'

# Ollama Settings (Default)
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_CHAT_MODEL=llama3.1
OLLAMA_EMBEDDING_MODEL=nomic-embed-text

# OpenAI Settings (Alternative)
OPENAI_API_KEY=your_key_here
OPENAI_CHAT_MODEL=gpt-4
OPENAI_EMBEDDING_MODEL=text-embedding-ada-002

# Quality Thresholds
MIN_RETRIEVAL_SCORE=0.7
MIN_ANSWER_QUALITY_SCORE=0.6
HALLUCINATION_THRESHOLD=0.3

# Server Settings
PORT=3000
HOST=localhost

📊 Quality Metrics

The chatbot provides comprehensive quality analysis for every response:

• Retrieval Score: How well retrieved documents match the query (0.0-1.0)
• Answer Quality: How well the answer addresses the query (0.0-1.0)
• Hallucination Score: Amount of unsupported information (0.0-1.0, lower is better)
• Context Relevance: How relevant the context is to the answer (0.0-1.0)
• Overall Score: Combined quality metric

Auto-Improvement

When quality scores fall below thresholds, the system automatically:

1. Identifies specific quality issues
2. Generates an improved response
3. Marks the response as "improved" in metadata

🛠️ Development

Running Tests

# Run example scripts
npm run dev examples/basic-usage.ts
npm run dev examples/document-ingestion.ts

Building

npm run build    # Compile TypeScript
npm run clean    # Clean build directory
npm run watch    # Watch mode for development

Project Structure

src/
├── chatbot.ts              # Main chatbot class
├── types.ts                # TypeScript interfaces
├── config.ts               # Configuration loader
├── api.ts                  # REST API server
├── cli.ts                  # CLI interface
├── providers/              # LLM provider implementations
│   ├── ollama.ts
│   └── openai.ts
├── vectorstore/            # Vector storage implementations
│   └── index.ts
├── utils/                  # Utility functions
│   └── document-processor.ts
└── workflow/               # Intelligent workflow engine
    └── chatbot-workflow.ts

🔍 Troubleshooting

Common Issues

1. Ollama Connection Error

# Make sure Ollama is running
ollama serve

# Check if models are available
ollama list

# Pull required models if missing
ollama pull llama3.1
ollama pull nomic-embed-text

2. File Upload Issues

• Ensure file size is under 10MB
• Supported formats: PDF, DOCX, TXT
• Check file permissions

3. Low Quality Responses

• Adjust quality thresholds in .env
• Upload more relevant documents
• Try rephrasing questions more specifically

4. Memory Issues

• Clear conversations: chatbot.clearConversation(id)
• Clear documents: chatbot.clearDocuments()
• Restart the application

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📝 License

MIT License - see LICENSE file for details.

🙏 Acknowledgments

• Ollama for local LLM inference
• OpenAI for cloud-based AI services
• The TypeScript community for excellent tooling

---

Built with ❤️ and TypeScript