Agents From Scratch: AI Email Assistant with Human-in-the-Loop Approval

Agents From Scratch is an intelligent assistant designed to streamline your inbox management. It analyzes incoming messages to determine the appropriate next step—whether that involves a quick reply or a calendar invitation—and drafts a response accordingly. Crucially, the assistant does not send messages autonomously. Every outgoing action is held in a queue for your personal review and approval. An advanced, memory-enhanced version of the system learns from your past decisions, eventually mirroring how you interact with specific senders and topics.

What It Does

• Email Categorization: Identifies whether a message requires a reply, a meeting request, or simply no action at all.
• Draft Composition: Generates context-aware responses based on the specific content of each email.
• Permission-Based Actions: Integrates directly with the Gmail API to send mail or book meetings, but only after receiving explicit user confirmation.
• Human-in-the-Loop: Critical tasks are queued as checkpoints. By requiring your confirmation, the system ensures it never acts without your oversight.
• Pattern Recognition: The memory-augmented version tracks your preferences over time, ensuring that recurring email types are handled according to your established habits.

Environment Setup

Python Version

This project requires Python 3.11 or newer to ensure full compatibility with LangGraph. You can verify your current version by running:

python3 --version

API Keys

• If you do not have an OpenAI key, you can obtain one via the OpenAI platform.
• You will also need to sign up for LangSmith.
• Once registered, generate a LangSmith API key from your dashboard.

Environment Variables

Create a .env file in the root directory of the project:

cp .env.example .env

Open the .env file and input your credentials:

LANGSMITH_API_KEY=your_langsmith_api_key
LANGSMITH_TRACING=true
LANGSMITH_PROJECT="interrupt-workshop"
OPENAI_API_KEY=your_openai_api_key

Alternatively, you can export these variables directly in your terminal:

export LANGSMITH_API_KEY=your_langsmith_api_key
export LANGSMITH_TRACING=true
export OPENAI_API_KEY=your_openai_api_key

Package Installation

For faster and more consistent dependency management, it is recommended to use uv. If you haven't installed it yet, run:

pip install uv

Next, synchronize the project dependencies:

uv sync --extra dev

Activate the virtual environment:

source .venv/bin/activate

If you prefer using standard pip, follow these steps:

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip
pip install -e .

Completing the installation step is essential. The package is registered as interrupt_workshop, but it uses the import name email_assistant. This configuration allows you to import modules from email_assistant throughout any part of the project.

Connecting the API

Gmail Integration and Deployment

Refer to the Gmail Tools README for detailed setup instructions. That guide will walk you through creating Google API credentials and explains the process for deploying the graph to the LangGraph platform. The complete implementation for Gmail-specific functions is located in src/email_assistant/email_assistant_hitl_memory_gmail.py.

Running Tests

The repository includes an automated test suite. These tests utilize LangSmith to evaluate the assistant's tool usage and the overall quality of its replies.

Run All Tests

python tests/run_all_tests.py

Test Results

Detailed logs will be available in LangSmith under the project name defined in your .env file (LANGSMITH_PROJECT). Within the LangSmith interface, you can:

• Visually inspect agent execution traces.
• Review granular evaluation metrics.
• Compare the performance of different agent versions.

Available Test Implementations

• email_assistant: The baseline email helper configuration.

Testing Notebooks

To verify that all project notebooks execute correctly without errors, run:

python tests/test_notebooks.py

Alternatively, you can use pytest:

pytest tests/test_notebooks.py -v