SE-Agent: Self-Evolving AI Agent Tops SWE-bench Verified

SE-Agent is a self-evolving framework designed to enhance the performance of LLM-based agents on complex, multi-step reasoning tasks. Rather than relying on a single-trajectory approach, this method utilizes three core operations to improve results: Revision, Recombination, and Refinement.

Revision is built on the principle of learning from failure. It analyzes unsuccessful trajectories, identifies conceptual blind spots, and generates a new strategy that differs fundamentally from the original attempt. This is not a simple retry, but a complete strategic rethink.

Recombination fuses the strengths of separate solution paths. By identifying high-performing segments from different trajectories and merging them, the framework allows the success of one attempt to resolve the weaknesses of another. This synergy produces results that exceed the capabilities of any individual part.

Refinement polishes the most promising trajectories. It removes redundancies, streamlines action sequences, and incorporates cautionary insights derived from a collective history of past errors.

Together, these operations expand the search space and help the agent avoid local optima. On the SWE-bench Verified benchmark, SE-Agent reached an 80% solve rate, delivering performance jumps of up to 112% across a wide range of Large Language Models.

Performance on SWE-bench Verified

SE-Agent currently holds the top rank among open-source frameworks on the SWE-bench Verified leaderboard.

The following table compares Pass@1 and Pass@5 metrics across different frameworks. In this context, SWE-Agent refers to the CodeAct-based framework, while SWE-Search utilizes Monte Carlo Tree Search.

SE-Agent consistently outperforms both baselines, with the performance gap widening as the number of attempts increases.

Installation

You can get SE-Agent running in approximately 30 seconds.

1. Clone and install

git clone https://github.com/JARVIS-Xs/SE-Agent.git
cd SE-Agent
pip install -e .

2. Set your API key

echo "DEEPSEEK_API_KEY=your_key_here" > .env

3. Run a demo (no API calls needed)

python SE/basic_run.py --mode demo

4. Run your first experiment

python SE/basic_run.py --mode execute

Expected output:

✅ SE-Agent initialized successfully
🔄 Starting self-evolution with 3 iterations  

How SE-Agent Works

SE-Agent applies three self-evolution operations that change how the agent approaches and resolves problems.

Three Core Operations

1. Revision — Failure-Driven Strategy Generation

This operation examines a failed trajectory to pinpoint errors, inefficiencies, or conceptual gaps. Through deep self-reflection, the agent produces a new solution that is architecturally distinct from the first. The goal is to develop an entirely different problem-solving paradigm rather than making minor tweaks.

2. Recombination — Cross-Trajectory Knowledge Synthesis

This operation constructs new trajectories by fusing high-quality segments from existing attempts. It explicitly capitalizes on the interdependencies between different runs, allowing a strength from one attempt to compensate for a flaw in another. This synergy allows the system to surpass the limits of any single trajectory.

3. Refinement — Risk-Aware Trajectory Polishing

Refinement sharpens promising trajectories using insights gathered from the entire pool of attempts. It removes unnecessary steps and compresses action sequences while integrating guidance that avoids known failure patterns and systemic blind spots.

A Concrete Example

Problem: The Astropy UnrecognizedUnit.__eq__ method throws an exception when compared to None. To follow Python's safe comparison conventions, it should return False.

Reward Function:
Reward(t, T) = α·TaskCompletion(t, T) + β·ReasoningQuality(t) + γ·Efficiency(t)

Scoring Components:

• LLM-based scores for reasoning quality and solution soundness.
• Rule-based checks for code completeness, syntax accuracy, and structural validity.

Outcome: From 10 initial trajectories, 5 high-quality paths are selected for further development.

Process Steps:

1. Generate 5 initial trajectories using various planning strategies.
2. Apply the Revision operation.
3. Filter results based on quality (Refinement selection).
4. Orchestrate Recombination to fuse knowledge across different trajectories.

Usage Examples

Basic Self-Evolution Experiment

strategy_config = {
    "iterations": [
        {"base_config": "baseline", "operator": None},
        {"base_config": "enhanced", "operator": "alternative_strategy"}, 
        {"base_config": "enhanced", "operator": "crossover"}
    ]
}

To run the evolution process:

python SE/basic_run.py --config SE/configs/se_configs/experiment.yaml --mode execute

Custom Operator Development

SE-Agent is designed to support flexible operator extensions.

from SE.operators import TemplateOperator, register_operator

class MyEvolutionOperator(TemplateOperator):
    def _generate_content(self, instance_info, problem_description, trajectory_data):
        # Implement your custom evolution strategy
        return "Your generated strategy content"

register_operator("my_operator", MyEvolutionOperator)

A comprehensive operator development guide is available at SE/operators.md, covering architecture, examples, and best practices.

Batch Processing

sweagent run-batch \
  --config config/default.yaml \
  --agent.model.name deepseek/deepseek-chat \
  --instances.subset verified \
  --instances.slice :10

Installation Options

Option 1: Pip (Recommended)

git clone https://github.com/JARVIS-Xs/SE-Agent.git
cd SE-Agent
pip install -e .

Option 2: Conda Environment

git clone https://github.com/JARVIS-Xs/SE-Agent.git
cd SE-Agent
conda create -n SE python=3.12
conda activate SE
pip install -e .

To verify the installation:

sweagent --help
python SE/test/run_operator_tests.py

API Key Configuration

Select a provider and create your .env file accordingly.

echo "DEEPSEEK_API_KEY=your_deepseek_key" > .env
# Or
echo "OPENAI_API_KEY=your_openai_key" > .env  
# Or
echo "ANTHROPIC_API_KEY=your_anthropic_key" > .env

SE-Agent fundamentally changes how agents solve difficult problems. Revision breaks through ineffective patterns, Recombination blends successful strategies, and Refinement ensures the final output is as efficient as possible. Its performance metrics on SWE-bench confirm the effectiveness of this self-evolving approach.