Mantis is a vision-language-action (VLA) model designed to help robots tackle complex tasks. Its performance relies on three core innovations: disentangled visual foresight, progressive training, and adaptive temporal ensembling.
First, the model extracts latent actions to map visual trajectories without adding computational overhead to the backbone. Second, it introduces different data types incrementally during training. This preserves—and often enhances—the language and reasoning capabilities inherent in the vision-language backbone. Third, Mantis dynamically adjusts its reliance on past information in real time. The result is stable control with fewer inference steps.
In standard robotic benchmarks, Mantis excels at both seen and unseen instructions, a result of its multi-stage pretraining pipeline and targeted fine-tuning across diverse datasets.
| Model | Description |
|---|---|
| Mantis-Base | Core model trained via a three-stage pretraining pipeline |
| Mantis-SSV2 | Pretrained on Something-Something-v2 during the first stage |
| Mantis-LIBERO | Fine-tuned specifically on the LIBERO dataset |
| Dataset | Description |
|---|---|
| Something-Something-v2 | Human action video dataset for stage-one pretraining |
| DROID-Lerobot | Robot dataset used in stages two and three |
| LLaVA-OneVision-1.5-Instruct-Data | Multimodal dataset for stage-three pretraining |
| LIBERO-Lerobot | LIBERO dataset used for fine-tuning |
git clone [email protected]:Yysrc/Mantis.git
cd Mantis
conda env create -f configs/environment_libero.yml
conda activate mantis_libero
git clone [email protected]:Lifelong-Robot-Learning/LIBERO.git
cd LIBERO
pip install -e .
cd ..
pip install -r experiments/libero/libero_requirements.txt
sh experiments/libero/run_libero_eval.sh
Adjust task_suite_name within the script to test different task suites. Use eval_mode to toggle between TE and ATE.
Download the LIBERO dataset and the base Mantis checkpoint.
Set up the training environment:
conda env create -f configs/environment_lerobot.yml
conda activate mantis_lerobot
git clone -b paszea/lerobot [email protected]:Yysrc/lerobot.git
cd lerobot
conda install ffmpeg=7.1.1 -c conda-forge
pip install -e .
Configure the files in configs/. Point dataset_root_dir to your LIBERO folder and resume_from_checkpoint to the Mantis base model.
Start the training process:
sh train.sh
HackGPT Enterprise Review: AI-Native Pentesting for Security Teams
Shendeng VPN Review: High-Speed Gaming, Video Streaming, and Unlimited Data
Eigent: Multi-Agent Workflow Desktop App with CAMEL and MCP
JoyAgent-JDGenie: An Open-Source Multi-Agent System for Direct Report Generation
Gemini-CLI-UI: A Web Interface for the Google Gemini CLI Coding Assistant
MemoryOS: Equip AI Agents with Persistent Recall via a Memory Hierarchy
InvenTree Inventory: Self-Hosted Stock Control with REST API
Crawl4AI RAG MCP Server: Web Crawling and Vector Search for AI Agents
Deep Search Lighting: Lightweight Web Search for LLMs
Ditto Clipboard Manager: Never Lose Your Copied Text Again
DBeaver: A Free Cross-Platform Database Tool (Plus CloudBeaver)
How to Install Unregistered Apps on Android