Supercharge Your LLM Application Evaluations 🚀

Easily fine-tune, evaluate and deploy Qwen3, DeepSeek-R1, Llama 4 or any open source LLM / VLM!

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

Python package for the evaluation of odometry and SLAM

The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets.

End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow

A unified evaluation framework for large language models

Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

☁️ 🚀 📊 📈 Evaluating state of the art in AI

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Multi-class confusion matrix library in Python

心理健康大模型 (LLM x Mental Health), Pre & Post-training & Dataset & Evaluation & Depoly & RAG, with InternLM / Qwen / Baichuan / DeepSeek / Mixtral / LLama / GLM series models

Evaluation code for various unsupervised automated metrics for Natural Language Generation.

Building blocks for rapid development of GenAI applications