AI Image Detector

A small, friendly open-source detector for AI-generated images. It is designed in the yt-dlp / rembg spirit: install it, run one command, get a probability and a reproducible report.

AI image detection is probabilistic. Treat the output as one signal, not as proof.
Other Quality Projects
AI Humanize Text:https://github.com/lynote-ai/humanize-text
AI Text Detector:https://github.com/lynote-ai/ai-text-detector

Model Choice

The default backend is UnivFD / UniversalFakeDetect: CLIP ViT-L/14 image features plus a tiny linear fake/real head. This is a strong practical default because the task-specific weight is tiny, the code path is understandable, and the CVPR 2023 paper showed good cross-generator generalization compared with older GAN-trained detectors.

This repo also ships hybrid, nonescape-mini, sentry-convnext-small, hybrid-plus, and ultra backends. hybrid blends UnivFD with a lightweight Hugging Face classifier, nonescape-mini and sentry-convnext-small adapt external open-source detectors, hybrid-plus ensembles our internal hybrid with Nonescape, and ultra adds Sentry on top. This has become the strongest practical route so far without training a new model from scratch.

The benchmark commands also support post-hoc threshold calibration objectives such as balanced_accuracy and f1. In practice, this has been one of the most effective low-risk levers for improving held-out performance.

Recent research has moved further. AIDE combines CLIP semantics with low-level frequency/noise features and reports gains on GenImage and AIGCDetectBenchmark. That is a good research target for a future backend, but UnivFD is currently the simplest robust default for an installable open-source tool.

Useful references:

• UniversalFakeDetect paper: https://openaccess.thecvf.com/content/CVPR2023/html/Ojha_Towards_Universal_Fake_Image_Detectors_That_Generalize_Across_Generative_Models_CVPR_2023_paper.html
• UniversalFakeDetect code: https://github.com/WisconsinAIVision/UniversalFakeDetect
• AIDE paper: https://arxiv.org/abs/2406.19435
• GenImage benchmark: https://github.com/GenImage-Dataset/GenImage
• Tiny-GenImage runnable subset: https://huggingface.co/datasets/TheKernel01/Tiny-GenImage
• CIFAKE benchmark paper: https://huggingface.co/papers/2303.14126

Install

Use Python 3.10+.

python -m venv .venv
source .venv/bin/activate
pip install -e .

Optional extras:

pip install -e '.[eval]'      # Hugging Face dataset benchmarks
pip install -e '.[hf]'        # generic Hugging Face image-classification backend
pip install -e '.[api]'       # FastAPI server
pip install -e '.[web]'       # Gradio UI
pip install -e '.[dev]'       # tests and linting

CLI Usage

Quick Start

Detect one image with the default backend:

aidetect detect image.jpg

Detect a folder recursively and save a CSV:

aidetect detect ./images --csv report.csv

Print JSON lines for scripting:

aidetect detect ./images --json

Which Backend Should I Use?

• univfd: simplest default, smallest moving parts, good baseline.
• sentry-convnext-small: strong single external detector.
• ultra: strongest current ensemble in this repo; best first choice when you want the highest practical accuracy.
• nonescape-mini: useful extra signal and part of the stronger ensembles.
• hf: generic Transformers image-classification path for standard Hugging Face checkpoints.

If you only want one recommendation:

aidetect detect image.jpg --backend ultra

If you want a simpler but still strong single model:

aidetect detect image.jpg --backend sentry-convnext-small

Use a Hugging Face image-classification model instead of UnivFD:

aidetect detect image.jpg --backend hf --hf-model capcheck/ai-image-detection

The generic hf backend expects a standard Transformers image-classification checkpoint. Some open-source detectors publish custom repos that need a dedicated adapter instead of --backend hf.

Use the hybrid backend:

aidetect detect image.jpg --backend hybrid --hybrid-univfd-weight 0.8

Use the external Nonescape Mini adapter directly:

aidetect detect image.jpg --backend nonescape-mini

Use the strongest current ensemble:

aidetect detect image.jpg --backend ultra

How To Read The Output

• probability_ai is the model's estimated likelihood that the image is AI-generated.
• label is the thresholded decision. By default, probability_ai >= 0.5 becomes ai.
• Treat borderline scores such as 0.45 to 0.55 as weak evidence, not proof.
• If a decision matters, compare at least two backends, especially sentry-convnext-small and ultra.

Python API

from aidetector import create_detector

detector = create_detector("univfd", device="auto")
result = detector.predict_path("image.jpg")
print(result.as_dict())

Web UI

pip install -e '.[web]'
aidetect serve

FastAPI

pip install -e '.[api]'
aidetect api --host 127.0.0.1 --port 8000

Then call:

curl -F "file=@image.jpg" http://127.0.0.1:8000/detect

Benchmarks

Small Local Smoke Test

To make the tradeoffs concrete, we ran the models in this repo against three local images in test_images/:

• ai-generated.png
• ai_retouched.png
• human.jpeg

Test images:

test_images/ai-generated.png

test_images/ai_retouched.png

test_images/human.jpeg

For this tiny smoke test, we treated the filenames as labels:

• ai-generated.png -> AI
• ai_retouched.png -> AI
• human.jpeg -> human

This is not a publishable benchmark. It is only a quick sanity check on a 3-image sample, but it is still useful for understanding how each backend tends to behave on obviously generated images versus edited or ambiguous ones.

Summary:

Per-backend results:

UnivFD / CLIP ViT-L/14

Accuracy on this sample: 1 / 3 = 0.333.

HF (capcheck/ai-image-detection)

Accuracy on this sample: 2 / 3 = 0.667.

Nonescape Mini

Accuracy on this sample: 2 / 3 = 0.667.

Sentry ConvNeXt Small

Accuracy on this sample: 2 / 3 = 0.667.

Ultra (hybrid-plus + sentry-convnext-small)

Accuracy on this sample: 2 / 3 = 0.667.

What this tells us:

• ultra is still the best default recommendation overall because it has the strongest larger-sample benchmark evidence in this repo.
• On this tiny 3-image test, sentry-convnext-small and ultra behaved almost the same.
• ai_retouched.png is the hard case in this sample. Multiple backends treated it more like a human-edited image than a fully synthetic one.
• human.jpeg is a good reminder that some detectors can overfire on real images. Here, hf and nonescape-mini both produced false positives.

Evaluate a GenImage-style folder where nature/ contains real images and ai/ contains generated images:

aidetect benchmark-folder /path/to/GenImage/Midjourney/val \
  --real-dir nature \
  --fake-dir ai \
  --output benchmarks/midjourney-val.json

Evaluate a Hugging Face dataset such as Tiny-GenImage:

pip install -e '.[eval]'
aidetect benchmark-hf TheKernel01/Tiny-GenImage \
  --split validation \
  --image-field image \
  --label-field label \
  --fake-label 1 \
  --max-samples 200 \
  --output benchmarks/tiny-genimage-univfd-200.json

The JSON report includes accuracy, balanced accuracy, precision, recall, F1, ROC AUC, confusion counts, a diagnostic threshold sweep, model metadata, dataset metadata, and per-image predictions.

For more defensible evaluation, calibrate a threshold on one split and evaluate on another:

aidetect benchmark-calibrated-folder /path/to/exported-folder \
  --backend univfd \
  --output benchmarks/univfd-calibrated.json

For multi-shard Tiny-GenImage evaluation with per-generator slices:

aidetect benchmark-tiny-genimage-local \
  /path/to/validation-00000-of-00004.parquet \
  /path/to/validation-00001-of-00004.parquet \
  /path/to/validation-00002-of-00004.parquet \
  /path/to/validation-00003-of-00004.parquet \
  --backend ultra \
  --optimize-metric f1 \
  --max-per-class-per-shard 100 \
  --output benchmarks/tiny-genimage-ultra-800-f1.json

If Hugging Face dataset metadata requests are flaky, you can work from a local Tiny-GenImage parquet shard:

aidetect prepare-tiny-genimage .cache/tiny-genimage-validation-200 \
  --local-parquet /path/to/validation-00000-of-00004.parquet \
  --max-per-class 100

aidetect benchmark-calibrated-folder .cache/tiny-genimage-validation-200 \
  --backend univfd \
  --real-dir real \
  --fake-dir ai \
  --output benchmarks/tiny-genimage-univfd-calibrated-200.json

Current local benchmark evidence is split into two levels.

Smoke benchmark on Tiny-GenImage validation shard data/validation-00000-of-00004.parquet, 20 real + 20 fake images:

Calibrated hold-out benchmark on the same shard family, exported as 100 real + 100 fake images and split deterministically into calibration/test sets:

Interpretation:

• The 40-image run is only a smoke test.
• The 200-image calibrated split is a stronger local benchmark because threshold selection happens on a separate calibration split before the test split is scored.
• It is still not a publication-grade claim. It is one shard, one deterministic split, and one local environment.
• These calibrated runs were executed on CPU in this workspace.

Current strongest local benchmark, calibrated on 4 Tiny-GenImage validation shards with up to 100 real + 100 fake images sampled per shard:

The important takeaway is that external detector ensembling helped more than any single internal threshold tweak. optimize=f1 still mattered, but the biggest jump came from combining our internal hybrid path with two external open-source detectors, first Nonescape and then Sentry.

Selected generator-vs-real slices from that same held-out split:

This is the honest picture: the strongest gains came from combining fast external detectors with our internal stack, then calibrating the final decision for f1. That materially improves overall balance and lifts weak generators, though performance is still generator-dependent and far from a universal guarantee.

Model Weights

On first use, the UnivFD backend downloads:

• CLIP ViT-L/14 OpenAI weights through open_clip_torch
• UniversalFakeDetect linear head from siddharthksah/deepsafe-weights/universalfakedetect/fc_weights.pth

You can also pass a local head checkpoint:

aidetect detect image.jpg --weight-path ./fc_weights.pth

Development

pip install -e '.[dev,eval,hf,api]'
pytest
ruff check .

Limitations

• No detector is universal. New generators, heavy recompression, screenshots, crops, edits, upscaling, and adversarial post-processing can change results.
• Benchmarks can overstate real-world reliability if the deployment data differs from the benchmark distribution.
• The tool currently detects whole-image synthetic likelihood. It does not localize edited regions.

Citation

If this helps your work, cite the original UnivFD paper:

@InProceedings{Ojha_2023_CVPR,
  author = {Ojha, Utkarsh and Li, Yuheng and Lee, Yong Jae},
  title = {Towards Universal Fake Image Detectors That Generalize Across Generative Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2023},
  pages = {24480-24489}
}