DRAEM MODEL - Heatmap shows everything is detected as anomaly

[Paucaster3]

Hi, I've been performing some tests with DRAEM model, as it is one of the best performing according to Deep Industrial Image Anomaly Detection: A Survey.

When predicting some tests images (with synthetic defects, as I don't have real samples), the resulting heatmap indicates that the entire image is abnormal.

I'm trying to understand the result, as it has no sense, as the images come from the same distribution. Am I missing something?

Training sample

Predicted sample

Training script

datamodule = Folder(
    name="container-dataset",
    root=DATASET_PATH,
    normal_dir="normal",
    abnormal_dir="abnormal",
    mask_dir="masks",
    normal_split_ratio=0.2,
    train_batch_size=BATCH_SIZE,
    eval_batch_size=BATCH_SIZE,
)

# Model
model = Draem(
    enable_sspcab=False,
    sspcab_lambda=0.1,
    anomaly_source_path=None,
    beta=(0.1, 1.0),
)

# Engine
engine = Engine(
    task=TaskType.SEGMENTATION,  # Since DRAEM performs segmentation for anomaly localization
    image_metrics=["AUPR", "AUROC"],  # Metrics for image-level evaluation
    pixel_metrics=["AUPR", "AUROC"],  # Metrics for pixel-level evaluation
    max_epochs=NUM_EPOCHS,
    accelerator="gpu",
    default_root_dir=RESULTS_PATH,
    log_every_n_steps=10,
)

# Train the model
engine.fit(model, datamodule=datamodule)

engine.export(model=model, export_type=ExportType.TORCH, export_root=RESULTS_PATH)

Prediction script

model = TorchInferencer(path_model)
   
metadata = {
    "image_threshold": 0.9, # threshold to label image as anomaly
    "pixel_threshold": 0.9, # threshold to label pixel as anomaly
    "task": "segmentation", # to return also boxes and box labels
}
result = model.predict(image_path, metadata=metadata)

[samet-akcay]

@Paucaster3, have you tried Engine.predict()? to see if you have consistent results.
There could be multiple reasons; (i) there could potentially be discrepancy between your lightning model and torch model that you are exporting. (ii) as you mentioned your synthetic anomalous are not distinct enough to draw a boundary between normal and abnormal, (iii) DRAEM may not be suitable for this use-case, and you might want to try some other models.

Have you tried any any of these?