Why get_num_threads() and set_num_threads(1) in engine.py? Is it essential?

[BCMoo]

Hi there,

I'm wondering what's the function of these code? Can i remove them?

Thanks!

https://github.com/pytorch/vision/blob/dab475720f116c42fc80e437b25496dfc94d2a8a/references/detection/engine.py#L72:L74

vision/references/detection/engine.py

Line 108 in dab4757

torch.set_num_threads(n_threads)

[fmassa]

Hi,

They used to be important specially in multi-gpu setup because the paste_mask_in_image was multi-threaded and would run on the CPU, where each thread would try to maximize the number of CPUs used, leading to significant slowdown.
As of today, it seems that this part of the codebase is run on the GPU, so maybe this is not needed anymore, but it would need to be benchmarked.

[nickponline]

I would be happy to test it - would this be the corrected vision that runs on the GPU?

@torch.no_grad()
def evaluate(model, data_loader, device):
    device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')    
    model.eval()
    metric_logger = utils.MetricLogger(delimiter="  ")
    header = 'Test:'

    coco = get_coco_api_from_dataset(data_loader.dataset)
    iou_types = _get_iou_types(model)
    coco_evaluator = CocoEvaluator(coco, iou_types)

    for images, targets in metric_logger.log_every(data_loader, 100, header):
        images = list(img.to(device) for img in images)

        torch.cuda.synchronize()
        model_time = time.time()
        outputs = model(images)

        outputs = [{k: v.to(cpu_device) for k, v in t.items()} for t in outputs]
        model_time = time.time() - model_time

        res = {target["image_id"].item(): output for target, output in zip(targets, outputs)}
        evaluator_time = time.time()
        coco_evaluator.update(res)
        evaluator_time = time.time() - evaluator_time
        metric_logger.update(model_time=model_time, evaluator_time=evaluator_time)

    # gather the stats from all processes
    metric_logger.synchronize_between_processes()
    print("Averaged stats:", metric_logger)
    coco_evaluator.synchronize_between_processes()

    # accumulate predictions from all images
    coco_evaluator.accumulate()
    coco_evaluator.summarize()
    return coco_evaluator

[fmassa]

@nickponline the test would be to remove the set_num_threads(1) and compare the difference in evaluation speed when running evaluation on 8 GPUs using distributed evaluation