Skip to content

Update reference scripts to use the "Batteries Included" utils #4281

Closed
@datumbox

Description

@datumbox

🚀 Feature

As part of the "Batteries Included" initiative (#3911) we are adding a number of new utils that can be used to produce SOTA results. Once those utils are landed, we should update our reference scripts to use them.

More specifically we need to:

  • Update the reference scripts to use PyTorch's new warmup schedulers.
    • The reference scripts should be updated to let users define the warmup_method (if any), the warmup_iters and the warmup_factor. The warmup scheduler should be chained with other existing schedulers.
    • The Object Detection recipe needs to be BC compatible and replace our custom linear warmup approach with the one from PyTorch.
    • Similarly the Video Classification recipe needs also to be BC compatible and replace our customer scheduler.
    • All other recipes (Classification and Segmentation), should be updated to optionally use warm up. The addition should be BC compatible and turned off by default.
  • Update the Classification reference to use PyTorch's new label smoothing implementation.
  • Update the Classification reference to use Mixup and Cutmix.
    • After implementing the Mixup and Cutmix augmentations, update the reference script of classification to use them on training. Adding Mixup and Cutmix #4379
    • The new mixup-alpha and cutmix-alpha arguments should have 0.0 default values to ensure BC.
  • Update the Classification reference to use EMA.
    • Add Exponential Moving Average support on the Classification scripts by extending the torch.optim.swa_utils.AveragedModel util and ensure the EMA model is evaluated at the end of each epoch.
    • Use a feature switch model-ema param to ensure BC and ensure we have a good model-ema-decay default value.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions