MoCo: Momentum Contrast for Unsupervised Visual Representation Learning
This implements training of MoCo v2 on the Imagenet dataset, mainly modified from facebookresearch/moco.
MoCo v2 Detail
See moco directory.
Requirements
- Install PyTorch (pytorch.org)
pip install -r requirements.txtNote: pillow recommends installing a newer version. If the corresponding torchvision version cannot be installed directly, you can use the source code to install the corresponding version. The source code reference link: Suggestion the pillow is 9.1.0 and the torchvision is 0.6.0- Download the ImageNet dataset following the official PyTorch ImageNet training code.
Training
Unsupervised Training
To do unsupervised pre-training of a ResNet-50 model on ImageNet in an 8P machine, run:
bash test/train_moco_8p.sh --data_path=[path of imagenet]
（此步骤需要训练2周左右时间。由于FP16无法使模型收敛，必须使用FP32，并且maxpool有精度问题，需要在cpu完成计算。）
Linear Classification
With a pre-trained model, to train a supervised linear classifier on frozen features/weights in an 8P machine, run: （预训练模型需要训练两周，时间太长，可从此处下载预训练模型）
bash test/train_full_8p.sh --data_path=[path of imagenet]
Training Results
| Acc@1 | Npu_nums | Epochs | AMP_Type | FPS |
|---|---|---|---|---|
| 67.272 | 8 | 100 | O1 | 3869 |
公网地址说明
代码涉及公网地址参考 public_address_statement.md