SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation (NeurIPS 2022)

The repository contains official Pytorch implementations of training and evaluation codes and pre-trained models for SegNext.

For Jittor user, https://github.com/Jittor/JSeg is a jittor version.

The paper is in Here.

The code is based on MMSegmentaion v0.24.1.

This is Pytorch implementation of " SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation" in PSFusion.

Fast Test

1. Download the pre-trained model from SegNext, unzip SegNext and put res_MSRS and res_MFNet in . /
2. Download the pre-processed MSRS and MFNet dataset from MSRS and MFNet, respectively. Then, put them in ./datasets
3. Refer to the original project configuration running environment SegNext.
4. Running python tools/test.py --aug-test (The default configuration is for the MFNet dataset, if you need to test the MSRS dataset please modify the related configuration file.)

To Train

Please refer to the original SegNext to start your training.

For a fast start tutorial, you can also refer to: SegNext Guide.

Citation

If you find our repo useful for your research, please consider citing our paper:

@article{guo2022segnext,
  title={SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation},
  author={Guo, Meng-Hao and Lu, Cheng-Ze and Hou, Qibin and Liu, Zhengning and Cheng, Ming-Ming and Hu, Shi-Min},
  journal={arXiv preprint arXiv:2209.08575},
  year={2022}
}


@article{guo2022visual,
  title={Visual Attention Network},
  author={Guo, Meng-Hao and Lu, Cheng-Ze and Liu, Zheng-Ning and Cheng, Ming-Ming and Hu, Shi-Min},
  journal={arXiv preprint arXiv:2202.09741},
  year={2022}
}


@inproceedings{
    ham,
    title={Is Attention Better Than Matrix Decomposition?},
    author={Zhengyang Geng and Meng-Hao Guo and Hongxu Chen and Xia Li and Ke Wei and Zhouchen Lin},
    booktitle={International Conference on Learning Representations},
    year={2021},
}

Results

Notes: ImageNet Pre-trained models can be found in TsingHua Cloud.

Rank 1 on Pascal VOC dataset: Leaderboard

ADE20K

Method	Backbone	Pretrained	Iters	mIoU(ss/ms)	Params	FLOPs	Config	Download
SegNeXt	MSCAN-T	IN-1K	160K	41.1/42.2	4M	7G	config	TsingHua Cloud
SegNeXt	MSCAN-S	IN-1K	160K	44.3/45.8	14M	16G	config	TsingHua Cloud
SegNeXt	MSCAN-B	IN-1K	160K	48.5/49.9	28M	35G	config	TsingHua Cloud
SegNeXt	MSCAN-L	IN-1K	160K	51.0/52.1	49M	70G	config	TsingHua Cloud

Cityscapes

Method	Backbone	Pretrained	Iters	mIoU(ss/ms)	Params	FLOPs	Config	Download
SegNeXt	MSCAN-T	IN-1K	160K	79.8/81.4	4M	56G	config	TsingHua Cloud
SegNeXt	MSCAN-S	IN-1K	160K	81.3/82.7	14M	125G	config	TsingHua Cloud
SegNeXt	MSCAN-B	IN-1K	160K	82.6/83.8	28M	276G	config	TsingHua Cloud
SegNeXt	MSCAN-L	IN-1K	160K	83.2/83.9	49M	578G	config	TsingHua Cloud

Notes: In this scheme, The number of FLOPs (G) is calculated on the input size of 512 $\times$ 512 for ADE20K, 2048 $\times$ 1024 for Cityscapes by torchprofile (recommended, highly accurate and automatic MACs/FLOPs statistics).

Installation

Install the dependencies and download ADE20K according to the guidelines in MMSegmentation.

pip install timm
cd SegNeXt
python setup.py develop

Training

We use 8 GPUs for training by default. Run:

./tools/dist_train.sh /path/to/config 8

Evaluation

To evaluate the model, run:

./tools/dist_test.sh /path/to/config /path/to/checkpoint_file 8 --eval mIoU

FLOPs

Install torchprofile using

pip install torchprofile

To calculate FLOPs for a model, run:

bash tools/get_flops.py /path/to/config --shape 512 512

Contact

For technical problem, please create an issue.

If you have any private question, please feel free to contact me via gmh20@mails.tsinghua.edu.cn.

Acknowledgment

Our implementation is mainly based on mmsegmentaion, Segformer and Enjoy-Hamburger. Thanks for their authors.

LICENSE

This repo is under the Apache-2.0 license. For commercial use, please contact the authors.