mmpose/projects/yolox_pose

YOLOX-Pose

This project implements a YOLOX-based human pose estimator, utilizing the approach outlined in YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss (CVPRW 2022). This pose estimator is lightweight and quick, making it well-suited for crowded scenes.

📌 For improved performance and compatibility, consider using YOLOX-Pose which is built into MMPose, which seamlessly integrates with MMPose's tools. To learn more about adopting YOLOX-Pose in your workflow, see the documentation: YOLOX-Pose.

Usage

Prerequisites

• Python 3.7 or higher

• PyTorch 1.6 or higher

• MMEngine v0.6.0 or higher

• MMCV v2.0.0rc4 or higher

• MMDetection v3.0.0rc6 or higher

• MMYOLO v0.5.0

• MMPose v1.0.0rc1 or higher

All the commands below rely on the correct configuration of PYTHONPATH, which should point to the project's directory so that Python can locate the module files. In yolox-pose/ root directory, run the following line to add the current directory to PYTHONPATH:

export PYTHONPATH=`pwd`:$PYTHONPATH

Inference

Users can apply YOLOX-Pose models to estimate human poses using the inferencer found in the MMPose core package. Use the command below:

python demo/inferencer_demo.py $INPUTS \
    --pose2d $CONFIG --pose2d-weights $CHECKPOINT --scope mmyolo \
    [--show] [--vis-out-dir $VIS_OUT_DIR] [--pred-out-dir $PRED_OUT_DIR]

For more information on using the inferencer, please see this document.

Here's an example code:

python demo/inferencer_demo.py ../../tests/data/coco/000000000785.jpg \
    --pose2d configs/yolox-pose_s_8xb32-300e_coco.py \
    --pose2d-weights https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth \
    --scope mmyolo --vis-out-dir vis_results

This will create an output image vis_results/000000000785.jpg, which appears like:

Training & Testing

Data Preparation

Prepare the COCO dataset according to the instruction.

Commands

To train with multiple GPUs:

bash tools/dist_train.sh $CONFIG 8 --amp

To train with slurm:

bash tools/slurm_train.sh $PARTITION $JOBNAME $CONFIG $WORKDIR --amp

To test with single GPU:

python tools/test.py $CONFIG $CHECKPOINT

To test with multiple GPUs:

bash tools/dist_test.sh $CONFIG $CHECKPOINT 8

To test with multiple GPUs by slurm:

bash tools/slurm_test.sh $PARTITION $JOBNAME $CONFIG $CHECKPOINT

Results

Results on COCO val2017

Model	Input Size	AP	AP50	AP75	AR	AR50	Download
YOLOX-tiny-Pose	416	0.518	0.799	0.545	0.566	0.841	model | log
YOLOX-s-Pose	640	0.632	0.875	0.692	0.676	0.907	model | log
YOLOX-m-Pose	640	0.685	0.897	0.753	0.727	0.925	model | log
YOLOX-l-Pose	640	0.706	0.907	0.775	0.747	0.934	model | log

We have only trained models with an input size of 640, as we couldn't replicate the performance enhancement mentioned in the paper when increasing the input size from 640 to 960. We warmly welcome any contributions if you can successfully reproduce the results from the paper!

NEW!

MMYOLO also supports YOLOX-Pose and achieves better performance. Their models are fully compatible with this project. Here are their results on COCO val2017:

Backbone	Size	Batch Size	AMP	RTMDet-Hyp	Mem (GB)	AP	Config	Download
YOLOX-tiny	416	8xb32	Yes	Yes	5.3	52.8	config	model | log
YOLOX-s	640	8xb32	Yes	Yes	10.7	63.7	config	model | log
YOLOX-m	640	8xb32	Yes	Yes	19.2	69.3	config	model | log
YOLOX-l	640	8xb32	Yes	Yes	30.3	71.1	config	model | log

Citation

If this project benefits your work, please kindly consider citing the original papers:

@inproceedings{maji2022yolo,
  title={YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss},
  author={Maji, Debapriya and Nagori, Soyeb and Mathew, Manu and Poddar, Deepak},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={2637--2646},
  year={2022}
}

@article{yolox2021,
  title={{YOLOX}: Exceeding YOLO Series in 2021},
  author={Ge, Zheng and Liu, Songtao and Wang, Feng and Li, Zeming and Sun, Jian},
  journal={arXiv preprint arXiv:2107.08430},
  year={2021}
}

Additionally, please cite our work as well:

@misc{mmpose2020,
    title={OpenMMLab Pose Estimation Toolbox and Benchmark},
    author={MMPose Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmpose}},
    year={2020}
}