移动paddle_detection
This commit is contained in:
@@ -0,0 +1,50 @@
|
||||
# Keypoint Inference Benchmark
|
||||
|
||||
## Benchmark on Server
|
||||
We tested benchmarks in different runtime environments。 See the table below for details.
|
||||
|
||||
| Model | CPU + MKLDNN (thread=1) | CPU + MKLDNN (thread=4) | GPU | TensorRT (FP32) | TensorRT (FP16) |
|
||||
| :------------------------ | :------: | :------: | :-----: | :---: | :---: |
|
||||
| LiteHRNet-18-256x192 | 88.8 ms | 40.7 ms | 4.4 ms | 2.0 ms | 1.8 ms |
|
||||
| LiteHRNet-18-384x288 | 188.0 ms | 79.3 ms | 4.8 ms | 3.6 ms | 3.2 ms |
|
||||
| LiteHRNet-30-256x192 | 148.4 ms | 69.0 ms | 7.1 ms | 3.1 ms | 2.8 ms |
|
||||
| LiteHRNet-30-384x288 | 309.8 ms | 133.5 ms | 8.2 ms | 6.0 ms | 5.3 ms |
|
||||
| PP-TinyPose-128x96 | 25.2 ms | 14.1 ms | 2.7 ms | 0.9 ms | 0.8 ms |
|
||||
| PP-TinyPose-256x192 | 82.4 ms | 36.1 ms | 3.0 ms | 1.5 ms | 1.1 ms |
|
||||
|
||||
**Notes:**
|
||||
- These tests above are based Python deployment.
|
||||
- The environment is NVIDIA T4 / PaddlePaddle(commit: 7df301f2fc0602745e40fa3a7c43ccedd41786ca) / CUDA10.1 / CUDNN7 / Python3.7 / TensorRT6.
|
||||
- The test is based on deploy/python/det_keypoint_unite_infer.py with image demo/000000014439.jpg. And input batch size for keypoint model is set to 8.
|
||||
- The time only includes inference time.
|
||||
|
||||
|
||||
| Model | CPU + MKLDNN (thread=1) | CPU + MKLDNN (thread=4) | GPU | TensorRT (FP32) | TensorRT (FP16) |
|
||||
| :------------------------ | :------: | :------: | :-----: | :---: | :---: |
|
||||
| DARK_HRNet_w32-256x192 | 363.93 ms | 97.38 ms | 4.13 ms | 3.74 ms | 1.75 ms |
|
||||
| DARK_HRNet_w32-384x288 | 823.71 ms | 218.55 ms | 9.44 ms | 8.91 ms | 2.96 ms |
|
||||
| HRNet_w32-256x192 | 363.67 ms | 97.64 ms | 4.11 ms | 3.71 ms | 1.72 ms |
|
||||
| HRNet_w32-256x256_mpii | 485.56 ms | 131.48 ms | 4.81 ms | 4.26 ms | 2.00 ms |
|
||||
| HRNet_w32-384x288 | 822.73 ms | 215.48 ms | 9.40 ms | 8.81 ms | 2.97 ms |
|
||||
| PP-TinyPose-128x96 | 24.06 ms | 13.05 ms | 2.43 ms | 0.75 ms | 0.72 ms |
|
||||
| PP-TinyPose-256x192 | 82.73 ms | 36.25 ms | 2.57 ms | 1.38 ms | 1.15 ms |
|
||||
|
||||
|
||||
**Notes:**
|
||||
- These tests above are based C++ deployment.
|
||||
- The environment is NVIDIA T4 / PaddlePaddle(commit: 7df301f2fc0602745e40fa3a7c43ccedd41786ca) / CUDA10.1 / CUDNN7 / Python3.7 / TensorRT6.
|
||||
- The test is based on deploy/python/det_keypoint_unite_infer.py with image demo/000000014439.jpg. And input batch size for keypoint model is set to 8.
|
||||
- The time only includes inference time.
|
||||
|
||||
## Benchmark on Mobile
|
||||
We tested benchmarks on Kirin and Qualcomm Snapdragon devices. See the table below for details.
|
||||
|
||||
| Model | Kirin 980 (1-thread) | Kirin 980 (4-threads) | Qualcomm Snapdragon 845 (1-thread) | Qualcomm Snapdragon 845 (4-threads) | Qualcomm Snapdragon 660 (1-thread) | Qualcomm Snapdragon 660 (4-threads) |
|
||||
| :------------------------ | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| PicoDet-s-192x192 (det) | 14.85 ms | 5.45 ms | 17.50 ms | 7.56 ms | 80.08 ms | 27.36 ms |
|
||||
| PicoDet-s-320x320 (det) | 38.09 ms | 12.00 ms | 45.26 ms | 17.07 ms | 232.81 ms | 58.68 ms |
|
||||
| PP-TinyPose-128x96 (pose) | 12.03 ms | 5.09 ms | 13.14 ms | 6.73 ms | 71.87 ms | 20.04 ms |
|
||||
|
||||
**Notes:**
|
||||
- These tests above are based Paddle Lite deployment, and version is v2.10-rc.
|
||||
- The time only includes inference time.
|
||||
@@ -0,0 +1,298 @@
|
||||
简体中文 | [English](README_en.md)
|
||||
|
||||
# 关键点检测系列模型
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22989727/205551833-a891a790-73c6-43cb-84f9-91553e9ef27b.gif" width='800'/>
|
||||
</div>
|
||||
|
||||
## 目录
|
||||
|
||||
- [简介](#简介)
|
||||
- [模型推荐](#模型推荐)
|
||||
- [模型库](#模型库)
|
||||
- [快速开始](#快速开始)
|
||||
- [环境安装](#1环境安装)
|
||||
- [数据准备](#2数据准备)
|
||||
- [训练与测试](#3训练与测试)
|
||||
- [单卡训练](#单卡训练)
|
||||
- [多卡训练](#多卡训练)
|
||||
- [模型评估](#模型评估)
|
||||
- [模型预测](#模型预测)
|
||||
- [模型部署](#模型部署)
|
||||
- [Top-Down模型联合部署](#top-down模型联合部署)
|
||||
- [Bottom-Up模型独立部署](#bottom-up模型独立部署)
|
||||
- [与多目标跟踪联合部署](#与多目标跟踪模型fairmot联合部署)
|
||||
- [完整部署教程及Demo](#完整部署教程及Demo)
|
||||
- [自定义数据训练](#自定义数据训练)
|
||||
- [BenchMark](#benchmark)
|
||||
|
||||
## 简介
|
||||
|
||||
PaddleDetection 中的关键点检测部分紧跟最先进的算法，包括 Top-Down 和 Bottom-Up 两种方法，可以满足用户的不同需求。Top-Down 先检测对象，再检测特定关键点。Top-Down 模型的准确率会更高，但速度会随着对象数量的增加而变慢。不同的是，Bottom-Up 首先检测点，然后对这些点进行分组或连接以形成多个人体姿势实例。Bottom-Up 的速度是固定的，不会随着物体数量的增加而变慢，但精度会更低。
|
||||
|
||||
同时，PaddleDetection 提供针对移动端设备优化的自研实时关键点检测模型 [PP-TinyPose](./tiny_pose/README.md)。
|
||||
|
||||
## 模型推荐
|
||||
|
||||
### 移动端模型推荐
|
||||
|
||||
| 检测模型 | 关键点模型 | 输入尺寸 | COCO数据集精度 | 平均推理耗时 (FP16) | 参数量 （M） | Flops (G) | 模型权重 | Paddle-Lite部署模型（FP16) |
|
||||
| :----------------------------------------------------------- | :------------------------------------ | :------------------------------: | :-----------------------------: | :------------------------------------: | --------------------------- | :-------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_192_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_128x96.yml) | 检测：192x192<br>关键点：128x96 | 检测mAP：29.0<br>关键点AP：58.1 | 检测耗时：2.37ms<br>关键点耗时：3.27ms | 检测：1.18<br/>关键点：1.36 | 检测：0.35<br/>关键点：0.08 | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams)<br>[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16.nb)<br>[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16.nb) |
|
||||
| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_256x192.yml) | 检测：320x320<br>关键点：256x192 | 检测mAP：38.5<br>关键点AP：68.8 | 检测耗时：6.30ms<br>关键点耗时：8.33ms | 检测：1.18<br/>关键点：1.36 | 检测：0.97<br/>关键点：0.32 | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams)<br>[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16.nb)<br>[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16.nb) |
|
||||
|
||||
|
||||
*详细关于PP-TinyPose的使用请参考[文档](./tiny_pose/README.md)。
|
||||
|
||||
### 服务端模型推荐
|
||||
|
||||
| 检测模型 | 关键点模型 | 输入尺寸 | COCO数据集精度 | 参数量 （M） | Flops (G) | 模型权重 |
|
||||
| :----------------------------------------------------------- | :----------------------------------------- | :------------------------------: | :-----------------------------: | :----------------------: | :----------------------: | :----------------------------------------------------------: |
|
||||
| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_384x288.yml) | 检测：640x640<br>关键点：384x288 | 检测mAP：49.5<br>关键点AP：77.8 | 检测：54.6<br/>关键点：28.6 | 检测：115.8<br/>关键点：17.3 | [检测](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)<br>[关键点](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) |
|
||||
| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_256x192.yml) | 检测：640x640<br>关键点：256x192 | 检测mAP：49.5<br>关键点AP：76.9 | 检测：54.6<br/>关键点：28.6 | 检测：115.8<br/>关键点：7.68 | [检测](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)<br>[关键点](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) |
|
||||
|
||||
|
||||
## 模型库
|
||||
|
||||
COCO数据集
|
||||
|
||||
| 模型 | 方案 |输入尺寸 | AP(coco val) | 模型下载 | 配置文件 |
|
||||
| :---------------- | -------- | :----------: | :----------------------------------------------------------: | ----------------------------------------------------| ------- |
|
||||
| PETR_Res50 |One-Stage| 512 | 65.5 | [petr_res50.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/petr_resnet50_16x2_coco.pdparams) | [config](./petr/petr_resnet50_16x2_coco.yml) |
|
||||
| HigherHRNet-w32 |Bottom-Up| 512 | 67.1 | [higherhrnet_hrnet_w32_512.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512.yml) |
|
||||
| HigherHRNet-w32 | Bottom-Up| 640 | 68.3 | [higherhrnet_hrnet_w32_640.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_640.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_640.yml) |
|
||||
| HigherHRNet-w32+SWAHR |Bottom-Up| 512 | 68.9 | [higherhrnet_hrnet_w32_512_swahr.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512_swahr.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512_swahr.yml) |
|
||||
| HRNet-w32 | Top-Down| 256x192 | 76.9 | [hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) | [config](./hrnet/hrnet_w32_256x192.yml) |
|
||||
| HRNet-w32 |Top-Down| 384x288 | 77.8 | [hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) | [config](./hrnet/hrnet_w32_384x288.yml) |
|
||||
| HRNet-w32+DarkPose |Top-Down| 256x192 | 78.0 | [dark_hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_256x192.pdparams) | [config](./hrnet/dark_hrnet_w32_256x192.yml) |
|
||||
| HRNet-w32+DarkPose |Top-Down| 384x288 | 78.3 | [dark_hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_384x288.pdparams) | [config](./hrnet/dark_hrnet_w32_384x288.yml) |
|
||||
| WiderNaiveHRNet-18 | Top-Down|256x192 | 67.6(+DARK 68.4) | [wider_naive_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/wider_naive_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/wider_naive_hrnet_18_256x192_coco.yml) |
|
||||
| LiteHRNet-18 |Top-Down| 256x192 | 66.5 | [lite_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_256x192_coco.yml) |
|
||||
| LiteHRNet-18 |Top-Down| 384x288 | 69.7 | [lite_hrnet_18_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_384x288_coco.yml) |
|
||||
| LiteHRNet-30 | Top-Down|256x192 | 69.4 | [lite_hrnet_30_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_256x192_coco.yml) |
|
||||
| LiteHRNet-30 |Top-Down| 384x288 | 72.5 | [lite_hrnet_30_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_384x288_coco.yml) |
|
||||
|Vitpose_base_simple |Top-Down| 256x192 | 77.7 | [vitpose_base_simple_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/vitpose_base_simple_256x192_coco.pdparams) | [config](./vit_pose/vitpose_base_simple_coco_256x192.yml) |
|
||||
|Vitpose_base |Top-Down| 256x192 | 78.2 | [vitpose_base_coco_256x192.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/vitpose_base_coco_256x192.pdparams) | [config](./vit_pose/vitpose_base_coco_256x192.yml) |
|
||||
|
||||
备注： 1.Top-Down模型测试AP结果基于GroundTruth标注框
|
||||
2.vitpose训练用[MAE](https://bj.bcebos.com/v1/paddledet/models/keypoint/mae_pretrain_vit_base.pdparams)做为预训练模型
|
||||
|
||||
MPII数据集
|
||||
| 模型 | 方案| 输入尺寸 | PCKh(Mean) | PCKh(Mean@0.1) | 模型下载 | 配置文件 |
|
||||
| :---- | ---|----- | :--------: | :------------: | :----------------------------------------------------------: | -------------------------------------------- |
|
||||
| HRNet-w32 | Top-Down|256x256 | 90.6 | 38.5 | [hrnet_w32_256x256_mpii.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x256_mpii.pdparams) | [config](./hrnet/hrnet_w32_256x256_mpii.yml) |
|
||||
|
||||
场景模型
|
||||
| 模型 | 方案 | 输入尺寸 | 精度 | 预测速度 |模型权重 | 部署模型 | 说明|
|
||||
| :---- | ---|----- | :--------: | :--------: | :------------: |:------------: |:-------------------: |
|
||||
| HRNet-w32 + DarkPose | Top-Down|256x192 | AP: 87.1 (业务数据集)| 单人2.9ms |[下载链接](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[下载链接](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) | 针对摔倒场景特别优化，该模型应用于[PP-Human](../../deploy/pipeline/README.md) |
|
||||
|
||||
|
||||
我们同时推出了基于LiteHRNet（Top-Down）针对移动端设备优化的实时关键点检测模型[PP-TinyPose](./tiny_pose/README.md), 欢迎体验。
|
||||
|
||||
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1、环境安装
|
||||
|
||||
请参考PaddleDetection [安装文档](../../docs/tutorials/INSTALL_cn.md)正确安装PaddlePaddle和PaddleDetection即可。
|
||||
|
||||
### 2、数据准备
|
||||
|
||||
目前KeyPoint模型支持[COCO](https://cocodataset.org/#keypoints-2017)数据集和[MPII](http://human-pose.mpi-inf.mpg.de/#overview)数据集，数据集的准备方式请参考[关键点数据准备](../../docs/tutorials/data/PrepareKeypointDataSet.md)。
|
||||
|
||||
关于config配置文件内容说明请参考[关键点配置文件说明](../../docs/tutorials/KeyPointConfigGuide_cn.md)。
|
||||
|
||||
- 请注意，Top-Down方案使用检测框测试时，需要通过检测模型生成bbox.json文件。COCO val2017的检测结果可以参考[Detector having human AP of 56.4 on COCO val2017 dataset](https://paddledet.bj.bcebos.com/data/bbox.json)，下载后放在根目录（PaddleDetection）下，然后修改config配置文件中`use_gt_bbox: False`后生效。然后正常执行测试命令即可。
|
||||
|
||||
### 3、训练与测试
|
||||
|
||||
#### 单卡训练
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
```
|
||||
|
||||
#### 多卡训练
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
```
|
||||
|
||||
#### 模型评估
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
|
||||
#当只需要保存评估预测的结果时，可以通过设置save_prediction_only参数实现，评估预测结果默认保存在output/keypoints_results.json文件中
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml --save_prediction_only
|
||||
```
|
||||
|
||||
#### 模型预测
|
||||
|
||||
注意：top-down模型只支持单人截图预测，如需使用多人图，请使用[联合部署推理]方式。或者使用bottom-up模型。
|
||||
|
||||
```shell
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/infer.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=./output/higherhrnet_hrnet_w32_512/model_final.pdparams --infer_dir=../images/ --draw_threshold=0.5 --save_txt=True
|
||||
```
|
||||
|
||||
#### 模型部署
|
||||
|
||||
##### Top-Down模型联合部署
|
||||
|
||||
```shell
|
||||
#导出检测模型
|
||||
python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams
|
||||
|
||||
#导出关键点模型
|
||||
python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams
|
||||
|
||||
#detector 检测 + keypoint top-down模型联合部署（联合推理只支持top-down方式）
|
||||
python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_384x288/ --video_file=../video/xxx.mp4 --device=gpu
|
||||
```
|
||||
|
||||
##### Bottom-Up模型独立部署
|
||||
|
||||
```shell
|
||||
#导出模型
|
||||
python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams
|
||||
|
||||
#部署推理
|
||||
python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5
|
||||
```
|
||||
|
||||
##### 与多目标跟踪模型FairMOT联合部署
|
||||
|
||||
```shell
|
||||
#导出FairMOT跟踪模型
|
||||
python tools/export_model.py -c configs/mot/fairmot/fairmot_dla34_30e_1088x608.yml -o weights=https://paddledet.bj.bcebos.com/models/mot/fairmot_dla34_30e_1088x608.pdparams
|
||||
|
||||
#用导出的跟踪和关键点模型Python联合预测
|
||||
python deploy/python/mot_keypoint_unite_infer.py --mot_model_dir=output_inference/fairmot_dla34_30e_1088x608/ --keypoint_model_dir=output_inference/higherhrnet_hrnet_w32_512/ --video_file={your video name}.mp4 --device=GPU
|
||||
```
|
||||
|
||||
**注意:**
|
||||
跟踪模型导出教程请参考[文档](../mot/README.md)。
|
||||
|
||||
### 完整部署教程及Demo
|
||||
|
||||
|
||||
我们提供了PaddleInference(服务器端)、PaddleLite(移动端)、第三方部署(MNN、OpenVino)支持。无需依赖训练代码，deploy文件夹下相应文件夹提供独立完整部署代码。 详见 [部署文档](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/README.md)介绍。
|
||||
|
||||
## 自定义数据训练
|
||||
|
||||
我们以[tinypose_256x192](./tiny_pose/README.md)为例来说明对于自定义数据如何修改：
|
||||
|
||||
#### 1、配置文件[tinypose_256x192.yml](../../configs/keypoint/tiny_pose/tinypose_256x192.yml)
|
||||
|
||||
基本的修改内容及其含义如下：
|
||||
|
||||
```
|
||||
num_joints: &num_joints 17 #自定义数据的关键点数量
|
||||
train_height: &train_height 256 #训练图片尺寸-高度h
|
||||
train_width: &train_width 192 #训练图片尺寸-宽度w
|
||||
hmsize: &hmsize [48, 64] #对应训练尺寸的输出尺寸，这里是输入[w,h]的1/4
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]] #关键点定义中左右对称的关键点，用于flip增强。若没有对称结构在 TrainReader 的 RandomFlipHalfBodyTransform 一栏中 flip_pairs 后面加一行 "flip: False"（注意缩紧对齐）
|
||||
num_joints_half_body: 8 #半身关键点数量，用于半身增强
|
||||
prob_half_body: 0.3 #半身增强实现概率，若不需要则修改为0
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] #上半身对应关键点id，用于半身增强中获取上半身对应的关键点。
|
||||
```
|
||||
|
||||
上述是自定义数据时所需要的修改部分，完整的配置及含义说明可参考文件：[关键点配置文件说明](../../docs/tutorials/KeyPointConfigGuide_cn.md)。
|
||||
|
||||
#### 2、其他代码修改（影响测试、可视化）
|
||||
- keypoint_utils.py中的sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62, .62, 1.07, 1.07,.87, .87, .89, .89]) / 10.0，表示每个关键点的确定范围方差，根据实际关键点可信区域设置，区域精确的一般0.25-0.5，例如眼睛。区域范围大的一般0.5-1.0，例如肩膀。若不确定建议0.75。
|
||||
- visualizer.py中的draw_pose函数中的EDGES，表示可视化时关键点之间的连接线关系。
|
||||
- pycocotools工具中的sigmas，同第一个keypoint_utils.py中的设置。用于coco指标评估时计算。
|
||||
|
||||
#### 3、数据准备注意
|
||||
- 训练数据请按coco数据格式处理。需要包括关键点[Nx3]、检测框[N]标注。
|
||||
- 请注意area>0，area=0时数据在训练时会被过滤掉。此外，由于COCO的评估机制，area较小的数据在评估时也会被过滤掉，我们建议在自定义数据时取`area = bbox_w * bbox_h`。
|
||||
|
||||
如有遗漏，欢迎反馈
|
||||
|
||||
|
||||
## 关键点稳定策略（仅适用于视频数据）
|
||||
使用关键点算法处理视频数据时，由于预测针对单帧图像进行，在视频结果上往往会有抖动的现象。在一些依靠精细化坐标的应用场景（例如健身计数、基于关键点的虚拟渲染等）上容易造成误检或体验不佳的问题。针对这个问题，在PaddleDetection关键点视频推理中加入了[OneEuro滤波器](http://www.lifl.fr/~casiez/publications/CHI2012-casiez.pdf)和EMA两种关键点稳定方式。实现将当前关键点坐标结果和历史关键点坐标结果结合计算，使得输出的点坐标更加稳定平滑。该功能同时支持在Python及C++推理中一键开启使用。
|
||||
|
||||
```bash
|
||||
# 使用Python推理
|
||||
python deploy/python/det_keypoint_unite_infer.py \
|
||||
--det_model_dir output_inference/picodet_s_320 \
|
||||
--keypoint_model_dir output_inference/tinypose_256x192 \
|
||||
--video_file test_video.mp4 --device gpu --smooth True
|
||||
|
||||
# 使用CPP推理
|
||||
./deploy/cpp/build/main --det_model_dir output_inference/picodet_s_320 \
|
||||
--keypoint_model_dir output_inference/tinypose_256x192 \
|
||||
--video_file test_video.mp4 --device gpu --smooth True
|
||||
```
|
||||
效果如下：
|
||||
|
||||
|
||||
|
||||
## BenchMark
|
||||
|
||||
我们给出了不同运行环境下的测试结果，供您在选用模型时参考。详细数据请见[Keypoint Inference Benchmark](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/keypoint/KeypointBenchmark.md)。
|
||||
|
||||
## 引用
|
||||
|
||||
```
|
||||
@inproceedings{cheng2020bottom,
|
||||
title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
|
||||
author={Bowen Cheng and Bin Xiao and Jingdong Wang and Honghui Shi and Thomas S. Huang and Lei Zhang},
|
||||
booktitle={CVPR},
|
||||
year={2020}
|
||||
}
|
||||
|
||||
@inproceedings{SunXLW19,
|
||||
title={Deep High-Resolution Representation Learning for Human Pose Estimation},
|
||||
author={Ke Sun and Bin Xiao and Dong Liu and Jingdong Wang},
|
||||
booktitle={CVPR},
|
||||
year={2019}
|
||||
}
|
||||
|
||||
@article{wang2019deep,
|
||||
title={Deep High-Resolution Representation Learning for Visual Recognition},
|
||||
author={Wang, Jingdong and Sun, Ke and Cheng, Tianheng and Jiang, Borui and Deng, Chaorui and Zhao, Yang and Liu, Dong and Mu, Yadong and Tan, Mingkui and Wang, Xinggang and Liu, Wenyu and Xiao, Bin},
|
||||
journal={TPAMI},
|
||||
year={2019}
|
||||
}
|
||||
|
||||
@InProceedings{Zhang_2020_CVPR,
|
||||
author = {Zhang, Feng and Zhu, Xiatian and Dai, Hanbin and Ye, Mao and Zhu, Ce},
|
||||
title = {Distribution-Aware Coordinate Representation for Human Pose Estimation},
|
||||
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
|
||||
month = {June},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
@inproceedings{Yulitehrnet21,
|
||||
title={Lite-HRNet: A Lightweight High-Resolution Network},
|
||||
author={Yu, Changqian and Xiao, Bin and Gao, Changxin and Yuan, Lu and Zhang, Lei and Sang, Nong and Wang, Jingdong},
|
||||
booktitle={CVPR},
|
||||
year={2021}
|
||||
}
|
||||
|
||||
@inproceedings{
|
||||
xu2022vitpose,
|
||||
title={ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation},
|
||||
author={Yufei Xu and Jing Zhang and Qiming Zhang and Dacheng Tao},
|
||||
booktitle={Advances in Neural Information Processing Systems},
|
||||
year={2022},
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,280 @@
|
||||
[简体中文](README.md) | English
|
||||
|
||||
# KeyPoint Detection Models
|
||||
|
||||
## Content
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Model Recommendation](#model-recommendation)
|
||||
- [Model Zoo](#model-zoo)
|
||||
- [Getting Start](#getting-start)
|
||||
- [Environmental Installation](#1environmental-installation)
|
||||
- [Dataset Preparation](#2dataset-preparation)
|
||||
- [Training and Testing](#3training-and-testing)
|
||||
- [Training on single GPU](#training-on-single-gpu)
|
||||
- [Training on multiple GPU](#training-on-multiple-gpu)
|
||||
- [Evaluation](#evaluation)
|
||||
- [Inference](#inference)
|
||||
- [Deploy Inference](#deploy-inference)
|
||||
- [Deployment for Top-Down models](#deployment-for-top-down-models)
|
||||
- [Deployment for Bottom-Up models](#deployment-for-bottom-up-models)
|
||||
- [Joint Inference with Multi-Object Tracking Model FairMOT](#joint-inference-with-multi-object-tracking-model-fairmot)
|
||||
- [Complete Deploy Instruction and Demo](#complete-deploy-instruction-and-demo)
|
||||
- [Train with custom data](#train-with-custom-data)
|
||||
- [BenchMark](#benchmark)
|
||||
|
||||
## Introduction
|
||||
|
||||
The keypoint detection part in PaddleDetection follows the state-of-the-art algorithm closely, including Top-Down and Bottom-Up methods, which can satisfy the different needs of users. Top-Down detects the object first and then detects the specific keypoint. Top-Down models will be more accurate, but slower as the number of objects increases. Differently, Bottom-Up detects the point first and then group or connect those points to form several instances of human pose. The speed of Bottom-Up is fixed, it won't slow down as the number of objects increases, but it will be less accurate.
|
||||
|
||||
At the same time, PaddleDetection provides a self-developed real-time keypoint detection model [PP-TinyPose](./tiny_pose/README_en.md) optimized for mobile devices.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22989727/205551833-a891a790-73c6-43cb-84f9-91553e9ef27b.gif" width='800'/>
|
||||
</div>
|
||||
|
||||
## Model Recommendation
|
||||
|
||||
### Mobile Terminal
|
||||
|
||||
|
||||
|
||||
|
||||
| Detection Model | Keypoint Model | Input Size | Accuracy of COCO | Average Inference Time (FP16) | Params (M) | Flops (G) | Model Weight | Paddle-Lite Inference Model（FP16) |
|
||||
| :----------------------------------------------------------- | :------------------------------------ | :-------------------------------------: | :--------------------------------------: | :-----------------------------------: | :--------------------------------: | :--------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_192_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_128x96.yml) | Detection：192x192<br>Keypoint：128x96 | Detection mAP：29.0<br>Keypoint AP：58.1 | Detection：2.37ms<br>Keypoint：3.27ms | Detection：1.18<br/>Keypoint：1.36 | Detection：0.35<br/>Keypoint：0.08 | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams)<br>[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16.nb)<br>[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16.nb) |
|
||||
| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_256x192.yml) | Detection：320x320<br>Keypoint：256x192 | Detection mAP：38.5<br>Keypoint AP：68.8 | Detection：6.30ms<br>Keypoint：8.33ms | Detection：1.18<br/>Keypoint：1.36 | Detection：0.97<br/>Keypoint：0.32 | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams)<br>[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16.nb)<br>[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16.nb) |
|
||||
|
||||
|
||||
*Specific documents of PP-TinyPose, please refer to [Document](./tiny_pose/README.md)。
|
||||
|
||||
### Terminal Server
|
||||
|
||||
|
||||
| Detection Model | Keypoint Model | Input Size | Accuracy of COCO | Params (M) | Flops (G) | Model Weight |
|
||||
| :----------------------------------------------------------- | :----------------------------------------- | :-------------------------------------: | :--------------------------------------: | :-----------------------------: | :-----------------------------: | :----------------------------------------------------------: |
|
||||
| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_384x288.yml) | Detection：640x640<br>Keypoint：384x288 | Detection mAP：49.5<br>Keypoint AP：77.8 | Detection：54.6<br/>Keypoint：28.6 | Detection：115.8<br/>Keypoint：17.3 | [Detection](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)<br>[Keypoint](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) |
|
||||
| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_256x192.yml) | Detection：640x640<br>Keypoint：256x192 | Detection mAP：49.5<br>Keypoint AP：76.9 | Detection：54.6<br/>Keypoint：28.6 | Detection：115.8<br/>Keypoint：7.68 | [Detection](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)<br>[Keypoint](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) |
|
||||
|
||||
|
||||
## Model Zoo
|
||||
|
||||
COCO Dataset
|
||||
| Model | Input Size | AP(coco val) | Model Download | Config File |
|
||||
| :---------------- | -------- | :----------: | :----------------------------------------------------------: | ----------------------------------------------------------- |
|
||||
| PETR_Res50 |One-Stage| 512 | 65.5 | [petr_res50.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/petr_resnet50_16x2_coco.pdparams) | [config](./petr/petr_resnet50_16x2_coco.yml) |
|
||||
| HigherHRNet-w32 | 512 | 67.1 | [higherhrnet_hrnet_w32_512.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512.yml) |
|
||||
| HigherHRNet-w32 | 640 | 68.3 | [higherhrnet_hrnet_w32_640.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_640.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_640.yml) |
|
||||
| HigherHRNet-w32+SWAHR | 512 | 68.9 | [higherhrnet_hrnet_w32_512_swahr.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512_swahr.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512_swahr.yml) |
|
||||
| HRNet-w32 | 256x192 | 76.9 | [hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) | [config](./hrnet/hrnet_w32_256x192.yml) |
|
||||
| HRNet-w32 | 384x288 | 77.8 | [hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) | [config](./hrnet/hrnet_w32_384x288.yml) |
|
||||
| HRNet-w32+DarkPose | 256x192 | 78.0 | [dark_hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_256x192.pdparams) | [config](./hrnet/dark_hrnet_w32_256x192.yml) |
|
||||
| HRNet-w32+DarkPose | 384x288 | 78.3 | [dark_hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_384x288.pdparams) | [config](./hrnet/dark_hrnet_w32_384x288.yml) |
|
||||
| WiderNaiveHRNet-18 | 256x192 | 67.6(+DARK 68.4) | [wider_naive_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/wider_naive_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/wider_naive_hrnet_18_256x192_coco.yml) |
|
||||
| LiteHRNet-18 | 256x192 | 66.5 | [lite_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_256x192_coco.yml) |
|
||||
| LiteHRNet-18 | 384x288 | 69.7 | [lite_hrnet_18_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_384x288_coco.yml) |
|
||||
| LiteHRNet-30 | 256x192 | 69.4 | [lite_hrnet_30_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_256x192_coco.yml) |
|
||||
| LiteHRNet-30 | 384x288 | 72.5 | [lite_hrnet_30_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_384x288_coco.yml) |
|
||||
| Vitpose_base_simple | 256x192 | 77.7 | [vitpose_base_simple_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/vitpose_base_simple_256x192_coco.pdparams) | [config](./vit_pose/vitpose_base_simple_coco_256x192.yml) |
|
||||
| Vitpose_base | 256x192 | 78.2 | [vitpose_base_coco_256x192.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/vitpose_base_coco_256x192.pdparams) | [config](./vit_pose/vitpose_base_coco_256x192.yml) |
|
||||
|
||||
Note：1.The AP results of Top-Down models are based on bounding boxes in GroundTruth.
|
||||
2.Vitpose training uses [MAE](https://bj.bcebos.com/v1/paddledet/models/keypoint/mae_pretrain_vit_base.pdparams) as the pre-training model
|
||||
|
||||
MPII Dataset
|
||||
| Model | Input Size | PCKh(Mean) | PCKh(Mean@0.1) | Model Download | Config File |
|
||||
| :---- | -------- | :--------: | :------------: | :----------------------------------------------------------: | -------------------------------------------- |
|
||||
| HRNet-w32 | 256x256 | 90.6 | 38.5 | [hrnet_w32_256x256_mpii.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x256_mpii.pdparams) | [config](./hrnet/hrnet_w32_256x256_mpii.yml) |
|
||||
|
||||
|
||||
Model for Scenes
|
||||
| Model | Strategy | Input Size | Precision | Inference Speed |Model Weights | Model Inference and Deployment | description|
|
||||
| :---- | ---|----- | :--------: | :-------: |:------------: |:------------: |:-------------------: |
|
||||
| HRNet-w32 + DarkPose | Top-Down|256x192 | AP: 87.1 (on internal dataset)| 2.9ms per person |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) | Especially optimized for fall scenarios, the model is applied to [PP-Human](../../deploy/pipeline/README.md) |
|
||||
|
||||
|
||||
We also release [PP-TinyPose](./tiny_pose/README_en.md), a real-time keypoint detection model optimized for mobile devices. Welcome to experience.
|
||||
|
||||
## Getting Start
|
||||
|
||||
### 1.Environmental Installation
|
||||
|
||||
Please refer to [PaddleDetection Installation Guide](../../docs/tutorials/INSTALL.md) to install PaddlePaddle and PaddleDetection correctly.
|
||||
|
||||
### 2.Dataset Preparation
|
||||
|
||||
Currently, KeyPoint Detection Models support [COCO](https://cocodataset.org/#keypoints-2017) and [MPII](http://human-pose.mpi-inf.mpg.de/#overview). Please refer to [Keypoint Dataset Preparation](../../docs/tutorials/data/PrepareKeypointDataSet_en.md) to prepare dataset.
|
||||
|
||||
About the description for config files, please refer to [Keypoint Config Guild](../../docs/tutorials/KeyPointConfigGuide_en.md).
|
||||
|
||||
- Note that, when testing by detected bounding boxes in Top-Down method, We should get `bbox.json` by a detection model. You can download the detected results for COCO val2017 [(Detector having human AP of 56.4 on COCO val2017 dataset)](https://paddledet.bj.bcebos.com/data/bbox.json) directly, put it at the root path (`PaddleDetection/`), and set `use_gt_bbox: False` in config file.
|
||||
|
||||
### 3.Training and Testing
|
||||
|
||||
#### Training on single GPU
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
```
|
||||
|
||||
#### Training on multiple GPU
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
```
|
||||
|
||||
#### Evaluation
|
||||
|
||||
```shell
|
||||
#COCO DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml
|
||||
|
||||
#MPII DataSet
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
|
||||
|
||||
#If you only need the prediction result, you can set --save_prediction_only. Then the result will be saved at output/keypoints_results.json by default.
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml --save_prediction_only
|
||||
```
|
||||
|
||||
#### Inference
|
||||
|
||||
Note：Top-down models only support inference for a cropped image with single person. If you want to do inference on image with several people, please see "joint inference by detection and keypoint". Or you can choose a Bottom-up model.
|
||||
|
||||
```shell
|
||||
CUDA_VISIBLE_DEVICES=0 python3 tools/infer.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=./output/higherhrnet_hrnet_w32_512/model_final.pdparams --infer_dir=../images/ --draw_threshold=0.5 --save_txt=True
|
||||
```
|
||||
|
||||
#### Deploy Inference
|
||||
|
||||
##### Deployment for Top-Down models
|
||||
|
||||
```shell
|
||||
#Export Detection Model
|
||||
python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams
|
||||
|
||||
|
||||
#Export Keypoint Model
|
||||
python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams
|
||||
|
||||
#Deployment for detector and keypoint, which is only for Top-Down models
|
||||
python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_384x288/ --video_file=../video/xxx.mp4 --device=gpu
|
||||
```
|
||||
|
||||
##### Deployment for Bottom-Up models
|
||||
|
||||
```shell
|
||||
#Export model
|
||||
python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams
|
||||
|
||||
|
||||
#Keypoint independent deployment, which is only for bottom-up models
|
||||
python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5
|
||||
```
|
||||
|
||||
##### Joint Inference with Multi-Object Tracking Model FairMOT
|
||||
|
||||
```shell
|
||||
#export FairMOT model
|
||||
python tools/export_model.py -c configs/mot/fairmot/fairmot_dla34_30e_1088x608.yml -o weights=https://paddledet.bj.bcebos.com/models/mot/fairmot_dla34_30e_1088x608.pdparams
|
||||
|
||||
#joint inference with Multi-Object Tracking model FairMOT
|
||||
python deploy/python/mot_keypoint_unite_infer.py --mot_model_dir=output_inference/fairmot_dla34_30e_1088x608/ --keypoint_model_dir=output_inference/higherhrnet_hrnet_w32_512/ --video_file={your video name}.mp4 --device=GPU
|
||||
```
|
||||
|
||||
**Note:**
|
||||
To export MOT model, please refer to [Here](../../configs/mot/README_en.md).
|
||||
|
||||
### Complete Deploy Instruction and Demo
|
||||
|
||||
We provide standalone deploy of PaddleInference(Server-GPU)、PaddleLite(mobile、ARM)、Third-Engine(MNN、OpenVino), which is independent of training codes。For detail, please click [Deploy-docs](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/README_en.md)。
|
||||
|
||||
## Train with custom data
|
||||
|
||||
We take an example of [tinypose_256x192](./tiny_pose/README_en.md) to show how to train with custom data.
|
||||
|
||||
#### 1、For configs [tinypose_256x192.yml](../../configs/keypoint/tiny_pose/tinypose_256x192.yml)
|
||||
|
||||
you may need to modify these for your job：
|
||||
|
||||
```
|
||||
num_joints: &num_joints 17 #the number of joints in your job
|
||||
train_height: &train_height 256 #the height of model input
|
||||
train_width: &train_width 192 #the width of model input
|
||||
hmsize: &hmsize [48, 64] #the shape of model output，usually 1/4 of [w,h]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]] #the correspondence between left and right keypoint id，used for flip transform。You can add an line(by "flip: False") behind of flip_pairs in RandomFlipHalfBodyTransform of TrainReader if you don't need it
|
||||
num_joints_half_body: 8 #The joint numbers of half body, used for half_body transform
|
||||
prob_half_body: 0.3 #The probability of half_body transform, set to 0 if you don't need it
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] #The joint ids of half(upper) body, used to get the upper joints in half_body transform
|
||||
```
|
||||
|
||||
For more configs, please refer to [KeyPointConfigGuide](../../docs/tutorials/KeyPointConfigGuide_en.md)。
|
||||
|
||||
#### 2、Others(used for test and visualization)
|
||||
- In keypoint_utils.py, please set: "sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62, .62, 1.07, 1.07,.87, .87, .89, .89]) / 10.0", the value indicate the variance of a joint locations，normally 0.25-0.5 means the location is highly accuracy，for example: eyes。0.5-1.0 means the location is not sure so much，for example: shoulder。0.75 is recommand if you not sure。
|
||||
- In visualizer.py, please set "EDGES" in draw_pose function，this indicate the line to show between joints for visualization。
|
||||
- In pycocotools you installed, please set "sigmas"，it is the same as that in keypoint_utils.py, but used for coco evaluation。
|
||||
|
||||
#### 3、Note for data preparation
|
||||
- The data should has the same format as Coco data, and the keypoints(Nx3) and bbox(N) should be annotated.
|
||||
- please set "area">0 in annotations files otherwise it will be skiped while training. Moreover, due to the evaluation mechanism of COCO, the data with small area may also be filtered out during evaluation. We recommend to set `area = bbox_w * bbox_h` when customizing your dataset.
|
||||
|
||||
|
||||
## BenchMark
|
||||
|
||||
We provide benchmarks in different runtime environments for your reference when choosing models. See [Keypoint Inference Benchmark](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/keypoint/KeypointBenchmark.md) for details.
|
||||
|
||||
## Reference
|
||||
|
||||
```
|
||||
@inproceedings{cheng2020bottom,
|
||||
title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
|
||||
author={Bowen Cheng and Bin Xiao and Jingdong Wang and Honghui Shi and Thomas S. Huang and Lei Zhang},
|
||||
booktitle={CVPR},
|
||||
year={2020}
|
||||
}
|
||||
|
||||
@inproceedings{SunXLW19,
|
||||
title={Deep High-Resolution Representation Learning for Human Pose Estimation},
|
||||
author={Ke Sun and Bin Xiao and Dong Liu and Jingdong Wang},
|
||||
booktitle={CVPR},
|
||||
year={2019}
|
||||
}
|
||||
|
||||
@article{wang2019deep,
|
||||
title={Deep High-Resolution Representation Learning for Visual Recognition},
|
||||
author={Wang, Jingdong and Sun, Ke and Cheng, Tianheng and Jiang, Borui and Deng, Chaorui and Zhao, Yang and Liu, Dong and Mu, Yadong and Tan, Mingkui and Wang, Xinggang and Liu, Wenyu and Xiao, Bin},
|
||||
journal={TPAMI},
|
||||
year={2019}
|
||||
}
|
||||
|
||||
@InProceedings{Zhang_2020_CVPR,
|
||||
author = {Zhang, Feng and Zhu, Xiatian and Dai, Hanbin and Ye, Mao and Zhu, Ce},
|
||||
title = {Distribution-Aware Coordinate Representation for Human Pose Estimation},
|
||||
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
|
||||
month = {June},
|
||||
year = {2020}
|
||||
}
|
||||
|
||||
@inproceedings{Yulitehrnet21,
|
||||
title={Lite-HRNet: A Lightweight High-Resolution Network},
|
||||
author={Yu, Changqian and Xiao, Bin and Gao, Changxin and Yuan, Lu and Zhang, Lei and Sang, Nong and Wang, Jingdong},
|
||||
booktitle={CVPR},
|
||||
year={2021}
|
||||
}
|
||||
|
||||
@inproceedings{
|
||||
xu2022vitpose,
|
||||
title={ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation},
|
||||
author={Yufei Xu and Jing Zhang and Qiming Zhang and Dacheng Tao},
|
||||
booktitle={Advances in Neural Information Processing Systems},
|
||||
year={2022},
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,139 @@
|
||||
use_gpu: true
|
||||
log_iter: 10
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/higherhrnet_hrnet_w32_512/model_final
|
||||
epoch: 300
|
||||
num_joints: &num_joints 17
|
||||
flip_perm: &flip_perm [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
|
||||
input_size: &input_size 512
|
||||
hm_size: &hm_size 128
|
||||
hm_size_2x: &hm_size_2x 256
|
||||
max_people: &max_people 30
|
||||
metric: COCO
|
||||
IouType: keypoints
|
||||
num_classes: 1
|
||||
|
||||
|
||||
#####model
|
||||
architecture: HigherHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
HigherHRNet:
|
||||
backbone: HRNet
|
||||
hrhrnet_head: HrHRNetHead
|
||||
post_process: HrHRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
eval_flip: true
|
||||
|
||||
HRNet:
|
||||
width: &width 32
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
HrHRNetHead:
|
||||
num_joints: *num_joints
|
||||
width: *width
|
||||
loss: HrHRNetLoss
|
||||
swahr: false
|
||||
|
||||
HrHRNetLoss:
|
||||
num_joints: *num_joints
|
||||
swahr: false
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.001
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [200, 260]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer: None
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
EvalDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
test_mode: true
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 8
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomAffine:
|
||||
max_degree: 30
|
||||
scale: [0.75, 1.5]
|
||||
max_shift: 0.2
|
||||
trainsize: [*input_size, *input_size]
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- KeyPointFlip:
|
||||
flip_prob: 0.5
|
||||
flip_permutation: *flip_perm
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- ToHeatmaps:
|
||||
num_joints: *num_joints
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
sigma: 2
|
||||
- TagGenerate:
|
||||
num_joints: *num_joints
|
||||
max_people: *max_people
|
||||
- NormalizePermute:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
batch_size: 20
|
||||
shuffle: true
|
||||
drop_last: true
|
||||
use_shared_memory: true
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
|
||||
TestReader:
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 10
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/higherhrnet_hrnet_w32_512_swahr/model_final
|
||||
epoch: 300
|
||||
num_joints: &num_joints 17
|
||||
flip_perm: &flip_perm [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
|
||||
input_size: &input_size 512
|
||||
hm_size: &hm_size 128
|
||||
hm_size_2x: &hm_size_2x 256
|
||||
max_people: &max_people 30
|
||||
metric: COCO
|
||||
IouType: keypoints
|
||||
num_classes: 1
|
||||
|
||||
|
||||
#####model
|
||||
architecture: HigherHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
HigherHRNet:
|
||||
backbone: HRNet
|
||||
hrhrnet_head: HrHRNetHead
|
||||
post_process: HrHRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
eval_flip: true
|
||||
|
||||
HRNet:
|
||||
width: &width 32
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
HrHRNetHead:
|
||||
num_joints: *num_joints
|
||||
width: *width
|
||||
loss: HrHRNetLoss
|
||||
swahr: true
|
||||
|
||||
HrHRNetLoss:
|
||||
num_joints: *num_joints
|
||||
swahr: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.001
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [200, 260]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer: None
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
EvalDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
test_mode: true
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 8
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomAffine:
|
||||
max_degree: 30
|
||||
scale: [0.75, 1.5]
|
||||
max_shift: 0.2
|
||||
trainsize: [*input_size, *input_size]
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- KeyPointFlip:
|
||||
flip_prob: 0.5
|
||||
flip_permutation: *flip_perm
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- ToHeatmaps:
|
||||
num_joints: *num_joints
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
sigma: 2
|
||||
- TagGenerate:
|
||||
num_joints: *num_joints
|
||||
max_people: *max_people
|
||||
- NormalizePermute:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
batch_size: 16
|
||||
shuffle: true
|
||||
drop_last: true
|
||||
use_shared_memory: true
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
|
||||
TestReader:
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,139 @@
|
||||
use_gpu: true
|
||||
log_iter: 10
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/higherhrnet_hrnet_w32_640/model_final
|
||||
epoch: 300
|
||||
num_joints: &num_joints 17
|
||||
flip_perm: &flip_perm [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
|
||||
input_size: &input_size 640
|
||||
hm_size: &hm_size 160
|
||||
hm_size_2x: &hm_size_2x 320
|
||||
max_people: &max_people 30
|
||||
metric: COCO
|
||||
IouType: keypoints
|
||||
num_classes: 1
|
||||
|
||||
|
||||
#####model
|
||||
architecture: HigherHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
HigherHRNet:
|
||||
backbone: HRNet
|
||||
hrhrnet_head: HrHRNetHead
|
||||
post_process: HrHRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
eval_flip: true
|
||||
|
||||
HRNet:
|
||||
width: &width 32
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
HrHRNetHead:
|
||||
num_joints: *num_joints
|
||||
width: *width
|
||||
loss: HrHRNetLoss
|
||||
swahr: false
|
||||
|
||||
HrHRNetLoss:
|
||||
num_joints: *num_joints
|
||||
swahr: false
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.001
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [200, 260]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer: None
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
EvalDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
test_mode: true
|
||||
return_bbox: False
|
||||
return_area: False
|
||||
return_class: False
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 8
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomAffine:
|
||||
max_degree: 30
|
||||
scale: [0.75, 1.5]
|
||||
max_shift: 0.2
|
||||
trainsize: [*input_size, *input_size]
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- KeyPointFlip:
|
||||
flip_prob: 0.5
|
||||
flip_permutation: *flip_perm
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
- ToHeatmaps:
|
||||
num_joints: *num_joints
|
||||
hmsize: [*hm_size, *hm_size_2x]
|
||||
sigma: 2
|
||||
- TagGenerate:
|
||||
num_joints: *num_joints
|
||||
max_people: *max_people
|
||||
- NormalizePermute:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
batch_size: 20
|
||||
shuffle: true
|
||||
drop_last: true
|
||||
use_shared_memory: true
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
|
||||
TestReader:
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- EvalAffine:
|
||||
size: *input_size
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,141 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w32_256x192/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 32
|
||||
loss: KeyPointMSELoss
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
bbox_file: bbox.json
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,145 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w32_384x288/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 384
|
||||
train_width: &train_width 288
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [72, 96]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 32
|
||||
loss: KeyPointMSELoss
|
||||
flip: true
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
bbox_file: bbox.json
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 32
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,141 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w48_256x192/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W48_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 48
|
||||
loss: KeyPointMSELoss
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
bbox_file: bbox.json
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,142 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w32_256x192/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 32
|
||||
loss: KeyPointMSELoss
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
bbox_file: bbox.json
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false #whether to fuse normalize layer into model while export model
|
||||
@@ -0,0 +1,132 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w32_256x256_mpii/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 16
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownMPIIEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 256
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [64, 64]
|
||||
flip_perm: &flip_perm [[0, 5], [1, 4], [2, 3], [10, 15], [11, 14], [12, 13]]
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 32
|
||||
loss: KeyPointMSELoss
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownMPIIDataset
|
||||
image_dir: images
|
||||
anno_path: annotations/mpii_train.json
|
||||
dataset_dir: dataset/mpii
|
||||
num_joints: *num_joints
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownMPIIDataset
|
||||
image_dir: images
|
||||
anno_path: annotations/mpii_val.json
|
||||
dataset_dir: dataset/mpii
|
||||
num_joints: *num_joints
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 4
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [7, 8, 9, 10, 11, 12, 13, 14, 15]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,142 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/hrnet_w32_384x288/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 384
|
||||
train_width: &train_width 288
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [72, 96]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/Trunc_HRNet_W32_C_pretrained.pdparams
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: HRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 32
|
||||
loss: KeyPointMSELoss
|
||||
flip: true
|
||||
|
||||
HRNet:
|
||||
width: *width
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
bbox_file: bbox.json
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/lite_hrnet_18_256x192_coco/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: false
|
||||
|
||||
LiteHRNet:
|
||||
network_type: lite_18
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/lite_hrnet_18_384x288_coco/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 384
|
||||
train_width: &train_width 288
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [72, 96]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: false
|
||||
|
||||
LiteHRNet:
|
||||
network_type: lite_18
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 3
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 32
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/lite_hrnet_30_256x192_coco/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: false
|
||||
|
||||
LiteHRNet:
|
||||
network_type: lite_30
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 4
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/lite_hrnet_30_384x288_coco/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 384
|
||||
train_width: &train_width 288
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [72, 96]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: false
|
||||
|
||||
LiteHRNet:
|
||||
network_type: lite_30
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 3
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 32
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,140 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/wider_naive_hrnet_18_256x192_coco/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: false
|
||||
|
||||
LiteHRNet:
|
||||
network_type: wider_naive
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.0
|
||||
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
- ToHeatmapsTopDown:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,254 @@
|
||||
use_gpu: true
|
||||
log_iter: 50
|
||||
save_dir: output
|
||||
snapshot_epoch: 1
|
||||
weights: output/petr_resnet50_16x2_coco/model_final
|
||||
epoch: 100
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: COCO
|
||||
num_classes: 1
|
||||
trainsize: &trainsize 512
|
||||
flip_perm: &flip_perm [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
|
||||
find_unused_parameters: False
|
||||
|
||||
#####model
|
||||
architecture: PETR
|
||||
pretrain_weights: https://bj.bcebos.com/v1/paddledet/models/pretrained/PETR_pretrained.pdparams
|
||||
|
||||
PETR:
|
||||
backbone:
|
||||
name: ResNet
|
||||
depth: 50
|
||||
variant: b
|
||||
norm_type: bn
|
||||
freeze_norm: True
|
||||
freeze_at: 0
|
||||
return_idx: [1,2,3]
|
||||
num_stages: 4
|
||||
lr_mult_list: [0.1, 0.1, 0.1, 0.1]
|
||||
neck:
|
||||
name: ChannelMapper
|
||||
in_channels: [512, 1024, 2048]
|
||||
kernel_size: 1
|
||||
out_channels: 256
|
||||
norm_type: "gn"
|
||||
norm_groups: 32
|
||||
act: None
|
||||
num_outs: 4
|
||||
bbox_head:
|
||||
name: PETRHead
|
||||
num_query: 300
|
||||
num_classes: 1 # only person
|
||||
in_channels: 2048
|
||||
sync_cls_avg_factor: true
|
||||
with_kpt_refine: true
|
||||
transformer:
|
||||
name: PETRTransformer
|
||||
as_two_stage: true
|
||||
encoder:
|
||||
name: TransformerEncoder
|
||||
encoder_layer:
|
||||
name: TransformerEncoderLayer
|
||||
d_model: 256
|
||||
attn:
|
||||
name: MSDeformableAttention
|
||||
embed_dim: 256
|
||||
num_heads: 8
|
||||
num_levels: 4
|
||||
num_points: 4
|
||||
dim_feedforward: 1024
|
||||
dropout: 0.1
|
||||
num_layers: 6
|
||||
decoder:
|
||||
name: PETR_TransformerDecoder
|
||||
num_layers: 3
|
||||
return_intermediate: true
|
||||
decoder_layer:
|
||||
name: PETR_TransformerDecoderLayer
|
||||
d_model: 256
|
||||
dim_feedforward: 1024
|
||||
dropout: 0.1
|
||||
self_attn:
|
||||
name: MultiHeadAttention
|
||||
embed_dim: 256
|
||||
num_heads: 8
|
||||
dropout: 0.1
|
||||
cross_attn:
|
||||
name: MultiScaleDeformablePoseAttention
|
||||
embed_dims: 256
|
||||
num_heads: 8
|
||||
num_levels: 4
|
||||
num_points: 17
|
||||
hm_encoder:
|
||||
name: TransformerEncoder
|
||||
encoder_layer:
|
||||
name: TransformerEncoderLayer
|
||||
d_model: 256
|
||||
attn:
|
||||
name: MSDeformableAttention
|
||||
embed_dim: 256
|
||||
num_heads: 8
|
||||
num_levels: 1
|
||||
num_points: 4
|
||||
dim_feedforward: 1024
|
||||
dropout: 0.1
|
||||
num_layers: 1
|
||||
refine_decoder:
|
||||
name: PETR_DeformableDetrTransformerDecoder
|
||||
num_layers: 2
|
||||
return_intermediate: true
|
||||
decoder_layer:
|
||||
name: PETR_TransformerDecoderLayer
|
||||
d_model: 256
|
||||
dim_feedforward: 1024
|
||||
dropout: 0.1
|
||||
self_attn:
|
||||
name: MultiHeadAttention
|
||||
embed_dim: 256
|
||||
num_heads: 8
|
||||
dropout: 0.1
|
||||
cross_attn:
|
||||
name: MSDeformableAttention
|
||||
embed_dim: 256
|
||||
num_levels: 4
|
||||
positional_encoding:
|
||||
name: PositionEmbedding
|
||||
num_pos_feats: 128
|
||||
normalize: true
|
||||
offset: -0.5
|
||||
loss_cls:
|
||||
name: Weighted_FocalLoss
|
||||
use_sigmoid: true
|
||||
gamma: 2.0
|
||||
alpha: 0.25
|
||||
loss_weight: 2.0
|
||||
reduction: "mean"
|
||||
loss_kpt:
|
||||
name: L1Loss
|
||||
loss_weight: 70.0
|
||||
loss_kpt_rpn:
|
||||
name: L1Loss
|
||||
loss_weight: 70.0
|
||||
loss_oks:
|
||||
name: OKSLoss
|
||||
loss_weight: 2.0
|
||||
loss_hm:
|
||||
name: CenterFocalLoss
|
||||
loss_weight: 4.0
|
||||
loss_kpt_refine:
|
||||
name: L1Loss
|
||||
loss_weight: 80.0
|
||||
loss_oks_refine:
|
||||
name: OKSLoss
|
||||
loss_weight: 3.0
|
||||
assigner:
|
||||
name: PoseHungarianAssigner
|
||||
cls_cost:
|
||||
name: FocalLossCost
|
||||
weight: 2.0
|
||||
kpt_cost:
|
||||
name: KptL1Cost
|
||||
weight: 70.0
|
||||
oks_cost:
|
||||
name: OksCost
|
||||
weight: 7.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [80]
|
||||
gamma: 0.1
|
||||
use_warmup: false
|
||||
# - !LinearWarmup
|
||||
# start_factor: 0.001
|
||||
# steps: 1000
|
||||
|
||||
OptimizerBuilder:
|
||||
clip_grad_by_norm: 0.1
|
||||
optimizer:
|
||||
type: AdamW
|
||||
regularizer:
|
||||
factor: 0.0001
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
return_mask: false
|
||||
|
||||
EvalDataset:
|
||||
!KeypointBottomUpCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
test_mode: true
|
||||
return_mask: false
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- PhotoMetricDistortion:
|
||||
brightness_delta: 32
|
||||
contrast_range: [0.5, 1.5]
|
||||
saturation_range: [0.5, 1.5]
|
||||
hue_delta: 18
|
||||
- KeyPointFlip:
|
||||
flip_prob: 0.5
|
||||
flip_permutation: *flip_perm
|
||||
- RandomAffine:
|
||||
max_degree: 30
|
||||
scale: [1.0, 1.0]
|
||||
max_shift: 0.
|
||||
trainsize: -1
|
||||
- RandomSelect: { transforms1: [ RandomShortSideRangeResize: { scales: [[400, 1400], [1400, 1400]]} ],
|
||||
transforms2: [
|
||||
RandomShortSideResize: { short_side_sizes: [ 400, 500, 600 ] },
|
||||
RandomSizeCrop: { min_size: 384, max_size: 600},
|
||||
RandomShortSideRangeResize: { scales: [[400, 1400], [1400, 1400]]} ]}
|
||||
batch_transforms:
|
||||
- NormalizeImage: {mean: *global_mean, std: *global_std, is_scale: True}
|
||||
- PadGT: {pad_img: True, minimum_gtnum: 1}
|
||||
- Permute: {}
|
||||
batch_size: 2
|
||||
shuffle: true
|
||||
drop_last: true
|
||||
use_shared_memory: true
|
||||
collate_batch: true
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- PETR_Resize: {img_scale: [[800, 1333]], keep_ratio: True}
|
||||
# - MultiscaleTestResize: {origin_target_size: [[800, 1333]], use_flip: false}
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
|
||||
TestReader:
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- EvalAffine: {size: 800}
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
@@ -0,0 +1,281 @@
|
||||
简体中文 | [English](README_en.md)
|
||||
|
||||
# PP-TinyPose
|
||||
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_demo.png"/>
|
||||
<center>图片来源：COCO2017开源数据集</center>
|
||||
</div>
|
||||
|
||||
## 最新动态
|
||||
- **2022.8.01：发布PP-TinyPose升级版。 在健身、舞蹈等场景的业务数据集端到端AP提升9.1**
|
||||
- 新增体育场景真实数据，复杂动作识别效果显著提升，覆盖侧身、卧躺、跳跃、高抬腿等非常规动作
|
||||
- 检测模型升级为[PP-PicoDet增强版](../../../configs/picodet/README.md)，在COCO数据集上精度提升3.1%
|
||||
- 关键点稳定性增强。新增滤波稳定方式，视频预测结果更加稳定平滑
|
||||
|
||||
|
||||
|
||||
## 简介
|
||||
PP-TinyPose是PaddleDetecion针对移动端设备优化的实时关键点检测模型，可流畅地在移动端设备上执行多人姿态估计任务。借助PaddleDetecion自研的优秀轻量级检测模型[PicoDet](../../picodet/README.md)，我们同时提供了特色的轻量级垂类行人检测模型。TinyPose的运行环境有以下依赖要求：
|
||||
- [PaddlePaddle](https://github.com/PaddlePaddle/Paddle)>=2.2
|
||||
|
||||
如希望在移动端部署，则还需要：
|
||||
- [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite)>=2.11
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_pipeline.png" width='800'/>
|
||||
</div>
|
||||
|
||||
## 部署案例
|
||||
|
||||
- [Android Fitness Demo](https://github.com/zhiboniu/pose_demo_android) 基于PP-TinyPose, 高效实现健身校准与计数功能。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22989727/205545098-fe6515af-3f1d-4303-bb4d-6e2141e42e2c.gif" width='636'/>
|
||||
</div>
|
||||
|
||||
- 欢迎扫码快速体验
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_app.png" width='220'/>
|
||||
</div>
|
||||
|
||||
|
||||
## 模型库
|
||||
|
||||
### Pipeline性能
|
||||
| 单人模型配置 | AP (业务数据集） | AP (COCO Val单人）| 单人耗时 (FP32) | 单人耗时 (FP16) |
|
||||
| :---------------------------------- | :------: | :------: | :---: | :---: |
|
||||
| PicoDet-S-Lcnet-Pedestrian-192\*192 + PP-TinyPose-128\*96 | 77.1 (+9.1) | 52.3 (+0.5) | 12.90 ms| 9.61 ms |
|
||||
|
||||
| 多人模型配置 | AP (业务数据集) | AP (COCO Val多人）| 6人耗时 (FP32) | 6人耗时 (FP16)|
|
||||
| :------------------------ | :-------: | :-------: | :---: | :---: |
|
||||
| PicoDet-S-Lcnet-Pedestrian-320\*320 + PP-TinyPose-128\*96 | 78.0 (+7.7) | 50.1 (-0.2) | 47.63 ms| 34.62 ms |
|
||||
|
||||
**说明**
|
||||
- 关键点检测模型的精度指标是基于对应行人检测模型检测得到的检测框。
|
||||
- 精度测试中去除了flip操作，且检测置信度阈值要求0.5。
|
||||
- 速度测试环境为qualcomm snapdragon 865，采用arm8下4线程推理。
|
||||
- Pipeline速度包含模型的预处理、推理及后处理部分。
|
||||
- 精度值的增量对比自历史版本中对应模型组合, 详情请见**历史版本-Pipeline性能**。
|
||||
- 精度测试中，为了公平比较，多人数据去除了6人以上（不含6人）的图像。
|
||||
|
||||
### 关键点检测模型
|
||||
| 模型 | 输入尺寸 | AP (业务数据集) | AP (COCO Val) | 参数量 | FLOPS |单人推理耗时 (FP32) | 单人推理耗时（FP16) | 配置文件 | 模型权重 | 预测部署模型 | Paddle-Lite部署模型（FP32) | Paddle-Lite部署模型（FP16) |
|
||||
| :---------- | :------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------------: | :-----------------: | :------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| PP-TinyPose | 128*96 | 84.3 | 58.4 | 1.32 M | 81.56 M | 4.57ms | 3.27ms | [Config](./tinypose_128x96.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_128x96.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_128x96.zip) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_128x96_fp32.nb) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_128x96_fp16.nb) |
|
||||
| PP-TinyPose | 256*192 | 91.0 | 68.3 | 1.32 M | 326.24M |14.07ms | 8.33ms | [Config](./tinypose_256x192.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192.zip) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192_fp32.nb) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192_fp16.nb) |
|
||||
|
||||
|
||||
### 行人检测模型
|
||||
| 模型 | 输入尺寸 | mAP (COCO Val-Person) | 参数量 | FLOPS | 平均推理耗时 (FP32) | 平均推理耗时 (FP16) | 配置文件 | 模型权重 | 预测部署模型 | Paddle-Lite部署模型（FP32) | Paddle-Lite部署模型（FP16) |
|
||||
| :------------------- | :------: | :------------: | :------------: | :------------: | :-----------------: | :-----------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| PicoDet-S-Lcnet-Pedestrian | 192*192 | 31.7 | 1.16 M | 170.03 M | 5.24ms | 3.66ms | [Config](../../picodet/application/pedestrian_detection/picodet_s_192_lcnet_pedestrian.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_192_lcnet_pedestrian.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_192_lcnet_pedestrian.zip) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_192_lcnet_pedestrian_fp32.nb) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_192_lcnet_pedestrian_fp16.nb) |
|
||||
| PicoDet-S-Lcnet-Pedestrian | 320*320 | 41.6 | 1.16 M | 472.07 M | 13.87ms | 8.94ms | [Config](../../picodet/application/pedestrian_detection/picodet_s_320_lcnet_pedestrian.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_320_lcnet_pedestrian.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_320_lcnet_pedestrian.zip) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_320_lcnet_pedestrian_fp32.nb) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_320_lcnet_pedestrian_fp16.nb) |
|
||||
|
||||
**说明**
|
||||
- 关键点检测模型与行人检测模型均使用`COCO train2017`, `AI Challenger trainset`以及采集的多姿态场景数据集作为训练集。关键点检测模型使用多姿态场景数据集作为测试集，行人检测模型采用`COCO instances val2017`作为测试集。
|
||||
- 关键点检测模型的精度指标所依赖的检测框为ground truth标注得到。
|
||||
- 关键点检测模型与行人检测模型均在4卡环境下训练，若实际训练环境需要改变GPU数量或batch size， 须参考[FAQ](../../../docs/tutorials/FAQ/README.md)对应调整学习率。
|
||||
- 推理速度测试环境为 Qualcomm Snapdragon 865，采用arm8下4线程推理得到。
|
||||
|
||||
## 历史版本
|
||||
|
||||
<details>
|
||||
<summary>2021版本</summary>
|
||||
|
||||
|
||||
### Pipeline性能
|
||||
| 单人模型配置 | AP (COCO Val 单人) | 单人耗时 (FP32) | 单人耗时 (FP16) |
|
||||
| :------------------------ | :------: | :---: | :---: |
|
||||
| PicoDet-S-Pedestrian-192\*192 + PP-TinyPose-128\*96 | 51.8 | 11.72 ms| 8.18 ms |
|
||||
| 其他优秀开源模型-192\*192 | 22.3 | 12.0 ms| - |
|
||||
|
||||
| 多人模型配置 | AP (COCO Val 多人) | 6人耗时 (FP32) | 6人耗时 (FP16)|
|
||||
| :------------------------ | :-------: | :---: | :---: |
|
||||
| PicoDet-S-Pedestrian-320\*320 + PP-TinyPose-128\*96 | 50.3 | 44.0 ms| 32.57 ms |
|
||||
| 其他优秀开源模型-256\*256 | 39.4 | 51.0 ms| - |
|
||||
|
||||
**说明**
|
||||
- 关键点检测模型的精度指标是基于对应行人检测模型检测得到的检测框。
|
||||
- 精度测试中去除了flip操作，且检测置信度阈值要求0.5。
|
||||
- 精度测试中，为了公平比较，多人数据去除了6人以上（不含6人）的图像。
|
||||
- 速度测试环境为qualcomm snapdragon 865，采用arm8下4线程、FP32推理得到。
|
||||
- Pipeline速度包含模型的预处理、推理及后处理部分。
|
||||
- 其他优秀开源模型的测试及部署方案，请参考[这里](https://github.com/zhiboniu/MoveNet-PaddleLite)。
|
||||
- 更多环境下的性能测试结果，请参考[Keypoint Inference Benchmark](../KeypointBenchmark.md)。
|
||||
|
||||
|
||||
### 关键点检测模型
|
||||
| 模型 | 输入尺寸 | AP (COCO Val) | 单人推理耗时 (FP32) | 单人推理耗时（FP16) | 配置文件 | 模型权重 | 预测部署模型 | Paddle-Lite部署模型（FP32) | Paddle-Lite部署模型（FP16) |
|
||||
| :---------- | :------: | :-----------: | :-----------------: | :-----------------: | :------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| PP-TinyPose | 128*96 | 58.1 | 4.57ms | 3.27ms | [Config](./tinypose_128x96.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.tar) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_lite.tar) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16_lite.tar) |
|
||||
| PP-TinyPose | 256*192 | 68.8 | 14.07ms | 8.33ms | [Config](./tinypose_256x192.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.tar) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_lite.tar) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16_lite.tar) |
|
||||
|
||||
### 行人检测模型
|
||||
| 模型 | 输入尺寸 | mAP (COCO Val-Person) | 平均推理耗时 (FP32) | 平均推理耗时 (FP16) | 配置文件 | 模型权重 | 预测部署模型 | Paddle-Lite部署模型（FP32) | Paddle-Lite部署模型（FP16) |
|
||||
| :------------------- | :------: | :------------: | :-----------------: | :-----------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
|
||||
| PicoDet-S-Pedestrian | 192*192 | 29.0 | 4.30ms | 2.37ms | [Config](../../picodet/legacy_model/application/pedestrian_detection/picodet_s_192_pedestrian.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.tar) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_lite.tar) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16_lite.tar) |
|
||||
| PicoDet-S-Pedestrian | 320*320 | 38.5 | 10.26ms | 6.30ms | [Config](../../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams) | [预测部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.tar) | [Lite部署模型](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_lite.tar) | [Lite部署模型(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16_lite.tar) |
|
||||
|
||||
|
||||
**说明**
|
||||
- 关键点检测模型与行人检测模型均使用`COCO train2017`和`AI Challenger trainset`作为训练集。关键点检测模型使用`COCO person keypoints val2017`作为测试集，行人检测模型采用`COCO instances val2017`作为测试集。
|
||||
- 关键点检测模型的精度指标所依赖的检测框为ground truth标注得到。
|
||||
- 关键点检测模型与行人检测模型均在4卡环境下训练，若实际训练环境需要改变GPU数量或batch size， 须参考[FAQ](../../../docs/tutorials/FAQ/README.md)对应调整学习率。
|
||||
- 推理速度测试环境为 Qualcomm Snapdragon 865，采用arm8下4线程推理得到。
|
||||
|
||||
|
||||
</details>
|
||||
|
||||
## 模型训练
|
||||
关键点检测模型与行人检测模型的训练集在`COCO`以外还扩充了[AI Challenger](https://arxiv.org/abs/1711.06475)数据集，各数据集关键点定义如下：
|
||||
```
|
||||
COCO keypoint Description:
|
||||
0: "Nose",
|
||||
1: "Left Eye",
|
||||
2: "Right Eye",
|
||||
3: "Left Ear",
|
||||
4: "Right Ear",
|
||||
5: "Left Shoulder,
|
||||
6: "Right Shoulder",
|
||||
7: "Left Elbow",
|
||||
8: "Right Elbow",
|
||||
9: "Left Wrist",
|
||||
10: "Right Wrist",
|
||||
11: "Left Hip",
|
||||
12: "Right Hip",
|
||||
13: "Left Knee",
|
||||
14: "Right Knee",
|
||||
15: "Left Ankle",
|
||||
16: "Right Ankle"
|
||||
|
||||
AI Challenger Description:
|
||||
0: "Right Shoulder",
|
||||
1: "Right Elbow",
|
||||
2: "Right Wrist",
|
||||
3: "Left Shoulder",
|
||||
4: "Left Elbow",
|
||||
5: "Left Wrist",
|
||||
6: "Right Hip",
|
||||
7: "Right Knee",
|
||||
8: "Right Ankle",
|
||||
9: "Left Hip",
|
||||
10: "Left Knee",
|
||||
11: "Left Ankle",
|
||||
12: "Head top",
|
||||
13: "Neck"
|
||||
```
|
||||
|
||||
由于两个数据集的关键点标注形式不同，我们将两个数据集的标注进行了对齐，仍然沿用COCO的标注形式，您可以下载[训练的参考列表](https://bj.bcebos.com/v1/paddledet/data/keypoint/aic_coco_train_cocoformat.json)并放在`dataset/`下使用。对齐两个数据集标注文件的主要处理如下：
|
||||
- `AI Challenger`关键点标注顺序调整至与COCO一致，统一是否标注/可见的标志位；
|
||||
- 舍弃了`AI Challenger`中特有的点位；将`AI Challenger`数据中`COCO`特有点位标记为未标注；
|
||||
- 重新排列了`image_id`与`annotation id`；
|
||||
利用转换为`COCO`形式的合并数据标注，执行模型训练：
|
||||
```bash
|
||||
# 关键点检测模型
|
||||
python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml
|
||||
|
||||
# 行人检测模型
|
||||
python3 -m paddle.distributed.launch tools/train.py -c configs/picodet/application/pedestrian_detection/picodet_s_320_pedestrian.yml
|
||||
```
|
||||
|
||||
## 部署流程
|
||||
### 实现部署预测
|
||||
1. 通过以下命令将训练得到的模型导出：
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/picodet/application/pedestrian_detection/picodet_s_192_pedestrian.yml --output_dir=outut_inference -o weights=output/picodet_s_192_pedestrian/model_final
|
||||
|
||||
python3 tools/export_model.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml --output_dir=outut_inference -o weights=output/tinypose_128x96/model_final
|
||||
```
|
||||
导出后的模型如：
|
||||
```
|
||||
picodet_s_192_pedestrian
|
||||
├── infer_cfg.yml
|
||||
├── model.pdiparams
|
||||
├── model.pdiparams.info
|
||||
└── model.pdmodel
|
||||
```
|
||||
您也可以直接下载模型库中提供的对应`预测部署模型`，分别获取得到行人检测模型和关键点检测模型的预测部署模型，解压即可。
|
||||
|
||||
2. 执行Python联合部署预测
|
||||
```bash
|
||||
# 预测一张图片
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --image_file={your image file} --device=GPU
|
||||
|
||||
# 预测多张图片
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --image_dir={dir of image file} --device=GPU
|
||||
|
||||
# 预测一个视频
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --video_file={your video file} --device=GPU
|
||||
```
|
||||
|
||||
3. 执行C++联合部署预测
|
||||
- 请先按照[C++端预测部署](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/deploy/cpp)，根据您的实际环境准备对应的`paddle_inference`库及相关依赖。
|
||||
- 我们提供了[一键编译脚本](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.3/deploy/cpp/scripts/build.sh)，您可以通过该脚本填写相关环境变量的位置，编译上述代码后，得到可执行文件。该过程中请保证`WITH_KEYPOINT=ON`.
|
||||
- 编译完成后，即可执行部署预测，例如：
|
||||
```bash
|
||||
# 预测一张图片
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --image_file={your image file} --device=GPU
|
||||
|
||||
# 预测多张图片
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --image_dir={dir of image file} --device=GPU
|
||||
|
||||
# 预测一个视频
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --video_file={your video file} --device=GPU
|
||||
```
|
||||
|
||||
### 实现移动端部署
|
||||
#### 直接使用我们提供的模型进行部署
|
||||
1. 下载模型库中提供的`Paddle-Lite部署模型`，分别获取得到行人检测模型和关键点检测模型的`.nb`格式文件。
|
||||
2. 准备Paddle-Lite运行环境, 可直接通过[PaddleLite预编译库下载](https://paddle-lite.readthedocs.io/zh/latest/quick_start/release_lib.html)获取预编译库，无需自行编译。如需要采用FP16推理，则需要下载FP16的预编译库。
|
||||
3. 编译模型运行代码，详细步骤见[Paddle-Lite端侧部署](../../../deploy/lite/README.md)。
|
||||
|
||||
#### 将训练的模型实现端侧部署
|
||||
如果您希望将自己训练的模型应用于部署，可以参考以下步骤：
|
||||
1. 将训练的模型导出
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/picodet/application/pedestrian_detection/picodet_s_192_pedestrian.yml --output_dir=outut_inference -o weights=output/picodet_s_192_pedestrian/model_final TestReader.fuse_normalize=true
|
||||
|
||||
python3 tools/export_model.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml --output_dir=outut_inference -o weights=output/tinypose_128x96/model_final TestReader.fuse_normalize=true
|
||||
```
|
||||
2. 转换为Lite模型（依赖[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite))
|
||||
|
||||
- 安装Paddle-Lite:
|
||||
```bash
|
||||
pip install paddlelite
|
||||
```
|
||||
- 执行以下步骤，以得到对应后缀为`.nb`的Paddle-Lite模型用于端侧部署:
|
||||
```
|
||||
# 1. 转换行人检测模型
|
||||
# FP32
|
||||
paddle_lite_opt --model_dir=inference_model/picodet_s_192_pedestrian --valid_targets=arm --optimize_out=picodet_s_192_pedestrian_fp32
|
||||
# FP16
|
||||
paddle_lite_opt --model_dir=inference_model/picodet_s_192_pedestrian --valid_targets=arm --optimize_out=picodet_s_192_pedestrian_fp16 --enable_fp16=true
|
||||
|
||||
# 2. 转换关键点检测模型
|
||||
# FP32
|
||||
paddle_lite_opt --model_dir=inference_model/tinypose_128x96 --valid_targets=arm --optimize_out=tinypose_128x96_fp32
|
||||
# FP16
|
||||
paddle_lite_opt --model_dir=inference_model/tinypose_128x96 --valid_targets=arm --optimize_out=tinypose_128x96_fp16 --enable_fp16=true
|
||||
```
|
||||
|
||||
3. 编译模型运行代码，详细步骤见[Paddle-Lite端侧部署](../../../deploy/lite/README.md)。
|
||||
|
||||
我们已提供包含数据预处理、模型推理及模型后处理的[全流程示例代码](../../../deploy/lite/)，可根据实际需求进行修改。
|
||||
|
||||
**注意**
|
||||
- 在导出模型时增加`TestReader.fuse_normalize=true`参数，可以将对图像的Normalize操作合并在模型中执行，从而实现加速。
|
||||
- FP16推理可实现更快的模型推理速度。若希望部署FP16模型，除模型转换步骤外，还需要编译支持FP16的Paddle-Lite预测库，详见[Paddle Lite 使用 ARM CPU 预测部署](https://paddle-lite.readthedocs.io/zh/latest/demo_guides/arm_cpu.html)。
|
||||
|
||||
## 关键点稳定策略（仅支持视频推理）
|
||||
请参考[关键点稳定策略](../README.md#关键点稳定策略仅适用于视频数据)。
|
||||
|
||||
## 优化策略
|
||||
TinyPose采用了以下策略来平衡模型的速度和精度表现：
|
||||
- 轻量级的姿态估计任务骨干网络，[wider naive Lite-HRNet](https://arxiv.org/abs/2104.06403)。
|
||||
- 更小的输入尺寸，以提升整体推理速度。
|
||||
- 加入Distribution-Aware coordinate Representation of Keypoints ([DARK](https://arxiv.org/abs/1910.06278))，以提升低分辨率热力图下模型的精度表现。
|
||||
- Unbiased Data Processing ([UDP](https://arxiv.org/abs/1911.07524))，使用无偏数据编解码提升模型精度。
|
||||
- Augmentation by Information Dropping ([AID](https://arxiv.org/abs/2008.07139v2))，通过添加信息丢失的数组增强，提升模型对关键点的定位能力。
|
||||
- FP16 推理, 实现更快的模型推理速度。
|
||||
@@ -0,0 +1,224 @@
|
||||
[简体中文](README.md) | English
|
||||
|
||||
# PP-TinyPose
|
||||
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_demo.png"/>
|
||||
<center>Image Source: COCO2017</center>
|
||||
</div>
|
||||
|
||||
## Introduction
|
||||
PP-TinyPose is a real-time keypoint detection model optimized by PaddleDetecion for mobile devices, which can smoothly run multi-person pose estimation tasks on mobile devices. With the excellent self-developed lightweight detection model [PicoDet](../../picodet/README.md), we also provide a lightweight pedestrian detection model. PP-TinyPose has the following dependency requirements:
|
||||
- [PaddlePaddle](https://github.com/PaddlePaddle/Paddle)>=2.2
|
||||
|
||||
If you want to deploy it on the mobile devives, you also need:
|
||||
- [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite)>=2.10
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_pipeline.png" width='800'/>
|
||||
</div>
|
||||
|
||||
## Deployment Case
|
||||
|
||||
- [Android Fitness Demo](https://github.com/zhiboniu/pose_demo_android) based on PP-TinyPose, which efficiently implements fitness calibration and counting.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/22989727/205545098-fe6515af-3f1d-4303-bb4d-6e2141e42e2c.gif" width='636'/>
|
||||
</div>
|
||||
|
||||
- Welcome to scan the QR code for quick experience.
|
||||
<div align="center">
|
||||
<img src="../../../docs/images/tinypose_app.png" width='220'/>
|
||||
</div>
|
||||
|
||||
|
||||
## Model Zoo
|
||||
### Keypoint Detection Model
|
||||
| Model | Input Size | AP (COCO Val) | Inference Time for Single Person (FP32)| Inference Time for Single Person（FP16) | Config | Model Weights | Deployment Model | Paddle-Lite Model（FP32) | Paddle-Lite Model（FP16)|
|
||||
| :------------------------ | :-------: | :------: | :------: |:---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| PP-TinyPose | 128*96 | 58.1 | 4.57ms | 3.27ms | [Config](./tinypose_128x96.yml) |[Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [Deployment Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.tar) | [Lite Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_lite.tar) | [Lite Model(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16_lite.tar) |
|
||||
| PP-TinyPose | 256*192 | 68.8 | 14.07ms | 8.33ms | [Config](./tinypose_256x192.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [Deployment Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.tar) | [Lite Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_lite.tar) | [Lite Model(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16_lite.tar) |
|
||||
|
||||
### Pedestrian Detection Model
|
||||
| Model | Input Size | mAP (COCO Val) | Average Inference Time (FP32)| Average Inference Time (FP16) | Config | Model Weights | Deployment Model | Paddle-Lite Model（FP32) | Paddle-Lite Model（FP16)|
|
||||
| :------------------------ | :-------: | :------: | :------: | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| PicoDet-S-Pedestrian | 192*192 | 29.0 | 4.30ms | 2.37ms | [Config](../../picodet/legacy_model/application/pedestrian_detection/picodet_s_192_pedestrian.yml) |[Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams) | [Deployment Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.tar) | [Lite Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_lite.tar) | [Lite Model(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16_lite.tar) |
|
||||
| PicoDet-S-Pedestrian | 320*320 | 38.5 | 10.26ms | 6.30ms | [Config](../../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams) | [Deployment Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.tar) | [Lite Model](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_lite.tar) | [Lite Model(FP16)](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16_lite.tar) |
|
||||
|
||||
|
||||
**Tips**
|
||||
- The keypoint detection model and pedestrian detection model are both trained on `COCO train2017` and `AI Challenger trainset`. The keypoint detection model is evaluated on `COCO person keypoints val2017`, and the pedestrian detection model is evaluated on `COCO instances val2017`.
|
||||
- The AP results of keypoint detection models are based on bounding boxes in GroundTruth.
|
||||
- Both keypoint detection model and pedestrian detection model are trained in a 4-GPU environment. In practice, if number of GPUs or batch size need to be changed according to the training environment, you should refer to [FAQ](../../../docs/tutorials/FAQ/README.md) to adjust the learning rate.
|
||||
- The inference time is tested on a Qualcomm Snapdragon 865, with 4 threads at arm8.
|
||||
|
||||
### Pipeline Performance
|
||||
| Model for Single-Pose | AP (COCO Val Single-Person) | Time for Single Person(FP32) | Time for Single Person(FP16) |
|
||||
| :------------------------ | :------: | :---: | :---: |
|
||||
| PicoDet-S-Pedestrian-192\*192 + PP-TinyPose-128\*96 | 51.8 | 11.72 ms| 8.18 ms |
|
||||
| Other opensource model-192\*192 | 22.3 | 12.0 ms| - |
|
||||
|
||||
| Model for Multi-Pose | AP (COCO Val Multi-Persons) | Time for Six Persons(FP32) | Time for Six Persons(FP16)|
|
||||
| :------------------------ | :-------: | :---: | :---: |
|
||||
| PicoDet-S-Pedestrian-320\*320 + PP-TinyPose-128\*96 | 50.3 | 44.0 ms| 32.57 ms |
|
||||
| Other opensource model-256\*256 | 39.4 | 51.0 ms| - |
|
||||
|
||||
**Tips**
|
||||
- The AP results of keypoint detection models are based on bounding boxes detected by corresponding detection model.
|
||||
- In accuracy evaluation, there is no flip, and threshold of bounding boxes is set to 0.5.
|
||||
- For fairness, in multi-persons test, we remove images with more than 6 people.
|
||||
- The inference time is tested on a Qualcomm Snapdragon 865, with 4 threads at arm8, FP32.
|
||||
- Pipeline time includes time for preprocess, inferece and postprocess.
|
||||
- About the deployment and testing for other opensource model, please refer to [Here](https://github.com/zhiboniu/MoveNet-PaddleLite).
|
||||
- For more performance data in other runtime environment, please refer to [Keypoint Inference Benchmark](../KeypointBenchmark.md).
|
||||
|
||||
## Model Training
|
||||
In addition to `COCO`, the trainset for keypoint detection model and pedestrian detection model also includes [AI Challenger](https://arxiv.org/abs/1711.06475). Keypoints of each dataset are defined as follows:
|
||||
```
|
||||
COCO keypoint Description:
|
||||
0: "Nose",
|
||||
1: "Left Eye",
|
||||
2: "Right Eye",
|
||||
3: "Left Ear",
|
||||
4: "Right Ear",
|
||||
5: "Left Shoulder,
|
||||
6: "Right Shoulder",
|
||||
7: "Left Elbow",
|
||||
8: "Right Elbow",
|
||||
9: "Left Wrist",
|
||||
10: "Right Wrist",
|
||||
11: "Left Hip",
|
||||
12: "Right Hip",
|
||||
13: "Left Knee",
|
||||
14: "Right Knee",
|
||||
15: "Left Ankle",
|
||||
16: "Right Ankle"
|
||||
|
||||
AI Challenger Description:
|
||||
0: "Right Shoulder",
|
||||
1: "Right Elbow",
|
||||
2: "Right Wrist",
|
||||
3: "Left Shoulder",
|
||||
4: "Left Elbow",
|
||||
5: "Left Wrist",
|
||||
6: "Right Hip",
|
||||
7: "Right Knee",
|
||||
8: "Right Ankle",
|
||||
9: "Left Hip",
|
||||
10: "Left Knee",
|
||||
11: "Left Ankle",
|
||||
12: "Head top",
|
||||
13: "Neck"
|
||||
```
|
||||
|
||||
Since the annatation format of these two datasets are different, we aligned their annotations to `COCO` format. You can download [Training List](https://bj.bcebos.com/v1/paddledet/data/keypoint/aic_coco_train_cocoformat.json) and put it at `dataset/`. To align these two datasets, we mainly did the following works:
|
||||
- Align the indexes of the `AI Challenger` keypoint to be consistent with `COCO` and unify the flags whether the keypoint is labeled/visible.
|
||||
- Discard the unique keypoints in `AI Challenger`. For keypoints not in this dataset but in `COCO`, set it to not labeled.
|
||||
- Rearranged `image_id` and `annotation id`.
|
||||
|
||||
Training with merged annotation file converted to `COCO` format:
|
||||
```bash
|
||||
# keypoint detection model
|
||||
python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml
|
||||
|
||||
# pedestrian detection model
|
||||
python3 -m paddle.distributed.launch tools/train.py -c configs/picodet/application/pedestrian_detection/picodet_s_320_pedestrian.yml
|
||||
```
|
||||
|
||||
## Model Deployment
|
||||
### Deploy Inference
|
||||
1. Export the trained model through the following command:
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/picodet/application/pedestrian_detection/picodet_s_192_pedestrian.yml --output_dir=outut_inference -o weights=output/picodet_s_192_pedestrian/model_final
|
||||
|
||||
python3 tools/export_model.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml --output_dir=outut_inference -o weights=output/tinypose_128x96/model_final
|
||||
```
|
||||
The exported model looks as:
|
||||
```
|
||||
picodet_s_192_pedestrian
|
||||
├── infer_cfg.yml
|
||||
├── model.pdiparams
|
||||
├── model.pdiparams.info
|
||||
└── model.pdmodel
|
||||
```
|
||||
You can also download `Deployment Model` from `Model Zoo` directly. And obtain the deployment models of pedestrian detection model and keypoint detection model, then unzip them.
|
||||
|
||||
2. Python joint inference by detection and keypoint
|
||||
```bash
|
||||
# inference for one image
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --image_file={your image file} --device=GPU
|
||||
|
||||
# inference for several images
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --image_dir={dir of image file} --device=GPU
|
||||
|
||||
# inference for a video
|
||||
python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/picodet_s_320_pedestrian --keypoint_model_dir=output_inference/tinypose_128x96 --video_file={your video file} --device=GPU
|
||||
```
|
||||
|
||||
3. C++ joint inference by detection and keypoint
|
||||
- First, please refer to [C++ Deploy Inference](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/deploy/cpp), prepare the corresponding `paddle_inference` library and related dependencies according to your environment.
|
||||
- We provide [Compile Script](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.3/deploy/cpp/scripts/build.sh). You can fill the location of the relevant environment variables in this script and excute it to compile the above codes. you can get an executable file. Please ensure `WITH_KEYPOINT=ON` during this process.
|
||||
- After compilation, you can do inference like:
|
||||
```bash
|
||||
# inference for one image
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --image_file={your image file} --device=GPU
|
||||
|
||||
# inference for several images
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --image_dir={dir of image file} --device=GPU
|
||||
|
||||
# inference for a video
|
||||
./build/main --model_dir=output_inference/picodet_s_320_pedestrian --model_dir_keypoint=output_inference/tinypose_128x96 --video_file={your video file} --device=GPU
|
||||
```
|
||||
|
||||
### Deployment on Mobile Devices
|
||||
#### Deploy directly using models we provide
|
||||
1. Download `Lite Model` from `Model Zoo` directly. And get the `.nb` format files of pedestrian detection model and keypoint detection model.
|
||||
2. Prepare environment for Paddle-Lite, you can obtain precompiled libraries from [PaddleLite Precompiled Libraries](https://paddle-lite.readthedocs.io/zh/latest/quick_start/release_lib.html). If FP16 is needed, you should download [Precompiled Libraries for FP16](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10-rc/inference_lite_lib.android.armv8_clang_c++_static_with_extra_with_cv_with_fp16.tiny_publish_427e46.zip).
|
||||
3. Compile the code to run models. The detail can be seen in [Paddle-Lite Deployment on Mobile Devices](../../../deploy/lite/README.md).
|
||||
|
||||
#### Deployment self-trained models on Mobile Devices
|
||||
If you want to deploy self-trained models, you can refer to the following steps:
|
||||
1. Export the trained model
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/picodet/application/pedestrian_detection/picodet_s_192_pedestrian.yml --output_dir=outut_inference -o weights=output/picodet_s_192_pedestrian/model_final TestReader.fuse_normalize=true
|
||||
|
||||
python3 tools/export_model.py -c configs/keypoint/tiny_pose/tinypose_128x96.yml --output_dir=outut_inference -o weights=output/tinypose_128x96/model_final TestReader.fuse_normalize=true
|
||||
```
|
||||
2. Convert to Lite Model（rely on [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite))
|
||||
|
||||
- Install Paddle-Lite:
|
||||
```bash
|
||||
pip install paddlelite
|
||||
```
|
||||
- Run the following commands to obtain `.nb` format models of Paddle-Lite:
|
||||
```
|
||||
# 1. Convert pedestrian detection model
|
||||
# FP32
|
||||
paddle_lite_opt --model_dir=inference_model/picodet_s_192_pedestrian --valid_targets=arm --optimize_out=picodet_s_192_pedestrian_fp32
|
||||
# FP16
|
||||
paddle_lite_opt --model_dir=inference_model/picodet_s_192_pedestrian --valid_targets=arm --optimize_out=picodet_s_192_pedestrian_fp16 --enable_fp16=true
|
||||
|
||||
# 2. keypoint detection model
|
||||
# FP32
|
||||
paddle_lite_opt --model_dir=inference_model/tinypose_128x96 --valid_targets=arm --optimize_out=tinypose_128x96_fp32
|
||||
# FP16
|
||||
paddle_lite_opt --model_dir=inference_model/tinypose_128x96 --valid_targets=arm --optimize_out=tinypose_128x96_fp16 --enable_fp16=true
|
||||
```
|
||||
|
||||
3. Compile the code to run models. The detail can be seen in [Paddle-Lite Deployment on Mobile Devices](../../../deploy/lite/README.md).
|
||||
|
||||
We provide [Example Code](../../../deploy/lite/) including data preprocessing, inferece and postpreocess. You can modify the codes according to your actual needs.
|
||||
|
||||
**Note:**
|
||||
- Add `TestReader.fuse_normalize=true` during the step of exporting model. The Normalize operation for the image will be executed in the model, which can achieve acceleration.
|
||||
- With FP16, we can get a faster inference speed. If you want to deploy the FP16 model, in addition to the model conversion step, you also need to compile the Paddle-Lite prediction library that supports FP16. The detail is in [Paddle Lite Deployment on ARM CPU](https://paddle-lite.readthedocs.io/zh/latest/demo_guides/arm_cpu.html).
|
||||
|
||||
## Optimization Strategies
|
||||
TinyPose adopts the following strategies to balance the speed and accuracy of the model:
|
||||
- Lightweight backbone network for pose estimation, [wider naive Lite-HRNet](https://arxiv.org/abs/2104.06403).
|
||||
- Smaller input size.
|
||||
- Distribution-Aware coordinate Representation of Keypoints ([DARK](https://arxiv.org/abs/1910.06278)), which can improve the accuracy of the model under the low-resolution heatmap.
|
||||
- Unbiased Data Processing ([UDP](https://arxiv.org/abs/1911.07524)).
|
||||
- Augmentation by Information Dropping ([AID](https://arxiv.org/abs/2008.07139v2)).
|
||||
- FP16 inference.
|
||||
@@ -0,0 +1,150 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/tinypose_128x96/model_final
|
||||
epoch: 420
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 128
|
||||
train_width: &train_width 96
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [24, 32]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
# AMP training
|
||||
init_loss_scaling: 32752
|
||||
master_grad: true
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: true
|
||||
|
||||
LiteHRNet:
|
||||
network_type: wider_naive
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.008
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [380, 410]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: ""
|
||||
anno_path: aic_coco_train_cocoformat.json
|
||||
dataset_dir: dataset
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.5
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- AugmentationbyInformantionDropping:
|
||||
prob_cutout: 0.5
|
||||
offset_factor: 0.05
|
||||
num_patch: 1
|
||||
trainsize: *trainsize
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 1
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 512
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false
|
||||
@@ -0,0 +1,147 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/tinypose_256x192/model_final
|
||||
epoch: 420
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: true
|
||||
|
||||
LiteHRNet:
|
||||
network_type: wider_naive
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [380, 410]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: ""
|
||||
anno_path: aic_coco_train_cocoformat.json
|
||||
dataset_dir: dataset
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
use_gt_bbox: True
|
||||
image_thre: 0.5
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.25
|
||||
rot: 30
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
- AugmentationbyInformantionDropping:
|
||||
prob_cutout: 0.5
|
||||
offset_factor: 0.05
|
||||
num_patch: 1
|
||||
trainsize: *trainsize
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 128
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 16
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false
|
||||
@@ -0,0 +1,145 @@
|
||||
use_gpu: true
|
||||
log_iter: 5
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/tinypose_256x256_hand/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 21
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOWholeBadyHandEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 256
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [64, 64]
|
||||
flip_perm: &flip_perm []
|
||||
|
||||
|
||||
#####model
|
||||
architecture: TopDownHRNet
|
||||
|
||||
TopDownHRNet:
|
||||
backbone: LiteHRNet
|
||||
post_process: HRNetPostProcess
|
||||
flip_perm: *flip_perm
|
||||
num_joints: *num_joints
|
||||
width: &width 40
|
||||
loss: KeyPointMSELoss
|
||||
use_dark: true
|
||||
|
||||
LiteHRNet:
|
||||
network_type: wider_naive
|
||||
freeze_at: -1
|
||||
freeze_norm: false
|
||||
return_idx: [0]
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
|
||||
#####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.002
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
milestones: [170, 200]
|
||||
gamma: 0.1
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
regularizer:
|
||||
factor: 0.0
|
||||
type: L2
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoWholeBodyHandDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/coco_wholebody_train_v1.0.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoWholeBodyHandDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/coco_wholebody_val_v1.0.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 2
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- TopDownRandomShiftBboxCenter:
|
||||
shift_prob: 0.3
|
||||
shift_factor: 0.16
|
||||
- TopDownRandomFlip:
|
||||
flip_prob: 0.5
|
||||
flip_perm: *flip_perm
|
||||
- TopDownGetRandomScaleRotation:
|
||||
rot_prob: 0.6
|
||||
rot_factor: 90
|
||||
scale_factor: 0.3
|
||||
# - AugmentationbyInformantionDropping:
|
||||
# prob_cutout: 0.5
|
||||
# offset_factor: 0.05
|
||||
# num_patch: 1
|
||||
# trainsize: *trainsize
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
- ToHeatmapsTopDown_DARK:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 128
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 128
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false
|
||||
@@ -0,0 +1,171 @@
|
||||
use_gpu: true
|
||||
log_iter: 50
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/vitpose_base_simple_coco_256x192/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: VitPose_TopDown
|
||||
pretrain_weights: https://bj.bcebos.com/v1/paddledet/models/keypoint/mae_pretrain_vit_base.pdparams
|
||||
VitPose_TopDown:
|
||||
backbone: ViT
|
||||
head: TopdownHeatmapSimpleHead
|
||||
post_process: VitPosePostProcess
|
||||
loss: KeyPointMSELoss
|
||||
flip_test: True
|
||||
|
||||
ViT:
|
||||
img_size: [256, 192]
|
||||
patch_size: 16
|
||||
embed_dim: 768
|
||||
depth: 12
|
||||
num_heads: 12
|
||||
ratio: 1
|
||||
mlp_ratio: 4
|
||||
qkv_bias: True
|
||||
drop_path_rate: 0.3
|
||||
epsilon: 0.000001
|
||||
|
||||
|
||||
TopdownHeatmapSimpleHead:
|
||||
in_channels: 768
|
||||
num_deconv_layers: 2
|
||||
num_deconv_filters: [256,256]
|
||||
num_deconv_kernels: [4,4]
|
||||
out_channels: 17
|
||||
shift_heatmap: False
|
||||
flip_pairs: *flip_perm
|
||||
extra: {final_conv_kernel: 1}
|
||||
|
||||
VitPosePostProcess:
|
||||
use_dark: True
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
gamma: 0.1
|
||||
milestones: [170, 200]
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
clip_grad_by_norm: 1.0
|
||||
optimizer:
|
||||
type: AdamWDL
|
||||
betas: [0.9, 0.999]
|
||||
weight_decay: 0.1
|
||||
num_layers: 12
|
||||
layer_decay: 0.75
|
||||
filter_bias_and_bn: True
|
||||
skip_decay_names: ['pos_embed','norm']
|
||||
set_param_lr_func: 'layerwise_lr_decay'
|
||||
|
||||
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
center_scale: 0.4
|
||||
|
||||
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
image_thre: 0.0
|
||||
use_gt_bbox: True
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 4
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
- ToHeatmapsTopDown_UDP:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: True
|
||||
drop_last: True
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false
|
||||
@@ -0,0 +1,164 @@
|
||||
use_gpu: true
|
||||
log_iter: 50
|
||||
save_dir: output
|
||||
snapshot_epoch: 10
|
||||
weights: output/vitpose_base_simple_coco_256x192/model_final
|
||||
epoch: 210
|
||||
num_joints: &num_joints 17
|
||||
pixel_std: &pixel_std 200
|
||||
metric: KeyPointTopDownCOCOEval
|
||||
num_classes: 1
|
||||
train_height: &train_height 256
|
||||
train_width: &train_width 192
|
||||
trainsize: &trainsize [*train_width, *train_height]
|
||||
hmsize: &hmsize [48, 64]
|
||||
flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
|
||||
|
||||
|
||||
#####model
|
||||
architecture: VitPose_TopDown
|
||||
pretrain_weights: https://bj.bcebos.com/v1/paddledet/models/keypoint/mae_pretrain_vit_base.pdparams
|
||||
VitPose_TopDown:
|
||||
backbone: ViT
|
||||
head: TopdownHeatmapSimpleHead
|
||||
post_process: VitPosePostProcess
|
||||
loss: KeyPointMSELoss
|
||||
flip_test: True
|
||||
|
||||
ViT:
|
||||
img_size: [256, 192]
|
||||
qkv_bias: True
|
||||
drop_path_rate: 0.3
|
||||
epsilon: 0.000001
|
||||
|
||||
|
||||
TopdownHeatmapSimpleHead:
|
||||
in_channels: 768
|
||||
num_deconv_layers: 0
|
||||
num_deconv_filters: []
|
||||
num_deconv_kernels: []
|
||||
upsample: 4
|
||||
shift_heatmap: False
|
||||
flip_pairs: *flip_perm
|
||||
extra: {final_conv_kernel: 3}
|
||||
|
||||
VitPosePostProcess:
|
||||
use_dark: True
|
||||
|
||||
KeyPointMSELoss:
|
||||
use_target_weight: true
|
||||
loss_scale: 1.0
|
||||
|
||||
####optimizer
|
||||
LearningRate:
|
||||
base_lr: 0.0005
|
||||
schedulers:
|
||||
- !PiecewiseDecay
|
||||
gamma: 0.1
|
||||
milestones: [170, 200]
|
||||
- !LinearWarmup
|
||||
start_factor: 0.001
|
||||
steps: 500
|
||||
|
||||
OptimizerBuilder:
|
||||
clip_grad_by_norm: 1.0
|
||||
optimizer:
|
||||
type: AdamWDL
|
||||
betas: [0.9, 0.999]
|
||||
weight_decay: 0.1
|
||||
num_layers: 12
|
||||
layer_decay: 0.75
|
||||
filter_bias_and_bn: True
|
||||
skip_decay_names: ['pos_embed','norm']
|
||||
set_param_lr_func: 'layerwise_lr_decay'
|
||||
|
||||
|
||||
|
||||
|
||||
#####data
|
||||
TrainDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: train2017
|
||||
anno_path: annotations/person_keypoints_train2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
center_scale: 0.4
|
||||
|
||||
|
||||
|
||||
EvalDataset:
|
||||
!KeypointTopDownCocoDataset
|
||||
image_dir: val2017
|
||||
anno_path: annotations/person_keypoints_val2017.json
|
||||
dataset_dir: dataset/coco
|
||||
num_joints: *num_joints
|
||||
trainsize: *trainsize
|
||||
pixel_std: *pixel_std
|
||||
image_thre: 0.0
|
||||
use_gt_bbox: True
|
||||
|
||||
TestDataset:
|
||||
!ImageFolder
|
||||
anno_path: dataset/coco/keypoint_imagelist.txt
|
||||
|
||||
worker_num: 4
|
||||
global_mean: &global_mean [0.485, 0.456, 0.406]
|
||||
global_std: &global_std [0.229, 0.224, 0.225]
|
||||
TrainReader:
|
||||
sample_transforms:
|
||||
- RandomFlipHalfBodyTransform:
|
||||
scale: 0.5
|
||||
rot: 40
|
||||
num_joints_half_body: 8
|
||||
prob_half_body: 0.3
|
||||
pixel_std: *pixel_std
|
||||
trainsize: *trainsize
|
||||
upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
|
||||
flip_pairs: *flip_perm
|
||||
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
- ToHeatmapsTopDown_UDP:
|
||||
hmsize: *hmsize
|
||||
sigma: 2
|
||||
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
shuffle: True
|
||||
drop_last: True
|
||||
|
||||
EvalReader:
|
||||
sample_transforms:
|
||||
- TopDownAffine:
|
||||
trainsize: *trainsize
|
||||
use_udp: true
|
||||
batch_transforms:
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 64
|
||||
|
||||
TestReader:
|
||||
inputs_def:
|
||||
image_shape: [3, *train_height, *train_width]
|
||||
sample_transforms:
|
||||
- Decode: {}
|
||||
- TopDownEvalAffine:
|
||||
trainsize: *trainsize
|
||||
- NormalizeImage:
|
||||
mean: *global_mean
|
||||
std: *global_std
|
||||
is_scale: true
|
||||
- Permute: {}
|
||||
batch_size: 1
|
||||
fuse_normalize: false
|
||||
Reference in New Issue
Block a user