Echo/fcb_photo_review
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

更换文档检测模型

Browse Source
This commit is contained in:
liuyebo
2024-08-27 14:42:45 +08:00
parent aea6f19951
commit 1514e09c40
2072 changed files with 254336 additions and 4967 deletions
Download Patch File Download Diff File Expand all files Collapse all files

121
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/README.md Normal file
View File

@@ -0,0 +1,121 @@
[English](README.md) | 简体中文
# PaddleDetection RKNPU2部署示例
## 1. 说明
RKNPU2 提供了一个高性能接口来访问 Rockchip NPU，支持如下硬件的部署
- RK3566/RK3568
- RK3588/RK3588S
- RV1103/RV1106
在RKNPU2上已经通过测试的PaddleDetection模型如下:
- Picodet
- PPYOLOE(int8)
- YOLOV8
如果你需要查看详细的速度信息，请查看[RKNPU2模型速度一览表](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/rknpu2.md)
## 2. 使用预导出的模型列表
### ONNX模型转RKNN模型
为了方便大家使用，我们提供了python脚本，通过我们预配置的config文件，你将能够快速地转换ONNX模型到RKNN模型
```bash
python tools/rknpu2/export.py --config_path tools/rknpu2/config/picodet_s_416_coco_lcnet_unquantized.yaml \
--target_platform rk3588
```
### RKNN模型列表
为了方便大家测试，我们提供picodet和ppyoloe两个模型，解压后即可使用:
| 模型名称 | 下载地址 |
|-----------------------------|-----------------------------------------------------------------------------------|
| picodet_s_416_coco_lcnet | https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/picodet_s_416_coco_lcnet.zip |
| ppyoloe_plus_crn_s_80e_coco | https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/ppyoloe_plus_crn_s_80e_coco.zip |
## 3. 自行导出PaddleDetection部署模型以及转换模型
RKNPU部署模型前需要将Paddle模型转换成RKNN模型，具体步骤如下:
* Paddle动态图模型转换为ONNX模型，请参考[PaddleDetection导出模型](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/EXPORT_MODEL.md)
,注意在转换时请设置**export.nms=True**.
* ONNX模型转换RKNN模型的过程，请参考[转换文档](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/export.md)进行转换。
### 3.1 模型转换example
#### 3.1.1 注意点
PPDetection模型在RKNPU2上部署时要注意以下几点:
* 模型导出需要包含Decode
* 由于RKNPU2不支持NMS，因此输出节点必须裁剪至NMS之前
* 由于RKNPU2 Div算子的限制，模型的输出节点需要裁剪至Div算子之前
#### 3.1.2 Paddle模型转换为ONNX模型
由于Rockchip提供的rknn-toolkit2工具暂时不支持Paddle模型直接导出为RKNN模型，因此需要先将Paddle模型导出为ONNX模型，再将ONNX模型转为RKNN模型。
```bash
# 以Picodet为例
# 下载Paddle静态图模型并解压
wget https://paddledet.bj.bcebos.com/deploy/Inference/picodet_s_416_coco_lcnet.tar
tar xvf picodet_s_416_coco_lcnet.tar
# 静态图转ONNX模型，注意，这里的save_file请和压缩包名对齐
paddle2onnx --model_dir picodet_s_416_coco_lcnet \
--model_filename model.pdmodel \
--params_filename model.pdiparams \
--save_file picodet_s_416_coco_lcnet/picodet_s_416_coco_lcnet.onnx \
--enable_dev_version True
# 固定shape
python -m paddle2onnx.optimize --input_model picodet_s_416_coco_lcnet/picodet_s_416_coco_lcnet.onnx \
--output_model picodet_s_416_coco_lcnet/picodet_s_416_coco_lcnet.onnx \
--input_shape_dict "{'image':[1,3,416,416], 'scale_factor':[1,2]}"
```
#### 3.1.3 编写yaml文件
**修改normalize参数**
如果你需要在NPU上执行normalize操作，请根据你的模型配置normalize参数，例如:
```yaml
mean:
-
- 123.675
- 116.28
- 103.53
std:
-
- 58.395
- 57.12
- 57.375
```
**修改outputs参数**
由于Paddle2ONNX版本的不同，转换模型的输出节点名称也有所不同，请使用[Netron](https://netron.app)对模型进行可视化，并找到以下蓝色方框标记的NonMaxSuppression节点，红色方框的节点名称即为目标名称。
## 4. 模型可视化
例如，使用Netron可视化后，得到以下图片:
![](https://ai-studio-static-online.cdn.bcebos.com/8bce6b904a6b479e8b30da9f7c719fad57517ffb2f234aeca3b8ace0761754d5)
找到蓝色方框标记的NonMaxSuppression节点，可以看到红色方框标记的两个节点名称为p2o.Div.79和p2o.Concat.9,因此需要修改outputs参数，修改后如下:
```yaml
outputs_nodes:
- 'p2o.Mul.179'
- 'p2o.Concat.9'
```
## 5. 详细的部署示例
- [RKNN总体部署教程](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/rknpu2.md)
- [C++部署](cpp)
- [Python部署](python)

11
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/cpp/CMakeLists.txt Normal file
View File

@@ -0,0 +1,11 @@
CMAKE_MINIMUM_REQUIRED(VERSION 3.10)
project(infer_demo)
set(CMAKE_CXX_STANDARD 14)
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
include(${FASTDEPLOY_INSTALL_DIR}/FastDeployConfig.cmake)
include_directories(${FastDeploy_INCLUDE_DIRS})
add_executable(infer_demo infer.cc)
target_link_libraries(infer_demo ${FastDeploy_LIBS})

47
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/cpp/README.md Normal file
View File

@@ -0,0 +1,47 @@
[English](README.md) | 简体中文
# PaddleDetection RKNPU2 C++部署示例
本目录下用于展示PaddleDetection系列模型在RKNPU2上的部署，以下的部署过程以PPYOLOE为例子。
## 1. 部署环境准备
在部署前，需确认以下两个步骤:
1. 软硬件环境满足要求
2. 根据开发环境，下载预编译部署库或者从头编译FastDeploy仓库
以上步骤请参考[RK2代NPU部署库编译](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/rknpu2.md)实现
## 2. 部署模型准备
模型转换代码请参考[模型转换文档](../README.md)
## 3. 运行部署示例
```bash
# 下载部署示例代码
git clone https://github.com/PaddlePaddle/PaddleDetection.git
cd PaddleDetection/deploy/fastdeploy/rockchip/rknpu2/cpp
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
# git checkout develop
# 编译部署示例
mkdir build && cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x
make -j8
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/ppyoloe_plus_crn_s_80e_coco.zip
unzip ppyoloe_plus_crn_s_80e_coco.zip
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
# 运行部署示例
# CPU推理
./infer_demo ./ppyoloe_plus_crn_s_80e_coco 000000014439.jpg 0
# RKNPU2推理
./infer_demo ./ppyoloe_plus_crn_s_80e_coco 000000014439.jpg 1
```
## 4. 更多指南
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisableNormalizeAndPermute(C++)或`disable_normalize_and_permute(Python)，在预处理阶段禁用归一化以及数据格式的转换。
- [Python部署](../python)
- [转换PaddleDetection RKNN模型文档](../README.md)

96
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/cpp/infer.cc Normal file
View File

@@ -0,0 +1,96 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"
void ONNXInfer(const std::string& model_dir, const std::string& image_file) {
std::string model_file = model_dir + "/ppyoloe_plus_crn_s_80e_coco.onnx";
std::string params_file;
std::string config_file = model_dir + "/infer_cfg.yml";
auto option = fastdeploy::RuntimeOption();
option.UseCpu();
auto format = fastdeploy::ModelFormat::ONNX;
auto model = fastdeploy::vision::detection::PPYOLOE(
model_file, params_file, config_file, option, format);
fastdeploy::TimeCounter tc;
tc.Start();
auto im = cv::imread(image_file);
fastdeploy::vision::DetectionResult res;
if (!model.Predict(im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
tc.End();
tc.PrintInfo("PPDet in ONNX");
std::cout << res.Str() << std::endl;
cv::imwrite("infer_onnx.jpg", vis_im);
std::cout << "Visualized result saved in ./infer_onnx.jpg" << std::endl;
}
void RKNPU2Infer(const std::string& model_dir, const std::string& image_file) {
auto model_file =
model_dir + "/ppyoloe_plus_crn_s_80e_coco_rk3588_quantized.rknn";
auto params_file = "";
auto config_file = model_dir + "/infer_cfg.yml";
auto option = fastdeploy::RuntimeOption();
option.UseRKNPU2();
auto format = fastdeploy::ModelFormat::RKNN;
auto model = fastdeploy::vision::detection::PPYOLOE(
model_file, params_file, config_file, option, format);
model.GetPreprocessor().DisablePermute();
model.GetPreprocessor().DisableNormalize();
model.GetPostprocessor().ApplyNMS();
auto im = cv::imread(image_file);
fastdeploy::vision::DetectionResult res;
fastdeploy::TimeCounter tc;
tc.Start();
if (!model.Predict(&im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}
tc.End();
tc.PrintInfo("PPDet in RKNPU2");
std::cout << res.Str() << std::endl;
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
cv::imwrite("infer_rknpu2.jpg", vis_im);
std::cout << "Visualized result saved in ./infer_rknpu2.jpg" << std::endl;
}
int main(int argc, char* argv[]) {
if (argc < 4) {
std::cout
<< "Usage: infer_demo path/to/model_dir path/to/image run_option, "
"e.g ./infer_demo ./model_dir ./test.jpeg"
<< std::endl;
return -1;
}
if (std::atoi(argv[3]) == 0) {
ONNXInfer(argv[1], argv[2]);
} else if (std::atoi(argv[3]) == 1) {
RKNPU2Infer(argv[1], argv[2]);
}
return 0;
}

41
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/python/README.md Normal file
View File

@@ -0,0 +1,41 @@
[English](README.md) | 简体中文
# PaddleDetection RKNPU2 Python部署示例
本目录下用于展示PaddleDetection系列模型在RKNPU2上的部署，以下的部署过程以PPYOLOE为例子。
## 1. 部署环境准备
在部署前，需确认以下步骤
- 1. 软硬件环境满足要求，RKNPU2环境部署等参考[FastDeploy环境要求](https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/rknpu2.md)
## 2. 部署模型准备
模型转换代码请参考[模型转换文档](../README.md)
## 3. 运行部署示例
本目录下提供`infer.py`快速完成PPYOLOE在RKNPU上部署的示例。执行如下脚本即可完成
```bash
# 下载部署示例代码
git clone https://github.com/PaddlePaddle/PaddleDetection.git
cd PaddleDetection/deploy/fastdeploy/rockchip/rknpu2/python
# 注意：如果当前分支找不到下面的fastdeploy测试代码，请切换到develop分支
# git checkout develop
# 下载图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/ppyoloe_plus_crn_s_80e_coco.zip
unzip ppyoloe_plus_crn_s_80e_coco.zip
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
# 运行部署示例
python3 infer.py --model_file ./ppyoloe_plus_crn_s_80e_coco/ppyoloe_plus_crn_s_80e_coco_rk3588_quantized.rknn \
--config_file ./ppyoloe_plus_crn_s_80e_coco/infer_cfg.yml \
--image_file 000000014439.jpg
```
# 4. 更多指南
RKNPU上对模型的输入要求是使用NHWC格式，且图片归一化操作会在转RKNN模型时，内嵌到模型中，因此我们在使用FastDeploy部署时，需要先调用DisableNormalizeAndPermute(C++)或`disable_normalize_and_permute(Python)，在预处理阶段禁用归一化以及数据格式的转换。
- [C++部署](../cpp)
- [转换PaddleDetection RKNN模型文档](../README.md)

68
paddle_detection/deploy/fastdeploy/rockchip/rknpu2/python/infer.py Normal file
View File

@@ -0,0 +1,68 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fastdeploy as fd
import cv2
import os
def parse_arguments():
import argparse
import ast
parser = argparse.ArgumentParser()
parser.add_argument(
"--model_file",
default="./ppyoloe_plus_crn_s_80e_coco/ppyoloe_plus_crn_s_80e_coco_rk3588_quantized.rknn",
help="Path of rknn model.")
parser.add_argument(
"--config_file",
default="./ppyoloe_plus_crn_s_80e_coco/infer_cfg.yml",
help="Path of config.")
parser.add_argument(
"--image_file",
type=str,
default="./000000014439.jpg",
help="Path of test image file.")
return parser.parse_args()
if __name__ == "__main__":
args = parse_arguments()
model_file = args.model_file
params_file = ""
config_file = args.config_file
# setup runtime
runtime_option = fd.RuntimeOption()
runtime_option.use_rknpu2()
model = fd.vision.detection.PPYOLOE(
model_file,
params_file,
config_file,
runtime_option=runtime_option,
model_format=fd.ModelFormat.RKNN)
model.preprocessor.disable_normalize()
model.preprocessor.disable_permute()
model.postprocessor.apply_nms()
# predict
im = cv2.imread(args.image_file)
result = model.predict(im)
print(result)
# visualize
vis_im = fd.vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("visualized_result.jpg", vis_im)
print("Visualized result save in ./visualized_result.jpg")