Echo/fcb_photo_review
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

更换文档检测模型

Browse Source
This commit is contained in:
liuyebo
2024-08-27 14:42:45 +08:00
parent aea6f19951
commit 1514e09c40
2072 changed files with 254336 additions and 4967 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
paddle_detection/deploy/serving/cpp/README.md Normal file
View File

@@ -0,0 +1,74 @@
# C++ Serving预测部署
## 1. 简介
Paddle Serving是飞桨开源的服务化部署框架，提供了C++ Serving和Python Pipeline两套框架，
C++ Serving框架更倾向于追求极致性能，Python Pipeline框架倾向于二次开发的便捷性。
旨在帮助深度学习开发者和企业提供高性能、灵活易用的工业级在线推理服务，助力人工智能落地应用。
更多关于Paddle Serving的介绍，可以参考[Paddle Serving官网repo](https://github.com/PaddlePaddle/Serving)。
本文档主要介绍利用C++ Serving框架实现模型（以yolov3_darknet53_270e_coco为例）的服务化部署。
## 2. C++ Serving预测部署
#### 2.1 C++ 服务化部署样例程序介绍
服务化部署的样例程序的目录地址为：`deploy/serving/cpp`
```shell
deploy/
├── serving/
│ ├── python/ # Python 服务化部署样例程序目录
│ │ ├──config.yml # 服务端模型预测相关配置文件
│ │ ├──pipeline_http_client.py # 客户端代码
│ │ ├──postprocess_ops.py # 用户自定义后处理代码
│ │ ├──preprocess_ops.py # 用户自定义预处理代码
│ │ ├──README.md # 说明文档
│ │ ├──web_service.py # 服务端代码
│ ├── cpp/ # C++ 服务化部署样例程序目录
│ │ ├──preprocess/ # C++ 自定义OP
│ │ ├──build_server.sh # C++ Serving 编译脚本
│ │ ├──serving_client.py # 客户端代码
│ │ └── ...
│ └── ...
└── ...
```
### 2.2 环境准备
安装Paddle Serving三个安装包的最新版本，
分别是：paddle-serving-client, paddle-serving-app和paddlepaddle(CPU/GPU版本二选一)。
```commandline
pip install paddle-serving-client
# pip install paddle-serving-server # CPU
pip install paddle-serving-server-gpu # GPU 默认 CUDA10.2 + TensorRT6，其他环境需手动指定版本号
pip install paddle-serving-app
# pip install paddlepaddle # CPU
pip install paddlepaddle-gpu
```
您可能需要使用国内镜像源（例如百度源, 在pip命令中添加`-i https://mirror.baidu.com/pypi/simple`）来加速下载。
Paddle Serving Server更多不同运行环境的whl包下载地址，请参考：[下载页面](https://github.com/PaddlePaddle/Serving/blob/v0.7.0/doc/Latest_Packages_CN.md)
PaddlePaddle更多版本请参考[官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)
### 2.3 服务化部署模型导出
导出步骤参考文档[PaddleDetection部署模型导出教程](../../EXPORT_MODEL.md),
导出服务化部署模型需要添加`--export_serving_model True`参数，导出示例如下:
```commandline
python tools/export_model.py -c configs/yolov3/yolov3_darknet53_270e_coco.yml \
--export_serving_model True \
-o weights=https://paddledet.bj.bcebos.com/models/yolov3_darknet53_270e_coco.pdparams
```
### 2.4 编译C++ Serving & 启动服务端模型预测服务
可使用一键编译脚本`deploy/serving/cpp/build_server.sh`进行编译
```commandline
bash deploy/serving/cpp/build_server.sh
```
当完成以上编译安装和模型导出后，可以按如下命令启动模型预测服务：
```commandline
python -m paddle_serving_server.serve --model output_inference/yolov3_darknet53_270e_coco/serving_server --op yolov3_darknet53_270e_coco --port 9997 &
```
如果需要自定义开发OP，请参考[文档](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/C%2B%2B_Serving/2%2B_model.md)进行开发
### 2.5 启动客户端访问
当成功启动了模型预测服务，可以按如下命令启动客户端访问服务：
```commandline
python deploy/serving/python/serving_client.py --serving_client output_inference/yolov3_darknet53_270e_coco/serving_client --image_file demo/000000014439.jpg --http_port 9997
```

70
paddle_detection/deploy/serving/cpp/build_server.sh Normal file
View File

@@ -0,0 +1,70 @@
#使用镜像：
#registry.baidubce.com/paddlepaddle/paddle:latest-dev-cuda10.1-cudnn7-gcc82
#编译Serving Server：
#client和app可以直接使用release版本
#server因为加入了自定义OP，需要重新编译
apt-get update
apt install -y libcurl4-openssl-dev libbz2-dev
wget https://paddle-serving.bj.bcebos.com/others/centos_ssl.tar && tar xf centos_ssl.tar && rm -rf centos_ssl.tar && mv libcrypto.so.1.0.2k /usr/lib/libcrypto.so.1.0.2k && mv libssl.so.1.0.2k /usr/lib/libssl.so.1.0.2k && ln -sf /usr/lib/libcrypto.so.1.0.2k /usr/lib/libcrypto.so.10 && ln -sf /usr/lib/libssl.so.1.0.2k /usr/lib/libssl.so.10 && ln -sf /usr/lib/libcrypto.so.10 /usr/lib/libcrypto.so && ln -sf /usr/lib/libssl.so.10 /usr/lib/libssl.so
# 安装go依赖
rm -rf /usr/local/go
wget -qO- https://paddle-ci.cdn.bcebos.com/go1.17.2.linux-amd64.tar.gz | tar -xz -C /usr/local
export GOROOT=/usr/local/go
export GOPATH=/root/gopath
export PATH=$PATH:$GOPATH/bin:$GOROOT/bin
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway@v1.15.2
go install github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger@v1.15.2
go install github.com/golang/protobuf/protoc-gen-go@v1.4.3
go install google.golang.org/grpc@v1.33.0
go env -w GO111MODULE=auto
# 下载opencv库
wget https://paddle-qa.bj.bcebos.com/PaddleServing/opencv3.tar.gz && tar -xvf opencv3.tar.gz && rm -rf opencv3.tar.gz
export OPENCV_DIR=$PWD/opencv3
# clone Serving
git clone https://github.com/PaddlePaddle/Serving.git -b develop --depth=1
cd Serving
export Serving_repo_path=$PWD
git submodule update --init --recursive
python -m pip install -r python/requirements.txt
# set env
export PYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())")
export PYTHON_LIBRARIES=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
export PYTHON_EXECUTABLE=`which python`
export CUDA_PATH='/usr/local/cuda'
export CUDNN_LIBRARY='/usr/local/cuda/lib64/'
export CUDA_CUDART_LIBRARY='/usr/local/cuda/lib64/'
export TENSORRT_LIBRARY_PATH='/usr/local/TensorRT6-cuda10.1-cudnn7/targets/x86_64-linux-gnu/'
# cp 自定义OP代码
\cp ../deploy/serving/cpp/preprocess/*.h ${Serving_repo_path}/core/general-server/op
\cp ../deploy/serving/cpp/preprocess/*.cpp ${Serving_repo_path}/core/general-server/op
# 编译Server, export SERVING_BIN
mkdir server-build-gpu-opencv && cd server-build-gpu-opencv
cmake -DPYTHON_INCLUDE_DIR=$PYTHON_INCLUDE_DIR \
-DPYTHON_LIBRARIES=$PYTHON_LIBRARIES \
-DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \
-DCUDA_TOOLKIT_ROOT_DIR=${CUDA_PATH} \
-DCUDNN_LIBRARY=${CUDNN_LIBRARY} \
-DCUDA_CUDART_LIBRARY=${CUDA_CUDART_LIBRARY} \
-DTENSORRT_ROOT=${TENSORRT_LIBRARY_PATH} \
-DOPENCV_DIR=${OPENCV_DIR} \
-DWITH_OPENCV=ON \
-DSERVER=ON \
-DWITH_GPU=ON ..
make -j32
python -m pip install python/dist/paddle*
export SERVING_BIN=$PWD/core/general-server/serving
cd ../../

309
paddle_detection/deploy/serving/cpp/preprocess/mask_rcnn_r50_fpn_1x_coco.cpp Normal file
View File

@@ -0,0 +1,309 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/mask_rcnn_r50_fpn_1x_coco.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int mask_rcnn_r50_fpn_1x_coco::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
Resize(&img, scale_factor_h, scale_factor_w, im_shape_h, im_shape_w);
Normalize(&img, mean_, scale_, is_scale_);
PadStride(&img, 32);
int input_shape_h = img.rows;
int input_shape_w = img.cols;
std::vector<float> input(1 * 3 * input_shape_h * input_shape_w, 0.0f);
Permute(img, input.data());
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// im_shape
std::vector<float> im_shape{static_cast<float>(im_shape_h),
static_cast<float>(im_shape_w)};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, im_shape.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_0(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_0;
tensor_in_0.name = "im_shape";
tensor_in_0.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_0.shape = {1, 2};
tensor_in_0.lod = in->at(0).lod;
tensor_in_0.data = paddleBuf_0;
real_in->push_back(tensor_in_0);
// image
in_num = 1 * 3 * input_shape_h * input_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_1(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_1;
tensor_in_1.name = "image";
tensor_in_1.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_1.shape = {1, 3, input_shape_h, input_shape_w};
tensor_in_1.lod = in->at(0).lod;
tensor_in_1.data = paddleBuf_1;
real_in->push_back(tensor_in_1);
// scale_factor
std::vector<float> scale_factor{scale_factor_h, scale_factor_w};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, scale_factor.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_2(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_2;
tensor_in_2.name = "scale_factor";
tensor_in_2.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_2.shape = {1, 2};
tensor_in_2.lod = in->at(0).lod;
tensor_in_2.data = paddleBuf_2;
real_in->push_back(tensor_in_2);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void mask_rcnn_r50_fpn_1x_coco::Resize(cv::Mat *img, float &scale_factor_h,
float &scale_factor_w, int &im_shape_h,
int &im_shape_w) {
// keep_ratio
int im_size_max = std::max(img->rows, img->cols);
int im_size_min = std::min(img->rows, img->cols);
int target_size_max = std::max(im_shape_h, im_shape_w);
int target_size_min = std::min(im_shape_h, im_shape_w);
float scale_min =
static_cast<float>(target_size_min) / static_cast<float>(im_size_min);
float scale_max =
static_cast<float>(target_size_max) / static_cast<float>(im_size_max);
float scale_ratio = std::min(scale_min, scale_max);
// scale_factor
scale_factor_h = scale_ratio;
scale_factor_w = scale_ratio;
// Resize
cv::resize(*img, *img, cv::Size(), scale_ratio, scale_ratio, 2);
im_shape_h = img->rows;
im_shape_w = img->cols;
}
void mask_rcnn_r50_fpn_1x_coco::Normalize(cv::Mat *img,
const std::vector<float> &mean,
const std::vector<float> &scale,
const bool is_scale) {
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
(*img).convertTo(*img, CV_32FC3, e);
for (int h = 0; h < img->rows; h++) {
for (int w = 0; w < img->cols; w++) {
img->at<cv::Vec3f>(h, w)[0] =
(img->at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img->at<cv::Vec3f>(h, w)[1] =
(img->at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img->at<cv::Vec3f>(h, w)[2] =
(img->at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
}
void mask_rcnn_r50_fpn_1x_coco::PadStride(cv::Mat *img, int stride_) {
// PadStride
if (stride_ <= 0)
return;
int rh = img->rows;
int rw = img->cols;
int nh = (rh / stride_) * stride_ + (rh % stride_ != 0) * stride_;
int nw = (rw / stride_) * stride_ + (rw % stride_ != 0) * stride_;
cv::copyMakeBorder(*img, *img, 0, nh - rh, 0, nw - rw, cv::BORDER_CONSTANT,
cv::Scalar(0));
}
void mask_rcnn_r50_fpn_1x_coco::Permute(const cv::Mat &img, float *data) {
// Permute
int rh = img.rows;
int rw = img.cols;
int rc = img.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw), i);
}
}
cv::Mat mask_rcnn_r50_fpn_1x_coco::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string mask_rcnn_r50_fpn_1x_coco::base64Decode(const char *Data,
int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(mask_rcnn_r50_fpn_1x_coco);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

72
paddle_detection/deploy/serving/cpp/preprocess/mask_rcnn_r50_fpn_1x_coco.h Normal file
View File

@@ -0,0 +1,72 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class mask_rcnn_r50_fpn_1x_coco
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(mask_rcnn_r50_fpn_1x_coco);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 1333;
int im_shape_w = 800;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void Resize(cv::Mat *img, float &scale_factor_h, float &scale_factor_w,
int &im_shape_h, int &im_shape_w);
void Normalize(cv::Mat *img, const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
void PadStride(cv::Mat *img, int stride_ = -1);
void Permute(const cv::Mat &img, float *data);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

258
paddle_detection/deploy/serving/cpp/preprocess/picodet_lcnet_1_5x_416_coco.cpp Normal file
View File

@@ -0,0 +1,258 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/picodet_lcnet_1_5x_416_coco.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int picodet_lcnet_1_5x_416_coco::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
std::vector<float> input(1 * 3 * im_shape_h * im_shape_w, 0.0f);
preprocess_det(img, input.data(), scale_factor_h, scale_factor_w, im_shape_h,
im_shape_w, mean_, scale_, is_scale_);
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// image
in_num = 1 * 3 * im_shape_h * im_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in;
tensor_in.name = "image";
tensor_in.dtype = paddle::PaddleDType::FLOAT32;
tensor_in.shape = {1, 3, im_shape_h, im_shape_w};
tensor_in.lod = in->at(0).lod;
tensor_in.data = paddleBuf;
real_in->push_back(tensor_in);
// scale_factor
std::vector<float> scale_factor{scale_factor_h, scale_factor_w};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, scale_factor.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_2(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_2;
tensor_in_2.name = "scale_factor";
tensor_in_2.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_2.shape = {1, 2};
tensor_in_2.lod = in->at(0).lod;
tensor_in_2.data = paddleBuf_2;
real_in->push_back(tensor_in_2);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void picodet_lcnet_1_5x_416_coco::preprocess_det(
const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean, const std::vector<float> &scale,
const bool is_scale) {
// scale_factor
scale_factor_h =
static_cast<float>(im_shape_h) / static_cast<float>(img.rows);
scale_factor_w =
static_cast<float>(im_shape_w) / static_cast<float>(img.cols);
// Resize
cv::Mat resize_img;
cv::resize(img, resize_img, cv::Size(im_shape_w, im_shape_h), 0, 0, 2);
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
cv::Mat img_fp;
(resize_img).convertTo(img_fp, CV_32FC3, e);
for (int h = 0; h < im_shape_h; h++) {
for (int w = 0; w < im_shape_w; w++) {
img_fp.at<cv::Vec3f>(h, w)[0] =
(img_fp.at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img_fp.at<cv::Vec3f>(h, w)[1] =
(img_fp.at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img_fp.at<cv::Vec3f>(h, w)[2] =
(img_fp.at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
// Permute
int rh = img_fp.rows;
int rw = img_fp.cols;
int rc = img_fp.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img_fp, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw),
i);
}
}
cv::Mat picodet_lcnet_1_5x_416_coco::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string picodet_lcnet_1_5x_416_coco::base64Decode(const char *Data,
int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(picodet_lcnet_1_5x_416_coco);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

69
paddle_detection/deploy/serving/cpp/preprocess/picodet_lcnet_1_5x_416_coco.h Normal file
View File

@@ -0,0 +1,69 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class picodet_lcnet_1_5x_416_coco
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(picodet_lcnet_1_5x_416_coco);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 416;
int im_shape_w = 416;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void preprocess_det(const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

282
paddle_detection/deploy/serving/cpp/preprocess/ppyolo_mbv3_large_coco.cpp Normal file
View File

@@ -0,0 +1,282 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/ppyolo_mbv3_large_coco.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int ppyolo_mbv3_large_coco::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
std::vector<float> input(1 * 3 * im_shape_h * im_shape_w, 0.0f);
preprocess_det(img, input.data(), scale_factor_h, scale_factor_w, im_shape_h,
im_shape_w, mean_, scale_, is_scale_);
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// im_shape
std::vector<float> im_shape{static_cast<float>(im_shape_h),
static_cast<float>(im_shape_w)};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, im_shape.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_0(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_0;
tensor_in_0.name = "im_shape";
tensor_in_0.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_0.shape = {1, 2};
tensor_in_0.lod = in->at(0).lod;
tensor_in_0.data = paddleBuf_0;
real_in->push_back(tensor_in_0);
// image
in_num = 1 * 3 * im_shape_h * im_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_1(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_1;
tensor_in_1.name = "image";
tensor_in_1.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_1.shape = {1, 3, im_shape_h, im_shape_w};
tensor_in_1.lod = in->at(0).lod;
tensor_in_1.data = paddleBuf_1;
real_in->push_back(tensor_in_1);
// scale_factor
std::vector<float> scale_factor{scale_factor_h, scale_factor_w};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, scale_factor.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_2(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_2;
tensor_in_2.name = "scale_factor";
tensor_in_2.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_2.shape = {1, 2};
tensor_in_2.lod = in->at(0).lod;
tensor_in_2.data = paddleBuf_2;
real_in->push_back(tensor_in_2);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void ppyolo_mbv3_large_coco::preprocess_det(const cv::Mat &img, float *data,
float &scale_factor_h,
float &scale_factor_w,
int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale,
const bool is_scale) {
// scale_factor
scale_factor_h =
static_cast<float>(im_shape_h) / static_cast<float>(img.rows);
scale_factor_w =
static_cast<float>(im_shape_w) / static_cast<float>(img.cols);
// Resize
cv::Mat resize_img;
cv::resize(img, resize_img, cv::Size(im_shape_w, im_shape_h), 0, 0, 2);
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
cv::Mat img_fp;
(resize_img).convertTo(img_fp, CV_32FC3, e);
for (int h = 0; h < im_shape_h; h++) {
for (int w = 0; w < im_shape_w; w++) {
img_fp.at<cv::Vec3f>(h, w)[0] =
(img_fp.at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img_fp.at<cv::Vec3f>(h, w)[1] =
(img_fp.at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img_fp.at<cv::Vec3f>(h, w)[2] =
(img_fp.at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
// Permute
int rh = img_fp.rows;
int rw = img_fp.cols;
int rc = img_fp.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img_fp, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw),
i);
}
}
cv::Mat ppyolo_mbv3_large_coco::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string ppyolo_mbv3_large_coco::base64Decode(const char *Data,
int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(ppyolo_mbv3_large_coco);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

69
paddle_detection/deploy/serving/cpp/preprocess/ppyolo_mbv3_large_coco.h Normal file
View File

@@ -0,0 +1,69 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class ppyolo_mbv3_large_coco
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(ppyolo_mbv3_large_coco);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 320;
int im_shape_w = 320;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void preprocess_det(const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

260
paddle_detection/deploy/serving/cpp/preprocess/ppyoloe_crn_s_300e_coco.cpp Normal file
View File

@@ -0,0 +1,260 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/ppyoloe_crn_s_300e_coco.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int ppyoloe_crn_s_300e_coco::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
std::vector<float> input(1 * 3 * im_shape_h * im_shape_w, 0.0f);
preprocess_det(img, input.data(), scale_factor_h, scale_factor_w, im_shape_h,
im_shape_w, mean_, scale_, is_scale_);
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// image
in_num = 1 * 3 * im_shape_h * im_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in;
tensor_in.name = "image";
tensor_in.dtype = paddle::PaddleDType::FLOAT32;
tensor_in.shape = {1, 3, im_shape_h, im_shape_w};
tensor_in.lod = in->at(0).lod;
tensor_in.data = paddleBuf;
real_in->push_back(tensor_in);
// scale_factor
std::vector<float> scale_factor{scale_factor_h, scale_factor_w};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, scale_factor.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_2(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_2;
tensor_in_2.name = "scale_factor";
tensor_in_2.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_2.shape = {1, 2};
tensor_in_2.lod = in->at(0).lod;
tensor_in_2.data = paddleBuf_2;
real_in->push_back(tensor_in_2);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void ppyoloe_crn_s_300e_coco::preprocess_det(const cv::Mat &img, float *data,
float &scale_factor_h,
float &scale_factor_w,
int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale,
const bool is_scale) {
// scale_factor
scale_factor_h =
static_cast<float>(im_shape_h) / static_cast<float>(img.rows);
scale_factor_w =
static_cast<float>(im_shape_w) / static_cast<float>(img.cols);
// Resize
cv::Mat resize_img;
cv::resize(img, resize_img, cv::Size(im_shape_w, im_shape_h), 0, 0, 2);
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
cv::Mat img_fp;
(resize_img).convertTo(img_fp, CV_32FC3, e);
for (int h = 0; h < im_shape_h; h++) {
for (int w = 0; w < im_shape_w; w++) {
img_fp.at<cv::Vec3f>(h, w)[0] =
(img_fp.at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img_fp.at<cv::Vec3f>(h, w)[1] =
(img_fp.at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img_fp.at<cv::Vec3f>(h, w)[2] =
(img_fp.at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
// Permute
int rh = img_fp.rows;
int rw = img_fp.cols;
int rc = img_fp.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img_fp, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw),
i);
}
}
cv::Mat ppyoloe_crn_s_300e_coco::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string ppyoloe_crn_s_300e_coco::base64Decode(const char *Data,
int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(ppyoloe_crn_s_300e_coco);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

69
paddle_detection/deploy/serving/cpp/preprocess/ppyoloe_crn_s_300e_coco.h Normal file
View File

@@ -0,0 +1,69 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class ppyoloe_crn_s_300e_coco
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(ppyoloe_crn_s_300e_coco);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 640;
int im_shape_w = 640;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void preprocess_det(const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

232
paddle_detection/deploy/serving/cpp/preprocess/tinypose_128x96.cpp Normal file
View File

@@ -0,0 +1,232 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/tinypose_128x96.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int tinypose_128x96::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
std::vector<float> input(1 * 3 * im_shape_h * im_shape_w, 0.0f);
preprocess_det(img, input.data(), scale_factor_h, scale_factor_w, im_shape_h,
im_shape_w, mean_, scale_, is_scale_);
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// image
in_num = 1 * 3 * im_shape_h * im_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in;
tensor_in.name = "image";
tensor_in.dtype = paddle::PaddleDType::FLOAT32;
tensor_in.shape = {1, 3, im_shape_h, im_shape_w};
tensor_in.lod = in->at(0).lod;
tensor_in.data = paddleBuf;
real_in->push_back(tensor_in);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void tinypose_128x96::preprocess_det(const cv::Mat &img, float *data,
float &scale_factor_h,
float &scale_factor_w, int im_shape_h,
int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale,
const bool is_scale) {
// Resize
cv::Mat resize_img;
cv::resize(img, resize_img, cv::Size(im_shape_w, im_shape_h), 0, 0, 1);
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
cv::Mat img_fp;
(resize_img).convertTo(img_fp, CV_32FC3, e);
for (int h = 0; h < im_shape_h; h++) {
for (int w = 0; w < im_shape_w; w++) {
img_fp.at<cv::Vec3f>(h, w)[0] =
(img_fp.at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img_fp.at<cv::Vec3f>(h, w)[1] =
(img_fp.at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img_fp.at<cv::Vec3f>(h, w)[2] =
(img_fp.at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
// Permute
int rh = img_fp.rows;
int rw = img_fp.cols;
int rc = img_fp.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img_fp, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw),
i);
}
}
cv::Mat tinypose_128x96::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string tinypose_128x96::base64Decode(const char *Data, int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(tinypose_128x96);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

69
paddle_detection/deploy/serving/cpp/preprocess/tinypose_128x96.h Normal file
View File

@@ -0,0 +1,69 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class tinypose_128x96
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(tinypose_128x96);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 128;
int im_shape_w = 96;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void preprocess_det(const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

282
paddle_detection/deploy/serving/cpp/preprocess/yolov3_darknet53_270e_coco.cpp Normal file
View File

@@ -0,0 +1,282 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "core/general-server/op/yolov3_darknet53_270e_coco.h"
#include "core/predictor/framework/infer.h"
#include "core/predictor/framework/memory.h"
#include "core/predictor/framework/resource.h"
#include "core/util/include/timer.h"
#include <algorithm>
#include <iostream>
#include <memory>
#include <sstream>
namespace baidu {
namespace paddle_serving {
namespace serving {
using baidu::paddle_serving::Timer;
using baidu::paddle_serving::predictor::InferManager;
using baidu::paddle_serving::predictor::MempoolWrapper;
using baidu::paddle_serving::predictor::PaddleGeneralModelConfig;
using baidu::paddle_serving::predictor::general_model::Request;
using baidu::paddle_serving::predictor::general_model::Response;
using baidu::paddle_serving::predictor::general_model::Tensor;
int yolov3_darknet53_270e_coco::inference() {
VLOG(2) << "Going to run inference";
const std::vector<std::string> pre_node_names = pre_names();
if (pre_node_names.size() != 1) {
LOG(ERROR) << "This op(" << op_name()
<< ") can only have one predecessor op, but received "
<< pre_node_names.size();
return -1;
}
const std::string pre_name = pre_node_names[0];
const GeneralBlob *input_blob = get_depend_argument<GeneralBlob>(pre_name);
if (!input_blob) {
LOG(ERROR) << "input_blob is nullptr,error";
return -1;
}
uint64_t log_id = input_blob->GetLogId();
VLOG(2) << "(logid=" << log_id << ") Get precedent op name: " << pre_name;
GeneralBlob *output_blob = mutable_data<GeneralBlob>();
if (!output_blob) {
LOG(ERROR) << "output_blob is nullptr,error";
return -1;
}
output_blob->SetLogId(log_id);
if (!input_blob) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed mutable depended argument, op:" << pre_name;
return -1;
}
const TensorVector *in = &input_blob->tensor_vector;
TensorVector *out = &output_blob->tensor_vector;
int batch_size = input_blob->_batch_size;
output_blob->_batch_size = batch_size;
VLOG(2) << "(logid=" << log_id << ") infer batch size: " << batch_size;
Timer timeline;
int64_t start = timeline.TimeStampUS();
timeline.Start();
// only support string type
char *total_input_ptr = static_cast<char *>(in->at(0).data.data());
std::string base64str = total_input_ptr;
cv::Mat img = Base2Mat(base64str);
cv::cvtColor(img, img, cv::COLOR_BGR2RGB);
// preprocess
std::vector<float> input(1 * 3 * im_shape_h * im_shape_w, 0.0f);
preprocess_det(img, input.data(), scale_factor_h, scale_factor_w, im_shape_h,
im_shape_w, mean_, scale_, is_scale_);
// create real_in
TensorVector *real_in = new TensorVector();
if (!real_in) {
LOG(ERROR) << "real_in is nullptr,error";
return -1;
}
int in_num = 0;
size_t databuf_size = 0;
void *databuf_data = NULL;
char *databuf_char = NULL;
// im_shape
std::vector<float> im_shape{static_cast<float>(im_shape_h),
static_cast<float>(im_shape_w)};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, im_shape.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_0(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_0;
tensor_in_0.name = "im_shape";
tensor_in_0.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_0.shape = {1, 2};
tensor_in_0.lod = in->at(0).lod;
tensor_in_0.data = paddleBuf_0;
real_in->push_back(tensor_in_0);
// image
in_num = 1 * 3 * im_shape_h * im_shape_w;
databuf_size = in_num * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, input.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_1(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_1;
tensor_in_1.name = "image";
tensor_in_1.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_1.shape = {1, 3, im_shape_h, im_shape_w};
tensor_in_1.lod = in->at(0).lod;
tensor_in_1.data = paddleBuf_1;
real_in->push_back(tensor_in_1);
// scale_factor
std::vector<float> scale_factor{scale_factor_h, scale_factor_w};
databuf_size = 2 * sizeof(float);
databuf_data = MempoolWrapper::instance().malloc(databuf_size);
if (!databuf_data) {
LOG(ERROR) << "Malloc failed, size: " << databuf_size;
return -1;
}
memcpy(databuf_data, scale_factor.data(), databuf_size);
databuf_char = reinterpret_cast<char *>(databuf_data);
paddle::PaddleBuf paddleBuf_2(databuf_char, databuf_size);
paddle::PaddleTensor tensor_in_2;
tensor_in_2.name = "scale_factor";
tensor_in_2.dtype = paddle::PaddleDType::FLOAT32;
tensor_in_2.shape = {1, 2};
tensor_in_2.lod = in->at(0).lod;
tensor_in_2.data = paddleBuf_2;
real_in->push_back(tensor_in_2);
if (InferManager::instance().infer(engine_name().c_str(), real_in, out,
batch_size)) {
LOG(ERROR) << "(logid=" << log_id
<< ") Failed do infer in fluid model: " << engine_name().c_str();
return -1;
}
int64_t end = timeline.TimeStampUS();
CopyBlobInfo(input_blob, output_blob);
AddBlobInfo(output_blob, start);
AddBlobInfo(output_blob, end);
return 0;
}
void yolov3_darknet53_270e_coco::preprocess_det(const cv::Mat &img, float *data,
float &scale_factor_h,
float &scale_factor_w,
int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale,
const bool is_scale) {
// scale_factor
scale_factor_h =
static_cast<float>(im_shape_h) / static_cast<float>(img.rows);
scale_factor_w =
static_cast<float>(im_shape_w) / static_cast<float>(img.cols);
// Resize
cv::Mat resize_img;
cv::resize(img, resize_img, cv::Size(im_shape_w, im_shape_h), 0, 0, 2);
// Normalize
double e = 1.0;
if (is_scale) {
e /= 255.0;
}
cv::Mat img_fp;
(resize_img).convertTo(img_fp, CV_32FC3, e);
for (int h = 0; h < im_shape_h; h++) {
for (int w = 0; w < im_shape_w; w++) {
img_fp.at<cv::Vec3f>(h, w)[0] =
(img_fp.at<cv::Vec3f>(h, w)[0] - mean[0]) / scale[0];
img_fp.at<cv::Vec3f>(h, w)[1] =
(img_fp.at<cv::Vec3f>(h, w)[1] - mean[1]) / scale[1];
img_fp.at<cv::Vec3f>(h, w)[2] =
(img_fp.at<cv::Vec3f>(h, w)[2] - mean[2]) / scale[2];
}
}
// Permute
int rh = img_fp.rows;
int rw = img_fp.cols;
int rc = img_fp.channels();
for (int i = 0; i < rc; ++i) {
cv::extractChannel(img_fp, cv::Mat(rh, rw, CV_32FC1, data + i * rh * rw),
i);
}
}
cv::Mat yolov3_darknet53_270e_coco::Base2Mat(std::string &base64_data) {
cv::Mat img;
std::string s_mat;
s_mat = base64Decode(base64_data.data(), base64_data.size());
std::vector<char> base64_img(s_mat.begin(), s_mat.end());
img = cv::imdecode(base64_img, cv::IMREAD_COLOR); // CV_LOAD_IMAGE_COLOR
return img;
}
std::string yolov3_darknet53_270e_coco::base64Decode(const char *Data,
int DataByte) {
const char DecodeTable[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0,
62, // '+'
0, 0, 0,
63, // '/'
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, // '0'-'9'
0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, // 'A'-'Z'
0, 0, 0, 0, 0, 0, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, // 'a'-'z'
};
std::string strDecode;
int nValue;
int i = 0;
while (i < DataByte) {
if (*Data != '\r' && *Data != '\n') {
nValue = DecodeTable[*Data++] << 18;
nValue += DecodeTable[*Data++] << 12;
strDecode += (nValue & 0x00FF0000) >> 16;
if (*Data != '=') {
nValue += DecodeTable[*Data++] << 6;
strDecode += (nValue & 0x0000FF00) >> 8;
if (*Data != '=') {
nValue += DecodeTable[*Data++];
strDecode += nValue & 0x000000FF;
}
}
i += 4;
} else // 回车换行,跳过
{
Data++;
i++;
}
}
return strDecode;
}
DEFINE_OP(yolov3_darknet53_270e_coco);
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

69
paddle_detection/deploy/serving/cpp/preprocess/yolov3_darknet53_270e_coco.h Normal file
View File

@@ -0,0 +1,69 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "core/general-server/general_model_service.pb.h"
#include "core/general-server/op/general_infer_helper.h"
#include "paddle_inference_api.h" // NOLINT
#include <string>
#include <vector>
#include "opencv2/core.hpp"
#include "opencv2/imgcodecs.hpp"
#include "opencv2/imgproc.hpp"
#include <chrono>
#include <iomanip>
#include <iostream>
#include <ostream>
#include <vector>
#include <cstring>
#include <fstream>
#include <numeric>
namespace baidu {
namespace paddle_serving {
namespace serving {
class yolov3_darknet53_270e_coco
: public baidu::paddle_serving::predictor::OpWithChannel<GeneralBlob> {
public:
typedef std::vector<paddle::PaddleTensor> TensorVector;
DECLARE_OP(yolov3_darknet53_270e_coco);
int inference();
private:
// preprocess
std::vector<float> mean_ = {0.485f, 0.456f, 0.406f};
std::vector<float> scale_ = {0.229f, 0.224f, 0.225f};
bool is_scale_ = true;
int im_shape_h = 608;
int im_shape_w = 608;
float scale_factor_h = 1.0f;
float scale_factor_w = 1.0f;
void preprocess_det(const cv::Mat &img, float *data, float &scale_factor_h,
float &scale_factor_w, int im_shape_h, int im_shape_w,
const std::vector<float> &mean,
const std::vector<float> &scale, const bool is_scale);
// read pics
cv::Mat Base2Mat(std::string &base64_data);
std::string base64Decode(const char *Data, int DataByte);
};
} // namespace serving
} // namespace paddle_serving
} // namespace baidu

125
paddle_detection/deploy/serving/cpp/serving_client.py Normal file
View File

@@ -0,0 +1,125 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import glob
import base64
import argparse
from paddle_serving_client import Client
from paddle_serving_client.proto import general_model_config_pb2 as m_config
import google.protobuf.text_format
parser = argparse.ArgumentParser(description="args for paddleserving")
parser.add_argument(
"--serving_client", type=str, help="the directory of serving_client")
parser.add_argument("--image_dir", type=str)
parser.add_argument("--image_file", type=str)
parser.add_argument("--http_port", type=int, default=9997)
parser.add_argument(
"--threshold", type=float, default=0.5, help="Threshold of score.")
args = parser.parse_args()
def get_test_images(infer_dir, infer_img):
"""
Get image path list in TEST mode
"""
assert infer_img is not None or infer_dir is not None, \
"--image_file or --image_dir should be set"
assert infer_img is None or os.path.isfile(infer_img), \
"{} is not a file".format(infer_img)
assert infer_dir is None or os.path.isdir(infer_dir), \
"{} is not a directory".format(infer_dir)
# infer_img has a higher priority
if infer_img and os.path.isfile(infer_img):
return [infer_img]
images = set()
infer_dir = os.path.abspath(infer_dir)
assert os.path.isdir(infer_dir), \
"infer_dir {} is not a directory".format(infer_dir)
exts = ['jpg', 'jpeg', 'png', 'bmp']
exts += [ext.upper() for ext in exts]
for ext in exts:
images.update(glob.glob('{}/*.{}'.format(infer_dir, ext)))
images = list(images)
assert len(images) > 0, "no image found in {}".format(infer_dir)
print("Found {} inference images in total.".format(len(images)))
return images
def postprocess(fetch_dict, fetch_vars, draw_threshold=0.5):
result = []
if "conv2d_441.tmp_1" in fetch_dict:
heatmap = fetch_dict["conv2d_441.tmp_1"]
print(heatmap)
result.append(heatmap)
else:
bboxes = fetch_dict[fetch_vars[0]]
for bbox in bboxes:
if bbox[0] > -1 and bbox[1] > draw_threshold:
print(f"{int(bbox[0])} {bbox[1]} "
f"{bbox[2]} {bbox[3]} {bbox[4]} {bbox[5]}")
result.append(f"{int(bbox[0])} {bbox[1]} "
f"{bbox[2]} {bbox[3]} {bbox[4]} {bbox[5]}")
return result
def get_model_vars(client_config_dir):
# read original serving_client_conf.prototxt
client_config_file = os.path.join(client_config_dir,
"serving_client_conf.prototxt")
with open(client_config_file, 'r') as f:
model_var = google.protobuf.text_format.Merge(
str(f.read()), m_config.GeneralModelConfig())
# modify feed_var to run core/general-server/op/
[model_var.feed_var.pop() for _ in range(len(model_var.feed_var))]
feed_var = m_config.FeedVar()
feed_var.name = "input"
feed_var.alias_name = "input"
feed_var.is_lod_tensor = False
feed_var.feed_type = 20
feed_var.shape.extend([1])
model_var.feed_var.extend([feed_var])
with open(
os.path.join(client_config_dir, "serving_client_conf_cpp.prototxt"),
"w") as f:
f.write(str(model_var))
# get feed_vars/fetch_vars
feed_vars = [var.name for var in model_var.feed_var]
fetch_vars = [var.name for var in model_var.fetch_var]
return feed_vars, fetch_vars
if __name__ == '__main__':
url = f"127.0.0.1:{args.http_port}"
logid = 10000
img_list = get_test_images(args.image_dir, args.image_file)
feed_vars, fetch_vars = get_model_vars(args.serving_client)
client = Client()
client.load_client_config(
os.path.join(args.serving_client, "serving_client_conf_cpp.prototxt"))
client.connect([url])
for img_file in img_list:
with open(img_file, 'rb') as file:
image_data = file.read()
image = base64.b64encode(image_data).decode('utf8')
fetch_dict = client.predict(
feed={feed_vars[0]: image}, fetch=fetch_vars)
result = postprocess(fetch_dict, fetch_vars, args.threshold)

20
paddle_detection/deploy/serving/cpp/serving_client_conf.prototxt Normal file
View File

@@ -0,0 +1,20 @@
feed_var {
name: "input"
alias_name: "input"
is_lod_tensor: false
feed_type: 20
shape: 1
}
fetch_var {
name: "multiclass_nms3_0.tmp_0"
alias_name: "multiclass_nms3_0.tmp_0"
is_lod_tensor: true
fetch_type: 1
shape: -1
}
fetch_var {
name: "multiclass_nms3_0.tmp_2"
alias_name: "multiclass_nms3_0.tmp_2"
is_lod_tensor: false
fetch_type: 2
}