Personnel Detection Algorithm Execution Guide

1. Personnel Detection Overview

Personnel detection is a deep-learning-based method for locating and detecting people. It is widely used in security, production safety, and other scenarios, and serves as the foundation for many algorithms such as perimeter intrusion detection, line-crossing detection, crowd gathering detection, loitering detection, and fall detection.

The performance of this personnel detection algorithm on the dataset is shown below.

Personnel detection algorithm	mAP@0.5
PERSON	0.79

The execution efficiency on the RV1126B board is as follows.

Algorithm type	Execution efficiency
PERSON	67ms

2. Quick Start

If you are reading this document for the first time, refer to the Getting Started Guide / Preparing and Updating the Development Compilation Environment, and follow the related steps to set up the compilation environment. Run the run script on the Ubuntu system on the PC side to enter the Docker development environment.

cd ~/linuxshare/work/rv1126b/jp/embedded/images/develop_environment
./run.sh 2204

Figure 2-1

2.1 Obtaining the Source Code

In the Docker development environment, create a management directory for storing the source code repository. Use git to clone the remote repository into the management directory.

cd /opt/linuxshare/work/rv1126b/jp/AI/demo/ai-algorithm
git clone https://github.com/csunltd/rv1126b-ai-toolkit.git

2.2 Model Placement

To run the algorithm demo, first download the personnel detection algorithm model. Copy the downloaded personnel detection algorithm model to the Release/ directory.

https://dl.dragonwake.com/download/rv1126b/AI/demo/03_person_detect.zip

Figure 2-2

2.3 Building the Sample

Move to the directory containing the sample and run the build. Because the dependent libraries are placed on the development board, keep /mnt mounted during cross-compilation.

cd rv1126b-ai-toolkit/Demos/algorithm-person_detect/
./build.sh cpres

sudo mount -t nfs -o vers=3,proto=tcp,mountproto=tcp,nolock,retrans=5,timeo=5 192.168.10.85:/ /mnt

Figure 2-3

2.4 Running the Sample and Checking Results

Copy the compiled files to the board, log in through serial debugging or SSH debugging, and run the sample. The result image can be copied back in the Docker development environment.

cp Release/* /mnt/userdata/Demo/algorithm-person/
cd /userdata/Demo/algorithm-person/
./test-person_detect person_detect.model test.jpg

cp /mnt/userdata/Demo/algorithm-person/result.jpg ./Release/

Figure 2-4

3. Personnel Detection API Description

3.1 Linking Method

To call the EASY EAI API library directly from a local project, add the following libraries and header files.

Item	Description
Header file directory	easyeai-api/algorithm/person_detect
Library file directory	easyeai-api/algorithm/person_detect
Library link parameter	-lperson_detect

3.2 Personnel Detection Initialization Function

The function prototype is as follows.

int person_detect_init(rknn_context *ctx, const char *path);

Item	Description
Function name	person_detect_init()
Header file	person_detect.h
Input parameter	ctx: rknn_context handle
Input parameter	path: algorithm model path
Return value	Return value on success: 0 / Return value on failure: -1
Description	None

3.3 Personnel Detection Execution Function

The function prototype is as follows.

int person_detect_run(rknn_context ctx, cv::Mat input_image, person_detect_result_group_t *detect_result_group);

Item	Description
Function name	person_detect_run()
Header file	person_detect.h
Input parameter	ctx: rknn_context handle
Input parameter	input_image: cv::Mat input image
Output parameter	detect_result_group: personnel detection result group
Return value	Return value on success: 0 / Return value on failure: -1
Description	None

3.4 Personnel Detection Release Function

The function prototype is as follows.

int person_detect_release(rknn_context ctx);

Item	Description
Function name	person_detect_release()
Header file	person_detect.h
Input parameter	ctx: rknn_context handle
Return value	Return value on success: 0 / Return value on failure: -1
Description	None

4. Personnel Detection Algorithm Sample

The sample directory is Demos/algorithm-person/test-person_detect.cpp. The processing flow is as follows.

Figure 4-1

The reference sample is as follows.

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include <sys/time.h>
#include"person_detect.h"

using namespace cv;
using namespace std;

static Scalar colorArray[10]={
    Scalar(255, 0, 0, 255),
    Scalar(0, 255, 0, 255),
    Scalar(0,0,139,255),
    Scalar(0,100,0,255),
    Scalar(139,139,0,255),
    Scalar(209,206,0,255),
    Scalar(0,127,255,255),
    Scalar(139,61,72,255),
    Scalar(0,255,0,255),
    Scalar(255,0,0,255),
};

int plot_one_box(Mat src, int x1, int x2, int y1, int y2, char *label, char colour)
{
    int tl = round(0.002 * (src.rows + src.cols) / 2) + 1;
    rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), colorArray[(unsigned char)colour], 3);

    int tf = max(tl -1, 1);

    int base_line = 0;
    cv::Size t_size = getTextSize(label, FONT_HERSHEY_SIMPLEX, (float)tl/3, tf, &base_line);
    int x3 = x1 + t_size.width;
    int y3 = y1 - t_size.height - 3;

    rectangle(src, cv::Point(x1, y1), cv::Point(x3, y3), colorArray[(unsigned char)colour], -1);
    putText(src, label, cv::Point(x1, y1 - 2), FONT_HERSHEY_SIMPLEX, (float)tl/3, cv::Scalar(255, 255, 255, 255), tf, 8);
    return 0;
}

int main(int argc, char **argv)
{
  if (argc != 3)
    {
        printf("%s <model_path> <image_path>\n", argv[0]);
        return -1;
    }

    const char *model_path = argv[1];
    const char *image_path = argv[2];

  /* パラメータ初期化 */
  detect_result_group_t detect_result_group;

  /* アルゴリズムモデル初期化 */
  rknn_context ctx;
  person_detect_init(&ctx, model_path);

  /* アルゴリズム実行 */
  cv::Mat src;
  src = cv::imread(image_path, 1);

  struct timeval start;
  struct timeval end;
  float time_use=0;

  gettimeofday(&start,NULL);

  person_detect_run(ctx, src, &detect_result_group);

  gettimeofday(&end,NULL);
  time_use=(end.tv_sec-start.tv_sec)*1000000+(end.tv_usec-start.tv_usec);//マイクロ秒
  printf("time_use is %f\n",time_use/1000);

  /* アルゴリズム結果を画像上に描画して保存します */
  // Draw Objects
  char text[256];
  for (int i = 0; i < detect_result_group.count; i++)
  {

    detect_result_t* det_result = &(detect_result_group.results[i]);
    if( det_result->prop < 0.4)
    {
      continue;
    }

    sprintf(text, "%s %.1f%%", det_result->name, det_result->prop * 100);
    printf("%s @ (%d %d %d %d) %f\n", det_result->name, det_result->box.left, det_result->box.top,
         det_result->box.right, det_result->box.bottom, det_result->prop);
    int x1 = det_result->box.left;
    int y1 = det_result->box.top;
    int x2 = det_result->box.right;
    int y2 = det_result->box.bottom;
    /*
    rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(255, 0, 0, 255), 3);
    putText(src, text, cv::Point(x1, y1 + 12), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
    */
    plot_one_box(src, x1, x2, y1, y2, text, i%10);
  }

  cv::imwrite("result.jpg", src);


  /* アルゴリズムモデルのリソースを解放します */
  person_detect_release(ctx);

  return 0;
}