YOLOv11-seg Training and Deployment Guide

Document Information

This tutorial explains how to train a YOLOv11-seg instance segmentation model and deploy it to the CSUN RV1126B board.

Item	Description
Document name	YOLOv11-seg Training and Deployment Guide
Company	Nissho Technology Co., Ltd.
URL	http://www.dragonwake.com
E-mail	info@dragonwake.com
Creation date	2026/06/08
Version	Ver1.0
Revision	Initial release

Revision History

No.	Version	Revision	Date
1	Ver1.0	Initial release	2026/06/08

The information in this document may be changed without prior notice for document improvement. For the latest version, refer to the company website: https://www.dragonwake.com.

Reproduction of this document in any form is strictly prohibited without written permission from Nissho Technology Co., Ltd.

1. YOLOv11-seg Overview

YOLOv11-seg is a recent member of the YOLO (You Only Look Once) family designed for real-time instance segmentation. While maintaining the high-speed inference characteristics of the YOLO family, it uses its network structure and segmentation head to perform pixel-level object detection and segmentation. It is suitable for use cases that require both accuracy and speed, such as autonomous driving, medical imaging, and industrial inspection.

This guide describes the training workflow for the YOLOv11-seg instance segmentation algorithm and the procedure for deploying it to the CSUN RV1126B board. Refer to previous articles for data annotation methods.

Figure1-1 Overall YOLOv11-seg training, conversion, and deployment workflow

2. Downloading the Materials

Download the materials and source code required for this manual from the following link.

AIDemo_yolov11-seg_All.zip

After extraction, the directory structure is as follows.

|-- 02-training        Training source code
|-- 03-model_convert   AI model conversion source code
|-- 04-AI_deploy       AI model deployment source code

Figure2-1 Directory structure after extracting the downloaded package

3. Training the YOLOv11-seg Model

The export-related part of the YOLOv11-seg training source code has been partially modified from the Ultralytics GitHub version. Therefore, use the training source code specified in this document.

3.1 Preparing the Dataset

Before starting YOLOv11-seg training, prepare a training dataset such as crack-seg. This dataset is also included in the training project package. The directory structure is shown below.

Figure3-1 crack-seg dataset directory structure

The crack-seg label data format is shown below. The first value in each row is the class ID, and the remaining values are polygon coordinates representing the contour.

Figure3-2 YOLO segmentation label format: class and polygon coordinates

If JSON annotation data needs to be converted to label format, use the ./data/json_2_yolo.py script.

3.2 Configuring Training Parameters

After data conversion, configure data.yaml, default.yaml, and yolo11.yaml for model training.

• data.yaml: Specifies training and validation data paths, the number of classes, and class names.
• default.yaml: Specifies YOLO11 training parameters. Adjust them as needed.
• yolo11.yaml: Defines the YOLO11 model structure. The number of classes must be changed for model training.

3.3 Model Training

After completing the above steps, open train.py and set the paths to data.yaml, default.yaml, and yolo11.yaml.

from pathlib import Path
import os
from ultralytics import YOLO
import ultralytics.data.utils as data_utils

# OMP 関連エラー（例: "OMP: Hint This means..."）が発生する場合に使用します。
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"

# Ultralytics のデータセット基準ディレクトリを相対パスに設定します。
# これにより、data.yaml の path: ../demo/crack-seg が
# ./datasets/../demo/crack-seg として解釈され、最終的に ./demo/crack-seg を参照します。
data_utils.DATASETS_DIR = Path("datasets")

if __name__ == '__main__':
    # 学習設定ファイル、データセット設定ファイル、モデル構造ファイルを指定します。
    # 本スクリプトは ultralytics_yolo11 のプロジェクトルートで実行してください。
    cfg = "./demo/crack-seg/default.yaml"
    data = "./demo/crack-seg/data.yaml"
    # weight = "./demo/weights/yolo11n-seg.pt"  # .pt または yolo11-seg.yaml を指定できます。
    weight = "./demo/crack-seg/yolo11-seg.yaml"

    print("YOLO11-seg セグメンテーションモデルの学習を開始します。")
    print(f"学習設定ファイル: {cfg}")
    print(f"データセット設定ファイル: {data}")
    print(f"モデルファイル: {weight}")

    model = YOLO(weight)

    results = model.train(
        data=data,
        cfg=cfg
    )

    print("YOLO11-seg セグメンテーションモデルの学習が完了しました。")

Run the following commands in the RV1126B Docker development environment to start training.

cd /home/csun/linuxshare/work/rv1126b/jp/embedded/images/develop_environment
./run.sh 2204
cd /opt/linuxshare/work/rv1126b/jp/AI/demo/Tutorial/yolov11-seg/02-training/ultralytics_yolo11
python train.py

Figure3-3 Terminal output at the start of YOLOv11-seg training

The evaluation results after training are shown below.

Figure3-4 Evaluation results after YOLOv11-seg training is completed

The generated model file is as follows.

./crack/train2/weights/best.pt

Figure3-5 best.pt and last.pt files generated after training

3.4 Model Inference on the PC Side

After training is completed, the training process and the model with the best validation result are saved to the project directory configured in default.yaml. Run predict-seg.py to perform an initial check of the model. predict-seg.py sets the model path, input image, inference device, and image size, then performs image preprocessing, model inference, and result visualization.

Check the actual model file path before running the script.

from ultralytics import YOLO

# 学習済みセグメンテーションモデルと推論画像を指定します。
# 本スクリプトは ultralytics_yolo11 のプロジェクトルートで実行してください。
model_path = "./crack/train2/weights/best.pt"
image_path = "./demo/crack-seg/test/images/3848.rf.eebe99038cd40502695607594e000258.jpg"

print("YOLO11-seg モデルの推論を開始します。")
print(f"入力モデル: {model_path}")
print(f"入力画像: {image_path}")

model = YOLO(model_path)  # 学習済みモデルを読み込みます。

# モデル推論を実行します。
results = model(image_path)

for result in results:
    boxes = result.boxes  # 検出ボックスの出力です。
    masks = result.masks  # セグメンテーションマスクの出力です。
    keypoints = result.keypoints  # 姿勢推定キーポイントの出力です。
    probs = result.probs  # 画像分類の確率出力です。
    obb = result.obb  # 回転ボックスの出力です。
    result.show()  # 画面に表示します。
    result.save(filename="result.jpg")  # 推論結果を保存します。

print("YOLO11-seg モデルの推論が完了しました。")

Run the following command.

python predict-seg.py

The execution result is shown below.

Figure3-6 Inference execution log on the PC side

Figure3-7 YOLOv11-seg inference result image on the PC side

3.5 Converting the PT Model to ONNX

Run export.py on the PC side to export the PT model to ONNX or to a format used for RKNN conversion. In the code example, format = 'rknn' is specified.

from ultralytics import YOLO

if __name__ == '__main__':
    # エクスポート形式を指定します。
    # 選択可能な形式例: 'torchscript', 'onnx', 'openvino', 'engine', 'coreml',
    # 'saved_model', 'pb', 'tflite', 'edgetpu', 'tfjs', 'paddle', 'ncnn', 'rknn'
    format = 'rknn'

    # 学習済みモデルを指定します。
    # 本スクリプトは ultralytics_yolo11 のプロジェクトルートで実行してください。
    weight = "./crack/train2/weights/best.pt"  # .pt または yolo11-seg.yaml を指定できます。

    print("YOLO11-seg モデルのエクスポートを開始します。")
    print(f"入力モデル: {weight}")
    print(f"エクスポート形式: {format}")

    model = YOLO(weight)
    results = model.export(format=format)

    print("YOLO11-seg モデルのエクスポートが完了しました。")

Run the following command.

python export.py

The execution result is shown below.

Figure3-8 Terminal output for exporting the PT model to a format used for RKNN conversion

The converted model file is saved in the same directory.

Figure3-9 best.onnx file generated after export

4. RKNN-Toolkit Model Conversion

4.1 Building the RKNN-Toolkit Model Conversion Environment

To run an ONNX model on the CSUN RV1126B board, the model must be converted to RKNN format. Therefore, prepare the RKNN-Toolkit model conversion environment in advance. TensorFlow, TensorFlow Lite, Caffe, Darknet, and other models can be converted in a similar way. This tutorial uses an ONNX model as the example.

For the environment setup procedure, refer to the AI Model Conversion Environment Setup Guide.

4.2 Converting the ONNX Model to an RKNN Model

This document supports evaluation and execution of models with the .rknn extension. Common trained models from TensorFlow, TensorFlow Lite, Caffe, Darknet, ONNX, PyTorch, and other frameworks can be converted to RKNN models using RKNN-Toolkit2. Models trained in other frameworks can also be converted to ONNX first and then converted to RKNN.

For detailed conversion procedures, refer to the RKNN Model Conversion Tutorial Example.

4.2.1 Extracting the Model Conversion Package

Download and extract the model conversion demo from “2. Downloading the Materials.”

cd /data/project/sales/csun/rv1126b/jp/AI/demo/Tutorial/yolov11-seg/03-model_convert
unzip quant_dataset.zip
tar -xJf yolov11_seg_model_convert.tar.xz

Figure4-1 Directory structure after extracting the model conversion package

4.2.2 Entering the Model Conversion Docker Environment

Use the following command to mount the working directory into the Docker image. The directory /data/project/sales/csun/rv1126b/jp/AI/demo/Tutorial is used as the working area and is mapped to /test in the container. The USB device is also mounted into the container using /dev/bus/usb:/dev/bus/usb.

docker run -t -i --privileged   -v /dev/bus/usb:/dev/bus/usb   -v /data/project/sales/csun/rv1126b/jp/AI/demo/Tutorial:/test   rknn-toolkit2:2.3.2-cp38 /bin/bash

Figure4-2 Starting the RKNN-Toolkit2 Docker container

4.2.3 Generating the Quantization Image List

In the Docker environment, go to the model conversion working directory.

cd /test/yolov11-seg/03-model_convert/yolov11_seg_model_convert

Run gen_list.py to generate the quantization image list.

python gen_list.py

Figure4-3 Execution result of the quantization image list generation command

The generated quantization image list is saved as pic_path.txt.

Figure4-4 Generated pic_path.txt quantization image list

4.2.4 Converting the ONNX Model to an RKNN Model

rknn_convert.py performs INT8 quantization by default. The main script is as follows.

import sys
from rknn.api import RKNN

# ONNX_MODEL = 'best.onnx'
ONNX_MODEL = 'yolov11s.onnx'
DATASET = './pic_path.txt'
RKNN_MODEL = './yolov11s_rv1126b.rknn'
# RKNN_MODEL = './dog_rope.rknn'
QUANTIZE_ON = True

if __name__ == '__main__':

    # Create RKNN object
    rknn = RKNN(verbose=False)

    # Pre-process config
    print('--> Config model')
    rknn.config(mean_values=[[0, 0, 0]], std_values=[
                    [255, 255, 255]], target_platform='rv1126b')
    print('done')

    # Load model
    print('--> Loading model')
    ret = rknn.load_onnx(model=ONNX_MODEL)
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')

    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization=QUANTIZE_ON, dataset=DATASET)
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')

    # Export rknn model
    print('--> Export rknn model')
    ret = rknn.export_rknn(RKNN_MODEL)
    if ret != 0:
        print('Export rknn model failed!')
        exit(ret)
    print('done')

    # Release
    rknn.release()

Place best.onnx in the model conversion directory and run the following commands to convert the model.

cp ../../02-training/ultralytics_yolo11/crack/train2/weights/best.onnx ./
python rknn_convert.py

Figure4-5 Log of converting the ONNX model to an RKNN model

After conversion succeeds, an RKNN model that can run on the CSUN RV1126B board is generated.

Figure4-6 RKNN model file generated after conversion

5. Deploying the YOLOv11-seg Model

5.1 Description of the Deployment Example

This section describes how to deploy the YOLOv11-seg model to the RV1126B board. The model used in this document is a sample model trained only briefly, and its accuracy is not guaranteed.

5.2 Preparation

5.2.1 Hardware Preparation

Prepare the RV1126B board, a Type-C data cable, and a LAN cable. Log in to the RV1126B board through SSH using MobaXterm. For details, refer to the getting started guide.

Connect using a LAN cable.

Figure5-1 Network connection using a LAN cable

Connect using a Type-C serial cable.

Figure5-2 Connection using a Type-C serial cable

5.2.2 Preparing the Development Environment

If this is your first time using the document, refer to the getting started guide and build the compilation environment according to the documented procedure.

On the PC-side Ubuntu system, run the run.sh script to enter the RV1126B compilation environment.

cd ~/develop_environment
./run.sh 2204

Figure5-3 Starting the RV1126B Docker development environment

5.3 Building the Sample Program

After moving the downloaded package to the RV1126B Docker development environment, run the following commands to extract it.

cd /opt/linuxshare/work/rv1126b/jp/AI/demo/Tutorial/yolov11-seg/04-AI_deploy
tar -xJf yolov11_seg_C_demo.tar.xz

The extracted files are shown below.

Figure5-4 Extracted AI deployment sample program

In the RV1126B Docker development environment, go to the sample program directory and build it.

sudo mount -t nfs -o vers=3,proto=tcp,mountproto=tcp,nolock,retrans=5,timeo=5 192.168.11.85:/ /mnt
cd /opt/linuxshare/work/rv1126b/jp/AI/demo/Tutorial/yolov11-seg/04-AI_deploy/yolov11_seg_C_demo/
./build.sh

Figure5-5 Build result of the YOLOv11-seg sample program

After compilation succeeds, copy the executable directory yolov11_seg_demo_release/ to the /userdata directory on the RV1126B board.

cp yolov11_seg_demo_release/ /mnt/userdata/ -rf

5.4 Running the YOLOv11-seg Model on the Development Board

Enter the board-side shell through serial debugging or SSH debugging, and go to the sample program deployment directory.

cd /userdata/yolov11_seg_demo_release/

Run the sample program with the following commands.

chmod 777 yolov11_seg_demo
./yolov11_seg_demo yolov11n_seg_rv1126b.rknn crack.jpg

The execution result is shown below.

Figure5-6 YOLOv11-seg demo execution result on the development board

The algorithm execution time is about 117 ms.

Figure5-7 Algorithm execution time on the development board

After execution, copy the test images from the RV1126B compilation environment using the following commands.

cp /mnt/userdata/yolov11_seg_demo_release/result.jpg .
cp /mnt/userdata/yolov11_seg_demo_release/mask_bgr.jpg .

Figure5-8 Command for copying inference result images to the PC

Figure5-9 Segmentation mask image generated on the board

Figure5-10 Crack segmentation result image generated on the board

The YOLOv11-seg model is now running correctly on the board.