RKNN Model Deployment API Reference

1. Basic Data Structure Definitions

This section describes 1. Basic Data Structure Definitions.

1.1 rknn_sdk_version

This section describes 1.1 rknn_sdk_version. Related API or structure: rknn_sdk_version.

Member	Data Type	Description
api_version	char[]	Description of api_version.
drv_version	char[]	Description of drv_version.

1.2 rknn_input_output_num

This section describes 1.2 rknn_input_output_num. Related API or structure: rknn_input_output_num.

Member	Data Type	Description
n_input	uint32_t	inputtensornumber。
n_output	uint32_t	inputtensornumber。

1.3 rknn_input_range

This section describes 1.3 rknn_input_range. Related API or structure: rknn_input_range.

Member	Data Type	Description
index	uint32_t	Description of index.
shape_number	uint32_t	Description of shape_number.
fmt	rknn_tensor_format	Description of fmt.
name	char[]	Description of name.
dyn_range	uint32_t[][]	Description of dyn_range.
n_dims	uint32_t	Description of n_dims.

1.4 rknn_tensor_attr

This section describes 1.4 rknn_tensor_attr.

Member	Data Type	Description
index	uint32_t	Description of index.
n_dims	uint32_t	tensor of dimensionnumber
dims	uint32_t[]	tensorshape
name	char[]	tensorname
n_elems	uint32_t	Description of n_elems.
size	uint32_t	Description of size.
fmt	rknn_tensor_format	Description of fmt. Related values: RKNN_TENSOR_NCHW RKNN_TENSOR_NHWC RKNN_TENSOR_NC1HWC2 RKNN_TENSOR_UNDEFINED.
type	rknn_tensor_type	Description of type. Related values: RKNN_TENSOR_FLOAT32 RKNN_TENSOR_FLOAT16 RKNN_TENSOR_INT8 RKNN_TENSOR_UINT8 RKNN_TENSOR_INT16 RKNN_TENSOR_UINT16 RKNN_TENSOR_INT32 RKNN_TENSOR_INT64 RKNN_TENSOR_BOOL.
qnt_type	rknn_tensor_qnt_type	Description of qnt_type. Related values: RKNN_TENSOR_QNT_NONE RKNN_TENSOR_QNT_DFP RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC.
fl	int8_t	Description of fl. Related values: RKNN_TENSOR_QNT_DFP.
scale	float	Description of scale. Related values: RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC.
w_stride	uint32_t	Description of w_stride.
size_with_stride	uint32_t	Description of size_with_stride.
pass_through	uint8_t	Description of pass_through.
h_stride	uint32_t	Description of h_stride.

1.5 rknn_perf_detail

This section describes 1.5 rknn_perf_detail. Related API or structure: rknn_perf_detail.

Member	Data Type	Description
perf_data	char*	Description of perf_data.
data_len	uint64_t	Description of data_len.

1.6 rknn_perf_run

This section describes 1.6 rknn_perf_run. Related API or structure: rknn_perf_run.

This section describes 1.6 rknn_perf_run.

Member	Data Type	Description
run_duration	int64_t	Description of run_duration.

1.7 rknn_mem_size

This section describes 1.7 rknn_mem_size. Related API or structure: rknn_mem_sizestructure.

Member	Data Type	Description
total_weight_size	uint32_t	Description of total_weight_size.
total_internal_size	uint32_t	Description of total_internal_size.
total_dma_allocated_size	uint64_t	Description of total_dma_allocated_size.
total_sram_size	uint32_t	Description of total_sram_size.
free_sram_size	uint32_t	Description of free_sram_size.
reserved[12]	uint32_t	Description of reserved[12].

1.8 rknn_tensor_mem

This section describes 1.8 rknn_tensor_mem. Related API or structure: rknn_tensor_mem.

Member	Data Type	Description
virt_addr	void*	Description of virt_addr.
phys_addr	uint64_t	Description of phys_addr.
fd	int32_t	Description of fd.
offset	int32_t	Description of offset.
size	int32_t	Description of size.
flags	int32_t	Description of flags. Related values: RKNN_TENSOR_MEMORY_FALGS_ALLOC_INSIDE RKNN_TENSOR_MEMORY_FLAGS_FROM_FD RKNN_TENSOR_MEMORY_FLAGS_FROM_PHYS.
priv_data	void*	Description of priv_data.

1.9 rknn_input

This section describes 1.9 rknn_input. Related API or structure: rknn_input, rknn_inputs_set.

Member	Data Type	Description
index	uint32_t	Description of index.
buf	void*	Description of buf.
size	uint32_t	Description of size.
pass_through	uint8_t	Description of pass_through.
type	rknn_tensor_type	inputdata of type。
fmt	rknn_tensor_format	inputdata of format。

1.10 rknn_output

This section describes 1.10 rknn_output. Related API or structure: rknn_output, rknn_outputs_get.

Member	Data Type	Description
want_float	uint8_t	Description of want_float.
is_prealloc	uint8_t	Description of is_prealloc.
index	uint32_t	Description of index.
buf	void*	Description of buf.
size	uint32_t	Description of size.

1.11 rknn_init_extend

This section describes 1.11 rknn_init_extend. Related API or structure: rknn_init_extend.

Member	Data Type	Description
ctx	rknn_context	Description of ctx.
real_model_offset	int32_t	Description of real_model_offset.
real_model_size	int32_t	Description of real_model_size.
model_buffer_fd	int32_t	Description of model_buffer_fd. Related values: RKNN_FLAG_MODEL_BUFFER_ZERO_COPY.
model_buffer_flags	int32_t	Description of model_buffer_flags. Related values: RKNN_FLAG_MODEL_BUFFER_ZERO_COPY.
reserved	uint8_t[]	Description of reserved.

1.12 rknn_run_extend

This section describes 1.12 rknn_run_extend. Related API or structure: rknn_run_extend.

Member	Data Type	Description
frame_id	uint64_t	Description of frame_id.
non_block	int32_t	Description of non_block.
timeout_ms	int32_t	Description of timeout_ms.
fence_fd	int32_t	Description of fence_fd.

1.13 rknn_output_extend

This section describes 1.13 rknn_output_extend. Related API or structure: rknn_output_extend.

Member	Data Type	Description
frame_id	int32_t	Description of frame_id.

1.14 rknn_custom_string

This section describes 1.14 rknn_custom_string. Related API or structure: rknn_custom_string.

Member	Data Type	Description
string	char[]	userdefinitionstring。

2. Basic API Reference

2.1 rknn_init

This section describes 2.1 rknn_init. Related API or structure: rknn_init, rknn_context, rknn_init_extend.

Item	Content
API	rknn_init
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, 0, NULL);

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_COLLECT_PERF_MASK.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_MEM_ALLOC_OUTSIDE.

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init.

Example:

rknn_context ctx_a, ctx_b;

rknn_init(&ctx_a, model_path_a, 0, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL);
rknn_query(ctx_a, RKNN_QUERY_MEM_SIZE, &mem_size_a, sizeof(mem_size_a));

rknn_init(&ctx_b, model_path_b, 0, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL);
rknn_query(ctx_b, RKNN_QUERY_MEM_SIZE, &mem_size_b, sizeof(mem_size_b));

max_internal_size = MAX(mem_size_a.total_internal_size, mem_size_b.total_internal_size);
internal_mem_max = rknn_create_mem(ctx_a, max_internal_size);

internal_mem_a = rknn_create_mem_from_fd(ctx_a, internal_mem_max->fd,
     internal_mem_max->virt_addr, mem_size_a.total_internal_size, 0);
rknn_set_internal_mem(ctx_a, internal_mem_a);

internal_mem_b = rknn_create_mem_from_fd(ctx_b, internal_mem_max->fd,
     internal_mem_max->virt_addr, mem_size_b.total_internal_size, 0);
rknn_set_internal_mem(ctx_b, internal_mem_b);

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_SHARE_WEIGHT_MEM.

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init. Related API or structure: config().

This section describes 2.1 rknn_init.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_SHARE_WEIGHT_MEM.

Example:

rknn_context ctx_a, ctx_b;
rknn_init(&ctx_a, model_path_a, 0, 0, NULL);

rknn_init_extend extend;
extend.ctx = ctx_a;
rknn_init(&ctx_b, model_path_b, 0, RKNN_FLAG_SHARE_WEIGHT_MEM, &extend);

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_COLLECT_MODEL_INFO_ONLY, rknn_query.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_INTERNAL_ALLOC_OUTSIDE.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_EXECUTE_FALLBACK_PRIOR_DEVICE_GPU.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_ENABLE_SRAM.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_SHARE_SRAM, RKNN_FLAG_ENABLE_SRAM.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_DISABLE_PROC_HIGH_PRIORITY.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_DISABLE_FLUSH_INPUT_MEM_CACHE, rknn_run.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_DISABLE_FLUSH_OUTPUT_MEM_CACHE, rknn_mem_sync, RKNN_MEMORY_SYNC_FROM_DEVICE.

This section describes 2.1 rknn_init. Related API or structure: RKNN_FLAG_MODEL_BUFFER_ZERO_COPY, rknn_init, rknn_create_mem, rknn_create_mem2.

This section describes 2.1 rknn_init. Related API or structure: RKNN_MEM_FLAG_ALLOC_NO_CONTEXT, rknn_create_mem2, rknn_destroy_mem.

Example:

rknn_tensor_mem* model_mem = rknn_create_mem2(ctx, model_size,
RKNN_MEM_FLAG_ALLOC_NO_CONTEXT);
memcpy(model_mem->virt_addr, model_data, model_size);
rknn_init_extend init_ext;
 memset(&init_ext, 0, sizeof(rknn_init_extend));
 init_ext.real_model_offset = 0;
 init_ext.real_model_size = model_size;
 init_ext.model_buffer_fd = model_mem->fd;
 init_ext.model_buffer_flags = model_mem->flags;
 int ret = rknn_init(&ctx, model_mem->virt_addr, model_size, RKNN_FLAG_MODEL_BUFFER_ZERO_COPY,
&init_ext);

// do rknn inference...

rknn_destroy_mem(ctx, model_mem);
rknn_destroy(ctx);

2.2 rknn_set_core_mask

This section describes 2.2 rknn_set_core_mask. Related API or structure: rknn_set_core_mask.

Item	Content
API	rknn_set_core_mask
Function	Description of Function.
Parameters	Description of Parameters. Related values: RKNN_NPU_CORE_AUTO RKNN_NPU_CORE_0 RKNN_NPU_CORE_1 RKNN_NPU_CORE_2 RKNN_NPU_CORE_0_1 RKNN_NPU_CORE_0_1_2 RKNN_NPU_CORE_ALL.
Return Value	Description of Return Value.

Example:

rknn_context ctx;
rknn_core_mask core_mask = RKNN_NPU_CORE_0;
int ret = rknn_set_core_mask(ctx, core_mask);

This section describes 2.2 rknn_set_core_mask. Related API or structure: RKNN_NPU_CORE_0_1, RKNN_NPU_CORE_0_1_2.

2.3 rknn_set_batch_core_num

This section describes 2.3 rknn_set_batch_core_num. Related API or structure: rknn_set_batch_core_num, rknn_batch_size.

Item	Content
API	rknn_set_batch_core_num
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_context ctx;
int ret = rknn_set_batch_core_num(ctx, 2);

2.4 rknn_dup_context

This section describes 2.4 rknn_dup_context. Related API or structure: rknn_dup_context.

Item	Content
API	rknn_dup_context
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_context ctx_in;
rknn_context ctx_out;
int ret = rknn_dup_context(&ctx_in, &ctx_out);

2.5 rknn_destroy

This section describes 2.5 rknn_destroy. Related API or structure: rknn_destroy, rknn_context.

Item	Content
API	rknn_destroy
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_context ctx;
int ret = rknn_destroy(ctx);

2.6 rknn_query

This section describes 2.6 rknn_query. Related API or structure: rknn_query.

Item	Content
API	rknn_query
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

This section describes 2.6 rknn_query.

Query Command	returned resultstructure	Description of Query Command.
RKNN_QUERY_IN_OUT_NUM	rknn_input_output_num	Description of RKNN_QUERY_IN_OUT_NUM.
RKNN_QUERY_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_INPUT_ATTR.
RKNN_QUERY_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_OUTPUT_ATTR.
RKNN_QUERY_PERF_DETAIL	rknn_perf_detail	Description of RKNN_QUERY_PERF_DETAIL. Related values: RKNN_FLAG_COLLECT_PERF_MASK.
RKNN_QUERY_PERF_RUN	rknn_perf_run	Description of RKNN_QUERY_PERF_RUN.
RKNN_QUERY_SDK_VERSION	rknn_sdk_version	SDKversion query。
RKNN_QUERY_MEM_SIZE	rknn_mem_size	Description of RKNN_QUERY_MEM_SIZE.
RKNN_QUERY_CUSTOM_STRING	rknn_custom_string	Description of RKNN_QUERY_CUSTOM_STRING.
RKNN_QUERY_NATIVE_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_INPUT_ATTR.
RKNN_QUERY_NATIVE_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_OUTPUT_ATTR.
RKNN_QUERY_NATIVE_NC1HWC2_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_NC1HWC2_INPUT_ATTR. Related values: RKNN_QUERY_NATIVE_INPUT_ATTR.
RKNN_QUERY_NATIVE_NC1HWC2_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_NC1HWC2_OUTPUT_ATTR. Related values: RKNN_QUERY_NATIVE_OUTPUT_ATTR.
RKNN_QUERY_NATIVE_NHWC_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_NHWC_INPUT_ATTR. Related values: RKNN_QUERY_NATIVE_INPUT_ATTR.
RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR. Related values: NHWC.
RKNN_QUERY_DEVICE_MEM_INFO	rknn_tensor_mem	Description of RKNN_QUERY_DEVICE_MEM_INFO.
RKNN_QUERY_INPUT_DYNAMIC_RANGE	rknn_input_range	Description of RKNN_QUERY_INPUT_DYNAMIC_RANGE.
RKNN_QUERY_CURRENT_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_CURRENT_INPUT_ATTR.
RKNN_QUERY_CURRENT_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_CURRENT_OUTPUT_ATTR.
RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR.
RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR	rknn_tensor_attr	Description of RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR.

This section describes 2.6 rknn_query.

1. Query SDK Version

This section describes 1. Query SDK Version. Related API or structure: RKNN_QUERY_SDK_VERSION, rknn_sdk_version.

Example:

rknn_sdk_version version;
ret = rknn_query(ctx, RKNN_QUERY_SDK_VERSION, &version, sizeof(rknn_sdk_version));
printf("sdk api version: %s\n", version.api_version);
printf("driver version: %s\n", version.drv_version);

2. input and output tensors of number of query

This section describes 2. input and output tensors of number of query. Related API or structure: rknn_init, RKNN_QUERY_IN_OUT_NUM, rknn_input_output_num.

Example:

rknn_input_output_num io_num;
ret = rknn_query(ctx, RKNN_QUERY_IN_OUT_NUM, &io_num, sizeof(io_num));
printf("model input num: %d, output num: %d\n", io_num.n_input, io_num.n_output);

3. input tensor of query（API）

This section describes 3. input tensor of query（API）. Related API or structure: rknn_init, RKNN_QUERY_INPUT_ATTR, rknn_tensor_attr.

Example:

rknn_tensor_attr input_attrs[io_num.n_input];
memset(input_attrs, 0, sizeof(input_attrs));
for (int i = 0; i < io_num.n_input; i++) {
    input_attrs[i].index = i;
    ret = rknn_query(ctx, RKNN_QUERY_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr));
}

4. output tensor of query（API）

This section describes 4. output tensor of query（API）. Related API or structure: rknn_init, RKNN_QUERY_OUTPUT_ATTR, rknn_tensor_attr.

Example:

rknn_tensor_attr output_attrs[io_num.n_output];
memset(output_attrs, 0, sizeof(output_attrs));
for (int i = 0; i < io_num.n_output; i++) {
    output_attrs[i].index = i;
    ret = rknn_query(ctx, RKNN_QUERY_OUTPUT_ATTR, &(output_attrs[i]), sizeof(rknn_tensor_attr));
}

5. Model Inference of layer of query

This section describes 5. Model Inference of layer of query. Related API or structure: rknn_run, rknn_query, RKNN_QUERY_PERF_DETAIL, rknn_init, RKNN_FLAG_COLLECT_PERF_MASK.

Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, RKNN_FLAG_COLLECT_PERF_MASK, NULL);
...
ret = rknn_run(ctx, NULL);
...
rknn_perf_detail perf_detail;
ret = rknn_query(ctx, RKNN_QUERY_PERF_DETAIL, &perf_detail, sizeof(perf_detail));

6. Model Inference of of query

This section describes 6. Model Inference of of query. Related API or structure: rknn_run, rknn_query, RKNN_QUERY_PERF_RUN.

Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, 0, NULL);
...
ret = rknn_run(ctx, NULL);
...
rknn_perf_run perf_run;
ret = rknn_query(ctx, RKNN_QUERY_PERF_RUN, &perf_run, sizeof(perf_run));

7. model of memoryuse of query

This section describes 7. model of memoryuse of query. Related API or structure: rknn_init, rknn_query, RKNN_QUERY_MEM_SIZE, RKNN_FLAG_MEM_ALLOC_OUTSIDE.

Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, RKNN_FLAG_MEM_ALLOC_OUTSIDE, NULL);
rknn_mem_size mem_size;
ret = rknn_query(ctx, RKNN_QUERY_MEM_SIZE, &mem_size, sizeof(mem_size));

8. modeluserdefinitionstring of query

This section describes 8. modeluserdefinitionstring of query. Related API or structure: rknn_init, rknn_query, RKNN_QUERY_CUSTOM_STRING.

Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, 0, NULL);
rknn_custom_string custom_string;
ret = rknn_query(ctx, RKNN_QUERY_CUSTOM_STRING, &custom_string, sizeof(custom_string));

9. nativeinput tensor of query（zero-copyAPI）

This section describes 9. nativeinput tensor of query（zero-copyAPI）. Related API or structure: rknn_init, RKNN_QUERY_NATIVE_INPUT_ATTR, RKNN_QUERY_NATIVE_NC1HWC2_INPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr input_attrs[io_num.n_input];
memset(input_attrs, 0, sizeof(input_attrs));
for (int i = 0; i < io_num.n_input; i++) {
   input_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_NATIVE_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr));
}

10. nativeoutput tensor of query（zero-copyAPI）

This section describes 10. nativeoutput tensor of query（zero-copyAPI）. Related API or structure: rknn_init, RKNN_QUERY_NATIVE_OUTPUT_ATTR, RKNN_QUERY_NATIVE_NC1HWC2_OUTPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr output_attrs[io_num.n_output];
memset(output_attrs, 0, sizeof(output_attrs));
for (int i = 0; i < io_num.n_output; i++) {
   output_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_NATIVE_OUTPUT_ATTR, &(output_attrs[i]), sizeof(rknn_tensor_attr));
}

11. NHWCformatnativeinput tensor of query（zero-copyAPI）

This section describes 11. NHWCformatnativeinput tensor of query（zero-copyAPI）. Related API or structure: rknn_init, RKNN_QUERY_NATIVE_NHWC_INPUT_ATTR, rknn_tensor_attr. Example:

rknn_context ctx;
int ret = rknn_init(&ctx, model_data, model_data_size, RKNN_FLAG_COLLECT_PERF_MASK, NULL);
...
ret = rknn_run(ctx, NULL);
...
rknn_perf_detail perf_detail;
ret = rknn_query(ctx, RKNN_QUERY_PERF_DETAIL, &perf_detail, sizeof(perf_detail));

12. NHWCformatnativeoutput tensor of query（zero-copyAPI）

This section describes 12. NHWCformatnativeoutput tensor of query（zero-copyAPI）. Related API or structure: rknn_init, RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr output_attrs[io_num.n_output];
memset(output_attrs, 0, sizeof(output_attrs));
for (int i = 0; i < io_num.n_output; i++) {
   output_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR,
             &(output_attrs[i]), sizeof(rknn_tensor_attr));
}

13. modelbuffer of memory of query

This section describes 13. modelbuffer of memory of query.

This section describes 13. modelbuffer of memory of query. Related API or structure: rknn_init, RKNN_QUERY_DEVICE_MEM_INFO. Example:

rknn_tensor_mem mem_info;
memset(&mem_info, 0, sizeof(mem_info));
ret = rknn_query(ctx, RKNN_QUERY_DEVICE_MEM_INFO, &mem_info, sizeof(mem_info));

14. RKNNmodel supportdynamicinputshapeinformation of query

This section describes 14. RKNNmodel supportdynamicinputshapeinformation of query.

This section describes 14. RKNNmodel supportdynamicinputshapeinformation of query. Related API or structure: rknn_init, RKNN_QUERY_INPUT_DYNAMIC_RANGE, rknn_input_range. Example:

rknn_input_range dyn_range[io_num.n_input];
memset(dyn_range, 0, io_num.n_input * sizeof(rknn_input_range));
for (uint32_t i = 0; i < io_num.n_input; i++) {
   dyn_range[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_INPUT_DYNAMIC_RANGE,
             &dyn_range[i], sizeof(rknn_input_range));
}

15. RKNNmodel currentlyuseinputdynamicshape of query

This section describes 15. RKNNmodel currentlyuseinputdynamicshape of query. Related API or structure: rknn_set_input_shapes, RKNN_QUERY_CURRENT_INPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr cur_input_attrs[io_num.n_input];
memset(cur_input_attrs, 0, io_num.n_input * sizeof(rknn_tensor_attr));
for (uint32_t i = 0; i < io_num.n_input; i++) {
   cur_input_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_CURRENT_INPUT_ATTR,
             &(cur_input_attrs[i]), sizeof(rknn_tensor_attr));
}

16. RKNNmodel currentlyuseoutputdynamicshape of query

This section describes 16. RKNNmodel currentlyuseoutputdynamicshape of query. Related API or structure: rknn_set_input_shapes, RKNN_QUERY_CURRENT_OUTPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr cur_output_attrs[io_num.n_output];
memset(cur_output_attrs, 0, io_num.n_output * sizeof(rknn_tensor_attr));
for (uint32_t i = 0; i < io_num.n_output; i++) {
   cur_output_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_CURRENT_OUTPUT_ATTR,
             &(cur_output_attrs[i]), sizeof(rknn_tensor_attr));
}

17. RKNNmodel currentlyusenativeinputdynamicshape of query

This section describes 17. RKNNmodel currentlyusenativeinputdynamicshape of query. Related API or structure: rknn_set_input_shapes, RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr cur_input_attrs[io_num.n_input];
memset(cur_input_attrs, 0, io_num.n_input * sizeof(rknn_tensor_attr));
for (uint32_t i = 0; i < io_num.n_input; i++) {
   cur_input_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR,
             &(cur_input_attrs[i]), sizeof(rknn_tensor_attr));
}

18. RKNNmodel currentlyusenativeoutputdynamicshape of query

This section describes 18. RKNNmodel currentlyusenativeoutputdynamicshape of query. Related API or structure: rknn_set_input_shapes, RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR, rknn_tensor_attr. Example:

rknn_tensor_attr cur_output_attrs[io_num.n_output];
memset(cur_output_attrs, 0, io_num.n_output * sizeof(rknn_tensor_attr));
for (uint32_t i = 0; i < io_num.n_output; i++) {
   cur_output_attrs[i].index = i;
   ret = rknn_query(ctx, RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR,
             &(cur_output_attrs[i]), sizeof(rknn_tensor_attr));
}

2.6 rknn_inputs_set

This section describes 2.6 rknn_inputs_set. Related API or structure: rknn_inputs_set, rknn_input.

Item	Content
API	rknn_inputs_set
Function	modelinputdata set。
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_input inputs[1];
memset(inputs, 0, sizeof(inputs));
inputs[0].index = 0;
inputs[0].type = RKNN_TENSOR_UINT8;
inputs[0].size = img_width * img_height * img_channels;
inputs[0].fmt = RKNN_TENSOR_NHWC;
inputs[0].buf = in_data;
inputs[0].pass_through = 0;
ret = rknn_inputs_set(ctx, 1, inputs);;

2.7 rknn_run

This section describes 2.7 rknn_run. Related API or structure: rknn_run, rknn_inputs_set.

Item	Content
API	rknn_run
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

ret = rknn_run(ctx, NULL);

2.8 rknn_outputs_get

This section describes 2.8 rknn_outputs_get. Related API or structure: rknn_outputs_get, rknn_output.

Item	Content
API	rknn_outputs_get
Function	Model Inferenceoutput get。
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_output outputs[io_num.n_output];
memset(outputs, 0, sizeof(outputs));
for (int i = 0; i < io_num.n_output; i++) {
    outputs[i].index = i;
    outputs[i].is_prealloc = 0;
    outputs[i].want_float = 1;
}
ret = rknn_outputs_get(ctx, io_num.n_output, outputs, NULL);

2.9 rknn_outputs_release

This section describes 2.9 rknn_outputs_release. Related API or structure: rknn_outputs_release, rknn_outputs_get.

Item	Content
API	rknn_outputs_release
Function	rknn_outputobject release。
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

ret = rknn_outputs_release(ctx, io_num.n_output, outputs);

2.10 rknn_create_mem_from_phys

This section describes 2.10 rknn_create_mem_from_phys. Related API or structure: rknn_create_mem_from_phys, rknn_tensor_mem.

Item	Content
API	rknn_create_mem_from_phys
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

// suppose we have got buffer information as input_phys, input_virt and size
rknn_tensor_mem* input_mems[1];
input_mems[0] = rknn_create_mem_from_phys(ctx, input_phys, input_virt, size);

2.11 rknn_create_mem_from_fd

This section describes 2.11 rknn_create_mem_from_fd. Related API or structure: rknn_create_mem_from_fd, rknn_tensor_mem.

Item	Content
API	rknn_create_mem_from_fd
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

2.12 rknn_create_mem

This section describes 2.12 rknn_create_mem. Related API or structure: rknn_create_mem, rknn_tensor_mem.

Item	Content
API	rknn_create_mem
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

// suppose we have got buffer size
rknn_tensor_mem* input_mems[1];
input_mems[0] = rknn_create_mem(ctx, size);

2.13 rknn_create_mem2

This section describes 2.13 rknn_create_mem2. Related API or structure: rknn_create_mem2, rknn_tensor_mem, rknn_create_mem.

Item	Content
API	rknn_create_mem2
Function	Description of Function.
Parameters	Description of Parameters. Related values: RKNN_FLAG_MEMORY_CACHEABLE RKNN_FLAG_MEMORY_NON_CACHEABLE RKNN_FLAG_MEMORY_FLAGS_DEFAULT RKNN_FLAG_MEMORY_CACHEABLE.
Return Value	Description of Return Value.

2.14 rknn_destroy_mem

This section describes 2.14 rknn_destroy_mem. Related API or structure: rknn_destroy_mem, rknn_tensor_mem.

Item	Content
API	rknn_destroy_mem
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_tensor_mem* input_mems[1];
int ret = rknn_destroy_mem(ctx, input_mems[0]);

2.15 rknn_set_weight_mem

This section describes 2.15 rknn_set_weight_mem. Related API or structure: rknn_tensor_mem, rknn_run, rknn_set_weight_mem.

Item	Content
API	rknn_set_weight_mem
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_tensor_mem* weight_mems[1];
int ret = rknn_set_weight_mem(ctx, weight_mems[0]);

2.16 rknn_set_internal_mem

This section describes 2.16 rknn_set_internal_mem. Related API or structure: rknn_tensor_mem, rknn_run, rknn_set_internal_mem.

Item	Content
API	rknn_set_internal_mem
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_tensor_mem* internal_tensor_mems[1];
int ret = rknn_set_internal_mem(ctx, internal_tensor_mems[0]);

2.17 rknn_set_io_mem

This section describes 2.17 rknn_set_io_mem. Related API or structure: rknn_tensor_mem, rknn_run, rknn_set_io_mem.

Item	Content
API	rknn_set_io_mem
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_tensor_attr output_attrs[1];
rknn_tensor_mem* output_mems[1];
ret = rknn_query(ctx, RKNN_QUERY_NATIVE_OUTPUT_ATTR, &(output_attrs[0]), sizeof(rknn_tensor_attr));
output_mems[0] = rknn_create_mem(ctx, output_attrs[0].size_with_stride);
rknn_set_io_mem(ctx, output_mems[0], &output_attrs[0]);

2.18 rknn_set_input_shape（deprecated）

This section describes 2.18 rknn_set_input_shape（deprecated）. Related API or structure: rknn_set_input_shapes.

Item	Content
API	rknn_set_input_shape
Function	Description of Function.
Parameters	None.
Return Value	None.

2.19 rknn_set_input_shapes

This section describes 2.19 rknn_set_input_shapes. Related API or structure: rknn_tensor_attr, rknn_query.

Item	Content
API	rknn_set_input_shapes
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

for (int i = 0; i < io_num.n_input; i++) {
    for (int j = 0; j < input_attrs[i].n_dims; ++j) {
This section describes the API usage and related parameters.
        input_attrs[i].dims[j] = dyn_range[i].dyn_range[0][j];
    }
}
ret = rknn_set_input_shapes(ctx, io_num.n_input, input_attrs);
if (ret < 0) {
    fprintf(stderr, "rknn_set_input_shapes error! ret=%d\n", ret);
    return -1;
}

2.20 rknn_mem_sync

This section describes 2.20 rknn_mem_sync. Related API or structure: rknn_create_mem.

Item	Content
API	rknn_mem_sync
Function	Description of Function.
Parameters	Description of Parameters. Related values: RKNN_MEMORY_SYNC_TO_DEVICE RKNN_MEMORY_SYNC_FROM_DEVICE RKNN_MEMORY_SYNC_BIDIRECTIONAL.
Return Value	Description of Return Value.

Example:

ret = rknn_mem_sync(ctx, &outputs[0].mem, RKNN_MEMORY_SYNC_FROM_DEVICE);
if (ret < 0) {
    fprintf(stderr, " rknn_mem_sync error! ret=%d\n", ret);
    return -1;
}

3. Matrix Multiplication Data Structure Definitions

3.1 rknn_matmul_info

This section describes 3.1 rknn_matmul_info. Related API or structure: rknn_matmul_info.

Member	Data Type	Description
M	int32_t	Description of M.
K	int32_t	Description of K.
N	int32_t	Description of N.
type	rknn_matmul_type	Description of type. Related values: RKNN_FLOAT16_MM_FLOAT16_TO_FLOAT32 float16 RKNN_INT8_MM_INT8_TO_INT32 RKNN_INT8_MM_INT8_TO_INT8 RKNN_FLOAT16_MM_FLOAT16_TO_FLOAT16 float16 RKNN_FLOAT16_MM_INT8_TO_FLOAT32 float16 RKNN_FLOAT16_MM_INT8_TO_FLOAT16 float16 float16 RKNN_FLOAT16_MM_INT4_TO_FLOAT32 float16 RKNN_FLOAT16_MM_INT4_TO_FLOAT16 float16 float16 RKNN_INT8_MM_INT8_TO_FLOAT32 RKNN_INT4_MM_INT4_TO_INT16 RKNN_INT8_MM_INT4_TO_INT32.
B_layout	int16_t	Description of B_layout. Related values: RKNN_MM_LAYOUT_NORM RKNN_MM_LAYOUT_NATIVE RKNN_MM_LAYOUT_TP_NORM.
B_quant_type	int16_t	Description of B_quant_type. Related values: RKNN_QUANT_TYPE_PER_LAYER_SYM RKNN_QUANT_TYPE_PER_LAYER_ASYM RKNN_QUANT_TYPE_PER_CHANNEL_SYM RKNN_QUANT_TYPE_PER_CHANNEL_ASYM RKNN_QUANT_TYPE_PER_GROUP_SYM RKNN_QUANT_TYPE_PER_GROUP_ASYM.
AC_layout	int16_t	Description of AC_layout. Related values: RKNN_MM_LAYOUT_NORM RKNN_MM_LAYOUT_NATIVE.
AC_quant_type	int16_t	Description of AC_quant_type. Related values: RKNN_QUANT_TYPE_PER_LAYER_SYM RKNN_QUANT_TYPE_PER_LAYER_ASYM.
iommu_domain_id	int32_t	Description of iommu_domain_id.
group_size	int16_t	Description of group_size.
reserved	int8_t[]	Description of reserved.

3.2 rknn_matmul_tensor_attr

This section describes 3.2 rknn_matmul_tensor_attr. Related API or structure: rknn_matmul_tensor_attr.

Member	Data Type	Description
name	char[]	Description of name.
n_dims	uint32_t	Description of n_dims.
dims	uint32_t[]	Description of dims.
size	uint32_t	Description of size.
type	rknn_tensor_type	Description of type.

3.3 rknn_matmul_io_attr

This section describes 3.3 rknn_matmul_io_attr. Related API or structure: rknn_matmul_io_attr.

Member	Data Type	Description
A	rknn_matmul_tensor_attr	Description of A.
B	rknn_matmul_tensor_attr	Description of B.
C	rknn_matmul_tensor_attr	Description of C.

3.4 rknn_quant_params

This section describes 3.4 rknn_quant_params. Related API or structure: rknn_quant_params, rknn_matmul_io_attr.

Member	Data Type	Description
name	char[]	Description of name.
scale	float*	Description of scale.
scale_len	int32_t	Description of scale_len.
zp	int32_t*	Description of zp.
zp_len	int32_t	Description of zp_len.

3.5 rknn_matmul_shape

This section describes 3.5 rknn_matmul_shape. Related API or structure: rknn_matmul_shape.

Member	Data Type	Description
M	int32_t	Description of M.
K	int32_t	Description of K.
N	int32_t	Description of N.

4. Matrix Multiplication API Reference

4.1 rknn_matmul_create

This section describes 4.1 rknn_matmul_create.

Item	Content
API	rknn_matmul_create
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_matmul_info info;
memset(&info, 0, sizeof(rknn_matmul_info));
info.M = 4;
info.K = 64;
info.N = 32;
info.type = RKNN_INT8_MM_INT8_TO_INT32;
info.B_layout = RKNN_MM_LAYOUT_NORM;
info.AC_layout = RKNN_MM_LAYOUT_NORM;
rknn_matmul_io_attr io_attr;
memset(&io_attr, 0, sizeof(rknn_matmul_io_attr));
int ret = rknn_matmul_create(&ctx, &info, &io_attr);
if (ret < 0) {
    printf("rknn_matmul_create fail! ret=%d\n", ret);
    return -1;
}

4.2 rknn_matmul_set_io_mem

This section describes 4.2 rknn_matmul_set_io_mem. Related API or structure: rknn_create_mem, rknn_tensor_mem, rknn_matmul_create, rknn_matmul_tensor_attr, rknn_matmul_info.

Item	Content
API	rknn_matmul_set_io_mem
Function	matrix multiplication of input/outputmemory set。
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

// Create A
rknn_tensor_mem* A = rknn_create_mem(ctx, io_attr.A.size);
if (A == NULL) {
    printf("rknn_create_mem fail!\n");
    return -1;
}
memset(A->virt_addr, 1, A->size);
// Set A
ret = rknn_matmul_set_io_mem(ctx, A, &io_attr.A);
if (ret < 0) {
    printf("rknn_matmul_set_io_mem fail! ret=%d\n", ret);
    return -1;
}

4.3 rknn_matmul_set_core_mask

This section describes 4.3 rknn_matmul_set_core_mask. Related API or structure: rknn_matmul_create.

Item	Content
API	rknn_matmul_set_core_mask
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_matmul_set_core_mask(ctx, RKNN_NPU_CORE_AUTO);

4.4 rknn_matmul_set_quant_params

This section describes 4.4 rknn_matmul_set_quant_params. Related API or structure: rknn_matmul_set_quant_params, rknn_quant_params, rknn_matmul_run.

Item	Content
API	rknn_matmul_set_quant_params
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

rknn_quant_params params_a;
memcpy(params_a.name, io_attr.A.name, RKNN_MAX_NAME_LEN);
params_a.scale_len = 1;
params_a.scale = (float *)malloc(params_a.scale_len * sizeof(float));
params_a.scale[0] = 0.2;
params_a.zp_len = 1;
params_a.zp = (int32_t *)malloc(params_a.zp_len * sizeof(int32_t));
params_a.zp[0] = 0;
rknn_matmul_set_quant_params(ctx, &params_a);

4.5 rknn_matmul_get_quant_params

This section describes 4.5 rknn_matmul_get_quant_params. Related API or structure: rknn_matmul_get_quant_params, rknn_matmul_type, RKNN_INT8_MM_INT8_TO_INT32.

Item	Content
API	rknn_matmul_get_quant_params
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

float b_scale;
rknn_matmul_get_quant_params(ctx, &params_b, &b_scale);

4.6 rknn_matmul_create_dyn_shape（deprecated）

This section describes 4.6 rknn_matmul_create_dyn_shape（deprecated）. Related API or structure: rknn_matmul_create_dynamic_shape.

Item	Content
API	rknn_matmul_create_dyn_shape
Function	Description of Function.
Parameters	None.
Return Value	None.

4.7 rknn_matmul_create_dynamic_shape

This section describes 4.7 rknn_matmul_create_dynamic_shape. Related API or structure: rknn_matmul_create_dynamic_shape, rknn_matmul_info, rknn_matmul_io_attr.

Item	Content
API	rknn_matmul_create_dynamic_shape
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

const int shape_num = 2;
rknn_matmul_shape shapes[shape_num];
for (int i = 0; i < shape_num; ++i) {
    shapes[i].M = i + 1;
    shapes[i].K = 64;
    shapes[i].N = 32;
}
rknn_matmul_io_attr io_attr[shape_num];
memset(io_attr, 0, sizeof(rknn_matmul_io_attr) * shape_num);
int ret = rknn_matmul_create_dynamic_shape(&ctx, &info, shape_num, shapes, io_attr);
if (ret < 0) {
    fprintf(stderr, " rknn_matmul_create_dynamic_shape fail! ret=%d\n", ret);
    return -1;
}

4.8 rknn_matmul_set_dynamic_shape

This section describes 4.8 rknn_matmul_set_dynamic_shape. Related API or structure: rknn_matmul_set_dynamic_shape, rknn_matmul_shape.

Item	Content
API	rknn_matmul_set_dynamic_shape
Function	matrix multiplicationshape set。
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

ret = rknn_matmul_set_dynamic_shape(ctx, &shapes[0]);
if (ret != 0) {
    fprintf(stderr, "rknn_matmul_set_dynamic_shapes fail!\n");
    return -1;
}

4.9 rknn_B_normal_layout_to_native_layout

This section describes 4.9 rknn_B_normal_layout_to_native_layout. Related API or structure: rknn_B_normal_layout_to_native_layout.

Item	Content
API	rknn_B_normal_layout_to_native_layout
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

int32_t subN = io_attr.B.dims[2];
int32_t subK = io_attr.B.dims[3];
rknn_B_normal_layout_to_native_layout(B_Matrix, B->virt_addr, K, N, subN, subK, &info);

4.10 rknn_matmul_run

This section describes 4.10 rknn_matmul_run. Related API or structure: rknn_matmul_set_io_mem, rknn_matmul_create.

Item	Content
API	rknn_matmul_run
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

int ret = rknn_matmul_run(ctx);

4.11 rknn_matmul_destroy

This section describes 4.11 rknn_matmul_destroy. Related API or structure: rknn_matmul_create.

Item	Content
API	rknn_matmul_destroy
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

int ret = rknn_matmul_destroy(ctx);

5. Custom Operator Data Structure Definitions

5.1 rknn_gpu_op_context

This section describes 5.1 rknn_gpu_op_context. Related API or structure: rknn_gpu_op_context.

Member	Data Type	Description
cl_context	void*	Description of cl_context.
cl_command_queue	void*	Description of cl_command_queue.
cl_kernel	void*	Description of cl_kernel.

5.2 rknn_custom_op_context

This section describes 5.2 rknn_custom_op_context. Related API or structure: rknn_custom_op_context.

Member	Data Type	Description
target	rknn_target_type	Description of target. Related values: RKNN_TARGET_TYPE_CPU RKNN_TARGET_TYPE_GPU.
internal_ctx	rknn_custom_op_interal_context	Description of internal_ctx.
gpu_ctx	rknn_gpu_op_context	Description of gpu_ctx.
priv_data	void*	Description of priv_data.

5.3 rknn_custom_op_tensor

This section describes 5.3 rknn_custom_op_tensor. Related API or structure: rknn_custom_op_tensor.

Member	Data Type	Description
attr	rknn_tensor_attr	Description of attr.
mem	rknn_tensor_mem	Description of mem.

5.4 rknn_custom_op_attr

This section describes 5.4 rknn_custom_op_attr. Related API or structure: rknn_custom_op_attr.

Member	Data Type	Description
name	char[]	Description of name.
dtype	rknn_tensor_type	Description of dtype.
n_elems	uint32_t	Description of n_elems.
data	void*	Description of data.

5.5 rknn_custom_op

This section describes 5.5 rknn_custom_op. Related API or structure: rknn_custom_op.

Member	Data Type	Description
version	uint32_t	Description of version.
target	rknn_target_type	Description of target.
op_type	char[]	Description of op_type.
cl_kernel_name	char[]	Description of cl_kernel_name.
cl_kernel_source	char*	Description of cl_kernel_source.
cl_source_size	uint64_t	Description of cl_source_size.
cl_build_options	char[]	Description of cl_build_options.
init	int ()(rknn_custom_op_context op_ctx, rknn_custom_op_tensor* inputs, uint32_t n_inputs, rknn_custom_op_tensor* outputs, uint32_t n_outputs);	Description of init.
prepare	int ()(rknn_custom_op_context op_ctx, rknn_custom_op_tensor* inputs, uint32_t n_inputs, rknn_custom_op_tensor* outputs, uint32_t n_outputs);	Description of prepare.
compute	int ()(rknn_custom_op_context op_ctx, rknn_custom_op_tensor* inputs, uint32_t n_inputs, rknn_custom_op_tensor* outputs, uint32_t n_outputs);	Description of compute.
compute_native	int ()(rknn_custom_op_context op_ctx, rknn_custom_op_tensor* inputs, uint32_t n_inputs, rknn_custom_op_tensor* outputs, uint32_t n_outputs);	Description of compute_native.
destroy	int ()(rknn_custom_op_context op_ctx);	Description of destroy.

6. Custom Operator API Reference

6.1 rknn_register_custom_ops

This section describes 6.1 rknn_register_custom_ops. Related API or structure: rknn_run.

Item	Content
API	rknn_register_custom_ops
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	Description of Return Value.

Example:

// CPU operators
rknn_custom_op user_op[2];
memset(user_op, 0, 2 * sizeof(rknn_custom_op));
strncpy(user_op[0].op_type, "cstSoftmax", RKNN_MAX_NAME_LEN - 1);
user_op[0].version = 1;
user_op[0].target = RKNN_TARGET_TYPE_CPU;
user_op[0].init = custom_op_init_callback;
user_op[0].compute = compute_custom_softmax_float32;
user_op[0].destroy = custom_op_destroy_callback;
strncpy(user_op[1].op_type, "ArgMax", RKNN_MAX_NAME_LEN - 1);
user_op[1].version = 1;
user_op[1].target = RKNN_TARGET_TYPE_CPU;
user_op[1].init = custom_op_init_callback;
user_op[1].compute = compute_custom_argmax_float32;
user_op[1].destroy = custom_op_destroy_callback;
ret = rknn_register_custom_ops(ctx, user_op, 2);
if (ret < 0) {
    printf("rknn_register_custom_ops fail! ret = %d\n", ret);
    return -1;
}

6.2 rknn_custom_op_get_op_attr

This section describes 6.2 rknn_custom_op_get_op_attr. Related API or structure: rknn_custom_op_attr.

Item	Content
API	rknn_custom_op_get_op_attr
Function	Description of Function.
Parameters	Description of Parameters.
Return Value	None.

Example:

rknn_custom_op_attr op_attr;
rknn_custom_op_get_op_attr(op_ctx, "axis", &op_attr);
if (op_attr.n_elems == 1 && op_attr.dtype == RKNN_TENSOR_INT64) {
   axis = ((int64_t*)op_attr.data)[0];
}
…

7. RKNN Return Value Error Codes

This section describes 7. RKNN Return Value Error Codes.

error code	Description.
RKNN_SUCC(0)	executionsuccess。
RKNN_ERR_FAIL(-1)	Description of RKNN_ERR_FAIL(-1).
RKNN_ERR_TIMEOUT(-2)	Description of RKNN_ERR_TIMEOUT(-2).
RKNN_ERR_DEVICE_UNAVAILABLE(-3)	Description of RKNN_ERR_DEVICE_UNAVAILABLE(-3).
RKNN_ERR_MALLOC_FAIL(-4)	Description of RKNN_ERR_MALLOC_FAIL(-4).
RKNN_ERR_PARAM_INVALID(-5)	Description of RKNN_ERR_PARAM_INVALID(-5).
RKNN_ERR_MODEL_INVALID(-6)	Description of RKNN_ERR_MODEL_INVALID(-6).
RKNN_ERR_CTX_INVALID(-7)	Description of RKNN_ERR_CTX_INVALID(-7).
RKNN_ERR_INPUT_INVALID(-8)	Description of RKNN_ERR_INPUT_INVALID(-8).
RKNN_ERR_OUTPUT_INVALID(-9)	Description of RKNN_ERR_OUTPUT_INVALID(-9).
RKNN_ERR_DEVICE_UNMATCH(-10)	Description of RKNN_ERR_DEVICE_UNMATCH(-10).
RKNN_ERR_INCOMPATILE_OPTIMIZATION_LEVEL_VERSION(-12)	Description of RKNN_ERR_INCOMPATILE_OPTIMIZATION_LEVEL_VERSION(-12).
RKNN_ERR_TARGET_PLATFORM_UNMATCH(-13)	Description of RKNN_ERR_TARGET_PLATFORM_UNMATCH(-13).