AidLite Python API

Model Data Types class DataType

For AidliteSDK, it handles different models from different frameworks, and each model has its own different input data types and different output data types. In the aforementioned usage workflow, you need to set the input and output data types of the model, which requires using this data type.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid DataType type
TYPE_UINT8	int	1	Unsigned byte data
TYPE_INT8	int	2	Byte data
TYPE_UINT32	int	3	Unsigned int32 data
TYPE_FLOAT32	int	4	Float data
TYPE_INT32	int	5	Int32 data
TYPE_INT64	int	6	Int64 data
TYPE_UINT64	int	7	Uint64 data
TYPE_INT16	int	8	Int16 data
TYPE_UINT16	int	9	Uint16 data
TYPE_FLOAT16	int	10	Float16 data
TYPE_BOOL	int	11	Bool data

Inference Implementation Type class ImplementType

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid ImplementType type
TYPE_MMKV	int	1	Implementation through MMKV (currently no related implementation, may be removed later)
TYPE_REMOTE	int	2	Backend implementation through IPC
TYPE_LOCAL	int	3	Backend implementation through local calls

Model Framework Type class FrameworkType

As mentioned earlier, AidLite SDK integrates multiple deep learning inference frameworks, so in the aforementioned usage workflow, you need to set which framework's model is currently being used, which requires using this framework type.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid FrameworkType type
TYPE_SNPE	int	1	SNPE 1.x (DLC) model
TYPE_TFLite	int	2	TFLite model
TYPE_RKNN	int	3	RKNN model
TYPE_QNN	int	4	QNN model
TYPE_SNPE2	int	5	SNPE 2.x (DLC) model
TYPE_NCNN	int	6	NCNN model
TYPE_MNN	int	7	MNN model
TYPE_TNN	int	8	TNN model
TYPE_PADDLE	int	9	Paddle model
TYPE_MS	int	10	MindSpore model
TYPE_ONNX	uint8_t	11	ONNX model
TYPE_QNN216	uint8_t	101	QNN2.16 model
TYPE_SNPE216	uint8_t	102	SNPE2.16 model
TYPE_QNN223	uint8_t	103	QNN2.23 model
TYPE_SNPE223	uint8_t	104	SNPE2.23 model
TYPE_QNN229	uint8_t	105	QNN2.29 model
TYPE_SNPE229	uint8_t	106	SNPE2.29 model
TYPE_QNN231	uint8_t	107	QNN2.31 model
TYPE_SNPE231	uint8_t	1068	SNPE2.31 model

[!TIP] 💡Note Special notes:

• If the user environment has multiple backend implementation software packages such as AidLite-QNN216/AidLite-QNN223/AidLite-QNN229 installed simultaneously, and the FrameworkType in the code is set to TYPE_QNN (i.e., without specifying which specific version of QNN backend to use), then the actual running QNN backend should be the one with the highest QNN version (i.e., AidLite-QNN229 will be prioritized to complete inference).
• Different QNN version backend software packages cannot be used simultaneously in the same program, which means if the program first specifies TYPE_QNN216 to complete inference, then TYPE_QNN223/TYPE_QNN229 cannot be used for inference.

Inference Acceleration Hardware Type class AccelerateType

For each deep learning inference framework, it may support running on different acceleration hardware units (such as SNPE/QNN models running on Qualcomm DSP computing units, RKNN models running on Rockchip NPU computing units), so in the aforementioned usage workflow, you need to set which computing unit the current model is expected to run on, which requires using this acceleration hardware unit type.

Member Variable Name	Type	Value	Description
TYPE_DEFAULT	int	0	Invalid AccelerateType type
TYPE_CPU	int	1	CPU general acceleration unit
TYPE_GPU	int	2	GPU general acceleration unit
TYPE_DSP	int	3	Qualcomm DSP acceleration unit
TYPE_NPU	int	4	NPU general acceleration unit

Log Level class LogLevel

AidliteSDK provides an interface for setting logs (which will be introduced later), which requires providing the interface with which log level to use currently, so this log level needs to be used.

Member Variable Name	Type	Value	Description
INFO	int	0	Information
WARNING	int	1	Warning
ERROR	int	2	Error
FATAL	int	3	Fatal error

Input/Output Information class TensorInfo

Each model has its own inputs and outputs. For models that developers are not familiar with, during development they may need to query the input and output Tensor information of the model to facilitate developers in performing pre and post-processing operations for model inference. AidLite provides developers with an interface to query detailed information about model inputs and outputs.

Member Variable List

The TensorInfo object is used to store detailed information about the model's input and output Tensors, including the following parameters:

Field 1	Field 2
Member Variable	version
Type	uint32_t
Default Value	No default value
Description	This member is used to identify the TensorInfo version. If more information needs to be extended in the future, the version number will be updated

Field 1	Field 2
Member Variable	index
Type	uint32_t
Default Value	No default value
Description	Index value of the Tensor

Field 1	Field 2
Member Variable	name
Type	std::string
Default Value	No default value
Description	Name string of the Tensor

Field 1	Field 2
Member Variable	element_count
Type	uint64_t
Default Value	0
Description	Number of data elements in the Tensor

Field 1	Field 2
Member Variable	element_type
Type	enum DataType
Default Value	DataType::TYPE_DEFAULT
Description	Type of data elements in the Tensor

Field 1	Field 2
Member Variable	dimensions
Type	uint32_t
Default Value	0
Description	Dimension value of data elements in the Tensor

Field 1	Field 2
Member Variable	shape
Type	std::vector< uint64_t >
Default Value	No default value
Description	Shape description of data elements in the Tensor

Model class Model

As mentioned earlier, before creating an inference interpreter, you need to set specific detailed parameters related to the model. The Model class is mainly used to record the model's file information, structural information, and model-related content during runtime.

Create Model Instance Object create_instance()

To set detailed information related to the model, you naturally need to have a model instance object first. This function is used to create a Model instance object.

Field 1	Field 2
API	create_instance
Description	Constructs a Model type instance object by passing the path name of the model file
Parameters	model_path: Path name of the model file
Return Value	Normal: Model instance object
Exception: None	—

# Create model object using inceptionv3_float32.dlc file in current path, error if return value is empty
model = aidlite.Model.create_instance(model_path=r"./inceptionv3.dlc")
if model is None:
    print("Create model failed !")
    return False

Set Model Properties set_model_properties()

After the model instance object is created successfully, you need to set the input and output data types of the model and the shape information of the input and output tensor data.

Field 1	Field 2
API	set_model_properties
Description	Set model properties, input and output data shape and data types.
Parameters	input_shapes: Shape array of input tensors, two-dimensional array structure.
input_data_type: Data type of input tensors, DataType enumeration type.	—
output_shapes: Shape array of output tensors, two-dimensional array structure.	—
output_data_type: Data type of output tensors, DataType enumeration type.	—
Return Value	Model file path string object corresponding to the Model object

# Data type uses the aforementioned DataType, input and output shapes are two-dimensional arrays
input_shapes=[[1,640,640,3]]
output_shapes=[[1,10,10,255], [1,20,20,255], [1,40,40,255]]
model.set_model_properties(input_shapes=input_shapes, input_data_type=aidlite.DataType.TYPE_FLOAT32, output_shapes=output_shapes, output_data_type=aidlite.DataType.TYPE_FLOAT32)

Get Model Path get_model_absolute_path()

Field 1	Field 2
API	get_model_absolute_path
Description	Used to get the existing path of the model file
Parameters	None
Return Value	Model file path string object corresponding to the Model object

model_path = model.get_model_absolute_path()

Get Model Framework Type get_model_type()

Field 1	Field 2
API	get_model_type
Description	Used to get the model type identifier, such as different model categories like DLC, RKNN, etc.
Parameters	None
Return Value	Model file path string object corresponding to the Model object

model_type = model.get_model_type()

Configuration class Config

As mentioned earlier, before creating an inference interpreter, in addition to setting specific Model information, you also need to set some configuration information for inference. The Config class is used to record configuration options that need to be set in advance, and these configuration items will be used at runtime.

Create Config Instance Object create_instance()

To set runtime configuration information, you naturally need to have a configuration instance object first. This function is used to create a Config instance object.

Field 1	Field 2
API	create_instance
Description	Used to construct an instance object of the Config class
Parameters	snpe_out_names: List of model output node names (optional)
number_of_threads: Number of threads, valid when greater than 0 (optional)	—
is_quantify_model: Whether it's a quantized model, 1 indicates quantized model for FAST (optional)	—
fast_timeout: Interface timeout (milliseconds, valid when greater than 0) for FAST (optional)	—
accelerate_type: Type of acceleration hardware (optional)	—
framework_type: Type of underlying deep learning framework (optional)	—
Return Value	Normal: Config instance object
Exception: None	—

# Create configuration instance object, error if return value is None
config = aidlite.Config.create_instance()
if config is None:
    print("Create config failed !")
    return False

Member Variable List

The Config object is used to set runtime configuration information, including the following parameters:

Field 1	Field 2
Member Variable	accelerate_type
Type	class AccelerateType
Default Value	AccelerateType.TYPE_CPU
Description	Type of acceleration hardware

---

Field 1	Field 2
Member Variable	implement_type
Type	class ImplementType
Default Value	ImplementType.TYPE_LOCAL
Description	Distinction of underlying function implementation approach

---

Field 1	Field 2
Member Variable	framework_type
Type	class FrameworkType
Default Value	FrameworkType.TYPE_DEFAULT
Description	Type of underlying inference framework

---

Field 1	Field 2
Member Variable	number_of_threads
Type	int
Default Value	-1
Description	Number of threads, valid when greater than 0

---

Field 1	Field 2
Member Variable	SNPE_out_names
Type	list
Default Value	None
Description	List of model output node names

---

Field 1	Field 2
Member Variable	is_quantify_model
Type	int
Default Value	0
Description	Whether it's a quantized model, 1 indicates quantized model (only Remote needs to be set)

---

Field 1	Field 2
Member Variable	remote_timeout
Type	int
Default Value	-1
Description	Interface timeout (milliseconds, valid when greater than 0) (only Remote needs to be set)

---

config.framework_type = aidlite.FrameworkType.TYPE_SNPE
config.accelerate_type = aidlite.AccelerateType.TYPE_DSP
config.is_quantify_model = 1
config.SNPE_out_names = ["InceptionV3/Softmax"]
config.fast_timeout = 1000

Context Class class Context

Used to store runtime context involved in the execution process, including Model objects and Config objects, and may be extended to include runtime data in the future.

Constructor Context()

Field 1	Field 2
API	Context
Description	Construct a Context instance object
Parameters	model: Model instance object
config: Config instance object	—
Return Value	Normal: Context instance object
Exception: None	—

context = aidlite.Context(model=model, config=config)

Get Model Member Variable get_model()

Field 1	Field 2
API	get_model
Description	Get the Model object managed by context
Parameters	None
Return Value	Model object

model = context.get_model()

Get Config Member Variable get_config()

Field 1	Field 2
API	get_config
Description	Get the Config object managed by context
Parameters	None
Return Value	Config object

config = context.get_config()

Interpreter Class class Interpreter

The Interpreter class object instance is the main entity that executes inference operations and is used to perform specific inference processes. In the inference workflow mentioned earlier, after creating the interpreter object, all operations are completed based on the interpreter object, making it the absolute core content of AidliteSDK.

Create Interpreter Instance Object create_instance()

To execute inference-related operations, the inference interpreter is essential. This function is used to build an instance object of the inference interpreter.

Field 1	Field 2
API	create_instance
Description	Use the various data managed by the Context object to construct an Interpreter type object.
Parameters	None
Return Value	Normal: Interpreter instance object
Exception: None	—

# Create interpreter object, return value of None indicates error
interpreter = aidlite.Interpreter.create_instance()
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Initialize init()

After the interpreter object is created, some initialization operations need to be performed (such as environment checking, resource construction, etc.).

Field 1	Field 2
API	create_instance
Description	Use the various data managed by the Context object to construct an Interpreter type object.
Parameters	context: Context object instance. It manages Model and Config objects, which contain model data, configuration data, etc.
Return Value	Normal: 0
Exception: Non-zero	—

# Initialize interpreter, non-zero return value indicates error
result = aidlite.interpreter.init()
if result != 0:
    printf("sample : interpreter->init() failed !")
    return False

Load Model load_model()

After the interpreter object completes the initialization operation, the required model file can be loaded for the interpreter to complete the model loading work. The subsequent inference process uses the loaded model resources.

Field 1	Field 2
API	load_model
Description	Complete model loading related work. Since the model file path has been set in the Model object previously, the model loading operation can be executed directly
Parameters	None
Return Value	Normal: 0
Exception: Non-zero	—

# Load model with interpreter, non-zero return value indicates error
result = interpreter.load_model()
if result != 0:
    print("sample : interpreter->load_model() failed !")
    return False

Get Input Tensor Details get_input_tensor_info()

If you want to get detailed information about the model's input Tensor, you can use this interface to obtain it.

Field 1	Field 2
API	get_input_tensor_info
Description	Get detailed information about input Tensors in the model
Return Value	Normal: Two-dimensional array of TensorInfo type data
Exception: None	—
Special Note	This interface is supported from version 2.2.5 onwards

# Get detailed information about model input Tensors
input_tensor_info = onnx_interpreter.get_input_tensor_info()
if len(input_tensor_info) == 0 :
    printf("interpreter get_input_tensor_info() failed !\n")
    return False
for gi, graph_tensor_info in enumerate(input_tensor_info):
    for ti, tensor_info in enumerate(graph_tensor_info):
        print(  f"Input  tensor : Graph[{gi}]-Tensor[{ti}]-name[{tensor_info.name}]"
                f"-element_count[{tensor_info.element_count}]-element_type[{tensor_info.element_type}]"
                f"-dimensions[{tensor_info.dimensions}]-shape{tensor_info.shape}")

Get Output Tensor Details get_output_tensor_info()

If you want to get detailed information about the model's output Tensor, you can use this interface to obtain it.

Field 1	Field 2
API	get_output_tensor_info
Description	Get detailed information about output Tensors in the model
Return Value	Normal: Two-dimensional array of TensorInfo type data
Exception: None	—
Special Note	This interface is supported from version 2.2.5 onwards

# Get detailed information about model output Tensors
output_tensor_info = onnx_interpreter.get_output_tensor_info()
if len(output_tensor_info) == 0 :
    printf("interpreter get_output_tensor_info() failed !\n")
    return False
for gi, graph_tensor_info in enumerate(output_tensor_info):
    for ti, tensor_info in enumerate(graph_tensor_info):
        print(  f"Output tensor : Graph[{gi}]-Tensor[{ti}]-name[{tensor_info.name}]"
                f"-element_count[{tensor_info.element_count}]-element_type[{tensor_info.element_type}]"
                f"-dimensions[{tensor_info.dimensions}]-shape{tensor_info.shape}")

Set Input Data set_input_tensor()

As mentioned when introducing the workflow, before setting input data, different preprocessing operations need to be completed for different models to adapt to specific models.

Field 1	Field 2
API	set_input_tensor
Description	Execute inference computation process
Parameters	in_tensor_tag: Index value or name string of input tensor, type is int or str
input_data: Binary data of input tensor, type is passed according to actual situation	—
Return Value	Normal: 0
Exception: Non-zero	—
Special Note	Parameter in_tensor_tag supports name string from version 2.2.5 onwards

# Set input data for inference, non-zero return value indicates error
result = interpreter.set_input_tensor(in_tensor_tag=1, input_data=obj)
#result = interpreter.set_input_tensor(in_tensor_tag="input_tensor_name", input_data=obj)
if result != 0:
    print("interpreter->set_input_tensor() failed !")
    return False

Execute Inference invoke()

As mentioned when introducing the workflow, after setting the input data, the next step is naturally to execute the inference running process on the input data.

Field 1	Field 2
API	invoke
Description	Execute inference computation process
Parameters	None
Return Value	Normal: 0
Exception: Non-zero	—

# Execute inference operation, non-zero return value indicates error
result = interpreter.invoke()
if result != 0:
    print("sample : interpreter->invoke() failed !")
    return False

Get Output Data get_output_tensor()

After inference is completed, the result data obtained from inference needs to be extracted. As mentioned when introducing the workflow, after extracting the result data, this result data can be processed to determine whether the result is correct.

Field 1	Field 2
API	get_output_tensor
Description	After successful inference, get inference result data
Parameters	out_tensor_tag: Index value or name string of result output tensor, type is int or str
output_type: Type of result output, optional parameter, defaults to aidlite.DataType.TYPE_FLOAT32 if not passed	—
Return Value	Normal: float result data.
Exception: None.	—
Special Note	Parameter out_tensor_tag supports name string from version 2.2.5 onwards

# Get inference result data, return value of None indicates error
out_data = interpreter.get_output_tensor(out_tensor_tag=1, output_type=aidlite.DataType.TYPE_INT32)
#out_data = interpreter.get_output_tensor(out_tensor_tag="output_tensor_name", output_type=aidlite.DataType.TYPE_INT32)
if out_data is None:
    print("interpreter->get_output_tensor() failed !")
    return False

Resource Release destroy()

As mentioned earlier, the interpreter object needs to perform init initialization operations and load model operations. Correspondingly, the interpreter also needs to perform some release operations to destroy the previously created resources.

Field 1	Field 2
API	destroy
Description	Complete necessary release operations
Parameters	None
Return Value	Normal: 0
Exception: Non-zero	—

# Execute interpreter release process, non-zero return value indicates error
result = interpreter.destroy()
if result != 0:
    print("sample : interpreter-> destroy() failed !")
    return False

Interpreter Builder Class class InterpreterBuilder

Unified creation function for Interpreter objects, create required interpreter objects through this class.

Build Interpreter build_interpreter_from_path()

Build inference interpreter object, can provide different parameters, the simplest is to provide only the path and name of the model file.

Field 1	Field 2
API	build_interpreter_from_path
Description	Directly create corresponding interpreter object through the path name of the model file. All related parameters use default values
Parameters	path: Path name of the model file
Return Value	Normal: Interpreter object instance
Exception: None	—

# Build interpreter by passing Model file path, return value of None indicates error
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_path(path=r"./640.dlc")
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Build Interpreter build_interpreter_from_model()

Build inference interpreter object, in addition to providing the model file path, you can also provide a Model object, so that you can not only set the path name of the model file, but also set the input and output data types of the model and the shape information of input and output data.

Field 1	Field 2
API	build_interpreter_from_model
Description	Create corresponding interpreter object by passing Model object. Config related parameters all use default values
Parameters	model: Model type object containing model-related data
Return Value	Normal: Interpreter object instance
Exception: None	—

# Build interpreter by passing Model object, return value of None indicates error
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_model(model=model)
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Build Interpreter build_interpreter_from_model_and_config()

Build inference interpreter object, in addition to the two methods mentioned above, you can also provide both Model object and Config object, so that you can not only provide model-related information, but also provide more runtime configuration parameters.

Field 1	Field 2
API	build_interpreter_from_model_and_config
Description	Create corresponding interpreter object by passing Model object and Config.
Parameters	model: Model type object containing model-related data
config: Config type object containing some configuration parameters	—
Return Value	Normal: Interpreter object instance
Exception: None	—

# Build interpreter by passing Model and Config objects, return value of None indicates error
interpreter = aidlite.InterpreterBuilder.build_interpreter_from_model_and_config(model=model, config=config)
if interpreter is None:
    print("Create Interpreter failed !")
    return False

Other Methods

Get SDK Version Information get_library_version()

Field 1	Field 2
API	get_library_version
Description	Used to get version-related information of the current AidLite-SDK
Parameters	None
Return Value	Returns version information string of the current AidLite-SDK

Get Python SDK Version Information get_library_version()

Field 1	Field 2
API	get_py_library_version
Description	Used to get version-related information of the current Py-AidLite-SDK
Parameters	None
Return Value	Returns version information string of the current Py-AidLite-SDK

Set Log Level set_log_level()

Field 1	Field 2
API	set_log_level
Description	Set the current minimum log level, output log data greater than or equal to this log level. By default, prints WARNING and above level logs.
Parameters	log_level: LogLevel type value
Return Value	Returns 0 by default

Log Output to Standard Terminal log_to_stderr()

Field 1	Field 2
API	log_to_stderr
Description	Set log information output to standard error terminal
Parameters	None
Return Value	Returns 0 by default

Log Output to Text File log_to_file()

Field 1	Field 2
API	log_to_file
Description	Set log information output to specified text file
Parameters	path_and_prefix: Storage path and name prefix for log file
also_to_stderr: Flag indicating whether to simultaneously output logs to stderr terminal, default value is False	—
Return Value	Normal: 0
Exception: Non-zero	—

Get Latest Log Information last_log_msg()

Field 1	Field 2
API	last_log_msg
Description	Get the latest log information for a specific log level, typically used to retrieve the latest error information
Parameters	log_level: Value of LogLevel type
Return Value	Latest log information