一. 概述
上篇文章介紹了 ONNX 重頭戲之一的 ”模組轉換”，將數種深度學習框架的模組都轉成 .onnx 格式，能夠更有效的應用模組(model) 資源 !! 而本篇所介紹的方向正好相反，是從 ONNX 官方網站取得豐富的模組資源(.onnx) 來應用。即利用該 模型(.onnx) 轉成想要的模組格式( .tflite / .caffe / .h5 )，比方說 ONNX 轉成 TensorFlow， 亦或是 ONNX 轉成 Caffe ，使得更靈活地對不同格式的模組進行交換 !! 而此篇章的最後，還會推薦一套檢視模組架構的軟體套件 !! 能夠給予開發者檢視上極大的便利 !! 如下圖所示，為系列博文之示意架構圖。此架構圖隸屬於 i.MX8M Plus 的方案博文中，並屬於機器學習內的推理引擎(Inference Engine) 的 ONNX 部分，目前章節介紹 “ONNX 模組資源應用與可視化工具” !!
若欲理解與架設 Tensorflow 框架，請參考另一個博文系列
大大通精彩博文 AI 即將來臨 !! 利用 Tensorflow 與 i.MX8 邁入新領域
大大通精彩博文 【 ATU Book-i.MX8系列】博文索引
ONNX 系列博文-文章架構示意圖 (1)
ONNX 系列博文-文章架構示意圖 (2)
二. ONNX 模型資源
除了將各種深度學習框架轉換為 ONNX 格式之外…
官方亦有提供一系列優化過的標準模型，讓初學開發者更快進行應用。下列提供三種較常見的模組類別，詳細請參照… (如下各小節所示)
官方資源 Model Zoo : https://github.com/onnx/models
(1) 影像分類 Image Classification
|
Model |
Description |
|
Light-weight deep neural network best suited for mobile and embedded vision applications. | |
|
A CNN model (up to 152 layers). Uses shortcut connections to achieve higher accuracy when classifying images. | |
|
A light-weight CNN model providing AlexNet level accuracy with 50x fewer parameters. | |
|
Deep CNN model(up to 19 layers). Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy when classifying images. | |
|
A Deep CNN model (up to 8 layers) where the input is an image and the output is a vector of 1000 numbers. | |
|
Deep CNN model (up to 22 layers). Comparatively smaller and faster than VGG and more accurate in detailing than AlexNet. | |
|
Deep CNN variation of AlexNet for Image Classification in Caffe where the max pooling precedes the local response normalization (LRN) so that the LRN takes less compute and memory. | |
|
Pure Caffe implementation of R-CNN for image classification. This model uses localization of regions to classify and extract features from images. | |
|
Model that has every layer connected to every other layer and passes on its own feature providing strong gradient flow and more diversified features. | |
|
This model is same as GoogLeNet, implemented through Caffe2 that has improved utilization of the computing resources inside the network and helps with the vanishing gradient problem. | |
|
Deep CNN model for Image Classification as an adaptation to Inception v1 with batch normalization. This model has reduced computational cost and improved image resolution compared to Inception v1. | |
|
Extremely computation efficient CNN model that is designed specifically for mobile devices. This model greatly reduces the computational cost and provides a ~13x speedup over AlexNet on ARM-based mobile devices. Compared to MobileNet, ShuffleNet achieves superior performance by a significant margin due to it's efficient structure. | |
|
Extremely computation efficient CNN model that is designed specifically for mobile devices. This network architecture design considers direct metric such as speed, instead of indirect metric like FLOP. | |
|
Deep CNN model (up to 8 layers) that increased the number of features that the network is capable of detecting that helps to pick image features at a finer level of resolution. | |
|
CNN model with an order of magnitude of few computations and parameters, while still acheiving state-of-the-art accuracy and better efficiency than previous ConvNets. |
(2) 物件偵測與分割 Object Detection & Image Segmentation
|
Model |
Description |
|
A real-time CNN for object detection that detects 20 different classes. A smaller version of the more complex full YOLOv2 network. | |
|
Single Stage Detector: real-time CNN for object detection that detects 80 different classes. | |
|
A variant of MobileNet that uses the Single Shot Detector (SSD) model framework. The model detects 80 different object classes and locates up to 10 objects in an image. | |
|
Increases efficiency from R-CNN by connecting a RPN with a CNN to create a single, unified network for object detection that detects 80 different classes. | |
|
A real-time neural network for object instance segmentation that detects 80 different classes. Extends Faster R-CNN as each of the 300 elected ROIs go through 3 parallel branches of the network: label prediction, bounding box prediction and mask prediction. | |
|
A real-time dense detector network for object detection that addresses class imbalance through Focal Loss. RetinaNet is able to match the speed of previous one-stage detectors and defines the state-of-the-art in two-stage detectors (surpassing R-CNN). | |
|
A CNN model for real-time object detection system that can detect over 9000 object categories. It uses a single network evaluation, enabling it to be more than 1000x faster than R-CNN and 100x faster than Faster R-CNN. This model is trained with COCO dataset and contains 80 classes. | |
|
A deep CNN model for real-time object detection that detects 80 different classes. A little bigger than YOLOv2 but still very fast. As accurate as SSD but 3 times faster. | |
|
A smaller version of YOLOv3 model. | |
|
Optimizes the speed and accuracy of object detection. Two times faster than EfficientDet. It improves YOLOv3's AP and FPS by 10% and 12%, respectively, with mAP50 of 52.32 on the COCO 2017 dataset and FPS of 41.7 on a Tesla V100. | |
|
Deep CNN based pixel-wise semantic segmentation model with >80% mIOU (mean Intersection Over Union). Trained on cityscapes dataset, which can be effectively implemented in self driving vehicle systems. |
(3) 身體 / 人臉 / 手勢分析 Body, Face & Gesture Analysis
|
Model |
Description |
|
A CNN based model for face recognition which learns discriminative features of faces and produces embeddings for input face images. | |
|
This model is a lightweight facedetection model designed for edge computing devices. | |
|
Deep CNN for emotion recognition trained on images of faces. |
三. ONNX 模組資源轉換
可以利用現有的 ONNX 模型將其應用至其他深度學習的框架上，比如說 ONNX 模組轉成 TensorFlow 的形式，亦或是轉成 PyTorch 形式。其好處是能透過 ONNX 達到硬體優化之外，同時亦須承擔模組轉換之損失資訊的風險，故以下提供幾個常用的轉換或應用方式。
(1) ONNX to Keras
官方資源 : https://github.com/gmalivenko/onnx2keras
代碼演示 :
import onnx
import keras
from onnx2keras import onnx_to_keras
onnx_model = onnx.load(ONNX_Model)
k_model = onnx_to_keras(onnx_model, [input_name])
keras_model = ONNX_Model[:-4] + 'h5'
keras.models.save_model(k_model, keras_model,overwrite=True,include_optimizer=True)
(2) ONNX to Caffe
官方資源 : https://github.com/onnx/tutorials/blob/master/tutorials/OnnxCaffe2Import.ipynb
代碼演示 :
import numpy as np
import onnx
import caffe2.python.onnx.backend
model = onnx.load(ONNX_Model) # Load the ONNX model
image = np.random.randn(1, nChannel, width, height).astype(np.float32)
outputs = caffe2.python.onnx.backend.run_model(model, [image]) # Run the ONNX model with Caffe2
(3) ONNX to Pytorch
官方資源 : https://pythonrepo.com/repo/fumihwh-onnx-pytorch
代碼演示 :
from onnx_pytorch import code_gen
code_gen.gen(ONNX_Model, "./")
(4) ONNX to CTNK
官方資源 : https://docs.aws.amazon.com/zh_tw/dlami/latest/devguide/tutorial-onnx-pytorch-cntk.html
代碼演示 :
import cntk as C
z = C.Function.load(ONNX_Model, format=C.ModelFormat.ONNX)
(5) ONNX to MXnet
官方資源 : https://github.com/onnx/tutorials/blob/master/tutorials/OnnxMxnetImport.ipynb
代碼演示 :
import mxnet as mx
import mxnet.contrib.onnx as onnx_mxnet
sym, arg_params, aux_params = onnx_mxnet.import_model(ONNX_Model)
mod = mx.mod.Module(symbol=sym, data_names=[input_name], label_names=None)
mod.bind(for_training=False, data_shapes=[(input_name , input_size)])
mod.set_params(arg_params=arg_params, aux_params=aux_params, allow_missing=True, allow_extra=True)
mod.save_checkpoint(OutputPath,0)
(6) ONNX to CoreML
官方資源 : https://github.com/onnx/tutorials/blob/master/tutorials/OnnxCoremlImport.ipynb
代碼演示 :
import sys
from onnx import onnx_pb
from onnx_coreml import convert
model_file = open(ONNX_Model, 'rb')
model_proto = onnx_pb.ModelProto()
model_proto.ParseFromString(model_file.read())
coreml_model = convert(model_proto, image_input_names=[input_name], image_output_names=[output_name])
coreml_model.save( ONNX_Model[:-4] + 'mlmodel' )
(7) ONNX to TensorFlow
官方資源 : https://github.com/onnx/tutorials/blob/master/tutorials/OnnxTensorflowImport.ipynb
代碼演示 :
import onnx
from onnx_tf.backend import prepare
onnx_model = onnx.load(ONNX_Model)
tf_rep = prepare(onnx_model)
tf_rep.export_graph(ONNX_Model[:-4] + 'pb')
四. ONNX 模型結構視覺化工具
ONNX 格式提供兩種可視覺化的 API ，讓使用者可以快速檢視模型與分析模型。
Netron 快速檢視模型
Netron是一個提供數種模型格式的檢視器。
支援模型格式 :
ONNX (.onnx) 、Keras ( .h5, .keras) 、TensorFlow Lite (.tflite) 、Caffe (.caffemodel) 、Darknet (.cfg) 、Core ML (.mlmodel) 、MNN (.mnn) 、MXNet (.model, -symbol.json) 、ncnn (.param) 、Caffe2 (predict_net.pb) 、PaddlePaddle (.zip, __model__) 、Barracuda (.nn) 、Tengine (.tmfile) 、TNN (.tnnproto) 、RKNN (.rknn) 、MindSpore Lite (.ms) 、UFF (.uff)
檢視模型方式 :
連結此網站 https://netron.app/ 即可 !!
檢視模型 :
五. 結語
經由上述的介紹，相信各位已經大致上了解如何取得官方所提供的現有模組資源 ONNX ，以及如何將其轉換成所需的深度學習框架的模組格式。然而，就如同上篇文章所敘述，ONNX 算是新穎的技術，在各模組轉換中難免會有失敗或是無法使用，這通常都與版本的匹配息息相關，這是須值得留意的 !! 本篇最後也介紹了極為簡單的 Netron 讓開發者可以快速地檢視模組 !! 而下一篇文章將會如何將 ONNX 模組進行優化以及推理引擎該如何應用!! 敬請期待 !!
六. 參考文件
[1] 官方網站 - ONNX
[2] 官方網站 - ONNX Tutorials
[3] 官方網站 – Netron
[4] 官方網站 – VisualDL
[5] 參考文獻 - VisualDL 工具简介
如有任何相關 ONNX 技術問題，歡迎至博文底下留言提問 !!
接下來還會分享更多 ONNX 的技術文章 !!敬請期待 【ATU Book-i.MX8 系列 - ONNX】系列 !!
評論