TVM性能评估分析（六）

 

 

 Figure 1.  The workflow of development PC, compile, deploy to the device, test, then modify the codes again to see whether it accelerates.

 

 

 Figure 2.   The Android APP takes shared library as input and runs compiled functions on the mobile phone. 

 

 

 Figure 3.  Build TVM functions and NDArrays on a remote device. The ability to cross-compile to different platforms makes it easy to develop on one platform and test on another.

 

 

 Figure 4.  The instruction to build for your Android device. Once the APK is built, sign it using apps/android_rpc/dev_tools and install it on the phone. 

 

 

 Figure 5.  The NNVM compiler support of TVM stack, we can now directly compile descriptions from deep learning frameworks and compile them to bare metal code that runs on AMD GPUs.

 

 

 Figure 6.  With ROCm backend, the generic workflow 

 

 

 Figure 7.   The ONNX library to load the ONNX model into the Protocol buffer object. 

 

 

 Figure 8.  An end to end compilation pipeline from front-end deep learning frameworks to bare metal hardwares.

 

Figure 9.  Typical workflow of NNVM Compiler

 

 

 Figure 10.  Separation of Optimization and Deployment

 

 

 Figure 11.  Time Cost of Inference on K80

 

 

 Figure 12.  The cost of inference on Raspberry PI