TensorFlow Android端编译过程记录

TensorFlow Android端编译过程记录

分享一下TensorFlow Android端编译全过程

*,一定要*,而且要保证下载速度的*!
首先说明:我是深度学习小白,关于深度学习TensorFlow不了解,因为公司需要将图像识别模型放在移动端设备上使用,所以才尝试编译它,万幸最后成功了!

参照

TensorFlow GitHub中的Android端编译指导
当然,编译过程中产生的问题都可以在issues中找找

编译环境

系统:mac os 10.14
bazel: 0.24.1
TensorFlow:1.14.0
Android SDK:23
Android NDK:17c
Python:3.6.3
jdk:1.8

环境搭建

1、安装Bazel 0.24.1

按照官网mac 上步骤安装即可,为什么安装0.24.1,因为其他的我试了,不合适。

2、安装jdk1.8

3、安装python 3.6.3

4、安装Android SDK 23

使用Android Studio或者官网下载安装即可

5、安装Android NDK

在官网下载或百度NDK 17下载后解压到sdk安装目录
为什么一定要是 17c ,因为其他的我都试了,然而只是浪费感情。

6、下载TensorFlow 1.14.0 release版

选择 1.14.0 的原因是,此版本我在mac上编译iOS和Android都成功了
PS:其他版本我也尝试过,编译过程中缺少文件。。。。

编译过程

1、清空编译缓存

如果之前编译过,最好是用一下,如果不用,不知道会出什么问题。

bazel clean --expunge

2、configure 配置TensorFlow编译过程中需要的编译环境

按照自己电脑配置情况按照步骤进行配置即可,以下是我的mac上的一些配置:
终端输入:

./configure

WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.24.1 installed.
Please specify the location of python. [Default is /usr/local/opt/python@2/bin/python2.7]: /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6

Found possible Python library paths:
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
Please input the desired Python library path to use.  Default is [/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages]

Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: n
No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: n
Clang will not be downloaded.

Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: 

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y
Searching for NDK and SDK installations.

Please specify the home path of the Android NDK to use. [Default is /Users/mask/library/Android/Sdk/ndk-bundle]: /Users/mask/Library/Android/sdk/android-ndk-r17c 

Please specify the (min) Android NDK API level to use. [Available levels: ['14', '15', '16', '17', '18', '19', '21', '22', '23', '24', '26', '27', '28']] [Default is 18]: 21

Please specify the home path of the Android SDK to use. [Default is /Users/mask/library/Android/Sdk]: /Users/mask/Library/Android/sdk 

Please specify the Android SDK API level to use. [Available levels: ['23', '26', '29']] [Default is 29]: 23

Please specify an Android build tools version to use. [Available versions: ['.DS_Store', '28.0.3', '29.0.2']] [Default is 29.0.2]: 28.0.3

Do you wish to build TensorFlow with iOS support? [y/N]: n
No iOS support will be enabled for TensorFlow.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=gdr         	# Build with GDR support.
	--config=verbs       	# Build with libverbs support.
	--config=ngraph      	# Build with Intel nGraph support.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=noaws       	# Disable AWS S3 filesystem support.
	--config=nogcp       	# Disable GCP support.
	--config=nohdfs      	# Disable HDFS support.
	--config=noignite    	# Disable Apache Ignite support.
	--config=nokafka     	# Disable Apache Kafka support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished

3、开始编译

终端输入

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so \
   --verbose_failures \
   --crosstool_top=//external:android/crosstool \
   --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
   --cxxopt=-std=c++11 \
   --cpu=armeabi-v7a

开始编译
漫长的等待,前面的编译一般不会出现问题,但是最后关头会出现:

ERROR: /Users/mask/Desktop/tensorflow-1.14.0Android/tensorflow/contrib/android/BUILD:60:1: Linking of rule '//tensorflow/contrib/android:libtensorflow_inference.so' failed (Exit 1): clang failed: error executing command 
  (cd /private/var/tmp/_bazel_mask/acbc0c8b666a3bbc6e9563ed36a12d36/execroot/org_tensorflow && \
  exec env - \
    ANDROID_BUILD_TOOLS_VERSION=28.0.3 \
    ANDROID_NDK_API_LEVEL=21 \
    ANDROID_NDK_HOME=/Users/mask/Library/Android/sdk/android-ndk-r17c \
    ANDROID_SDK_API_LEVEL=23 \
    ANDROID_SDK_HOME=/Users/mask/Library/Android/sdk \
    PATH=/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/mask/bin:/Users/mask/Library/Android/sdk/android-ndk-r17c/ \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 \
    PYTHON_LIB_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages \
    TF_CONFIGURE_IOS=0 \
    TF_DOWNLOAD_CLANG=0 \
    TF_NEED_CUDA=0 \
    TF_NEED_OPENCL_SYCL=0 \
    TF_NEED_ROCM=0 \
  external/androidndk/ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -shared -o bazel-out/armeabi-v7a-opt/bin/tensorflow/contrib/android/libtensorflow_inference.so -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/contrib/android/libandroid_tensorflow_inference_jni.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/java/src/main/native/libnative.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libandroid_tensorflow_lib.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/kernels/libandroid_tensorflow_kernels.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libandroid_tensorflow_lib_lite.lo -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libplatform_base.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/hash/libhash.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/hash/libcity.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/types/libbad_variant_access.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/container/libraw_hash_set.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/types/libbad_optional_access.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/container/libhashtablez_sampler.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/synchronization/libsynchronization.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/synchronization/libgraphcycles_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libstacktrace.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libsymbolize.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libdebugging_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/debugging/libdemangle_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libmalloc_internal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/libtime.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/internal/cctz/libtime_zone.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/time/internal/cctz/libcivil_time.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/strings/libstrings.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/strings/libinternal.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libthrow_delegate.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libbase.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libdynamic_annotations.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/base/libspinlock_wait.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/numeric/libint128.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libstats_calculator_portable.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/double_conversion/libdouble-conversion.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/farmhash_archive/libfarmhash.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/nsync/libnsync_cpp.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/libprotos_all_proto_cc_impl.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/tensorflow/core/liberror_codes_proto_cc_impl.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/fft2d/libfft2d.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/protobuf_archive/libprotobuf.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/protobuf_archive/libprotobuf_lite.a -Wl,-no-whole-archive -Wl,-whole-archive bazel-out/armeabi-v7a-opt/bin/external/zlib_archive/libzlib.a -Wl,-no-whole-archive external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libandroid_support.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++_static.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libc++abi.a external/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/libunwind.a -landroid -latomic -ldl -llog -lm -z defs -s -Wl,--gc-sections -Wl,--version-script,tensorflow/contrib/android/jni/version_script.lds -ldl -lz -pthread -pthread -pthread -framework Foundation -lm -lm -target armv7-none-linux-androideabi -Wl,--fix-cortex-a8 -gcc-toolchain external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64 -no-canonical-prefixes -Lexternal/androidndk/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a -static-libgcc '--sysroot=external/androidndk/ndk/platforms/android-21/arch-arm')
Execution platform: @bazel_tools//platforms:host_platform
external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot open Foundation: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Target //tensorflow/contrib/android:libtensorflow_inference.so failed to build
INFO: Elapsed time: 2413.791s, Critical Path: 324.00s
INFO: 942 processes: 942 local.
FAILED: Build did NOT complete successfully

解决方式
打开文件:bazel-tensorflow/external/com_google_absl/absl/time/internal/cctz/Build.bazel
删除以下Blocks:

config_setting(
	name = "osx",
	constraint_values = [
		"@bazel_tools//platforms:osx",
	],
)

config_setting(
     name = "ios",
     constraint_values = [
         "@bazel_tools//platforms:ios",
     ],
 )

linkopts = select({
     ":osx": [
         "-framework Foundation",
     ],
     ":ios": [
         "-framework Foundation",
     ],
     "//conditions:default": [],
 }),

删除后重新输入,编译

bazel build -c opt //tensorflow/contrib/android:libtensorflow_inference.so \
   --verbose_failures \
   --crosstool_top=//external:android/crosstool \
   --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
   --cxxopt=-std=c++11 \
   --cpu=armeabi-v7a

接下来 .so 文件会编译成功!
位置:bazel-bin/tensorflow/contrib/android/libtensorflow_inference.so
最好是复制一份,因为接下来编译 .jar 会覆盖目录
编译 .jar
终端输入:

bazel build //tensorflow/contrib/android:android_tensorflow_inference_java

此过程很快,一般不会出现问题,编译结果:
bazel-bin/tensorflow/contrib/android/libandroid_tensorflow_inference_java.jar
当然还有libandroid_tensorflow_inference_java-native-header.jar

补充说明

我在交付Android开发人员使用时一些问题:
1、如需支持其他cpu ,请修改编译命令,单独编译:–cpu = armeabi …
2、如armeabi-v7a等不支持64位运算的cpu,运行pb模型时,如果模型使用了64位运算,会出错~,这个如何解决:要不找厂家,要不找开发,自己选。。。

上一篇:linux – 如何在bazel中创建目录结构


下一篇:bazel remote executor--- buildfarm