categories: [asr]
tags: C++ asr kaldi
在MacOS 下跑 kaldi
brew install automake llvm cmake sox libtool subversion
基本安装
Common build problems · pyenv/pyenv Wiki;
brew install pyenv
pyenv install -v 2.7.18
# Set the python version.
pyenv global 2.7.18
# Export PATH if necessary.
export PATH="$(pyenv root)/shims:${PATH}"
Add if necessary.:
echo 'PATH=$(pyenv root)/shims:$PATH' >> ~/.zshrc
还得是用 xcode 的 sdk, brew 很多都显示找不到对应的包, 用了 CPPFLAG也不行
==> /opt/homebrew/bin/pyenv install 2.7.18
Downloading openssl-1.1.1v.tar.gz...
-> https://www.openssl.org/source/openssl-1.1.1v.tar.gz
Installing openssl-1.1.1v...
Installed openssl-1.1.1v to /Users/zorch/.pyenv/versions/2.7.18
Downloading readline-8.0.tar.gz...
-> https://ftpmirror.gnu.org/readline/readline-8.0.tar.gz
Installing readline-8.0...
Installed readline-8.0 to /Users/zorch/.pyenv/versions/2.7.18
Downloading Python-2.7.18.tar.xz...
-> https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz
Installing Python-2.7.18...
patching file configure
patching file configure.ac
patching file setup.py
patching file 'Mac/Tools/pythonw.c'
patching file setup.py
patching file 'Doc/library/ctypes.rst'
patching file 'Lib/test/test_str.py'
patching file 'Lib/test/test_unicode.py'
patching file 'Modules/_ctypes/_ctypes.c'
patching file 'Modules/_ctypes/callproc.c'
patching file 'Modules/_ctypes/ctypes.h'
patching file 'Modules/_ctypes/callproc.c'
patching file setup.py
patching file 'Mac/Modules/qt/setup.py'
patching file setup.py
python-build: use zlib from xcode sdk
Installed Python-2.7.18 to /Users/zorch/.pyenv/versions/2.7.18
还需要加入环境变量
==> pyenv global 2.7.18
√ ~/code/c-cpp_proj/kaldi/tools
==> python2
Python 2.7.18 (default, Dec 16 2023, 14:23:06)
[GCC Apple LLVM 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
==> which python2
/Users/zorch/.pyenv/shims/python2
安装 openblas
gfortran 11 ARM-darwin (Apple M1) build failure · Issue #3222 · OpenMathLib/OpenBLAS;
改版本
==> vi extras/install_openblas.sh
OPENBLAS_VERSION=0.3.20
之前是 0.3.13
==> sw_vers -productVersion
14.1
√ ~/code/c-cpp_proj/kaldi/tools
==> MACOSX_DEPLOYMENT_TARGET=14.1 extras/install_openblas.sh
结果
OpenBLAS build complete. (BLAS CBLAS LAPACK LAPACKE)
OS ... Darwin
Architecture ... arm64
BINARY ... 64bit
C compiler ... CLANG (cmd & version : Apple clang version 15.0.0 (clang-1500.1.0.2.5))
Fortran compiler ... GFORTRAN (cmd & version : GNU Fortran (Homebrew GCC 13.2.0) 13.2.0)
-n Library Name ... libopenblas_armv8-r0.3.20.a
(Single-threading)
WARNING: If you plan to use the dynamic library libopenblas_armv8-r0.3.20.dylib, you must run:
"make PREFIX=/your_installation_path/ install".
(or set PREFIX in Makefile.rule and run make install.
If you want to move the .dylib to a new location later, make sure you change
the internal name of the dylib with:
install_name_tool -id /new/absolute/path/to/libopenblas_armv8-r0.3.20.dylib libopenblas_armv8-r0.3.20.dylib
To install the library, you can run "make PREFIX=/path/to/your/installation install".
/Applications/Xcode.app/Contents/Developer/usr/bin/make -j 11 -f Makefile.install install
Generating openblas_config.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Generating f77blas.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Generating cblas.h in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Copying LAPACKE header files to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/include
Copying the static library to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib
Copying the shared library to /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib
Generating openblas.pc in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/pkgconfig
Generating OpenBLASConfig.cmake in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Generating OpenBLASConfigVersion.cmake in /Users/zorch/code/c-cpp_proj/kaldi/tools/OpenBLAS/install/lib/cmake/openblas
Install OK!
OpenBLAS is installed successfully.
issue 永远滴神
检验
==> extras/check_dependencies.sh
extras/check_dependencies.sh: all OK.
正餐开始
# optional, exec if you have built the old version
# make -j clean depend
cd src/
./configure --use-cuda=no
make -j11
跑一个例子试试
cd egs/yesno/s5
==> make test -j10
更新: 2024-11-06
最新的编译不用这么麻烦了, 安装python2 之后直接用 cmake 安装即可(看 INSTALL.md)
vi cmake/INSTALL.md
## Build with Ninja Generator
```bash
mkdir -p build && cd build
cmake -GNinja -DCMAKE_INSTALL_PREFIX=../dist ..
cmake --build . --target install
```
执行之后会在`./dist/bin` 中生成可执行的工具
安装之后有个小坑, 我这里环境是 MacOS14.6 的, m2 , 执行下面的测试脚本之后提示 dylib 找不到
cd egs/yesno/s5
./run.sh
dyld[90630]: Library not loaded: @rpath/libfstscript.16.dylib
Referenced from: <6A97203D-EF56-3A63-A568-9B24B38894CF> /Users/xx/code/asr_learning/kaldi/dist/bin/fstcompile
Reason: no LC_RPATH's found
zsh: abort
但是这个共享库是存在的, 后来问了一下 GPT, 用下面的脚本给填了一下 rpath 就可以执行了
for i in `ls -l ~/code/asr_learning/kaldi/dist/bin | tr -s ' ' | cut -d' ' -f9`;
do
install_name_tool -add_rpath /Users/x/code/asr_learning/kaldi/dist/lib "/Users/x/code/asr_learning/kaldi/dist/bin/$i" ;
done
注意这里用绝对路径, 否则找不到, 用户名改成自己的.
最后的部分日志信息.
fstdeterminizestar --use-log=true
fsttablecompose data/lang_test_tg/L_disambig.fst data/lang_test_tg/G.fst
fstminimizeencoded
fstpushspecial
fstisstochastic data/lang_test_tg/tmp/LG.fst
0.534295 0.533859
[info]: LG not stochastic.
fstcomposecontext --context-size=1 --central-position=0 --read-disambig-syms=data/lang_test_tg/phones/disambig.int --write-disambig-syms=data/lang_test_tg/tmp/disambig_ilabels_1_0.int data/lang_test_tg/tmp/ilabels_1_0.6521 data/lang_test_tg/tmp/LG.fst
fstisstochastic data/lang_test_tg/tmp/CLG_1_0.fst
0.534295 0.533859
[info]: CLG not stochastic.
make-h-transducer --disambig-syms-out=exp/mono0a/graph_tgpr/disambig_tid.int --transition-scale=1.0 data/lang_test_tg/tmp/ilabels_1_0 exp/mono0a/tree exp/mono0a/final.mdl
fstminimizeencoded
fsttablecompose exp/mono0a/graph_tgpr/Ha.fst data/lang_test_tg/tmp/CLG_1_0.fst
fstdeterminizestar --use-log=true
fstrmsymbols exp/mono0a/graph_tgpr/disambig_tid.int
fstrmepslocal
fstisstochastic exp/mono0a/graph_tgpr/HCLGa.fst
0.5342 -0.000144671
HCLGa is not stochastic
add-self-loops --self-loop-scale=0.1 --reorder=true exp/mono0a/final.mdl exp/mono0a/graph_tgpr/HCLGa.fst
steps/decode.sh --nj 1 --cmd utils/run.pl exp/mono0a/graph_tgpr data/test_yesno exp/mono0a/decode_test_yesno
decode.sh: feature type is delta
steps/diagnostic/analyze_lats.sh --cmd utils/run.pl exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_alignments.log
Overall, lattice depth (10,50,90-percentile)=(1,1,1) and mean=1.1
steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_lattice_depth_stats.log
local/score.sh --cmd utils/run.pl data/test_yesno exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
local/score.sh: scoring with word insertion penalty=0.0,0.5,1.0
%WER 0.00 [ 0 / 232, 0 in , 0 del, 0 ub ] exp/mono0a/decode_te t_ye no/wer_11_1.0
wer=0