1 问题描述
在使用cargo进行复杂软件安装时, 依赖比较多的库, 需要快速下载, 出现了该问题“Couldn‘t resolve host name (Could not resolve host: crates)”, 错误日志如下:
1 $ cargo install mdbook 2 Updating `git://mirrors.ustc.edu.cn/crates.io-index` index 3 Installing mdbook v0.4.4 4 Downloaded byteorder v1.3.4 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) 5 Downloaded inotify-sys v0.1.4 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) 6 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 7 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 8 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 9 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 10 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 11 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 12 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 13 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 14 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 15 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 16 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 17 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 18 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 19 warning: spurious network error (2 tries remaining): [6] Couldn‘t resolve host name (Could not resolve host: crates) 20 Downloaded input_buffer v0.3.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) 21 。。。// 跳过部分日志 22 error: failed to compile `mdbook v0.4.4`, intermediate artifacts can be found at `/tmp/cargo-installzrH2PU` 23 24 Caused by: 25 failed to download from `https://crates-io.proxy.ustclug.org/api/v1/crates/tokio-tungstenite/0.11.0/download` 26 27 Caused by: 28 [6] Couldn‘t resolve host name (Could not resolve host: crates)
看到该问题, 还特意检查了网络, 设置了dns,
在文件 /etc/resolv.conf 中增加了阿里的dns域名服务器, 问题没有解决;
nameserver 223.5.5.5 nameserver 223.6.6.6
查找资料找到了问题根因和解决办法;
2 解决办法
临时规避,禁止并行化下载安装, 命令行输入: CARGO_HTTP_MULTIPLEXING=false cargo install mdbook
$ CARGO_HTTP_MULTIPLEXING=false cargo install mdbook pdating `git://mirrors.ustc.edu.cn/crates.io-index` index Installing mdbook v0.4.4 Downloaded cfg-if v0.1.10 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded bytes v0.5.6 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded futures-io v0.3.8 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded http-body v0.3.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded markup5ever_rcdom v0.1.0 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded num-integer v0.1.44 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded phf_shared v0.8.0 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded rand_pcg v0.2.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) Downloaded serde_urlencoded v0.6.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`) 。。。 Compiling markup5ever_rcdom v0.1.0 Compiling ammonia v3.1.0 Compiling hyper v0.13.9 Compiling warp v0.2.5 Compiling mdbook v0.4.4 Finished release [optimized] target(s) in 3m 01s Installing /home/neo/.cargo/bin/mdbook Installed package `mdbook v0.4.4` (executable `mdbook`)
编译时间略长, 也还可以接受。。。
3 引申阅读
问题根因分析:
详见:https://github.com/ustclug/discussions/issues/294
主要原因如下:
knight42 commented on Apr 30 我找到这个问题的原因了,是因为我们的 nginx 会根据 client ip 对请求数作限制,默认是 4r/s 即每秒 4 个请求,如果超过的话就会返回 503。而我们的 nginx 配置里有这么一段 error_page 500 502 503 504 =302 $scheme://$proxy_host$uri; 即当上游返回 503 的时候,nginx 会让 client 302 到 $scheme://$proxy_host$uri,而这个时候 nginx 还没开始做 proxy,$proxy_host 应该还是空的,
所以 client 会看到 nginx 返回的 302 的响应里的 location 是 https:///crates/futures-io/futures-io-0.3.4.crate. 目前我先对 crates-io.proxy.ustclug.org 稍微放开了一下限制 limit_req_zone $binary_remote_addr zone=rust-crates:10m rate=20r/s; limit_req zone=rust-crates burst=150; 同时调整了一下配置,减少了一次 302 跳转,@s977120 你现在可以试试看有没有问题。 不过感觉这个只是临时的 workaround,要彻底解决应该要让 cargo 限制一下并发请求数。
最终解决方案, 要控制cargo下载时的并发数, 估计要看一下源码,确认如何实现的。当前尝试使用cargo install -j 4 时,问题依旧;
使用说明如下:
$ cargo install --help cargo-install Install a Rust binary. Default location is $HOME/.cargo/bin USAGE: cargo install [OPTIONS] [--] [crate]... OPTIONS: -q, --quiet No output printed to stdout --version <VERSION> Specify a version to install --git <URL> Git URL to install the specified crate from --branch <BRANCH> Branch to use when installing from git --tag <TAG> Tag to use when installing from git --rev <SHA> Specific commit to use when installing from git --path <PATH> Filesystem path to local crate to install --list list all installed packages and their versions -j, --jobs <N> Number of parallel jobs, defaults to # of CPUs -f, --force Force overwriting existing crates or binaries --no-track Do not save tracking information --features <FEATURES>... Space or comma separated list of features to activate --all-features Activate all available features --no-default-features Do not activate the `default` feature --profile <PROFILE-NAME> Install artifacts with the specified profile --debug Build in debug mode instead of release mode --bin <NAME>... Install only the specified binary --bins Install all binaries --example <NAME>... Install only the specified example --examples Install all examples --target <TRIPLE>... Build for the target triple --target-dir <DIRECTORY> Directory for all generated artifacts --root <DIR> Directory to install packages into --index <INDEX> Registry index to install from --registry <REGISTRY> Registry to use -v, --verbose Use verbose output (-vv very verbose/build.rs output) --color <WHEN> Coloring: auto, always, never --frozen Require Cargo.lock and cache are up to date --locked Require Cargo.lock is up to date --offline Run without accessing the network -Z <FLAG>... Unstable (nightly-only) flags to Cargo, see ‘cargo -Z help‘ for details -h, --help Prints help information ARGS: <crate>... Run `cargo help install` for more detailed information.