httpclient的两个重要的参数maxPerRoute及MaxTotal
httpclient的连接池3个参数
HTTP请求时connectionRequestTimeout 、connectionTimeout、socketTimeout三个超时时间的含义
1.connectionRequestTimout：指从连接池获取连接的timeout
2.connetionTimeout：指客户端和服务器建立连接的timeout，
就是http请求的三个阶段，一：建立连接；二：数据传送；三，断开连接。超时后会ConnectionTimeOutException
3.socketTimeout：指客户端和服务器建立连接后，客户端从服务器读取数据的timeout，超出后会抛出SocketTimeOutException
httpclient封装了java中进行http网络请求的底层实现，是一个被广泛使用的组件。
httpclient是支持池化机制的，这两个参数maxPerRoute及MaxTotal就是表示池化设置的。
服务之间发送http请求常用的有Apache的Fluent以及spring的restTemplate等。对httpclient进行封装的有：Apache的Fluent、es的restHighLevelClient、spring的restTemplate等。
以ES的restHighLevelClient为例说明
/**
* Creates a new {@link RestClient} based on the provided configuration.
*/
public RestClient build() {
if (failureListener == null) {
failureListener = new RestClient.FailureListener();
}
CloseableHttpAsyncClient httpClient = AccessController.doPrivileged(new PrivilegedAction<CloseableHttpAsyncClient>() {
@Override
public CloseableHttpAsyncClient run() {
return createHttpClient();
}
});
RestClient restClient = new RestClient(httpClient, defaultHeaders, nodes,
pathPrefix, failureListener, nodeSelector, strictDeprecationMode);
httpClient.start();
return restClient;
}
private CloseableHttpAsyncClient createHttpClient() {
//default timeouts are all infinite
RequestConfig.Builder requestConfigBuilder = RequestConfig.custom()
.setConnectTimeout(DEFAULT_CONNECT_TIMEOUT_MILLIS)
.setSocketTimeout(DEFAULT_SOCKET_TIMEOUT_MILLIS);
if (requestConfigCallback != null) {
requestConfigBuilder = requestConfigCallback.customizeRequestConfig(requestConfigBuilder);
} try {
HttpAsyncClientBuilder httpClientBuilder = HttpAsyncClientBuilder.create().setDefaultRequestConfig(requestConfigBuilder.build())
//default settings for connection pooling may be too constraining
.setMaxConnPerRoute(DEFAULT_MAX_CONN_PER_ROUTE).setMaxConnTotal(DEFAULT_MAX_CONN_TOTAL)
.setSSLContext(SSLContext.getDefault())
.setTargetAuthenticationStrategy(new PersistentCredentialsAuthenticationStrategy());
if (httpClientConfigCallback != null) {
httpClientBuilder = httpClientConfigCallback.customizeHttpClient(httpClientBuilder);
} final HttpAsyncClientBuilder finalBuilder = httpClientBuilder;
return AccessController.doPrivileged(new PrivilegedAction<CloseableHttpAsyncClient>() {
@Override
public CloseableHttpAsyncClient run() {
return finalBuilder.build();
}
});
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("could not create the default ssl context", e);
}
}
例子2：Apache的Fluent，其Executor类
/**
* An Executor for fluent requests
* <p/>
* A {@link PoolingHttpClientConnectionManager} with maximum 100 connections per route and
* a total maximum of 200 connections is used internally.
*/
//最大100 connections per route 以及 最大200个 connection CONNMGR = new PoolingHttpClientConnectionManager(sfr); CONNMGR.setDefaultMaxPerRoute(100); CONNMGR.setMaxTotal(200); CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();
maxPerRoute及MaxTotal参数含义
maxPerRoute及MaxTotal这两个参数的含义是什么呢？
下面用测试代码说明一下
测试端
public class HttpFluentUtil {
private Logger logger = LoggerFactory.getLogger(HttpFluentUtil.class);
private final static int MaxPerRoute = 2;
private final static int MaxTotal = 4;
final static PoolingHttpClientConnectionManager CONNMGR;
final static HttpClient CLIENT;
final static Executor EXECUTOR;
static {
LayeredConnectionSocketFactory ssl = null;
try {
ssl = SSLConnectionSocketFactory.getSystemSocketFactory();
} catch (final SSLInitializationException ex) {
final SSLContext sslcontext;
try {
sslcontext = SSLContext.getInstance(SSLConnectionSocketFactory.TLS);
sslcontext.init(null, null, null);
ssl = new SSLConnectionSocketFactory(sslcontext);
} catch (final SecurityException ignore) {
} catch (final KeyManagementException ignore) {
} catch (final NoSuchAlgorithmException ignore) {
}
}
final Registry<ConnectionSocketFactory> sfr = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", ssl != null ? ssl : SSLConnectionSocketFactory.getSocketFactory()).build();
CONNMGR = new PoolingHttpClientConnectionManager(sfr);
CONNMGR.setDefaultMaxPerRoute(MaxPerRoute);
CONNMGR.setMaxTotal(MaxTotal);
CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();
EXECUTOR = Executor.newInstance(CLIENT);
}
public static String Get(String uri, int connectTimeout, int socketTimeout) throws IOException {
return EXECUTOR.execute(Request.Get(uri).connectTimeout(connectTimeout).socketTimeout(socketTimeout))
.returnContent().asString();
}
public static String Post(String uri, StringEntity stringEntity, int connectTimeout, int socketTimeout)
throws IOException {
return EXECUTOR.execute(Request.Post(uri).socketTimeout(socketTimeout)
.addHeader("Content-Type", "application/json").body(stringEntity)).returnContent().asString();
}
public static void main(String[] args) {
HttpUtil httpUtil = new HttpUtil();
String url = "http://localhost:9064/app/test"; // 服务端sleep 5秒再返回
for (int i = 0; i < 5; i++) { // MaxPerRoute若设置为2，则5线程分3组返回（2、2、1），共15秒
new Thread(new Runnable() {
@Override
public void run() {
try {
String result = HttpFluentUtil.Get(url, 2000, 2000);
System.out.println(result);
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
}
}
服务器端
很简单的springmvc
@GetMapping(value="test")
public String test() throws InterruptedException {
Thread.sleep(1000);
return "1";
}
测试1：测试端MaxPerRoute=5 MaxTotal=4
服务器端结果
可以看到先接收4个请求，处理完成后，再接收下一次剩余的1个请求。即其一次最多接收MaxTotal次请求
测试2：测试端MaxPerRoute=2 MaxTotal=5
服务器端结果
可以看到接收2个请求，2个请求，1个请求，即说明maxPerRoute意思是某一个服务每次能并行接收的请求数量。
什么场景下要设置？
知道了两个参数的含义，那么在什么情况下要对这两个参数进行设置呢？
比如说下面的场景
服务1要通过Fluent调用服务2的接口。服务1发送了400个请求，但由于Fluent默认只支持maxPerRoute=100，MaxTotal=200，比如接口执行时间为500ms，由于maxPerRoute=100，所以要分为100,100,100,100分四批来执行，全部执行完成需要2000ms。而如果maxPerRoute设置为400，全部执行完需要500ms。在这种情况下（提供并发能力时）就要对这两个参数进行设置了。
设置的方法
1、Apache Fluent可以使用上面测试的HttpFluentUtil工具类来执行请求
2、RestTemplate类似使用下面的方式
@Bean
public HttpClient httpClient() {
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
connectionManager.setMaxTotal(restTemplateProperties.getMaxTotal());
connectionManager.setDefaultMaxPerRoute(restTemplateProperties.getDefaultMaxPerRoute());
connectionManager.setValidateAfterInactivity(restTemplateProperties.getValidateAfterInactivity());
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(restTemplateProperties.getSocketTimeout())
.setConnectTimeout(restTemplateProperties.getConnectTimeout())
.setConnectionRequestTimeout(restTemplateProperties.getConnectionRequestTimeout())
.build();
return HttpClientBuilder.create()
.setDefaultRequestConfig(requestConfig)
.setConnectionManager(connectionManager)
.build(); }
@Bean
public ClientHttpRequestFactory httpRequestFactory() {
return new HttpComponentsClientHttpRequestFactory(httpClient());
}
@Bean
public RestTemplate restTemplate() {
return new RestTemplate(httpRequestFactory());
}
其中RestTemplateProperties通过配置文件来配置
max-total
default-max-per-route
connect-timeout 获取连接超时
connection-request-timeout 请求超时
socket-timeout 读超时
3、ES的restHighLevelClient设置
@Bean
public RestHighLevelClient restHighLevelClient(){
//解析hostlist配置信息
String[] split = hostlist.split(",");
//创建HttpHost数组，其中存放es主机和端口的配置信息
HttpHost[] httpHostArray = new HttpHost[split.length];
for(int i=0;i<split.length;i++){
String item = split[i];
httpHostArray[i] = new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http");
}
//创建RestHighLevelClient客户端
//return new RestHighLevelClient(RestClient.builder(httpHostArray));//.setMaxRetryTimeoutMillis(5 * 60 * 1000)); //超时时间设为5分钟);
RestClientBuilder builder = RestClient.builder(httpHostArray); builder.setRequestConfigCallback(requestConfigBuilder -> {
requestConfigBuilder.setConnectTimeout(connectTimeoutMillis);
requestConfigBuilder.setSocketTimeout(socketTimeoutMillis);
requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeoutMillis);
return requestConfigBuilder;
}); builder.setHttpClientConfigCallback(httpClientBuilder -> {
httpClientBuilder.setMaxConnTotal(maxConnectTotal);
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
return httpClientBuilder;
}); return new RestHighLevelClient(builder); }
里面的5个变量，通过spring的@Value注入，这里省略。
总结：
max-total：连接池里的最大连接数
default-max-per-route：某一个/每服务每次能并行接收的请求数量
connect-timeout 从连接池里获取连接超时时间
connection-request-timeout 请求超时时间
socket-timeout 读超时时间
参考：https://blog.csdn.net/u013905744/java/article/details/94714696