后端使用的jetty服务
报错
2022-01-11 17:20:24.362 WARN 4784 --- [ctor-http-nio-3] r.netty.http.client.HttpClientConnect : [id: 0x3eaaea4e, L:0.0.0.0/0.0.0.0:58790 ! R:localhost/127.0.0.1:8090] The connection observed an error
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
2022-01-11 17:20:24.362 ERROR 4784 --- [ctor-http-nio-3] a.w.r.e.AbstractErrorWebExceptionHandler : [31e53fe7-21772] 500 Server Error for HTTP GET "/status"
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
|_ checkpoint ⇢ HTTP GET "/status" [ExceptionHandlingWebHandler]
Stack trace:
说明
- 通过网关服务
Spring Cloud Gateway
调用后端服务Jetty
报异常
原因
- 后端服务socket超时之后自动断开连接,网关服务从HttpClient连接池中取出了已经断开的连接进行请求
复现
- Spring Cloud GateWay : 2.2.1.RELEASE
- Spring Cloud : Hoxton.SR1
- Spring Boot : 2.3.4.RELEASE
- Server使用SpringBoot模拟
GateWay
spring:
application:
name: gateway
cloud:
gateway:
httpclient:
pool:
max-idle-time: 60000ms
routes:
- id: dev
uri: ${service-url.dev-service}/status
predicates:
- Path=/status
service-url:
dev-service: http://localhost:8080
server:
port: 8098
Server
spring:
application:
name: dev
server:
tomcat:
connection-timeout: 100ms
Jmeter
Threads:300
period seconds:5
Loop Count:20
说明:
- max-idle-time:最大空闲连接时间
- connection-timeout:socket read()超时时间。SpringBoot : server.tomcat.connection-timeout配置解析
- GateWay的max-idle-time:60000ms之后才释放连接,Server的connection-timeout在100ms之后就断开连接,此时GateWay会从连接池取已经断开的连接请求,以致报错
解决
- 因为Server是Provider, GateWay是Consumer,尽可能保证Consumer先于Provider断开连接,设置max-idle-time的时间不大于connection-timeout
- Jetty服务
- Jetty修改
jetty.http.idleTimeout
或http.timeout
。默认30000ms
- Jetty修改
参考
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response解决方案
【排障手记】WebClient调用抛出异常:PrematureCloseException: Connection prematurely closed BEFORE response
https://github.com/reactor/reactor-netty/pull/1442
https://github.com/reactor/reactor-netty/issues/1639
https://github.com/reactor/reactor-netty/issues/796
https://github.com/reactor/reactor-netty/issues/1092