第一次出现:是thrift的python client去请求server,发现偶尔出现这个问题
第二次:接入第三方的api,去请求数据时,发现一个接入方的api第一次总是报这个错,当时又没有做处理,导致获得信息置空,入缓存后数据就是错误的。做了一个更改就是retry三次,得到解决。
第三次:最近去抓appstore的应用指数又重新出现该问题,使用HttpRequestRetryHandler 重试,设置到20次都无一次成功。使用curl测试时也是失败
curl: (56) Failure when receiving data from the peer
堆栈错误信息:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
|
[app][index-error]: ScreenAnts HD ; priority empty 2014 - 01 - 26
14 : 59 : 30 , 668
- I/O exception (java.net.SocketException) caught when processing request: Connection reset
2014 - 01 - 26
14 : 59 : 30 , 703
- Retrying request
2014 - 01 - 26
14 : 59 : 30 , 668
- [SimpleHttpClient] myRetryHandler, IOException:Connection reset
2014 - 01 - 26
14 : 59 : 30 , 668
- I/O exception (java.net.SocketException) caught when processing request: Connection reset
2014 - 01 - 26
14 : 59 : 30 , 704
- Retrying request
2014 - 01 - 26
14 : 59 : 30 , 668
- I/O exception (java.net.SocketException) caught when processing request: Connection reset
2014 - 01 - 26
14 : 59 : 30 , 704
- Retrying request
2014 - 01 - 26
14 : 59 : 30 , 704
- [SimpleHttpClient] myRetryHandler, IOException:Connection reset
2014 - 01 - 26
14 : 59 : 30 , 704
- [SimpleHttpClient] read http: //search.itunes.apple.com/WebObjects/MZSearchHints.woa/wa/hints?q=Deadliest+Animals failed
java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java: 185 )
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java: 136 )
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java: 152 )
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java: 270 )
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java: 140 )
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java: 57 )
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java: 260 )
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java: 161 )
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43 )
at java.lang.reflect.Method.invoke(Method.java: 616 )
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java: 138 )
at $Proxy7.receiveResponseHeader(Unknown Source)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java: 271 )
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java: 123 )
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java: 253 )
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java: 194 )
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java: 85 )
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java: 108 )
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java: 186 )
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java: 82 )
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java: 106 )
at com.fanxer.aso.index.utils.SimpleHttpClient.getContent(SimpleHttpClient.java: 135 )
at com.fanxer.aso.index.task.AppIndexTask.getXmlBytesFromSearchLink(AppIndexTask.java: 163 )
at com.fanxer.aso.index.task.AppIndexTask.access$ 000 (AppIndexTask.java: 40 )
at com.fanxer.aso.index.task.AppIndexTask$ 1 .call(AppIndexTask.java: 98 )
at com.fanxer.aso.index.task.AppIndexTask$ 1 .call(AppIndexTask.java: 93 )
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java: 334 )
at java.util.concurrent.FutureTask.run(FutureTask.java: 166 )
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1110 )
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: 603 )
at java.lang.Thread.run(Thread.java: 679 )
|
http://bbs.csdn.net/topics/210061352 解释了四种socket异常:
java.net.SocketException: (Connection reset或者Connect reset by peer:Socket write error)。该异常在客户端和服务器端均有可能发生,引起该异常的原因有两个,第一个就是如果一端的Socket被关闭(或主动关闭或者因为异常退出而引起的关闭),另一端仍发送数据,发送的第一个数据包引发该异常(Connect reset by peer)。另一个是一端退出,但退出时并未关闭该连接,另一端如果在从连接中读数据则抛出该异常(Connection reset)。简单的说就是在连接断开后的读和写操作引起的。
经多次测试发现,50个线程并发,最大的连接时间超过了90秒,平均请求结果仅有400KB,很奇怪的现象。猜测是appstore端连接时间过长直接断开连接(是我被连90s也要断啊)。修改下超时,只能让请求更快恢复,
RetryExec.execute 时仍然无法正常连接。
InternalHttpClient继承了CloseableHttpClient 实现了父类的抽象方法
protected abstract CloseableHttpResponse doExecute(HttpHost target,
HttpRequest request,
HttpContext context) throws IOException,
ClientProtocolException;
对request做了封装,host、config和route确定后,转入execChain
execChain是一系列责任链
RedirectExec ->
RetryExec ->
ProtocolExec
MainClientExec
转给 HttpRequestExecutor 执行请求