Remember an httpclient pit

origin

I saw the server yesterday afternoon , There's a bunch of 500 Response code of . Look at it carefully .

throwable:org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 146,256; received: 139,121)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.Reader.read(Reader.java:140)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:227)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:308)
	at com.zuhao.uhaozutool.base.CommonHttpClient.getHttpClientResult(CommonHttpClient.java:524)
	at com.zuhao.uhaozutool.base.CommonHttpClient.doPost(CommonHttpClient.java:390)
	at com.zuhao.uhaozutool.base.CommonHttpClient.doJsonPost(CommonHttpClient.java:216)
 Copy code 

Errors are common , In a nutshell Httpclient When reading the response , client / Server side Disconnected . As a result, the data is read-only and an error is reported (expected: 146,256; received: 139,121).

Jieyuan

So check whether there is any code in the code that closes the connection in advance , Check both the client and the server .

It is found that there is no explicit way to close the connection at the root in the code , It's about using PoolingHttpClientConnectionManager Unified management .

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(1, TimeUnit.MINUTES);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);

defaultHttpClient = HttpClients.custom()
        .setConnectionManager(cm).build();
 Copy code 

So look for it PoolingHttpClientConnectionManager Related configuration . Only found 3 There are three common settings .

1. new PoolingHttpClientConnectionManager(1, TimeUnit.MINUTES), Set the maximum available duration of a single connection in the constructor .
2. cm.setMaxTotal(200);, Set the maximum number of connections in the whole connection pool .
3. cm.setDefaultMaxPerRoute(20);, Set individual host/ domain name maximum connection .

No, i giao, There is no setting at all , Then check the server code , Just the most basic springboot project , Even less likely to actively disconnect . Then I tried a series of parameter settings , All invalid .

So re observe the log , Found a suspicious place .

Meow every time (expected: 146,256; received: 139,121),received All are 139,121. That's ridiculous , How to accept the same data size every time ？？

So I turned to thinking , Is there a set to respond to the maximum acceptance size？ Found a circle , No configuration was found .

Think about it. , Wrong! ,httpclient.excute The success of , It's just EntityUtils.toString When it failed . The requests were executed successfully , Why does it fail to read ？ Check the source code and find , It turns out that the real read response content is EntityUtils.toString in . Yes, of course , It's just an episode .

private static String toString( final HttpEntity entity, final ContentType contentType) throws IOException {
    final InputStream inStream = entity.getContent();
    if (inStream == null) {
        return null;
    }
    try {
        Args.check(entity.getContentLength() <= Integer.MAX_VALUE,
                "HTTP entity too large to be buffered in memory");
        int capacity = (int)entity.getContentLength();
        if (capacity < 0) {
            capacity = DEFAULT_BUFFER_SIZE;
        }
        Charset charset = null;
        if (contentType != null) {
            charset = contentType.getCharset();
            if (charset == null) {
                final ContentType defaultContentType = ContentType.getByMimeType(contentType.getMimeType());
                charset = defaultContentType != null ? defaultContentType.getCharset() : null;
            }
        }
        if (charset == null) {
            charset = HTTP.DEF_CONTENT_CHARSET;
        }
        final Reader reader = new InputStreamReader(inStream, charset);
        final CharArrayBuffer buffer = new CharArrayBuffer(capacity);
        final char[] tmp = new char[1024];
        int l;
        while((l = reader.read(tmp)) != -1) {
            buffer.append(tmp, 0, l);
        }
        return buffer.toString();
    } finally {
        inStream.close();
    }
}
 Copy code 

At this time in Google I saw a brother on the show saying , He had a similar problem , Because nginx The configured cache cannot hold the entire response body,nginx The enabled user does not have write permission in the directory , Cause more than... Each time body, Only the size of the cache is returned body.

I understand , But also shocked .

Edge out .

Find it quickly , View the cache configuration of the server domain name .

Quickly check the meaning of each configuration .www.cnblogs.com/wshenjin/p/…

Now I really can't understand , but 139121/1024 About equal to 128, Feeling 128k That field is very suspicious , Try to change it , again nginx after , Problem solving .

complex bug Often only simple configuration is required .

Sum up , The solution to this problem ：

1. Check whether the client closes the connection in advance
2. Check whether the execution time of the server is too long, resulting in timeout, etc
3. Check nginx Related configuration