current position:Home>Remember an httpclient pit

Remember an httpclient pit

2021-08-27 07:42:28 Soldier Zhang Jian

origin

I saw the server yesterday afternoon , There's a bunch of 500 Response code of . Look at it carefully .

throwable:org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 146,256; received: 139,121)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.Reader.read(Reader.java:140)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:227)
	at org.apache.http.util.EntityUtils.toString(EntityUtils.java:308)
	at com.zuhao.uhaozutool.base.CommonHttpClient.getHttpClientResult(CommonHttpClient.java:524)
	at com.zuhao.uhaozutool.base.CommonHttpClient.doPost(CommonHttpClient.java:390)
	at com.zuhao.uhaozutool.base.CommonHttpClient.doJsonPost(CommonHttpClient.java:216)
 Copy code 

Errors are common , In a nutshell Httpclient When reading the response , client / Server side Disconnected . As a result, the data is read-only and an error is reported (expected: 146,256; received: 139,121).

Jieyuan

So check whether there is any code in the code that closes the connection in advance , Check both the client and the server .

It is found that there is no explicit way to close the connection at the root in the code , It's about using PoolingHttpClientConnectionManager Unified management .

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(1, TimeUnit.MINUTES);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);

defaultHttpClient = HttpClients.custom()
        .setConnectionManager(cm).build();
 Copy code 

So look for it PoolingHttpClientConnectionManager Related configuration . Only found 3 There are three common settings .

  1. new PoolingHttpClientConnectionManager(1, TimeUnit.MINUTES), Set the maximum available duration of a single connection in the constructor .
  2. cm.setMaxTotal(200);, Set the maximum number of connections in the whole connection pool .
  3. cm.setDefaultMaxPerRoute(20);, Set individual host/ domain name maximum connection .

No, i giao, There is no setting at all , Then check the server code , Just the most basic springboot project , Even less likely to actively disconnect . Then I tried a series of parameter settings , All invalid .

So re observe the log , Found a suspicious place .

Meow every time (expected: 146,256; received: 139,121),received All are 139,121. That's ridiculous , How to accept the same data size every time ??

So I turned to thinking , Is there a set to respond to the maximum acceptance size? Found a circle , No configuration was found .

Think about it. , Wrong! ,httpclient.excute The success of , It's just EntityUtils.toString When it failed . The requests were executed successfully , Why does it fail to read ? Check the source code and find , It turns out that the real read response content is EntityUtils.toString in . Yes, of course , It's just an episode .

private static String toString( final HttpEntity entity, final ContentType contentType) throws IOException {
    final InputStream inStream = entity.getContent();
    if (inStream == null) {
        return null;
    }
    try {
        Args.check(entity.getContentLength() <= Integer.MAX_VALUE,
                "HTTP entity too large to be buffered in memory");
        int capacity = (int)entity.getContentLength();
        if (capacity < 0) {
            capacity = DEFAULT_BUFFER_SIZE;
        }
        Charset charset = null;
        if (contentType != null) {
            charset = contentType.getCharset();
            if (charset == null) {
                final ContentType defaultContentType = ContentType.getByMimeType(contentType.getMimeType());
                charset = defaultContentType != null ? defaultContentType.getCharset() : null;
            }
        }
        if (charset == null) {
            charset = HTTP.DEF_CONTENT_CHARSET;
        }
        final Reader reader = new InputStreamReader(inStream, charset);
        final CharArrayBuffer buffer = new CharArrayBuffer(capacity);
        final char[] tmp = new char[1024];
        int l;
        while((l = reader.read(tmp)) != -1) {
            buffer.append(tmp, 0, l);
        }
        return buffer.toString();
    } finally {
        inStream.close();
    }
}
 Copy code 

At this time in Google I saw a brother on the show saying , He had a similar problem , Because nginx The configured cache cannot hold the entire response body,nginx The enabled user does not have write permission in the directory , Cause more than... Each time body, Only the size of the cache is returned body.

image.png

I understand , But also shocked .

Edge out .

Find it quickly , View the cache configuration of the server domain name .

lALPDgtYx9d1IurNApzNAjE_561_668.png

Quickly check the meaning of each configuration .www.cnblogs.com/wshenjin/p/…

Now I really can't understand , but 139121/1024 About equal to 128, Feeling 128k That field is very suspicious , Try to change it , again nginx after , Problem solving .

complex bug Often only simple configuration is required .

Sum up , The solution to this problem :

  1. Check whether the client closes the connection in advance
  2. Check whether the execution time of the server is too long, resulting in timeout, etc
  3. Check nginx Related configuration

copyright notice
author[Soldier Zhang Jian],Please bring the original link to reprint, thank you.
https://en.qdmana.com/2021/08/20210827074223696O.html

Random recommended