tomcat和压测应用分开部署

压测工具使用自研的基于web的jpress ，支持动态调整压测线程 （增加或减少都可以）

两台物理机  32C | 256G CentOS 7.4.1708

Tomcat/8.5.32


tomcat配置



<Executor
  name="tomcatThreadPool"
  namePrefix="catalina-exec-"
  maxThreads="500"
  minSpareThreads="30"
  maxIdleTime="60000"
  prestartminSpareThreads = "true"
  maxQueueSize = "100"
 />
 
<Connector port="7020"
               protocol="org.apache.coyote.http11.Http11Nio2Protocol"
               executor="tomcatThreadPool"
               maxConnections="10000"
               enableLookups="false"
               acceptCount="100"
               connectionTimeout="20000"
               useBodyEncodingForURI="true"
               URIEncoding="UTF-8"
               redirectPort="8443"
               tcpNoDelay="true"
               acceptorThreadCount="2"
               server="tomcat8.5"
               />



jvm参数设置


JAVA_OPTS="$JAVA_OPTS $JSSE_OPTS"
# JAVA_OPTS="$JAVA_OPTS -javaagent:/home/codefun007/soft/benchmark7020/prom/jmx_prometheus_javaagent-0.11.0.jar=7021:/home/codefun007/soft/benchmark7020/prom/tomcat.yml"
JAVA_OPTS="$JAVA_OPTS -Xmx4096m -Xms4096m -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/home/codefun007/soft/benchmark7020/gc.log"
JAVA_OPTS="$JAVA_OPTS -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=3 -XX:GCLogFileSize=10M"
 
#JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC"
#JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC"
# JAVA_OPTS="$JAVA_OPTS -XX:+UseSerialGC"

gc没配置 ，默认使用 并行收集器

jcmd pid  VM.flags   查看最终的jvm参数


分别测试了三种协议 
Http11Nio2Protocol  
Http11NioProtocol  
HTTP/1.1 

推荐使用 Http11Nio2Protocol ， tps 能到13W， 性能 较 Http11NioProtocol  和 HTTP/1.1 提升 50% 以上

并且需要更少的 执行线程  13w tps 只需要 34个执行线程


jstack 90549|grep catalina-exec-|wc -l


Http11NioProtocol  和  HTTP/1.1 性能差不多 9W不到

使用了所有的执行线程  500个

任务队列 满了之后  狂刷日志
[http-nio-7020-ClientPoller-1] org.apache.tomcat.util.net.AbstractEndpoint.processSocket Executor rejected socket [org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@2acad7be:org.apache.tomcat.util.net.NioChannel@1b674d1e:java.nio.channels.SocketChannel[connected local=/10.57.31.22:7020 remote=/10.57.31.23:19780]] for processing
 java.util.concurrent.RejectedExecutionException: Work queue full.

打印大量错误日志

du -d 1 -h
42M	./bin
232K	./conf
7.6M	./lib
253G	./logs
0	./temp
2.6M	./webapps
32K	./work
368K	./prom
253G	.