多线程即内存溢出处理方式

2022-11-18 22:53:21

一、什么是多线程
二、多线程的创建方式
三、为什么使用线程池

四、内存溢出怎么办

OOM:OutOfMemory(内存溢出)

开发中常见异常:

1.*Error

实例:递归调用后方法特别多,将栈空间撑爆

public class *ErrorDemo{
	public static void main(){
		test();
	}
	private static void test(){
		//递归调用
		test();
	}
}

结果:

Exception in thread "main" java.lang.*Error
	at com.coolcoding.boot.oom.*ErrorDemo.test(*ErrorDemo.java:13)
	at com.coolcoding.boot.oom.*ErrorDemo.test(*ErrorDemo.java:13)
	at com.coolcoding.boot.oom.*ErrorDemo.test(*ErrorDemo.java:13)

2.java.lang.OutOfMemoryError:Java heap space

为了调试方便先设置最大堆内存和初始堆内存大小
JVM调试参数设置和说明

-Xms	初始堆内存	一般为内存的1/16
-Xms	最大堆内存	一般为内存的1/4

设置:-Xms:10m ; -Xmx:10m
案例:

public class JavaHeapSpaceDemo {

    public static void main(String[] args) {
        String str = "xxxxxxxxxdddddddddddd";

        while (true){
        //循环创建字符串对象
            str += str + new Random().nextInt(1111111111) + new Random().nextInt(222222222);
            //从常量池中获取字符串，若不存在，则创建一个字符串放到常量池中
            str.intern();
        }
    }
}

结果:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
  at java.util.Arrays.copyOf(Arrays.java:3332)
  at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
  at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:674)
  at java.lang.StringBuilder.append(StringBuilder.java:208)
  at com.coolcoding.boot.oom.JavaHeapSpaceDemo.main(JavaHeapSpaceDemo.java:14)

3.java.lang.OutOfMemoryError:GC overhead limit exceeded

简单理解:GC回收时间过长,时间过多耗费在GC中,但是回收效果不佳。
原理:

回收过长指的是超过98%的时间用来做GC,并且回收了不到2%的堆内存
连续多次GC,都回收了不到2%的极端情况下才会抛出异常,
如不抛出异常,GC清理后的内存也会很快在此填满,迫使GC在此执行,
就此形成恶性循环

JVM参数设置:
-Xms:10m;-Xmx:10m;-XX:+PrintGCDetails ;-XX:MaxDirectMemory=5m
案例:

public class GCOverheadDemo {

    public static void main(String[] args) {
        int i = 0;
        List<String> list = new ArrayList<>();
        try {
            while (true){
                list.add(String.valueOf(++i).intern());
            }
        }catch (Throwable e){
            e.printStackTrace();
            throw e;
        }

    }
}

结果:

[Full GC (Ergonomics) [PSYoungGen: 2048K->2048K(2560K)] [ParOldGen: 7049K->7046K(7168K)] 9097K->9094K(9728K), [Metaspace: 3886K->3886K(1056768K)], 0.0530256 secs] [Times: user=0.14 sys=0.00, real=0.05 secs] 
[Full GC (Ergonomics) java.lang.OutOfMemoryError: GC overhead limit exceeded
[PSYoungGen: 2048K->2048K(2560K)] [ParOldGen: 7050K->7046K(7168K)] 9098K->9094K(9728K), [Metaspace: 3886K->3886K(1056768K)], 0.0493460 secs] [Times: user=0.13 sys=0.00, real=0.05 secs] 
[Full GC (Ergonomics)   at java.lang.Integer.toString(Integer.java:401)
             //回收前2048k -> 回收后2048k 可见GC回收效果不佳
[PSYoungGen: 2048K->2048K(2560K)
[Full GC (Ergonomics) [PSYoungGen: 2048K->0K(2560K)] [ParOldGen: 7020K->1119K(7168K)] 9068K->1119K(9728K), [Metaspace: 3892K->3892K(1056768K)], 0.0176793 secs] [Times: user=0.02 sys=0.00, real=0.02 secs] 
  at java.lang.String.valueOf(String.java:3099)
  at com.coolcoding.boot.oom.GCOverheadDemo.main(GCOverheadDemo.java:16)
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
  at java.lang.Integer.toString(Integer.java:401)
  at java.lang.String.valueOf(String.java:3099)
  at com.coolcoding.boot.oom.GCOverheadDemo.main(GCOverheadDemo.java:16)
Heap

4.java.lang.OutOfMemoryError:Direct buffer memory

直接内存溢出
原因分析:
直接内存崩溃,此处元空间并不在虚拟机中,而是使用本地内存,与GC无关。
常见于NIO程序中,使用ByteBuffer来读取和写入数据,这是基于通道channel和缓冲区buffer的IO方式,可以使用Native函数直接分配堆外内存,通过存储在JAVA推里面的DirectByteBuffer对象作为这块内存的引用进行操作。该方式在某些常见中能提高性能,因为避免了java堆和native堆中来回拷贝数据。
例如:

ByteBuffer.allocate(capability)	堆内内存	属于GC管辖,由于需要拷贝所以速度相对较慢
ByteBuffer.allocateDirect(capability)	本地内存	不属于GC管辖,由于不需要拷贝所以速度较快

如果不断分配本地内存,堆内存很少使用,那么JVM就不需要执行GC,DirectByteBuffer对象们就不会被回收,此时堆内存充足,但是本地内存即将耗尽,那么再次尝试分配本地内存就会出现OOM
配置参数:
-Xms:10m;-Xmx:10m; -XX:+PrintGCDetails;+XX:MaxDirectMemorySize=5m

示例:

public class DirectBufferMemoryDemo {

    public static void main(String[] args) {
        System.out.println("本地内存 = " + (VM.maxDirectMemory() / 1024 / 1024) + "MB");
        try{
            TimeUnit.SECONDS.sleep(3);}catch (InterruptedException e){e.printStackTrace();}
        //allocateDirect 分配直接内存
        ByteBuffer.allocateDirect(6 * 1024 * 1024);
    }
}

结果:

"D:\Program Files\Java\jdk1.8.0_144\bin\java.exe" -Xms10m -Xmx10m -XX:+PrintGCDetails -XX:MaxDirectMemorySize=5m "-javaagent:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3\lib\idea_rt.jar=49710:D:\Program Files\JetBrains\IntelliJ IDEA 2019.3\bin" -Dfile.encoding=UTF-8 -classpath "D:\Program Files\Java\jdk1.8.0_144\jre\lib\charsets.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\deploy.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\access-bridge-64.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\cldrdata.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\dnsns.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jaccess.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\jfxrt.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\localedata.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\nashorn.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\student.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunec.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunjce_provider.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunmscapi.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\sunpkcs11.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\ext\zipfs.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\javaws.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\jce.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\jfr.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\jfxswt.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\jsse.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\management-agent.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\plugin.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\resources.jar;D:\Program Files\Java\jdk1.8.0_144\jre\lib\rt.jar;E:\workspace\springboot-demo\target\classes;D:\mavenRepository\org\springframework\boot\spring-boot-starter-web\2.2.6.RELEASE\spring-boot-starter-web-2.2.6.RELEASE.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter\2.2.6.RELEASE\spring-boot-starter-2.2.6.RELEASE.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter-logging\2.2.6.RELEASE\spring-boot-starter-logging-2.2.6.RELEASE.jar;D:\mavenRepository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\mavenRepository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\mavenRepository\org\apache\logging\log4j\log4j-to-slf4j\2.12.1\log4j-to-slf4j-2.12.1.jar;D:\mavenRepository\org\apache\logging\log4j\log4j-api\2.12.1\log4j-api-2.12.1.jar;D:\mavenRepository\org\slf4j\jul-to-slf4j\1.7.30\jul-to-slf4j-1.7.30.jar;D:\mavenRepository\org\yaml\snakeyaml\1.25\snakeyaml-1.25.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter-json\2.2.6.RELEASE\spring-boot-starter-json-2.2.6.RELEASE.jar;D:\mavenRepository\com\fasterxml\jackson\core\jackson-databind\2.10.3\jackson-databind-2.10.3.jar;D:\mavenRepository\com\fasterxml\jackson\core\jackson-annotations\2.10.3\jackson-annotations-2.10.3.jar;D:\mavenRepository\com\fasterxml\jackson\core\jackson-core\2.10.3\jackson-core-2.10.3.jar;D:\mavenRepository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.10.3\jackson-datatype-jdk8-2.10.3.jar;D:\mavenRepository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.10.3\jackson-datatype-jsr310-2.10.3.jar;D:\mavenRepository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.10.3\jackson-module-parameter-names-2.10.3.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter-validation\2.2.6.RELEASE\spring-boot-starter-validation-2.2.6.RELEASE.jar;D:\mavenRepository\jakarta\validation\jakarta.validation-api\2.0.2\jakarta.validation-api-2.0.2.jar;D:\mavenRepository\org\hibernate\validator\hibernate-validator\6.0.18.Final\hibernate-validator-6.0.18.Final.jar;D:\mavenRepository\org\jboss\logging\jboss-logging\3.4.1.Final\jboss-logging-3.4.1.Final.jar;D:\mavenRepository\com\fasterxml\classmate\1.5.1\classmate-1.5.1.jar;D:\mavenRepository\org\springframework\spring-web\5.2.5.RELEASE\spring-web-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-beans\5.2.5.RELEASE\spring-beans-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-webmvc\5.2.5.RELEASE\spring-webmvc-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-aop\5.2.5.RELEASE\spring-aop-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-context\5.2.5.RELEASE\spring-context-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-expression\5.2.5.RELEASE\spring-expression-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\boot\spring-boot-devtools\2.2.6.RELEASE\spring-boot-devtools-2.2.6.RELEASE.jar;D:\mavenRepository\org\springframework\boot\spring-boot\2.2.6.RELEASE\spring-boot-2.2.6.RELEASE.jar;D:\mavenRepository\org\springframework\boot\spring-boot-autoconfigure\2.2.6.RELEASE\spring-boot-autoconfigure-2.2.6.RELEASE.jar;D:\mavenRepository\org\projectlombok\lombok\1.18.12\lombok-1.18.12.jar;D:\mavenRepository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\mavenRepository\org\springframework\spring-core\5.2.5.RELEASE\spring-core-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-jcl\5.2.5.RELEASE\spring-jcl-5.2.5.RELEASE.jar;D:\mavenRepository\cn\hutool\hutool-all\5.1.0\hutool-all-5.1.0.jar;D:\mavenRepository\org\apache\poi\poi-ooxml\4.1.2\poi-ooxml-4.1.2.jar;D:\mavenRepository\org\apache\poi\poi\4.1.2\poi-4.1.2.jar;D:\mavenRepository\commons-codec\commons-codec\1.13\commons-codec-1.13.jar;D:\mavenRepository\org\apache\commons\commons-collections4\4.4\commons-collections4-4.4.jar;D:\mavenRepository\org\apache\commons\commons-math3\3.6.1\commons-math3-3.6.1.jar;D:\mavenRepository\com\zaxxer\SparseBitSet\1.2\SparseBitSet-1.2.jar;D:\mavenRepository\org\apache\poi\poi-ooxml-schemas\4.1.2\poi-ooxml-schemas-4.1.2.jar;D:\mavenRepository\org\apache\xmlbeans\xmlbeans\3.1.0\xmlbeans-3.1.0.jar;D:\mavenRepository\org\apache\commons\commons-compress\1.19\commons-compress-1.19.jar;D:\mavenRepository\com\github\virtuald\curvesapi\1.06\curvesapi-1.06.jar;D:\mavenRepository\com\101tec\zkclient\0.11\zkclient-0.11.jar;D:\mavenRepository\org\apache\zookeeper\zookeeper\3.4.13\zookeeper-3.4.13.jar;D:\mavenRepository\org\slf4j\slf4j-log4j12\1.7.30\slf4j-log4j12-1.7.30.jar;D:\mavenRepository\log4j\log4j\1.2.17\log4j-1.2.17.jar;D:\mavenRepository\jline\jline\0.9.94\jline-0.9.94.jar;D:\mavenRepository\org\apache\yetus\audience-annotations\0.5.0\audience-annotations-0.5.0.jar;D:\mavenRepository\io\netty\netty\3.10.6.Final\netty-3.10.6.Final.jar;D:\mavenRepository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;D:\mavenRepository\com\google\guava\guava\29.0-jre\guava-29.0-jre.jar;D:\mavenRepository\com\google\guava\failureaccess\1.0.1\failureaccess-1.0.1.jar;D:\mavenRepository\com\google\guava\listenablefuture\9999.0-empty-to-avoid-conflict-with-guava\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\mavenRepository\com\google\code\findbugs\jsr305\3.0.2\jsr305-3.0.2.jar;D:\mavenRepository\org\checkerframework\checker-qual\2.11.1\checker-qual-2.11.1.jar;D:\mavenRepository\com\google\errorprone\error_prone_annotations\2.3.4\error_prone_annotations-2.3.4.jar;D:\mavenRepository\com\google\j2objc\j2objc-annotations\1.3\j2objc-annotations-1.3.jar;D:\mavenRepository\com\baomidou\dynamic-datasource-spring-boot-starter\3.0.0\dynamic-datasource-spring-boot-starter-3.0.0.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter-aop\2.2.6.RELEASE\spring-boot-starter-aop-2.2.6.RELEASE.jar;D:\mavenRepository\org\aspectj\aspectjweaver\1.9.5\aspectjweaver-1.9.5.jar;D:\mavenRepository\mysql\mysql-connector-java\8.0.19\mysql-connector-java-8.0.19.jar;D:\mavenRepository\org\springframework\boot\spring-boot-starter-jdbc\2.2.6.RELEASE\spring-boot-starter-jdbc-2.2.6.RELEASE.jar;D:\mavenRepository\com\zaxxer\HikariCP\3.4.2\HikariCP-3.4.2.jar;D:\mavenRepository\org\springframework\spring-jdbc\5.2.5.RELEASE\spring-jdbc-5.2.5.RELEASE.jar;D:\mavenRepository\org\springframework\spring-tx\5.2.5.RELEASE\spring-tx-5.2.5.RELEASE.jar;D:\mavenRepository\org\mybatis\spring\boot\mybatis-spring-boot-starter\1.3.5\mybatis-spring-boot-starter-1.3.5.jar;D:\mavenRepository\org\mybatis\spring\boot\mybatis-spring-boot-autoconfigure\1.3.5\mybatis-spring-boot-autoconfigure-1.3.5.jar;D:\mavenRepository\org\mybatis\mybatis\3.4.6\mybatis-3.4.6.jar;D:\mavenRepository\org\mybatis\mybatis-spring\1.3.3\mybatis-spring-1.3.3.jar" com.coolcoding.boot.oom.DirectBufferMemoryDemo
[GC (Allocation Failure) [PSYoungGen: 2048K->504K(2560K)] 2048K->983K(9728K), 0.0023899 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
本地内存 = 5MB
[GC (Allocation Failure) [PSYoungGen: 2552K->504K(2560K)] 3031K->1543K(9728K), 0.0035927 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
[GC (System.gc()) [PSYoungGen: 781K->504K(2560K)] 1820K->1591K(9728K), 0.0013692 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
[Full GC (System.gc()) [PSYoungGen: 504K->0K(2560K)] [ParOldGen: 1087K->1333K(7168K)] 1591K->1333K(9728K), [Metaspace: 4029K->4029K(1056768K)], 0.0155318 secs] [Times: user=0.03 sys=0.00, real=0.02 secs] 
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
  at java.nio.Bits.reserveMemory(Bits.java:694)
  at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
  at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
  at com.coolcoding.boot.oom.DirectBufferMemoryDemo.main(DirectBufferMemoryDemo.java:17)
Heap
 PSYoungGen      total 2560K, used 53K [0x00000000ffd00000, 0x0000000100000000, 0x0000000100000000)
  eden space 2048K, 2% used [0x00000000ffd00000,0x00000000ffd0d748,0x00000000fff00000)
  from space 512K, 0% used [0x00000000fff00000,0x00000000fff00000,0x00000000fff80000)
  to   space 512K, 0% used [0x00000000fff80000,0x00000000fff80000,0x0000000100000000)
 ParOldGen       total 7168K, used 1333K [0x00000000ff600000, 0x00000000ffd00000, 0x00000000ffd00000)
  object space 7168K, 18% used [0x00000000ff600000,0x00000000ff74d618,0x00000000ffd00000)
 Metaspace       used 4061K, capacity 4568K, committed 4864K, reserved 1056768K
  class space    used 450K, capacity 460K, committed 512K, reserved 1048576K

5.java.lang.OutOfMemoryError:unable to create new native thread

高并发情况下会出现该异常,该异常与对应的平台有关
原因分析:
一个应用进程创建太多的线程,超过系统承载极限。如Linux默认允许单个进程可以创建的线程数是1024个。
解决方法:

降低应用程序创建线程的数量,分析应用是否真的需要创建那么多线程,将线程数降到最低
修改服务器配置,如修改Linux服务器配置,扩大Linux默认限制

案例:

public class UnableCreateNewThreadDemo {

    public static void main(String[] args) {
        //不断for循环创建线程
        for (int i = 0; ; i++) {
            new Thread(()->{
                //设置Integer.MAX_VALUE 以保持线程还在运行中
                try{ TimeUnit.SECONDS.sleep(Integer.MAX_VALUE);}catch (InterruptedException e){e.printStackTrace();}
            }).start();
        }
    }
}

试验:
将该文件上传放到linux 下
执行javac -d 文件名
执行java 包名.文件名启动测试应用
达到系统负载后则会抛出java.lang.OutOfMemoryError:unable to create new native thread
若要修改系统配置可以先查看对应用户的线程限制
linux下查看对应用户的线程限制个数

//查看限制个数
ulimit -u
//编辑修改
vim /etc/security/limits.d/90-nproc.conf

如修改对z3用户的配置

6.java.lang.OutOfMemoryError:Metaspace

在cmd中执行,一下命令可以查看默认的JVM参数基本配置,可以查看metaspaceSize的大小

java -XX:+PrintFlagsInitial

metaspace存放数据:(永久代JAVA8之后被metaspace取代了)

虚拟机加载的类信息
常量池
静态变量
即使编译后的代码
JVM参数配置

-XX:MetaspaceSize=8m -XX:MAXMetaspaceSize=8m

案例:

public class MetaspaceOOMTest {
    //准备一个静态类
    static class OOMTest {
    }

    ;

    public static void main(String[] args) {
        int i = 0;
        try {
            while (true) {
                //循环不断创建静态类来填充metaspace的空间
                i++;
                Enhancer enhancer = new Enhancer();
                enhancer.setSuperclass(OOMTest.class);
                enhancer.setUseCache(false);
                enhancer.setCallback(new MethodInterceptor() {
                    @Override
                    public Object intercept(Object o, Method method, Object[] objects, MethodProxy methodProxy) throws Throwable {
                        return methodProxy.invoke(o, args);
                    }
                });
                enhancer.create();
            }
        } catch (Exception e) {
        //记录查看创建了多少个静态类OOMTest
            System.out.println(i + "次后发生异常");
            e.printStackTrace();
        }
    }
}

结果:

279次后发生异常
org.springframework.cglib.core.CodeGenerationException: java.lang.OutOfMemoryError-->Metaspace
  at org.springframework.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:538)
  at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:363)
  at org.springframework.cglib.proxy.Enhancer.generate(Enhancer.java:582)
  at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:131)
  at org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:319)
  at org.springframework.cglib.proxy.Enhancer.createHelper(Enhancer.java:569)
  at org.springframework.cglib.proxy.Enhancer.create(Enhancer.java:384)
  at com.coolcoding.boot.oom.MetaspaceOOMTest.main(MetaspaceOOMTest.java:34)
Caused by: java.lang.OutOfMemoryError: Metaspace
  at java.lang.ClassLoader.defineClass1(Native Method)
  at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
  at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.springframework.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:535)
  ... 7 more

码农公寓