线上使用java做hive的任务调度，通过java Runtime 类的exec方法来执行hive的job。最近发现有job卡死的情况。具体表现是调度的脚本挂起，而且还可能导致hive的lock不能正常释放。
使用jstack打印java的thread信息：
发现如下的lock，最终定位到waitFor函数。
|
1
2
3
4
5
6
7
8
|
"main" prio=10 tid=0x000000005b24c800 nid=0x280e in Object.wait() [0x00002b3dee8e7000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000eb6f88b8> (a java.lang.UNIXProcess)
at java.lang.Object.wait(Object.java:485)
at java.lang.UNIXProcess.waitFor(UNIXProcess.java:165)
- locked <0x00000000eb6f88b8> (a java.lang.UNIXProcess)
at calltest.main(calltest.java:8)
|
在java的api中发现关于的Runtime类的描述，每个jvm都有一个Runtime类的实例，Runtime类会起一个单独的进程来运行相关的命令
Every Java application has a single instance of class Runtimethat allows the application to interface with
the environment in which the application is running.
Executes the specified string command in a separate process with the specified environment and working directory.
同时，发现如下的解释：
即由exec生成的进程没有自己的console，会和父进程有IO操作联系，在输出数据比较大的时候，可能会导致缓冲区写满，从而导致进程死锁，而当前进程由于设置了waitFor会一直在等待子进程结束，从而当前子进程也进入阻塞状态。
The created subprocess does not have its own terminal or console. All its standard io (i.e. stdin, stdout, stderr) operations will be redirected to the parent process through three streams (getOutputStream(), getInputStream(), getErrorStream()). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
waitFor方法：
causes the current thread to wait, if necessary, until the process represented by this Process object has terminated. This method returns immediately if the subprocess has already terminated. If the subprocess has not yet terminated, the calling thread will be blocked until the subprocess exits.
解决方法：
可以在调用waitFor方法之前，为子进程创建两个线程，来读取标准输出和标准错误输出即可。
本文转自菜菜光 51CTO博客，原文链接：http://blog.51cto.com/caiguangguang/1352708，如需转载请自行联系原作者