[Android] [Java] Process 创建+控制+分析 经验浅谈
无论是Android亦或者Java中或多或少需要调用底层的一些命令，执行一些参数；
此时我们需要用到Java的Process来创建一个子进程，之所以是子进程是因为此进程依赖于发起创建请求的进程，如果发起者被Kill那个子进程也将Kill。
对于Process相信使用过的朋友一定不会陌生，它具有如下特点：
1.创建简单
2.控制难
3.容易导致无法创建子进程
4.如果是多线程那么很有可能造成内存溢出
以上现象如果你只是偶尔使用一次，创建一个进程或许你什么都没有感觉到，但是如果你使用了多线程，进行了大量的创建，以上问题你都会遇到。
相关：http://blog.csdn.net/qiujuer/article/details/38142273，http://blog.csdn.net/qiujuer/article/details/38086071
这两个星期一直在研究上面的问题，我要做的软件是在Android中进行TraceRoute，由于手机不可能完全Root所以不能采用JNI来发送ICMP请求的方式，最终只能使用创建进程方式进行；具体实现思路是：使用PING命令来PING百度等地址，在PING命令中加入TTL，得到每一次的IP地址，当IP地址与目标IP地址符合时退出，并且还需要单独PING一次每一跳的延迟和丢包。
单线程：PING 百度 TTL=1 =》 得到IP，PING IP 得到延迟丢包，改变TTL，进行下一次PING，直到所得到的IP与目标（百度）一样时停止。按照上面的思路一次需要创建两个子进程，一般到百度时TTL大约为12跳左右，所以就是2*12=24个子进程；如果是在单线程下简单明了，但是速度慢，整个过程大约需要1分钟左右。
多线程：同时发起3个线程进行3跳测试TTL=（1，2，3），测试完成后测试下一批数据TTL=（4，5，6），如果也是12跳的话，那么也是24个子进程，但是整体耗时将会为1/3.可见此时效率较高。
但是多线程需要考虑的是线程的同步问题，以及得到数据后的写入问题，这些赞不谈，只谈进程问题。经过我的测试假如现在测试100个网站的TraceRoute数据，在上层控制一次测试4个网站，底层实现并发3个线程，此时在一定时间内将会同时存在34个进程。按照平均每个网站12跳来算：122100=240个子进程，需要的子线程为12100=120个。
这个时候问题来了，假如现在程序子进程不正常了，遇到了一个一定的问题导致进程无法执行完成，此时你的现象是：一个子进程卡住，随后创建的所有子进程都卡住。假如最上层线程做了任务时间限制，那么到时间后将会尝试销毁，但是你会发现无法销毁，所持有的线程也不会销毁。但是上层以为销毁掉了，然后继续进行下一批的数据测试，此时你的线程数量会逐渐增加，如果100任务下来你的线程或许会达到34100=1200如果有前期没有这样的情况那个就是一半：600个线程左右，如果后期还有任务将会继续增加但是却永远不会销毁，但是我们知道JVM的内存是有限的，所以此时将会出现内存溢出。
以上就是我遇到的问题，我最先改为了等待线程完全返回后再进行下一批数据测试，此时内存溢出是解决了，但是任务却一直卡住在哪里了，永远也不走。我就在想要解决这一的问题需要解决根本上的问题才行，经过研究我发现在程序创建了子进程后JVM将会创建一个子进程管理线程：“ProcessManager”：
正常情况下该线程状态为Native，但是如果创建大量子进程后有可能会出现此线程为Monitor状态，过一段时间后所有创建子进程的线程状态也将会变为Monitor状态，然后将一直死锁，后面创建线程也是继续死锁，无法继续。
通过查看ProcessManager源码发现，其中启动了一个线程用于监听子进程状态，同时管理子进程，比如输出消息以及关闭子进程等操作，具体如下：
/**
* Copyright (C) 2007 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package java.lang;
import java.io.File;
import java.io.FileDescriptor;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.WeakReference;
import java.util.HashMap;
import java.util.Map;
import java.util.Arrays;
import java.util.logging.Logger;
import java.util.logging.Level;
/***
* Manages child processes.
*
* <p>Harmony's native implementation (for comparison purposes):
* http://tinyurl.com/3ytwuq
*/
final class ProcessManager {
/***
* constant communicated from native code indicating that a
* child died, but it was unable to determine the status
*/
private static final int WAIT_STATUS_UNKNOWN = -1;
/***
* constant communicated from native code indicating that there
* are currently no children to wait for
*/
private static final int WAIT_STATUS_NO_CHILDREN = -2;
/***
* constant communicated from native code indicating that a wait()
* call returned -1 and set an undocumented (and hence unexpected) errno
*/
private static final int WAIT_STATUS_STRANGE_ERRNO = -3;
/***
* Initializes native static state.
*/
static native void staticInitialize();
static {
staticInitialize();
}
/***
* Map from pid to Process. We keep weak references to the Process objects
* and clean up the entries when no more external references are left. The
* process objects themselves don't require much memory, but file
* descriptors (associated with stdin/out/err in this case) can be
* a scarce resource.
*/
private final Map<Integer, ProcessReference> processReferences
= new HashMap<Integer, ProcessReference>();
/*** Keeps track of garbage-collected Processes. */
private final ProcessReferenceQueue referenceQueue
= new ProcessReferenceQueue();
private ProcessManager() {
// Spawn a thread to listen for signals from child processes.
Thread processThread = new Thread(ProcessManager.class.getName()) {
@Override
public void run() {
watchChildren();
}
};
processThread.setDaemon(true);
processThread.start();
}
/***
* Kills the process with the given ID.
*
* @parm pid ID of process to kill
*/
private static native void kill(int pid) throws IOException;
/***
* Cleans up after garbage collected processes. Requires the lock on the
* map.
*/
void cleanUp() {
ProcessReference reference;
while ((reference = referenceQueue.poll()) != null) {
synchronized (processReferences) {
processReferences.remove(reference.processId);
}
}
}
/***
* Listens for signals from processes and calls back to
* {@link #onExit(int,int)}.
*/
native void watchChildren();
/***
* Called by {@link #watchChildren()} when a child process exits.
*
* @param pid ID of process that exited
* @param exitValue value the process returned upon exit
*/
void onExit(int pid, int exitValue) {
ProcessReference processReference = null;
synchronized (processReferences) {
cleanUp();
if (pid >= 0) {
processReference = processReferences.remove(pid);
} else if (exitValue == WAIT_STATUS_NO_CHILDREN) {
if (processReferences.isEmpty()) {
/**
* There are no eligible children; wait for one to be
* added. The wait() will return due to the
* notifyAll() call below.
*/
try {
processReferences.wait();
} catch (InterruptedException ex) {
// This should never happen.
throw new AssertionError("unexpected interrupt");
}
} else {
/**
* A new child was spawned just before we entered
* the synchronized block. We can just fall through
* without doing anything special and land back in
* the native wait().
*/
}
} else {
// Something weird is happening; abort!
throw new AssertionError("unexpected wait() behavior");
}
}
if (processReference != null) {
ProcessImpl process = processReference.get();
if (process != null) {
process.setExitValue(exitValue);
}
}
}
/***
* Executes a native process. Fills in in, out, and err and returns the
* new process ID upon success.
*/
static native int exec(String[] command, String[] environment,
String workingDirectory, FileDescriptor in, FileDescriptor out,
FileDescriptor err, boolean redirectErrorStream) throws IOException;
/***
* Executes a process and returns an object representing it.
*/
Process exec(String[] taintedCommand, String[] taintedEnvironment, File workingDirectory,
boolean redirectErrorStream) throws IOException {
// Make sure we throw the same exceptions as the RI.
if (taintedCommand == null) {
throw new NullPointerException();
}
if (taintedCommand.length == 0) {
throw new IndexOutOfBoundsException();
}
// Handle security and safety by copying mutable inputs and checking them.
String[] command = taintedCommand.clone();
String[] environment = taintedEnvironment != null ? taintedEnvironment.clone() : null;
SecurityManager securityManager = System.getSecurityManager();
if (securityManager != null) {
securityManager.checkExec(command[0]);
}
// Check we're not passing null Strings to the native exec.
for (String arg : command) {
if (arg == null) {
throw new NullPointerException();
}
}
// The environment is allowed to be null or empty, but no element may be null.
if (environment != null) {
for (String env : environment) {
if (env == null) {
throw new NullPointerException();
}
}
}
FileDescriptor in = new FileDescriptor();
FileDescriptor out = new FileDescriptor();
FileDescriptor err = new FileDescriptor();
String workingPath = (workingDirectory == null)
? null
: workingDirectory.getPath();
// Ensure onExit() doesn't access the process map before we add our
// entry.
synchronized (processReferences) {
int pid;
try {
pid = exec(command, environment, workingPath, in, out, err, redirectErrorStream);
} catch (IOException e) {
IOException wrapper = new IOException("Error running exec()."
+ " Command: " + Arrays.toString(command)
+ " Working Directory: " + workingDirectory
+ " Environment: " + Arrays.toString(environment));
wrapper.initCause(e);
throw wrapper;
}
ProcessImpl process = new ProcessImpl(pid, in, out, err);
ProcessReference processReference
= new ProcessReference(process, referenceQueue);
processReferences.put(pid, processReference);
/**
* This will wake up the child monitor thread in case there
* weren't previously any children to wait on.
*/
processReferences.notifyAll();
return process;
}
}
static class ProcessImpl extends Process {
/*** Process ID. */
final int id;
final InputStream errorStream;
/*** Reads output from process. */
final InputStream inputStream;
/*** Sends output to process. */
final OutputStream outputStream;
/*** The process's exit value. */
Integer exitValue = null;
final Object exitValueMutex = new Object();
ProcessImpl(int id, FileDescriptor in, FileDescriptor out,
FileDescriptor err) {
this.id = id;
this.errorStream = new ProcessInputStream(err);
this.inputStream = new ProcessInputStream(in);
this.outputStream = new ProcessOutputStream(out);
}
public void destroy() {
try {
kill(this.id);
} catch (IOException e) {
Logger.getLogger(Runtime.class.getName()).log(Level.FINE,
"Failed to destroy process " + id + ".", e);
}
}
public int exitValue() {
synchronized (exitValueMutex) {
if (exitValue == null) {
throw new IllegalThreadStateException(
"Process has not yet terminated.");
}
return exitValue;
}
}
public InputStream getErrorStream() {
return this.errorStream;
}
public InputStream getInputStream() {
return this.inputStream;
}
public OutputStream getOutputStream() {
return this.outputStream;
}
public int waitFor() throws InterruptedException {
synchronized (exitValueMutex) {
while (exitValue == null) {
exitValueMutex.wait();
}
return exitValue;
}
}
void setExitValue(int exitValue) {
synchronized (exitValueMutex) {
this.exitValue = exitValue;
exitValueMutex.notifyAll();
}
}
@Override
public String toString() {
return "Process[id=" + id + "]";
}
}
static class ProcessReference extends WeakReference<ProcessImpl> {
final int processId;
public ProcessReference(ProcessImpl referent,
ProcessReferenceQueue referenceQueue) {
super(referent, referenceQueue);
this.processId = referent.id;
}
}
static class ProcessReferenceQueue extends ReferenceQueue<ProcessImpl> {
@Override
public ProcessReference poll() {
// Why couldn't they get the generics right on ReferenceQueue? :(
Object reference = super.poll();
return (ProcessReference) reference;
}
}
static final ProcessManager instance = new ProcessManager();
/*** Gets the process manager. */
static ProcessManager getInstance() {
return instance;
}
/*** Automatically closes fd when collected. */
private static class ProcessInputStream extends FileInputStream {
private FileDescriptor fd;
private ProcessInputStream(FileDescriptor fd) {
super(fd);
this.fd = fd;
}
@Override
public void close() throws IOException {
try {
super.close();
} finally {
synchronized (this) {
if (fd != null && fd.valid()) {
try {
ProcessManager.close(fd);
} finally {
fd = null;
}
}
}
}
}
}
/*** Automatically closes fd when collected. */
private static class ProcessOutputStream extends FileOutputStream {
private FileDescriptor fd;
private ProcessOutputStream(FileDescriptor fd) {
super(fd);
this.fd = fd;
}
@Override
public void close() throws IOException {
try {
super.close();
} finally {
synchronized (this) {
if (fd != null && fd.valid()) {
try {
ProcessManager.close(fd);
} finally {
fd = null;
}
}
}
}
}
}
/*** Closes the given file descriptor. */
private static native void close(FileDescriptor fd) throws IOException;
}
在其中有一个“ native void watchChildren();”方法，此方法为线程主方法，具体实现可以看看JNI，在其中回调了方法：“ void onExit(int pid, int exitValue);” 在方法中：
void onExit(int pid, int exitValue) {
ProcessReference processReference = null;
synchronized (processReferences) {
cleanUp();
if (pid >= 0) {
processReference = processReferences.remove(pid);
} else if (exitValue == WAIT_STATUS_NO_CHILDREN) {
if (processReferences.isEmpty()) {
/**
* There are no eligible children; wait for one to be
* added. The wait() will return due to the
* notifyAll() call below.
*/
try {
processReferences.wait();
} catch (InterruptedException ex) {
// This should never happen.
throw new AssertionError("unexpected interrupt");
}
} else {
/**
* A new child was spawned just before we entered
* the synchronized block. We can just fall through
* without doing anything special and land back in
* the native wait().
*/
}
} else {
// Something weird is happening; abort!
throw new AssertionError("unexpected wait() behavior");
}
}
if (processReference != null) {
ProcessImpl process = processReference.get();
if (process != null) {
process.setExitValue(exitValue);
}
}
}
此方法作用是删除子进程队列中子进程同时通知子进程 ProcessImpl已完成。
但是在方法：“watchChildren（）”中如果出现System.in缓冲期满的情况那么进程将无法正常结束，它将一直等待缓冲区有空间存在，而缓冲区又是公共区间，如果一个出现等待那么后续子进程也将全部等待，如果缓冲区无法清空，那么所有子进程将会全部死锁掉。这就是导致子进程卡死的凶手。
知道问题关键点那么就会有人想办法解决，例如：
//...读取数据...
process.waitFor();
//....再次读取
这样的方式看似很好，但是你有没有想过有些数据无法及时返回，所以在 waitfor()之前读取很有可能没有数据导致进行 waitfor()等待，这时我们可以看看源码：
public int waitFor() throws InterruptedException {
synchronized (exitValueMutex) {
while (exitValue == null) {
exitValueMutex.wait();
}
return exitValue;
}
}
void setExitValue(int exitValue) {
synchronized (exitValueMutex) {
this.exitValue = exitValue;
exitValueMutex.notifyAll();
}
}
这里可以看见假如没有退出值将会进行等待，直到通知发生，但是通知想要发生必须要靠“ ProcessManager ”线程来告诉你。但是假如在等待过程中出现了大量的数据，导致 System.IN 满了，此时“ ProcessManager ”线程很傻很傻的进入了等待状态中，也将无法进行通知，而这边也就无法往下走，无法到达第二次读取，所以第二次读取就很随机了，在大量数据下第二次读取基本上就是摆设，也就是说无法正常的执行，最终也将导致死锁。
解决办法也很简单，创建线程后我们可以创建一个线程来专门读取信息，直到“ProcessManager”线程通知结束的时候，才退出线程。
首先我们看看Process提供的“exitValue()”方法：
public int exitValue() {
synchronized (exitValueMutex) {
if (exitValue == null) {
throw new IllegalThreadStateException(
"Process has not yet terminated.");
}
return exitValue;
}
}
可见在” exitValue “没有值时将会抛出异常而不会阻塞，所以可以得出：” exitValue() “与” waitfor() “都可以用于判断线程是否完成，但是一个是阻塞的一个是不阻塞的方法，在线程中当然使用不阻塞的来完成我们的工作：
/**
* 实例化一个ProcessModel
*
* @param process Process
*/
private ProcessModel(Process process) {
//init
this.process = process;
//get
out = process.getOutputStream();
in = process.getInputStream();
err = process.getErrorStream();
//in
if (in != null) {
isInReader = new InputStreamReader(in);
bInReader = new BufferedReader(isInReader, BUFFER_LENGTH);
}
sbReader = new StringBuilder();
//start read thread
readThread();
}
....................
//读取结果
private void read() {
String str;
//read In
try {
while ((str = bInReader.readLine()) != null) {
sbReader.append(str);
sbReader.append(BREAK_LINE);
}
} catch (Exception e) {
e.printStackTrace();
Logs.e(TAG, e.getMessage());
}
}
/**
* 启动线程进行异步读取结果
*/
private void readThread() {
Thread thread = new Thread(new Runnable() {
@Override
public void run() {
//
while (true) {
try {
process.exitValue();
//read last
read();
break;
} catch (IllegalThreadStateException e) {
read();
}
StaticFunction.sleepIgnoreInterrupt(300);
}
//read end
int len;
if (in != null) {
try {
while ((len = in.read(BUFFER)) > 0) {
Logs.d(TAG, String.valueOf(len));
}
} catch (IOException e) {
e.printStackTrace();
Logs.e(TAG, e.getMessage());
}
}
//close
close();
//done
isDone = true;
}
});
thread.setName("DroidTestAgent.Test.TestModel.ProcessModel:ReadThread");
thread.setDaemon(true);
thread.start();
}
当创建进程后把进程丢进我建立的类中实例化为一个进程管理类，随后启动线程，线程执行中调用进程的” exitValue()“ ，如果异常就进入读取数据，直到不异常时再次读取一次最后数据，随后退出循环，退出后还读取了一次底层的数据（这个其实可以不用要,纯属心理作用！）。最后写入完成标记。其中” StaticFunction.sleepIgnoreInterrupt(300); “是我写的静态方法用于休眠等待而已，也就是 Sleep ，只不过加入了 try catch 。
当然光是读取IN流是不行的，还有Error流，这个时候就需要两个线程来完成，一个也行。不过我为了简单采用了：ProcessBuilder类创建进程并重定向了错误流到IN流中，这样简化了操作。
而使用ProcessBuilder类需要注意的是同一个ProcessBuilder实例创建子进程的时候是需要进行线程同步操作的，因为如果并发操作将会导致进程参数错误等现象发生，所以建议加上线程互斥来实现，但是不建议重复创建ProcessBuilder实例，创建那么多实例，何不把所有子进程放在一个ProcessBuilder实例里边。减少内存消耗啊，手机伤不起啊。
有必要提出的是，当线程判断结束的时候，也就是退出值（exitvalue）有值得时候此时其实在”ProcessManager“线程中已经杀掉了进程了，此时在进程中其实没有此进程了，有的也就是执行后的数据流而已。所以正常结束情况下无需自己调用”destroy()“方法，调用后将会触发异常，说没有找到此进程。
public void destroy() {
try {
kill(this.id);
} catch (IOException e) {
Logger.getLogger(Runtime.class.getName()).log(Level.FINE,
"Failed to destroy process " + id + ".", e);
}
}
终于讲完了，累啊；
最后给大家分享我自己弄得一个类（ProcessModel），大家喜欢就直接拿去，如果有好的建议希望大家提出来：
import com.droidtestagent.journal.Logs;
import com.droidtestagent.util.StaticFunction;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
/**
* Create By Qiujuer
* 2014-08-05
* <p/>
* 执行命令行语句进程管理封装
*/
public class ProcessModel {
private static final String TAG = "ProcessModel";
//换行符
private static final String BREAK_LINE;
//错误缓冲
private static final byte[] BUFFER;
//缓冲区大小
private static final int BUFFER_LENGTH;
//创建进程时需要互斥进行
private static final Lock lock = new ReentrantLock();
//ProcessBuilder
private static final ProcessBuilder prc;
final private Process process;
final private InputStream in;
final private InputStream err;
final private OutputStream out;
final private StringBuilder sbReader;
private BufferedReader bInReader = null;
private InputStreamReader isInReader = null;
private boolean isDone;
/**
* 静态变量初始化
*/
static {
BREAK_LINE = "\n";
BUFFER_LENGTH = 128;
BUFFER = new byte[BUFFER_LENGTH];
prc = new ProcessBuilder();
}
/**
* 实例化一个ProcessModel
*
* @param process Process
*/
private ProcessModel(Process process) {
//init
this.process = process;
//get
out = process.getOutputStream();
in = process.getInputStream();
err = process.getErrorStream();
//in
if (in != null) {
isInReader = new InputStreamReader(in);
bInReader = new BufferedReader(isInReader, BUFFER_LENGTH);
}
sbReader = new StringBuilder();
//start read thread
readThread();
}
/**
* 执行命令
*
* @param params 命令参数 eg: "/system/bin/ping", "-c", "4", "-s", "100","www.qiujuer.net"
*/
public static ProcessModel create(String... params) {
Process process = null;
try {
lock.lock();
process = prc.command(params)
.redirectErrorStream(true)
.start();
} catch (IOException e) {
e.printStackTrace();
} finally {
//sleep 100
StaticFunction.sleepIgnoreInterrupt(100);
lock.unlock();
}
if (process == null)
return null;
return new ProcessModel(process);
}
/**
* 通过Android底层实现进程关闭
*
* @param process 进程
*/
public static void kill(Process process) {
int pid = getProcessId(process);
if (pid != 0) {
try {
android.os.Process.killProcess(pid);
} catch (Exception e) {
try {
process.destroy();
} catch (Exception ex) {
//ex.printStackTrace();
}
}
}
}
/**
* 获取进程的ID
*
* @param process 进程
* @return id
*/
public static int getProcessId(Process process) {
String str = process.toString();
try {
int i = str.indexOf("=") + 1;
int j = str.indexOf("]");
str = str.substring(i, j);
return Integer.parseInt(str);
} catch (Exception e) {
return 0;
}
}
//读取结果
private void read() {
String str;
//read In
try {
while ((str = bInReader.readLine()) != null) {
sbReader.append(str);
sbReader.append(BREAK_LINE);
}
} catch (Exception e) {
e.printStackTrace();
Logs.e(TAG, e.getMessage());
}
}
/**
* 启动线程进行异步读取结果
*/
private void readThread() {
Thread thread = new Thread(new Runnable() {
@Override
public void run() {
//while to end
while (true) {
try {
process.exitValue();
//read last
read();
break;
} catch (IllegalThreadStateException e) {
read();
}
StaticFunction.sleepIgnoreInterrupt(300);
}
//read end
int len;
if (in != null) {
try {
while ((len = in.read(BUFFER)) > 0) {
Logs.d(TAG, String.valueOf(len));
}
} catch (IOException e) {
e.printStackTrace();
Logs.e(TAG, e.getMessage());
}
}
//close
close();
//done
isDone = true;
}
});
thread.setName("DroidTestAgent.Test.TestModel.ProcessModel:ReadThread");
thread.setDaemon(true);
thread.start();
}
/**
* 获取执行结果
*
* @return 结果
*/
public String getResult() {
//waite process setValue
try {
process.waitFor();
} catch (Exception e) {
e.printStackTrace();
Logs.e(TAG, e.getMessage());
}
//until startRead en
while (true) {
if (isDone)
break;
StaticFunction.sleepIgnoreInterrupt(100);
}
//return
if (sbReader.length() == 0)
return null;
else
return sbReader.toString();
}
/**
* 关闭所有流
*/
private void close() {
//close out
if (out != null) {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
//err
if (err != null) {
try {
err.close();
} catch (IOException e) {
e.printStackTrace();
}
}
//in
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (isInReader != null) {
try {
isInReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (bInReader != null) {
try {
bInReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
/**
* 销毁
*/
public void destroy() {
//process
try {
process.destroy();
} catch (Exception ex) {
kill(process);
}
}
}
