JavaCV 采集摄像头和麦克风数据推送到流媒体服务器

越来越觉得放弃JavaCV FFmpeg native API,直接使用JavaCV二次封装的API开发是很明智的选择,使用JavaCV二次封装的API开发避免了各种内存操作不当引起的crash。

上一次介绍了 JavaCV 采集摄像头及桌面视频数据,这次介绍一下如何采集摄像头和麦克风数据推送到流媒体服务器。

引入依赖

跟上一次一样,这里使用的还是最新的JavaCV库(1.5.5).

<properties>
<javacpp.version>1.5.5</javacpp.version>
</properties>
<dependencies>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv</artifactId>
<version>${javacpp.version}</version>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv-platform</artifactId>
<version>${javacpp.version}</version>
</dependency>
</dependencies>

视频采集录制

JavaCV 采集摄像头及桌面视频数据 介绍了两种采集摄像头数据的方法,这里采用第一种,即使用 OpencvFrameGrabber 采集摄像头数据。

public class VideoRecorder implements Runnable {

  private static final int VIDEO_DEVICE_INDEX = 0;
private FFmpegFrameRecorder recorder;
private int width, height;
public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
this.recorder = recorder;
this.width = width;
this.height = height;
} @Override
public void run() {
try {
OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
grabber.setImageWidth(width);
grabber.setImageHeight(height);
grabber.start(); long startTS = 0, videoTS = 0;
Frame frame = null;
while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
if (startTS == 0) {
startTS = System.currentTimeMillis();
}
videoTS = 1000 * (System.currentTimeMillis() - startTS);
if (videoTS > recorder.getTimestamp()) {
recorder.setTimestamp(videoTS);
}
recorder.record(frame);
} grabber.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}

音频采集录制

音频采集直接使用JDK的API即可,直接读麦克风的数据塞给录制器,采样率为44100,16bit,小端模式。

public class AudioRecoder implements Runnable {

  private FFmpegFrameRecorder recorder;
private int channels;
private int sampleRate;
public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
this.recorder = recorder;
this.sampleRate = sampleRate;
this.channels = channels;
} @Override
public void run() {
try {
AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
line.open(format);
line.start(); int sampleRate = (int) format.getSampleRate();
int numChannels = format.getChannels();
byte[] buffer = new byte[sampleRate * numChannels]; while (!Thread.interrupted()) {
int nBytesRead = 0;
while (nBytesRead == 0) {
nBytesRead = line.read(buffer, 0, line.available());
}
int nSamplesRead = nBytesRead / 2;
short[] samples = new short[nSamplesRead];
ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
recorder.recordSamples(sampleRate, numChannels, sBuff);
}
} catch (Exception e) {
e.printStackTrace();
}
} }

录制推流

这里的录制推流涉及音频和视频,音频和视频分开线程采集并录制,这样可以降低延迟,在录制推流方面还可以通过设置录制参数降低延迟:

recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");

启动录制推流器,视频采集录制线程和音频采集录制线程,开始推流:

recorder.start();
Thread vt = new Thread(new VideoRecorder(recorder, width, height));
Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels)); vt.start();
at.start(); vt.join();
at.join();

效果展示

可以使用VLC拉取程序的推流查看效果:

JavaCV 采集摄像头和麦克风数据推送到流媒体服务器

拉流的音视频流信息:

JavaCV 采集摄像头和麦克风数据推送到流媒体服务器

录制推流完整源码可以在公众号上获取

=========================================================

关注公众号 “HiIT青年” 发送 “javacv-recoder” 获取。(如果没有收到回复,可能是你之前取消过关注。)

JavaCV 采集摄像头和麦克风数据推送到流媒体服务器

关注公众号,阅读更多文章。

上一篇:JavaCV 采集摄像头及桌面视频数据


下一篇:Linux下用FFMPEG采集usb摄像头到RTMP