与我们熟知的MP3格式一样,AAC是一种音频编码格式,对比MP3格式,AAC在缩小30%的前题下可以提供更好的音质。这篇博客的主要内容就是通过AudioRecorder录制PCM音频,再通过MediaCodec将PCM数据硬编码为AAC格式的音频。
通常我们使用MediaCodec的流程如下:
MediaCodec的使用流程:
- createEncoderByType/createDecoderByType
- configure
- start
- while(1) {
- dequeueInputBuffer
- queueInputBuffer
- dequeueOutputBuffer
- releaseOutputBuffer
- }
- stop
- release
编解码器一个比较经典的工作原理图如下:
图中的Client一般就是我们开发者,解释一下就是:我们从Codec中拿到拿到空的input buffer,然后填充上我们需要进行编码的数据,再输送给Codec,Codec对数据进行编解码,编解码完成后,Codec将处理好的数据放进output buffer,我们取出后再清空返还给Codec,形成一个环形结构。可以看作一个生产者-消费者模式。
下面我们的编码流程也基本遵守上面的这个流程。为了便于大家理解,我画个流程图讲一下我整个代码的逻辑:
录音和编码分别在两个线程中进行,两个线程通过一个ArrayBlockQueue(这是一个线程安全的队列,想了解更多自己动手)队列共享数据,录音线程中的AudioRecorder通过read()将一帧数据put()到队尾,编码线程中的MediaCodec再通过take()取出队首的一帧数据进行编码。
首先看录音线程,为了使代码更加简洁易懂,我会省略掉一些代码,完整代码会在文末贴出。
/**
* 录音线程
*/
public class AudioRecorder extends Thread {
private AudioRecord mAudioRecord;
private boolean isRecording;
private int minBufferSize;
public AudioRecorder() {
isRecording = true;
initRecorder();
}
@Override
public void run() {
super.run();
startRecording();
}
/**
* 初始化录音
*/
public void initRecorder(){
minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat);
mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, sampleRateInHz, channelConfig, audioFormat, minBufferSize);
if (mAudioRecord.getState() != AudioRecord.STATE_INITIALIZED) {
isRecording = false;
return;
}
}
/**
* 开始录音
*/
public void startRecording(){
if (mAudioRecord == null){
return;
}
mAudioRecord.startRecording();
while (isRecording) {
//自定义的一个类,用来存储一帧pcm数据,即byte[],下面给出具体定义,很简单
AudioDate audioDate = new AudioDate();
audioDate.buffer = ByteBuffer.allocateDirect(minBufferSize);
audioDate.size = mAudioRecord.read(audioDate.buffer, minBufferSize);
try {
if (queue != null) {
queue.put(audioDate);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
release();
}
}
录音线程比较简单,主要是先初始化录音器在initRecorder()中,然后通过AudioRecorder的read方法,获取到一帧数据,通过queue.put放入队尾。
然后是编码线程。
/**
* 音频编码线程
*/
public class AudioEncorder extends Thread {
private MediaCodec mEncorder;
private Boolean isEncording = false;
private int minBufferSize;
private OutputStream mFileStream;
public AudioEncorder() {
isEncording = true;
initEncorder();
}
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
@Override
public void run() {
super.run();
startEncording();
}
/**
* 初始化编码器
*/
private void initEncorder(){
minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat);
try {
mEncorder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
} catch (IOException e) {
e.printStackTrace();
}
MediaFormat format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRateInHz, channelConfig);
format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC);
format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
format.setInteger(MediaFormat.KEY_BIT_RATE, 96000);
format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, minBufferSize * 2);
mEncorder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
}
/**
* 开始编码
*/
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
public void startEncording(){
if (mEncorder == null){
return;
}
mEncorder.start();
try {
mFileStream = new FileOutputStream(getSDPath() + "/aac_encode.aac");
MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo();
AudioDate audioDate;
while (isEncording) {
// 从队列中取出录音的一帧音频数据
audioDate = getAudioDate();
if (audioDate == null) {
continue;
}
// 取出InputBuffer,填充音频数据,然后输送到编码器进行编码
int inputBufferIndex = mEncorder.dequeueInputBuffer(0);
if (inputBufferIndex >= 0) {
ByteBuffer inputBuffer = mEncorder.getInputBuffer(inputBufferIndex);
inputBuffer.clear();
inputBuffer.put(audioDate.buffer);
mEncorder.queueInputBuffer(inputBufferIndex, 0, audioDate.size, System.nanoTime(), 0);
}
// 取出编码好的一帧音频数据,然后给这一帧添加ADTS头
int outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0);
while (outputBufferIndex >= 0) {
int outBitsSize = mBufferInfo.size;
int outPacketSize = outBitsSize + 7; // ADTS头部是7个字节
ByteBuffer outputBuffer = mEncorder.getOutputBuffer(outputBufferIndex);
outputBuffer.position(mBufferInfo.offset);
outputBuffer.limit(mBufferInfo.offset + outBitsSize);
byte[] outData = new byte[outPacketSize];
addADTStoPacket(outData, outPacketSize);
outputBuffer.get(outData, 7, outBitsSize);
outputBuffer.position(mBufferInfo.offset);
mFileStream.write(outData);
mEncorder.releaseOutputBuffer(outputBufferIndex, false);
outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0);
}
}
release();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* 添加ADTS头
* @param packet
* @param packetLen
*/
private void addADTStoPacket(byte[] packet, int packetLen) {
int profile = 2; //AAC LC
int freqIdx = 4; //44100 根据不同的采样率修改这个值
int chanCfg = 2; //CPE
packet[0] = (byte) 0xFF;
packet[1] = (byte) 0xF9;
packet[2] = (byte) (((profile - 1) << 6) + (freqIdx << 2) + (chanCfg >> 2));
packet[3] = (byte) (((chanCfg & 3) << 6) + (packetLen >> 11));
packet[4] = (byte) ((packetLen & 0x7FF) >> 3);
packet[5] = (byte) (((packetLen & 7) << 5) + 0x1F);
packet[6] = (byte) 0xFC;
}
}
和录音线程一样,先进行编码器的初始化在initEncorder()中,这里的参数MediaFormat比较关键,针对具体的编码格式有不同的编码格式有不同的参数,其他的编码方式自行参阅官方文档,这里的AAC编码器参数我是经过测试可行的。
然后是最关键的一步–编码。在startEncording()方法中,所有的流程完全对应上面我列出的MediaCodec的使用流程,可以对照上面的流程来阅读代码。
编码流程中非常重要的一步就是为每一帧AAC音频添加ADTS头,和WAV格式的音频不同,AAC为每一帧音频都添加了一个ADTS头,使得解码器可以从任意一帧开始解码,有时我们遇到无法编码的AAC文件无法播放,可能就是因为我们没有为其添加ADTS头。通过上面的addADTStoPacket()方法便可以未每一帧AAC音频添加ADTS头了。
至此,我就列出了我认为在AAC编码的过程中所需要注意的所有点。在完成这个功能的时候,我也参照了一些别人的做法,以及我自己的一些理解与实践,也遇到了一些令人头大的错误(例如:同样的PCM数据,放进队列中再取出来写入文件,就出现了很大的杂音)如有错误的地方,还请留言指正。
下面是完整代码:
package com.example.sisyphus.audiovideolearning;
import android.Manifest;
import android.app.Activity;
import android.content.pm.PackageManager;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.MediaCodec;
import android.media.MediaCodecInfo;
import android.media.MediaFormat;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Bundle;
import android.os.Environment;
import android.support.annotation.Nullable;
import android.support.annotation.RequiresApi;
import android.support.v4.app.ActivityCompat;
import android.view.MotionEvent;
import android.view.View;
import android.widget.Button;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.util.concurrent.ArrayBlockingQueue;
/**
* Created by sisyphus on 2018/8/8. * 学习 MediaCodec API,完成音频 AAC 硬编、硬解
*
* MediaCodec的使用流程:
* - createEncoderByType/createDecoderByType
* - configure * - start * - while(1) { * - dequeueInputBuffer * - queueInputBuffer * - dequeueOutputBuffer * - releaseOutputBuffer * } * - stop * - release */public class AACCodecActivity extends Activity {
private final int sampleRateInHz = 44100;
private final int channelConfig = 1;
private final int audioFormat = AudioFormat.ENCODING_PCM_16BIT;
private class AudioDate {
private ByteBuffer buffer;
private int size;
}
public static String[] MICROPHONE = {Manifest.permission.RECORD_AUDIO};
public static String[] STORAGE = {Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.WRITE_EXTERNAL_STORAGE};
private AudioRecorder mAudioRecorder;
private AudioEncorder mAudioEncorder;
private ArrayBlockingQueue queue;
private Button btnStartRecording;
@Override
protected void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_aac_codec);
btnStartRecording = findViewById(R.id.btn_start_aac_encode);
queue = new ArrayBlockingQueue<>(1024);
mAudioRecorder = new AudioRecorder();
mAudioEncorder = new AudioEncorder();
btnStartRecording.setOnTouchListener(new View.OnTouchListener() {
@Override
public boolean onTouch(View view, MotionEvent motionEvent) {
int action = motionEvent.getAction();
if (action == MotionEvent.ACTION_DOWN) {
checkRecordPermission();
btnStartRecording.setText("正在录制");
startRecord();
} else if (action == MotionEvent.ACTION_UP) {
btnStartRecording.setText("录音");
stopRecord();
}
return false;
}
});
}
private void startRecord() {
mAudioRecorder.start();
mAudioEncorder.start();
}
private void stopRecord() {
mAudioRecorder.stopRecording();
mAudioEncorder.stopEncording();
}
/**
* 录音线程
*/
public class AudioRecorder extends Thread {
private AudioRecord mAudioRecord;
private boolean isRecording;
private int minBufferSize;
public AudioRecorder() {
isRecording = true;
initRecorder();
}
@Override
public void run() {
super.run();
startRecording();
}
/**
* 初始化录音
*/
public void initRecorder(){
minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat);
mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, sampleRateInHz, channelConfig, audioFormat, minBufferSize);
if (mAudioRecord.getState() != AudioRecord.STATE_INITIALIZED) {
isRecording = false;
return; }
}
/**
* 释放资源
*/
public void release() {
if (mAudioRecord != null && mAudioRecord.getState() == AudioRecord.STATE_INITIALIZED) {
mAudioRecord.stop();
}
}
/**
* 开始录音
*/
public void startRecording(){
if (mAudioRecord == null){
return;
}
mAudioRecord.startRecording();
while (isRecording) {
AudioDate audioDate = new AudioDate();
audioDate.buffer = ByteBuffer.allocateDirect(minBufferSize);
audioDate.size = mAudioRecord.read(audioDate.buffer, minBufferSize);
try {
if (queue != null) {
queue.put(audioDate);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
release();
}
/**
* 结束录音
*/
public void stopRecording() {
isRecording = false;
}
}
/**
* 音频编码线程
*/
public class AudioEncorder extends Thread {
private MediaCodec mEncorder;
private Boolean isEncording = false;
private int minBufferSize;
private OutputStream mFileStream;
public AudioEncorder() {
isEncording = true;
initEncorder();
}
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
@Override
public void run() {
super.run();
startEncording();
}
/**
* 初始化编码器
*/
private void initEncorder(){
minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat);
try {
mEncorder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC);
} catch (IOException e) {
e.printStackTrace();
}
MediaFormat format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRateInHz, channelConfig);
format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC);
format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
format.setInteger(MediaFormat.KEY_BIT_RATE, 96000);
format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, minBufferSize * 2);
mEncorder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
}
/**
* 开始编码
*/
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
public void startEncording(){
if (mEncorder == null){
return;
}
mEncorder.start();
try {
mFileStream = new FileOutputStream(getSDPath() + "/aac_encode.aac");
MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo();
AudioDate audioDate;
while (isEncording) {
// 从队列中取出录音的一帧音频数据
audioDate = getAudioDate();
if (audioDate == null) {
continue;
}
// 取出InputBuffer,填充音频数据,然后输送到编码器进行编码
int inputBufferIndex = mEncorder.dequeueInputBuffer(0);
if (inputBufferIndex >= 0) {
ByteBuffer inputBuffer = mEncorder.getInputBuffer(inputBufferIndex);
inputBuffer.clear();
inputBuffer.put(audioDate.buffer);
mEncorder.queueInputBuffer(inputBufferIndex, 0, audioDate.size, System.nanoTime(), 0);
}
// 取出编码好的一帧音频数据,然后给这一帧添加ADTS头
int outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0);
while (outputBufferIndex >= 0) {
int outBitsSize = mBufferInfo.size;
int outPacketSize = outBitsSize + 7; // ADTS头部是7个字节
ByteBuffer outputBuffer = mEncorder.getOutputBuffer(outputBufferIndex);
outputBuffer.position(mBufferInfo.offset);
outputBuffer.limit(mBufferInfo.offset + outBitsSize);
byte[] outData = new byte[outPacketSize];
addADTStoPacket(outData, outPacketSize);
outputBuffer.get(outData, 7, outBitsSize);
outputBuffer.position(mBufferInfo.offset);
mFileStream.write(outData);
mEncorder.releaseOutputBuffer(outputBufferIndex, false);
outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0);
}
}
release();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* 停止编码
*/
public void stopEncording() {
isEncording = false;
}
/**
* 从队列中取出一帧待编码的音频数据
* @return
*/
public AudioDate getAudioDate(){
if (queue != null){
try {
return queue.take();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
return null;
}
/**
* 添加ADTS头
* @param packet
* @param packetLen
*/
private void addADTStoPacket(byte[] packet, int packetLen) {
int profile = 2; //AAC LC
int freqIdx = 4; //44100 根据不同的采样率修改这个值
int chanCfg = 2; //CPE
packet[0] = (byte) 0xFF;
packet[1] = (byte) 0xF9;
packet[2] = (byte) (((profile - 1) << 6) + (freqIdx << 2) + (chanCfg >> 2));
packet[3] = (byte) (((chanCfg & 3) << 6) + (packetLen >> 11));
packet[4] = (byte) ((packetLen & 0x7FF) >> 3);
packet[5] = (byte) (((packetLen & 7) << 5) + 0x1F);
packet[6] = (byte) 0xFC;
}
/**
* 释放资源
*/
public void release() {
if (mFileStream != null) {
try {
mFileStream.flush();
mFileStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (mEncorder != null) {
mEncorder.stop();
}
}
}
public String getSDPath() {
// 判断是否挂载
if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
return Environment.getExternalStorageDirectory().getAbsolutePath();
}
return Environment.getRootDirectory().getAbsolutePath();
}
public void checkRecordPermission() {
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, MICROPHONE, 1);
return; }
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, STORAGE, 1);
return; }
}
}