使用librosa,我为我的音频文件创建了mfcc,如下所示:
import librosa
y, sr = librosa.load('myfile.wav')
print y
print sr
mfcc=librosa.feature.mfcc(y=y, sr=sr)
我还有一个文本文件,其中包含与音频对应的手动注释[start,stop,tag],如下所示:
0.0 2.0 sound1
2.0 4.0 sound2
4.0 6.0 silence
6.0 8.0 sound1
题:
如何将生成的librosa生成的mfcc与文本文件中的注释结合起来.
最终目标是,我想结合对应于标签的mfcc,并传递
它到神经网络.
因此,神经网络将mfcc和相应的标签作为训练数据.
如果它是一维的,我可以有N列N值,最后一列Y带有Class标签.
但我很困惑如何继续,因为mfcc具有类似的形状
(16,X)或
(20,Y).
所以我不知道如何将两者结合起来.
我的样本mfcc在这里:https://gist.github.com/manbharae/0a53f8dfef6055feef1d8912044e1418
请帮忙谢谢.
更新:目标是训练神经网络,以便在将来遇到它时识别新的声音.
我用Google搜索,发现mfcc非常适合演讲.然而,我的音频有语音,但我想识别非语音.是否有其他推荐的音频功能用于通用音频分类/识别任务?
解决方法:
请尝试以下方法.解释包含在代码中.
import numpy
import librosa
# The following function returns a label index for a point in time (tp)
# this is psuedo code for you to complete
def getLabelIndexForTime(tp):
# search the loaded annoations for what label corresponsons to the given time
# convert the label to an index that represents its unqiue value in the set
# ie.. 'sound1' = 0, 'sound2' = 1, ...
#print tp #for debug
label_index = 0 #replace with logic above
return label_index
if __name__ == '__main__':
# Load the waveforms samples and convert to mfcc
raw_samples, sample_rate = librosa.load('Front_Right.wav')
mfcc = librosa.feature.mfcc(y=raw_samples, sr=sample_rate)
print 'Wave duration is %4.2f seconds' % (len(raw_samples)/float(sample_rate))
# Create the network's input training data, X
# mfcc is organized (feature, sample) but the net needs (sample, feature)
# X is mfcc reorganized to (sample, feature)
X = numpy.moveaxis(mfcc, 1, 0)
print 'mfcc.shape:', mfcc.shape
print 'X.shape: ', X.shape
# Note that 512 samples is the default 'hop_length' used in calculating
# the mfcc so each mfcc spans 512/sample_rate seconds.
mfcc_samples = mfcc.shape[1]
mfcc_span = 512/float(sample_rate)
print 'MFCC calculated duration is %4.2f seconds' % (mfcc_span*mfcc_samples)
# for 'n' network input samples, calculate the time point where they occur
# and get the appropriate label index for them.
# Use +0.5 to get the middle of the mfcc's point in time.
Y = []
for sample_num in xrange(mfcc_samples):
time_point = (sample_num + 0.5) * mfcc_span
label_index = getLabelIndexForTime(time_point)
Y.append(label_index)
Y = numpy.array(Y)
# Y now contains the network's output training values
# !Note for some nets you may need to convert this to one-hot format
print 'Y.shape: ', Y.shape
assert Y.shape[0] == X.shape[0] # X and Y have the same number of samples
# Train the net with something like...
# model.fit(X, Y, ... #ie.. for a Keras NN model
我应该提到的是,这里的Y数据旨在用于具有softmax输出的网络,该输出可以用整数标签数据进行训练. Keras模型接受了sparse_categorical_crossentropy损失函数(我相信损失函数在内部将其转换为单热编码).其他框架要求以单热编码格式传递Y训练标签.这种情况比较常见.有很多关于如何进行转换的例子.对于你的情况,你可以做一些像……
Yoh = numpy.zeros(shape=(Y.shape[0], num_label_types), dtype='float32')
for i, val in enumerate(Y):
Yoh[i, val] = 1.0
至于mfcc是否可以接受非语音分类,我希望它们可以工作,但你可能想尝试修改它们的参数,即.. librosa允许你做一些像n_mfcc = 40这样你得到40个特征而不是20个.有趣的是,您可以尝试使用相同大小的简单FFT替换mfcc(512个样本),看看哪个效果最好.