《A chorus section detection method for musical audio signals and its application to a music listenin

Abstract:

重复的副歌识别对音乐理解的计算模型(computational model)至关重要,应用层面有:音乐副歌识别预览,音乐检索等。

传统检测的难点:变调,起始点和结束点(both ends)检测。

作者提出的方法RefrailD,可双端检测所有副歌,引入perceptually motivated acoustic feature可以检测变调的chorus

在流行歌曲数据集中测试100首歌曲,正确识别80首

同时 paper还介绍了基于该方法的一个应用,可直接跳转到chorus进行播放的软件

 

关键词

chorus detection,chroma vector,music-playback interface,music structure,music understanding

introduction

没啥干货

related work

【3】-【5】提取指定长度的副歌,【3】采用clustering techniques和HMMs根据声学特征来categorize 1m的片段,most frequent category被视为副歌,【4】采用相似度在节奏段中进行比较,【5】计算了similarity matrix of acoustic features of short frames。以上方法计算出来的副歌长度都是给定的。

其他paper bla bla bla

以上所有方法都不能将所有chorus检测出来,也不能对变调的chorus进行处理

chorus section detection problem

作者提出的fact:chorus sections are usually the most repeated sections of a song. 问题在于计算机无法识别重复段落(总会有差异)

问题一:acoustic features and similarity

判断一个section是否重复可以基于提取的acoustic feature来比较相似度(排除乐器变换)。simple power spectrums信号或者MFCC不能满足这个条件。

问题二:重复度判定规则

相似度要达到多少,才能判定其为重复段落,可根据歌曲类型动态设置

问题三:判断重复段落的起始位置:

歌曲结构ABCBCC 不能判定为BC,而是C

问题四:检测变调段落

常规acoustic feature会因为变调而改变

 

method

 《A chorus section detection method for musical audio signals and its application to a music listenin

首先分窗口提取chroma vector,chrome vector对乐器改变健壮性强。对这些vector进行相似度判定。chrome vector的每个元素对应12平分律当中的一个音,并且频率量级的计算横跨6个8度音阶(解决问题一)

借鉴【17】提出的discriminant criterion采用动态自适应的threshold判定并列举出所有重复段落 (解决问题二)

为了将重复段落分组并识别出both ends,需要分析段落与整首歌曲的对应关系(解决问题三)

由于chroma vector的元素对应了 一组类别的音符,所以变调前和变调后差别不大(解决问题四)

最后对所有成组的重复段落进行判定,副歌概率最高的被识别为副歌

 

 A. 提取声学特征

提取chroma vector的流程如下所示,作者描述它是perceptually-motivated的【18】:

《A chorus section detection method for musical audio signals and its application to a music listenin

乐器改变和变调等因素对chroma vector影响很小,【22,23】用其来识别和弦

 

B 相似度判定

《A chorus section detection method for musical audio signals and its application to a music listenin

 

 C 列举出重复段落

《A chorus section detection method for musical audio signals and its application to a music listenin中可以提取出成对的重复段落。将《A chorus section detection method for musical audio signals and its application to a music listenin绘制在等腰直角三角形的坐标系中,坐标轴分别为time和lag

《A chorus section detection method for musical audio signals and its application to a music listenin

 

平行于time轴的线段表示了高相似度的《A chorus section detection method for musical audio signals and its application to a music listenin《A chorus section detection method for musical audio signals and its application to a music listenin    T1和T2之间的时间段表示为[T1,T2],则线段(T1,L1)到(T1,L2)即(t=[T1,T2],l=L1)表示[T1,T2]的内容与[T1-L1,T2-L1]的内容是相似的。

我们需要找出坐标图中所有的平行线段,对于给定的lag l,我们通过下面公式计算其在时间区间t=[T1,T2]上包含平行线的可能性:

《A chorus section detection method for musical audio signals and its application to a music listenin

计算之前,首先将坐标图上的相似度点《A chorus section detection method for musical audio signals and its application to a music listenin做normalize,减去一个局部均值消除噪音。

1.对坐标中每个点《A chorus section detection method for musical audio signals and its application to a music listenin,延展《A chorus section detection method for musical audio signals and its application to a music listenin(这里取15个点,1.2s)计算其6个方向的均值何最大最小值

2. 如果左,右均值就是最大值,则该点可判定为在平行线上,并将其减去《A chorus section detection method for musical audio signals and its application to a music listenin的最小值以突出

3.否则《A chorus section detection method for musical audio signals and its application to a music listenin是噪音,将其减去《A chorus section detection method for musical audio signals and its application to a music listenin的最大值以降噪

然后从《A chorus section detection method for musical audio signals and its application to a music listenin中寻找导数改变符号的极值点(从正到负)

《A chorus section detection method for musical audio signals and its application to a music listenin 

Ksize设置为4(0.32s),计算前的预处理涉及一些通道计算,没看太明白

 

 D. 重复段落分组 integrate repeated sections

设想一个段落如果重复了n次(n>=3),则分组中的线段数量应为(n-1)*n/2。分组遵循规则:

1.两个线段包含相同的片段[Tsi,Tei]

2. 两个端点的时间差异性要小于一定阈值threshhold

每个group表示为《A chorus section detection method for musical audio signals and its application to a music listenin

 

 

 《A chorus section detection method for musical audio signals and its application to a music listenin

 

 

 

副歌判定选取规则:

副歌得分判定公式为:

《A chorus section detection method for musical audio signals and its application to a music listenin

Vi作为$lamdaij$的总和(以segmentation的长度加权计算)。Dlen为常数(1.4s)。$lamdaij$作为片段是否为副歌的概率,其估算遵循以下猜想:

1. 副歌长度符合一定规则,一般为7.7s-40s,不在此范围内,则$lamdaij$设为0

2. 副歌组中一般会有一段更接近结尾,如果是,则该组的$lamdaij$翻倍

3.副歌中一般含有两段子重复段落,如果是的话,$lamdaij$加权 

 

上一篇:javascript – 检测移动设备


下一篇:【Demo 1】基于object_detection API的行人检测 1:环境与依赖