NMS算法的理解
NonMaximumSuppression 非极大值抑制
当预测网络预测出bbox的位置之后,一定会产生很多种可能。每一个bbox包括位置信息和置信度(概率),这个时候就需要根据nms的来排除掉一些冗余的bbox。
例如,人脸检测算法得到了8个人脸检测框,这8个检测框中明显是由两个人同时有两个框的,这样就产生了冗余,需要利用nms将这些多余的框去掉。
代码:
//人脸检测结果数据结构bbox
typedef struct FaceRect{
float x1;
float y1;
float x2;
float y2;
float score; /**< Larger score should mean higher confidence. */
} FaceRect;
排序:
// compare score
bool CompareBBox(const FaceInfo & a, const FaceInfo & b) {
return a.bbox.score > b.bbox.score;
}
nms代码:
std::vector<FaceInfo> NonMaximumSuppression(std::vector<FaceInfo>& bboxes,
float thresh,char methodType){
std::vector<FaceInfo> bboxes_nms;
std::sort(bboxes.begin(), bboxes.end(), CompareBBox);//按照score降序排列
int32_t select_idx = 0;
int32_t num_bbox = static_cast<int32_t>(bboxes.size());
std::vector<int32_t> mask_merged(num_bbox, 0);
bool all_merged = false;
while (!all_merged) {
while (select_idx < num_bbox && mask_merged[select_idx] == 1)
select_idx++;
if (select_idx == num_bbox) {
all_merged = true;
continue;NM
}
bboxes_nms.push_back(bboxes[select_idx]);
mask_merged[select_idx] = 1;
FaceRect select_bbox = bboxes[select_idx].bbox;
float area1 = static_cast<float>((select_bbox.x2-select_bbox.x1+1) * (select_bbox.y2-select_bbox.y1+1));
float x1 = static_cast<float>(select_bbox.x1);
float y1 = static_cast<float>(select_bbox.y1);
float x2 = static_cast<float>(select_bbox.x2);
float y2 = static_cast<float>(select_bbox.y2);
select_idx++;
for (int32_t i = select_idx; i < num_bbox; i++) {
if (mask_merged[i] == 1)
continue;
FaceRect& bbox_i = bboxes[i].bbox;
float x = std::max<float>(x1, static_cast<float>(bbox_i.x1));
float y = std::max<float>(y1, static_cast<float>(bbox_i.y1));
float w = std::min<float>(x2, static_cast<float>(bbox_i.x2)) - x + 1;
float h = std::min<float>(y2, static_cast<float>(bbox_i.y2)) - y + 1;
if (w <= 0 || h <= 0)
continue;
float area2 = static_cast<float>((bbox_i.x2-bbox_i.x1+1) * (bbox_i.y2-bbox_i.y1+1));
float area_intersect = w * h;
switch (methodType) {
case 'u':
if (static_cast<float>(area_intersect) / (area1 + area2 - area_intersect) > thresh)
mask_merged[i] = 1;
break;
case 'm':
if (static_cast<float>(area_intersect) / std::min(area1 , area2) > thresh)
mask_merged[i] = 1;
break;
default:
break;
}
}
}
return bboxes_nms;
}
大体思路
首先对8个bbox进行降序排列,排序的依据就是bbox.score的值,从大到小依次排好。
std::sort(bboxes.begin(), bboxes.end(), CompareBBox);//按照score降序排列
分数最高的肯定是要保留下来的
bboxes_nms.push_back(bboxes[select_idx]);//最初select_idx = 0
然后进行一个 (人脸数 -1)次的循环,依次判断后续的人脸的和第一个人脸的IOU,如果大于阈值,那么就将这个人脸pass掉,如果后面的人脸和第一个人脸的IOU为0那么就跳过,先不处理。
if (static_cast<float>(area_intersect) / (area1 + area2 - area_intersect) > thresh)
mask_merged[i] = 1;//这里利用mesk_merged做标记,标记为1的证明大于阈值,需要排除
依次循环判断,最后将冗余的框排除。
但是nms在目标检测的有些时候会影响到一定的召回率,例如下面这种情况
有一篇论文专门针对nms进行了优化,称之为soft-nms。下篇文章再总结下soft-nms