电脑设计比赛：用Python 3实现在直播弹幕中防止遮挡的深度学习人体语义分割技巧

简介
实例分割已成为机器视觉研究中比较重要、复杂和具有挑战性的领域之一。为了预测对象类标签和特定于像素的对象实例掩码，它对各种图像中出现的对象实例的不同类进行本地化。实例分割的目的主要是帮助机器人，自动驾驶，监视等。

实例分割同时利用目标检测和语义分割的结果，通过目标检测提供的目标最高置信度类别的索引，将语义分割中目标对应的Mask抽取出来。实例分割顾名思义，就是把一个类别里具体的一个个对象（具体的一个个例子）分割出来。

Mask R-CNN算法
本项目使用Mask R-CNN算法来进行图像实例分割。
网络结构图：

Mask R-CNN，一个相对简单和灵活的实例分割模型。该模型通过目标检测进行了实例分割，同时生成了高质量的掩模。通常，Faster
R-CNN有一个用于识别物体边界框的分支。Mask R-CNN并行添加了一个对象蒙版预测分支作为改进。使用FPN主干的head架构如图所示。

关键代码

​

    ##利用不同的颜色为每个instance标注出mask，根据box的坐标在instance的周围画上矩形
    ##根据class_ids来寻找到对于的class_names。三个步骤中的任何一个都可以去掉，比如把mask部分
    ##去掉，那就只剩下box和label。同时可以筛选出class_ids从而显示制定类别的instance显示,下面
    ##这段就是用来显示人的，其实也就把人的id选出来，然后记录它们在输入ids中的相对位置，从而得到
    ##相对应的box与mask的准确顺序
    def display_instances_person(image, boxes, masks, class_ids, class_names,
                          scores=None, title="",
                          figsize=(16, 16), ax=None):
        """
        the funtion perform a role for displaying the persons who locate in the image
        boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates.
        masks: [height, width, num_instances]
        class_ids: [num_instances]
        class_names: list of class names of the dataset
        scores: (optional) confidence scores for each box
        figsize: (optional) the size of the image.
        """
        #compute the number of person
        temp = []
        for i, person in enumerate(class_ids):
            if person == 1:
                temp.append(i)
            else:
                pass
        person_number = len(temp)
        
        person_site = {}
        
        for i in range(person_number):
            person_site[i] = temp[i]


        NN = boxes.shape[0]   
        # Number of person'instances
        #N = boxes.shape[0]
        N = person_number
        if not N:
            print("\n*** No person to display *** \n")
        else:
           # assert boxes.shape[0] == masks.shape[-1] == class_ids.shape[0]
            pass
     
        if not ax:
            _, ax = plt.subplots(1, figsize=figsize)
     
        # Generate random colors
        colors = random_colors(NN)
     
        # Show area outside image boundaries.
        height, width = image.shape[:2]
        ax.set_ylim(height + 10, -10)
        ax.set_xlim(-10, width + 10)
        ax.axis('off')
        ax.set_title(title)
     
        masked_image = image.astype(np.uint32).copy()
        for a in range(N):
            
            color = colors[a]
            i = person_site[a]

            # Bounding box
            if not np.any(boxes[i]):
                # Skip this instance. Has no bbox. Likely lost in image cropping.
                continue
            y1, x1, y2, x2 = boxes[i]
            p = patches.Rectangle((x1, y1), x2 - x1, y2 - y1, linewidth=2,
                                  alpha=0.7, linestyle="dashed",
                                  edgecolor=color, facecolor='none')
            ax.add_patch(p)
     
            # Label
            class_id = class_ids[i]
            score = scores[i] if scores is not None else None
            label = class_names[class_id]
            x = random.randint(x1, (x1 + x2) // 2)
            caption = "{} {:.3f}".format(label, score) if score else label
            ax.text(x1, y1 + 8, caption,
                    color='w', size=11, backgroundcolor="none")
            
             # Mask
            mask = masks[:, :, i]
            masked_image = apply_mask(masked_image, mask, color)
     
            # Mask Polygon
            # Pad to ensure proper polygons for masks that touch image edges.
            padded_mask = np.zeros(
                (mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
            padded_mask[1:-1, 1:-1] = mask
            contours = find_contours(padded_mask, 0.5)
            for verts in contours:
                # Subtract the padding and flip (y, x) to (x, y)
                verts = np.fliplr(verts) - 1
                p = Polygon(verts, facecolor="none", edgecolor=color)
                ax.add_patch(p)
           
        ax.imshow(masked_image.astype(np.uint8))
        plt.show()