Face Segmentation

Face Segmentation

2017, Feb 28    

Face Segmentation is not a very well defined problem. We define face segmentation to include the visible part of the face excluding the neck, ears, hair, long beards, and any object that might obscure it. This project implements face segmentation using a fully convolutional neural network as described in our paper.

Code has been made available at: https://github.com/YuvalNirkin/face_segmentation

Overview

Our method uses a FCN to segment the visible parts of faces from their context and occlusion. We used the FCN-8s-VGG architecture, fine-tuned for segmentation, for which we fuse information at different locations from layers with different strides

Face Segmentation CNN (FCN-8s-VGG)

Training

The network was trained on IARPA Janus CS2 dataset (excluding subjects that are also in LFW) using a novel process for collecting ground truth face segmentations, involving our tool for semi-supervised Face video segmentation.

Face Segmentation CNN (FCN-8s-VGG)

We generate additional synthetic images by augmenting 3D models of glasses and microphones:

Face Segmentation CNN (FCN-8s-VGG)

We couldn’t find quality 3D models of hands, so for augmenting hands we used the EgoHands dataset, which included images of hands and their segmentations:

Face Segmentation CNN (FCN-8s-VGG)