Learning Disentangled Features for NeRF-Based Face Reconstruction

I. Overview

HeadNeRF can render photorealistic, controllable faces, but fitting its latent codes to each image is slow and prone to overfitting. HeadNeRF+ replaces iterative fitting with a learned encoder that directly predicts the disentangled reconstruction features.

The framework also introduces explicit semantic face-part guidance even though the underlying NeRF does not expose a conventional mesh.

II. Key Contributions

• Predicts HeadNeRF’s disentangled identity, expression, and appearance features directly from an input image.
• Adds a lightweight semantic face-segmentation network to expose facial-part structure.
• Uses a facial-part loss to improve reconstruction accuracy and local visual quality.

III. Methodology

A face encoder estimates the latent parameters consumed by a pretrained HeadNeRF renderer. A lightweight segmentation branch supplies semantic facial regions, and part-aware losses guide the encoder toward more accurate local reconstruction.

IV. Main Findings

The experiments report much lower reconstruction time than per-image fitting together with improved reconstruction accuracy and visual quality.

@inproceedings{yan2023learning,
  title={Learning disentangled features for NERF-based face reconstruction},
  author={Yan, Peizhi and Ward, Rabab and Wang, Dan and Tang, Qiang and Du, Shan},
  booktitle={2023 IEEE International Conference on Image Processing (ICIP)},
  pages={1135--1139},
  year={2023},
  organization={IEEE}
}