NEO-3DF

01 — Overview

Overview

NEO-3DF treats reconstruction and editing as connected tasks. Its face model is divided into semantic parts, each with intuitive controls such as nose height, so users can adjust local shape after reconstructing a face from one image.

A differentiable blending module adjusts the shape and placement of the parts so the assembled 3D face aligns more closely with the original photograph.

02 — Contributions

Key Contributions

• 01

  Introduces independent semantic face-part submodels with local, interpretable editing controls.

• 02

  Uses differentiable part blending to improve both editing continuity and 3D-to-2D alignment.

• 03

  Connects face reconstruction and post-reconstruction editing in one optimization framework.

03 — Method

Method

Each semantic face region is represented by a controllable submodel. The part parameters can be edited independently, while a differentiable blender assembles the parts and optimizes their shapes and placements against the source image.

04 — Evaluation

Results

The paper reports more intuitive local editing than prior global face models and a 14% improvement in 3D-to-2D alignment IoU.

05 — Reference

Citation

@InProceedings{Yan_2022_NEO3DF,
  author    = {Yan, Peizhi and Gregson, James and Tang, Qiang and Ward, Rabab and Xu, Zhan and Du, Shan},
  title     = {NEO-3DF: Novel Editing-Oriented 3D Face Creation and Reconstruction},
  booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)},
  year      = {2022},
  pages     = {486--502}
}

06 — Thanks

Acknowledgements

The original project acknowledges support from the University of British Columbia Okanagan under grant GR017752.