2025 01 v.55 213-222
融合扩散模型技术的文物面部三维模型孔洞修补
基金项目(Foundation):
国家自然科学基金(62271393);
文化和旅游部重点实验室项目(1222000812,cr2021K01);
西安市社会发展科技创新示范项目(2024JH-CXSF-0014);
虚拟现实技术与系统全国重点实验室(北京航空航天大学)开放基金(VRLAB2024C02)
邮箱(Email):
mqzhou@bnu.edu.cn;
DOI:
10.16152/j.cnki.xdxbzr.2025-01-018
中文作者单位:
西北大学文化遗产数字化国家地方联合工程研究中心;西北大学可视化技术研究所;
摘要(Abstract):
在三维几何建模领域，孔洞指三维模型表面的不完整性，通常产生在数据采集、处理或转换过程中，这些孔洞的存在可能会对模型的几何完整性和视觉质量造成显著影响。在文化遗产保护领域，传统的孔洞修复技术侧重于几何表面的修复，而忽略了纹理信息的修复，纹理信息对于恢复文物的真实感和材质特性至关重要。文中提出了一种结合几何和纹理修复的扩散模型方法，特别适用于文物面部模型的修复。由于缺乏文物面部模型的基准真值，使用相似的残缺黄种人面部模型进行评估，鉴于公开可用的黄种人面部数据集较为稀缺，构建了一个黄种人面部数据集，包含约20 000张黄种人面部图像、对应的三维模型及其纹理，以及渲染得到的高保真面部图像。首先，在构建的数据集上进行测试，以验证所提方法的有效性。结果显示，与现有基准方法相比，该方法在面部图像修复方面取得了显著改进，并且实现了对面部三维模型几何表面和纹理信息的同步修复。其次，将所提出的修复技术应用于文化遗产中的文物面部模型，通过实验验证该方法在文物面部模型的几何和纹理修复方面均展现出了优异的性能。研究结果为文物面部模型的数字化修复提供了一种有效的技术手段，有助于提高文物保护和数字化展示的质量。
关键词(KeyWords):
三维模型孔洞修补;文物面部修复;扩散模型;图像修复
| 132 | 0 | 15 |
| 下载次数 | 被引频次 | 阅读次数 |
参考文献
[1] LIEPA P.Filling holes in meshes[C]//In Proceedings of the 2003 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing.Aachen:ACM,2003:200-205.
[2] 温佩芝，雷永庆，孙梦龙.三维重建网格模型的缺陷孔洞识别与修复方法[J].计算机应用研究，2020,37(4):1234-1238.WEN P Z,LEI Y Q,SUN M L.Defect and hole identification and repair methods for 3D reconstructed mesh models[J].Application Research of Computers,2020,37(4):1234-1238.
[3] 谢倩茹，耿国华.三维模型孔洞修补方法的研究[J].计算机应用研究，2013,30(10):3175-3177.XIE Q R,GENG G H.Research on the method of repairing holes in 3D models[J].Application Research of Computers,2013,30(10):3175-3177.
[4] DAVIS J,MARSCHNER S R,GARR M,et al.Filling holes in complex surfaces using volumetric diffusion[C]//In Proceedings of First International Symposium on 3D Data Processing Visualization and Transmission.Padua:IEEE,2002:428-441.
[5] BISCHOFF S,KOBBELT L.Structure preserving CAD model repair[J].Computer Graphics Forum,2005,24(3):527-536.
[6] WANG X,LIU X,LU L,et al.Automatic hole-filling of CAD models with feature-preserving[J].Computers & Graphics,2012,36(2):101-110.
[7] 刘震，王艳宾，白丽丽，等.曲面细节特征保持的三维模型孔洞修复方法[J].计算机辅助设计与图形学学报，2016,28(12):2052-2059.LIU Z,WANG Y B,BAI L L,et al.3D model hole repair method for preserving surface detail features[J].Journal of Computer Aided Design and Graphics,2016,28(12):2052-2059.
[8] 王鹏飞，徐敏峰，辛士庆，等.鲁棒的水密流形网格修复[J].计算机辅助设计与图形学学报，2024,36(7):1047-1056.WANG P F,XU M F,XIN S Q,et al.Robust watertight manifold mesh repair[J].Journal of Computer-Aided Design and Computer Graphics,2024,36(7):1047-1056.
[9] HO J,JAIN A,ABBEEL P.Denoising diffusion probabilistic models[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[10] DHARIWAL P,NICHOL A.Diffusion models beat GANs on image synthesis[J].Advances in Neural Information Processing Systems,2021,34:8780-8794.
[11] RAMESH A,DHARIWAL P,NICHOL A,et al.Hierarchical text-conditional image generation with CLIP latents[EB/OL].(2022-04-13)[2024-07-02].https://arxiv.org/abs/2204.06125v1.
[12] 彭进业，余喆，屈书毅，等.基于深度学习的图像修复方法研究综述[J].西北大学学报(自然科学版),2023,53(6):943-963.PENG J Y,YU Z,QU S Y,et al.A survey of image inpainting methods based on deep learning[J].Journal of Northwest University (Natural Science Edition),2023,53(6):943-963.
[13] LUGMAYR A,DANELLJAN M,ROMERO A,et al.RePaint:Inpainting using denoising diffusion probabilistic models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).New Orleans:IEEE,2022:11451-11461.
[14] ROUT L,PARULEKAR A,CARAMANIS C,et al.A theoretical justification for image inpainting using denoising diffusion probabilistic models[EB/OL].(2023-02-02)[2024-07-02].https://arxiv.org/abs/2302.01217v1.
[15] ZHANG G H,JI J B,ZHANG Y,et al.Towards coherent image inpainting using denoising diffusion implicit models[C]//Proceedings of the 40th International Conference on Machine Learning.Honolulu:ACM,2023:41164-41193.
[16] YUE Z S,WANG J Y,LOY C C.Efficient diffusion model for image restoration by residual shifting[EB/OL].(2024-03-12)[2024-07-02].https://arxiv.org/abs/2403.07319v3.
[17] CORNEANU C,GADDE R,MARTINEZ A M.LatentPaint:Image inpainting in latent space with diffusion models[C]//2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).Waikoloa:IEEE,2024:4322-4331.
[18] DURRER A,WOLLEB J,BIEDER F,et al.Denoising diffusion models for 3D healthy brain tissue inpainting[C]//MICCAI Workshop on Deep Generative Models.Cham:Springer Nature Switzerland,2024:87-97.
[19] SONG J M,MENG C L,ERMON S.Denoising diffusion implicit models[EB/OL].(2020-10-06)[2024-07-02].https://arxiv.org/abs/2010.02502v4.
[20] DEJA K,KUZINA A,TRZCINSKI T,et al.On analyzing generative and denoising capabilities of diffusion-based deep generative models[EB/OL].(2022-05-31)[2024-07-02].https://arxiv.org/abs/2206.00070v1.
[21] ZHANG K,LI Y W,ZUO W M,et al.Plug-and-play image restoration with deep denoiser prior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(10):6360-6376.
[22] ZHU Y Z,ZHANG K,LIANG J Y,et al.Denoising diffusion models for plug-and-play image restoration[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Vancouver:IEEE,2023:1219-1229.
[23] ALEXANDER O,ROGERS M,LAMBETH W,et al.The digital emily project:Photoreal facial modeling and animation[C]//ACM SIGGRAPH 2009 Courses.New Orleans:ACM,2009:1-15.
[24] CAO X,CHEN Z,CHEN A P,et al.Sparse photometric 3D face reconstruction guided by morphable models[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:4635-4644.
[25] MA W C,HAWKINS T,PEERS P,et al.Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination[C]//Proceedings of the 18th Eurographics Conference on Rendering Techniques.Grenoble:ACM,2007:183-194.
[26] DIB A,THéBAULT C,AHN J,et al.Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV).Montreal:IEEE,2021:12799-12809.
[27] GECER B,PLOUMPIS S,KOTSIA I,et al.GANFIT:Generative adversarial network fitting for high fidelity 3D face reconstruction[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach:IEEE,2019:1155-1164.
[28] GECER B,PLOUMPIS S,KOTSIA I,et al.Fast-GANFIT:Generative adversarial network for high fidelity 3D face reconstruction[EB/OL].(2021-05-16)[2024-07-02].https://arxiv.org/abs/2105.07474v1.
[29] LATTAS A,MOSCHOGLOU S,GECER B,et al.AvatarMe:Realistically renderable 3D facial reconstruction "In-the-wild"[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle:IEEE,2020:760-769.
[30] LEI B W,REN J Q,FENG M Y,et al.A hierarchical representation network for accurate and detailed face reconstruction from in-the-wild images[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Vancouver:IEEE,2023:394-403.
[31] KARRAS T,LAINE S,AILA T M.A style-based generator architecture for generative adversarial networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach:IEEE,2019:4401-4410.
[2] 温佩芝，雷永庆，孙梦龙.三维重建网格模型的缺陷孔洞识别与修复方法[J].计算机应用研究，2020,37(4):1234-1238.WEN P Z,LEI Y Q,SUN M L.Defect and hole identification and repair methods for 3D reconstructed mesh models[J].Application Research of Computers,2020,37(4):1234-1238.
[3] 谢倩茹，耿国华.三维模型孔洞修补方法的研究[J].计算机应用研究，2013,30(10):3175-3177.XIE Q R,GENG G H.Research on the method of repairing holes in 3D models[J].Application Research of Computers,2013,30(10):3175-3177.
[4] DAVIS J,MARSCHNER S R,GARR M,et al.Filling holes in complex surfaces using volumetric diffusion[C]//In Proceedings of First International Symposium on 3D Data Processing Visualization and Transmission.Padua:IEEE,2002:428-441.
[5] BISCHOFF S,KOBBELT L.Structure preserving CAD model repair[J].Computer Graphics Forum,2005,24(3):527-536.
[6] WANG X,LIU X,LU L,et al.Automatic hole-filling of CAD models with feature-preserving[J].Computers & Graphics,2012,36(2):101-110.
[7] 刘震，王艳宾，白丽丽，等.曲面细节特征保持的三维模型孔洞修复方法[J].计算机辅助设计与图形学学报，2016,28(12):2052-2059.LIU Z,WANG Y B,BAI L L,et al.3D model hole repair method for preserving surface detail features[J].Journal of Computer Aided Design and Graphics,2016,28(12):2052-2059.
[8] 王鹏飞，徐敏峰，辛士庆，等.鲁棒的水密流形网格修复[J].计算机辅助设计与图形学学报，2024,36(7):1047-1056.WANG P F,XU M F,XIN S Q,et al.Robust watertight manifold mesh repair[J].Journal of Computer-Aided Design and Computer Graphics,2024,36(7):1047-1056.
[9] HO J,JAIN A,ABBEEL P.Denoising diffusion probabilistic models[J].Advances in Neural Information Processing Systems,2020,33:6840-6851.
[10] DHARIWAL P,NICHOL A.Diffusion models beat GANs on image synthesis[J].Advances in Neural Information Processing Systems,2021,34:8780-8794.
[11] RAMESH A,DHARIWAL P,NICHOL A,et al.Hierarchical text-conditional image generation with CLIP latents[EB/OL].(2022-04-13)[2024-07-02].https://arxiv.org/abs/2204.06125v1.
[12] 彭进业，余喆，屈书毅，等.基于深度学习的图像修复方法研究综述[J].西北大学学报(自然科学版),2023,53(6):943-963.PENG J Y,YU Z,QU S Y,et al.A survey of image inpainting methods based on deep learning[J].Journal of Northwest University (Natural Science Edition),2023,53(6):943-963.
[13] LUGMAYR A,DANELLJAN M,ROMERO A,et al.RePaint:Inpainting using denoising diffusion probabilistic models[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).New Orleans:IEEE,2022:11451-11461.
[14] ROUT L,PARULEKAR A,CARAMANIS C,et al.A theoretical justification for image inpainting using denoising diffusion probabilistic models[EB/OL].(2023-02-02)[2024-07-02].https://arxiv.org/abs/2302.01217v1.
[15] ZHANG G H,JI J B,ZHANG Y,et al.Towards coherent image inpainting using denoising diffusion implicit models[C]//Proceedings of the 40th International Conference on Machine Learning.Honolulu:ACM,2023:41164-41193.
[16] YUE Z S,WANG J Y,LOY C C.Efficient diffusion model for image restoration by residual shifting[EB/OL].(2024-03-12)[2024-07-02].https://arxiv.org/abs/2403.07319v3.
[17] CORNEANU C,GADDE R,MARTINEZ A M.LatentPaint:Image inpainting in latent space with diffusion models[C]//2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).Waikoloa:IEEE,2024:4322-4331.
[18] DURRER A,WOLLEB J,BIEDER F,et al.Denoising diffusion models for 3D healthy brain tissue inpainting[C]//MICCAI Workshop on Deep Generative Models.Cham:Springer Nature Switzerland,2024:87-97.
[19] SONG J M,MENG C L,ERMON S.Denoising diffusion implicit models[EB/OL].(2020-10-06)[2024-07-02].https://arxiv.org/abs/2010.02502v4.
[20] DEJA K,KUZINA A,TRZCINSKI T,et al.On analyzing generative and denoising capabilities of diffusion-based deep generative models[EB/OL].(2022-05-31)[2024-07-02].https://arxiv.org/abs/2206.00070v1.
[21] ZHANG K,LI Y W,ZUO W M,et al.Plug-and-play image restoration with deep denoiser prior[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(10):6360-6376.
[22] ZHU Y Z,ZHANG K,LIANG J Y,et al.Denoising diffusion models for plug-and-play image restoration[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Vancouver:IEEE,2023:1219-1229.
[23] ALEXANDER O,ROGERS M,LAMBETH W,et al.The digital emily project:Photoreal facial modeling and animation[C]//ACM SIGGRAPH 2009 Courses.New Orleans:ACM,2009:1-15.
[24] CAO X,CHEN Z,CHEN A P,et al.Sparse photometric 3D face reconstruction guided by morphable models[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE,2018:4635-4644.
[25] MA W C,HAWKINS T,PEERS P,et al.Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination[C]//Proceedings of the 18th Eurographics Conference on Rendering Techniques.Grenoble:ACM,2007:183-194.
[26] DIB A,THéBAULT C,AHN J,et al.Towards high fidelity monocular face reconstruction with rich reflectance using self-supervised learning and ray tracing[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV).Montreal:IEEE,2021:12799-12809.
[27] GECER B,PLOUMPIS S,KOTSIA I,et al.GANFIT:Generative adversarial network fitting for high fidelity 3D face reconstruction[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach:IEEE,2019:1155-1164.
[28] GECER B,PLOUMPIS S,KOTSIA I,et al.Fast-GANFIT:Generative adversarial network for high fidelity 3D face reconstruction[EB/OL].(2021-05-16)[2024-07-02].https://arxiv.org/abs/2105.07474v1.
[29] LATTAS A,MOSCHOGLOU S,GECER B,et al.AvatarMe:Realistically renderable 3D facial reconstruction "In-the-wild"[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle:IEEE,2020:760-769.
[30] LEI B W,REN J Q,FENG M Y,et al.A hierarchical representation network for accurate and detailed face reconstruction from in-the-wild images[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Vancouver:IEEE,2023:394-403.
[31] KARRAS T,LAINE S,AILA T M.A style-based generator architecture for generative adversarial networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach:IEEE,2019:4401-4410.
基本信息:
DOI：10.16152/j.cnki.xdxbzr.2025-01-018
中图分类号:K854;TP391.41
引用信息:
[1]王峥嵘,刘鑫达,周明全.融合扩散模型技术的文物面部三维模型孔洞修补[J].西北大学学报(自然科学版),2025,55(01):213-222.DOI:10.16152/j.cnki.xdxbzr.2025-01-018.
基金信息:
国家自然科学基金(62271393); 文化和旅游部重点实验室项目(1222000812,cr2021K01); 西安市社会发展科技创新示范项目(2024JH-CXSF-0014); 虚拟现实技术与系统全国重点实验室(北京航空航天大学)开放基金(VRLAB2024C02)