2025 01 v.55 168-179
注意力机制的秦腔视频去噪算法
基金项目(Foundation):
国家自然科学基金(62271393);
文化和旅游部重点实验室项目(1222000812,cr2021K01);
西安市社会发展科技创新示范项目(2024JH-CXSF-0014)
邮箱(Email):
ghgeng@nwu.edu.cn;
DOI:
10.16152/j.cnki.xdxbzr.2025-01-014
中文作者单位:
西北大学文化遗产数字化国家地方联合工程研究中心;西北大学可视化技术研究所;
摘要(Abstract):
秦腔，作为中国传统戏曲艺术的瑰宝，拥有深厚的历史底蕴。然而，秦腔早期的影像资料常受噪声和失真影响，导致画质不佳，严重妨碍了秦腔数字档案的保存品质。目前应用的视频去噪技术在处理秦腔那色彩丰富、纹理复杂的服饰时，往往没有充分利用视频帧序列的时间连贯性，使得去噪效果并不理想，难以有效保留视频帧的核心特征。基于注意力机制的秦腔视频去噪算法开展研究，针对现有视频去噪算法忽略帧间时序相关性导致效果不佳的问题，提出了一种新的视频去噪算法，该算法利用双门控注意力机制进行时序信息的融合。首先，通过时序融合模块，将视频连续帧的时序信息进行有效整合；其次，利用双门控注意力去噪网络精确识别并消除时序上的噪声；最后，通过多头交互注意力精炼模块进一步细化特征，以消除去噪过程中可能产生的伪影并恢复丢失的细节，从而提升去噪后图像的质量。实验结果表明，与DVDNet、ViDeNN以及FastDVDNet等现有方法相比，该方法可以更好地利用视频的时序信息，达到干净且高效的秦腔视频去噪效果。
关键词(KeyWords):
秦腔;视频去噪;注意力机制;时序融合
| 69 | 0 | 6 |
| 下载次数 | 被引频次 | 阅读次数 |
参考文献
[1] 周明全，耿国华，武仲科.文化遗产数字化保护技术及应用[M].北京：高等教育出版社，2011.
[2] 耿国华，何雪磊，王美丽，等.文化遗产活化关键技术研究进展[J].中国图象图形学报，2022,27(6):1988-2007.GENG G H,HE X L,WANG M L,et al.Research progress on key technologies of cultural heritage activation[J].Journal of Image and Graphics,2022,27(6):1988-2007.
[3] TICO M.Multi-frame image denoising and stabilization[C]//2008 16th European Signal Processing Conference.Lausanne:IEEE,2008:1-4.
[4] LIU X,ZHU Y,LIU L,et al.Feature-suppressed contrast for self-supervised food pre-training[C]//Proceedings of the 31st ACM International Conference on Multimedia.Ottawa:ACM,2023:4359-4367.
[5] MAGGIONI M,HUANG Y,LI C,et al.Efficient multi-stage video denoising with recurrent spatio-temporal fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Virtual:IEEE,2021:3466-3475.
[6] DAVY A,EHRET T,MOREL J M,et al.A non-local CNN for video denoising[C]//2019 IEEE International Conference on Image Processing (ICIP).Taipei:IEEE,2019:2409-2413.
[7] TASSANO M,DELON J,VEIT T.Dvdnet:A fast network for deep video denoising[C]//2019 IEEE International Conference on Image Processing (ICIP).Taipei:IEEE,2019:1805-1809.
[8] XUE T,CHEN B,WU J,et al.Video enhancement with task-oriented flow[J].International Journal of Computer Vision,2019,127(8):1106-1125.
[9] WANG C,ZHOU S K,CHENG Z W.First image then video:A two-stage network for spatiotemporal video denoising[EB/OL].(2020-01-22)[2024-06-20].https://arxiv.org/abs/2001.00346v2.
[10] MEHTA S,KUMAR A,REDA F,et al.Evrnet:Efficient video restoration on edge devices[C]//Proceedings of the 29th ACM International Conference on Multimedia.Virtual:ACM,2021:983-992.
[11] TASSANO M,DELON J,VEIT T.Fastdvdnet:Towards real-time deep video denoising without flow estimation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:1354-1363.
[12] VAKSMAN G,ELAD M,MILANFAR P.Patch craft:Video denoising by deep modeling and patch matching[C]//2021 IEEE/CVF International Conference on Computer Vision.Montreal:IEEE,2021:2157-2166.
[13] MAGGIONI M,HUANG Y,LI C,et al.Efficient multi-stage video denoising with recurrent spatio-temporal fusion[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE,2021:3466-3475.
[14] SONG M,ZHANG Y,AYDIN T O.Tempformer:Temporally consistent transformer for video denoising[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:481-496.
[15] LI D,SHI X,ZHANG Y,et al.A simple baseline for video restoration with grouped spatial-temporal shift[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver:IEEE,2023:9822-9832.
[16] LIANG J,CAO J,FAN Y,et al.Vrt:A video restoration transformer[J].IEEE Transactions on Image Processing,2024,33:2171-2182.
[17] ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE,2021:14821-14831.
[18] COOLEY J W,TUKEY J W.An algorithm for the machine calculation of complex Fourier series[J].Mathematics of Computation,1965,19(90):297-301.
[19] LECUN Y,BOSER B,DENKER J S,et al.Backpropagation applied to handwritten zip code recognition[J].Neural Computation,1989,1(4):541-551.
[20] PONT-TUSET J,PERAZZI F,CAELLES S,et al.The 2017 DAVIS challenge on video object segmentation[EB/OL].(2018-03-01)[2024-06-20].https://arxiv.org/abs/1704.00675v3.
[21] CLAUS M,VAN GEMERT J.Videnn:Deep blind video denoising[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Long Beach:IEEE,2019:1843-1852.
[2] 耿国华，何雪磊，王美丽，等.文化遗产活化关键技术研究进展[J].中国图象图形学报，2022,27(6):1988-2007.GENG G H,HE X L,WANG M L,et al.Research progress on key technologies of cultural heritage activation[J].Journal of Image and Graphics,2022,27(6):1988-2007.
[3] TICO M.Multi-frame image denoising and stabilization[C]//2008 16th European Signal Processing Conference.Lausanne:IEEE,2008:1-4.
[4] LIU X,ZHU Y,LIU L,et al.Feature-suppressed contrast for self-supervised food pre-training[C]//Proceedings of the 31st ACM International Conference on Multimedia.Ottawa:ACM,2023:4359-4367.
[5] MAGGIONI M,HUANG Y,LI C,et al.Efficient multi-stage video denoising with recurrent spatio-temporal fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Virtual:IEEE,2021:3466-3475.
[6] DAVY A,EHRET T,MOREL J M,et al.A non-local CNN for video denoising[C]//2019 IEEE International Conference on Image Processing (ICIP).Taipei:IEEE,2019:2409-2413.
[7] TASSANO M,DELON J,VEIT T.Dvdnet:A fast network for deep video denoising[C]//2019 IEEE International Conference on Image Processing (ICIP).Taipei:IEEE,2019:1805-1809.
[8] XUE T,CHEN B,WU J,et al.Video enhancement with task-oriented flow[J].International Journal of Computer Vision,2019,127(8):1106-1125.
[9] WANG C,ZHOU S K,CHENG Z W.First image then video:A two-stage network for spatiotemporal video denoising[EB/OL].(2020-01-22)[2024-06-20].https://arxiv.org/abs/2001.00346v2.
[10] MEHTA S,KUMAR A,REDA F,et al.Evrnet:Efficient video restoration on edge devices[C]//Proceedings of the 29th ACM International Conference on Multimedia.Virtual:ACM,2021:983-992.
[11] TASSANO M,DELON J,VEIT T.Fastdvdnet:Towards real-time deep video denoising without flow estimation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:1354-1363.
[12] VAKSMAN G,ELAD M,MILANFAR P.Patch craft:Video denoising by deep modeling and patch matching[C]//2021 IEEE/CVF International Conference on Computer Vision.Montreal:IEEE,2021:2157-2166.
[13] MAGGIONI M,HUANG Y,LI C,et al.Efficient multi-stage video denoising with recurrent spatio-temporal fusion[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE,2021:3466-3475.
[14] SONG M,ZHANG Y,AYDIN T O.Tempformer:Temporally consistent transformer for video denoising[C]//European Conference on Computer Vision.Cham:Springer Nature Switzerland,2022:481-496.
[15] LI D,SHI X,ZHANG Y,et al.A simple baseline for video restoration with grouped spatial-temporal shift[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver:IEEE,2023:9822-9832.
[16] LIANG J,CAO J,FAN Y,et al.Vrt:A video restoration transformer[J].IEEE Transactions on Image Processing,2024,33:2171-2182.
[17] ZAMIR S W,ARORA A,KHAN S,et al.Multi-stage progressive image restoration[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE,2021:14821-14831.
[18] COOLEY J W,TUKEY J W.An algorithm for the machine calculation of complex Fourier series[J].Mathematics of Computation,1965,19(90):297-301.
[19] LECUN Y,BOSER B,DENKER J S,et al.Backpropagation applied to handwritten zip code recognition[J].Neural Computation,1989,1(4):541-551.
[20] PONT-TUSET J,PERAZZI F,CAELLES S,et al.The 2017 DAVIS challenge on video object segmentation[EB/OL].(2018-03-01)[2024-06-20].https://arxiv.org/abs/1704.00675v3.
[21] CLAUS M,VAN GEMERT J.Videnn:Deep blind video denoising[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Long Beach:IEEE,2019:1843-1852.
基本信息:
DOI：10.16152/j.cnki.xdxbzr.2025-01-014
中图分类号:J825;TP391.41
引用信息:
[1]师秦高雪,杨超然,刘鑫达等.注意力机制的秦腔视频去噪算法[J].西北大学学报(自然科学版),2025,55(01):168-179.DOI:10.16152/j.cnki.xdxbzr.2025-01-014.
基金信息:
国家自然科学基金(62271393); 文化和旅游部重点实验室项目(1222000812,cr2021K01); 西安市社会发展科技创新示范项目(2024JH-CXSF-0014)