基于广义视觉的结构损伤巡检智能体导航方法

徐阳; 胡澍东; 杨广硕; 鲍跃全; 李惠

doi:10.3724/j.gyjzG25070801

基于广义视觉的结构损伤巡检智能体导航方法

doi: 10.3724/j.gyjzG25070801

徐阳^1,2,
胡澍东²,
杨广硕²,
鲍跃全^1,2,
李惠^1,2

1. 土木工程智能防灾减灾工信部重点实验室, 哈尔滨 150090;
2. 哈尔滨工业大学土木工程学院, 哈尔滨 150090

基金项目:

国家自然科学基金重大项目课题（52192661）。

详细信息

作者简介:
徐阳，博士，副教授，主要从事结构健康监测计算机视觉研究。电子信箱：xyce@hit.edu.cn

计量
- 文章访问数: 129
- HTML全文浏览量: 27
- PDF下载量: 10
- 被引次数: 0
出版历程
- 收稿日期: 2025-07-08
- 网络出版日期: 2025-09-12

A Generalized Vision-Based Intelligent Agent Navigation for Structural Damage Inspection

1. Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology, Harbin 150090, China;
2. School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China

摘要

摘要: 传统结构巡检依赖于人工目视检查，高度依赖经验，缺乏安全性保障，时间人力成本昂贵。近年来，机器人、计算机视觉、深度学习等新技术为结构智能巡检提供了崭新手段。为此提出了基于广义视觉的结构巡检智能体导航方法，建立了融合建筑结构损伤信息的视觉导航智能体交互试验环境，设计了光学图像和深度信息多模态融合感知网络，构建了基于多辅助任务的深度强化学习自主导航决策模型。具体地，采用轻量化MiDaS网络实现单目图像的实时深度估计，通过特征融合机制将光学图像信息与深度图进行多模态融合，继而传递给视觉导航模型；基于A3C算法提出了结构损伤驱动的深度强化学习导航架构，嵌入了LSTM长期记忆模块和价值网络解耦模块，设计了碰撞预测和奖励预测辅助任务，实现了基于多模态信息的视觉导航高效增强。跨场景试验验证结果表明：该研究提出的视觉导航模型在多种结构损伤环境下均展现出良好的有效性和泛化能力，实现了由场景感知图像到无地图导航策略的精准映射，解决了传统导航方法缺乏长期记忆功能、新环境泛化能力不足等难题。
- 结构损伤巡检 /
- 智能体导航 /
- 计算机视觉 /
- 深度强化学习 /
- 多模态信息融合 /
- 辅助任务增强
Abstract: Conventional structural inspection highly relies on human inspector with engineering experience， lacks safety guarantee， and is time-labor-consuming. Recently， advanced robotics， computer vision， and deep learning techniques have provided innovative solutions. This study proposed a generalized vision-based intelligent agent navigation for structural damage inspection， and established an interactive experimental environment for visual navigation agent integrating structural damage. A multi-modal information fusion network of optical image and depth information was designed， and a deep reinforcement learning network was developed based on the A3C algorithm with auxiliary tasks of collision prediction and reward prediction， in which a long-term memory module was embedded before the output layers of policy network and value network， and a decoupling module of value network was introduced based on universal successor representation. The model performance， effectiveness， and generalization ability were validated under various structural damage scenarios， and the results showed that it could achieve accurate mapping from observed images to navigation strategies and address conventional navigation limitations of lacking long-term memory and poor generalization ability in open environments.
- structural damage inspection /
- intelligent agent navigation /
- computer vision /
- deep reinforcement learning /
- multi-modal information fusion /
- auxiliary task enhancement

HTML全文

参考文献(23)

[1]	鲍跃全，李惠. 人工智能时代的土木工程［J］. 土木工程学报，2019，52（5）:1-11.
[2]	SPENCER Jr B F，HOSKERE V，NARAZAKI Y. Advances in computer vision-based civil infrastructure inspection and monitoring［J］. Engineering，2019，5（2）:199-222.
[3]	SUN L M，SHANG Z，XIA Y，et al. Review of bridge structural health monitoring aided by big data and artificial intelligence:from condition assessment to damage detection［J］. Journal of Structural Engineering，2020，146（5），04020073.
[4]	BAO Y Q，LI H. Machine learning paradigm for structural health monitoring［J］. Structural Health Monitoring，2021，20（4）:1353-1372.
[5]	徐阳，金晓威，李惠. 土木工程智能科学与技术研究现状及展望［J］. 建筑结构学报，2022，43（9）:23-35.
[6]	XU Y，QIAN W L，LI N，et al. Typical advances of artificial intelligence in civil engineering［J］. Advances in Structural Engineering，2022，25（16）:3405-3424.
[7]	XU Y，WEI S Y，BAO Y Q，et al. Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network［J］. Structural Control and Health Monitoring，2019，26（3），e2313.
[8]	丁威，俞珂，舒江鹏. 基于深度学习和无人机的混凝土结构裂缝检测方法［J］. 土木工程学报，2021，54（增刊1）:1-12.
[9]	岳清瑞，徐刚，刘晓刚. 桥梁裂缝智能识别与监测方法研究［J］. 中国公路学报，2024，37（2）:16-28.
[10]	PRASANNA P，DANA K J，GUCUNSKI N，et al. Automated crack detection on concrete bridges［J］. IEEE Transactions on Automation Science and Engineering，2014，13（2）:591-599.
[11]	LI Y，LI H，WANG H. Pixel-wise crack detection using deep local pattern predictor for robot application［J］. Sensors，2018，18（9），3042.
[12]	KOUZEHGAR M，TAMILSELVAM Y K，HEREDIA M V，et al. Self-reconfigurable façade-cleaning robot equipped with deep-learning-based crack detection based on convolutional neural networks［J］. Automation in Construction，2019，108，102959.
[13]	JIANG S，ZHANG J. Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system［J］. Computer-Aided Civil and Infrastructure Engineering，2020，35（6）:549-564.
[14]	JANG K，AN Y K，KIM B，et al. Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot［J］. Computer-Aided Civil and Infrastructure Engineering，2021，36（1）:14-29.
[15]	XU Y，FAN Y L，BAO Y Q，et al. Task-aware meta-learning paradigm for universal structural damage segmentation using limited images［J］. Engineering Structures，2023，284，115917.
[16]	POTENZA F，RINALDI C，OTTAVIANO E，et al. A robotics and computer-aided procedure for defect evaluation in bridge inspection［J］. Journal of Civil Structural Health Monitoring，2020，10（3）:471-484.
[17]	RAMALINGAM B，HAYAT A A，ELARA M R，et al. Deep learning-based pavement inspection using self-reconfigurable robot［J］. Sensors，2021，21（8），2595.
[18]	YUAN C，XIONG B，LI X，et al. A novel intelligent inspection robot with deep stereo vision for three-dimensional concrete damage detection and quantification［J］. Structural Health Monitoring，2022，21（3）:788-802.
[19]	MENG S，GAO Z，ZHOU Y，et al. Real-time automatic crack detection method based on drone［J］. Computer-Aided Civil and Infrastructure Engineering，2023，38（7）:849-872.
[20]	XU Y，FAN Y L，LI H. Lightweight semantic segmentation of complex structural damage recognition for actual bridges［J］. Structural Health Monitoring，2023，22（5）:3250-3269.
[21]	HOSKERE V，NARAZAKI Y，SPENCER Jr B F. Physics-based graphics models in 3D synthetic environments as autonomous vision-based inspection testbeds［J］. Sensors，2022，22（2），532.
[22]	XU Y，QIAO W D，ZHAO J，et al. Vision-based multi-level synthetical evaluation of seismic damage for RC structural components:a multi-task learning approach［J］. Earthquake Engineering and Engineering Vibration，2023，22（1）:69-85.
[23]	胡澍东. 扫描式结构损伤检测机器人深度强化学习视觉导航方法［D］. 哈尔滨:哈尔滨工业大学，2023.