Integrating Object Detection and Vision-Language Model for Mobile Scaffolding Safety Compliance Monitoring
- 주제(키워드) Mobile scaffolding , Object detection , Vision-language model , Safety compliance , KOSHA
- 주제(DDC) 690
- 발행기관 아주대학교 일반대학원
- 지도교수 문성곤
- 발행년도 2026
- 학위수여년월 2026. 8
- 학위명 석사
- 학과 및 전공 일반대학원 건설시스템공학과
- 실제URI http://www.dcollection.net/handler/ajou/000000036287
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
Mobile scaffolding is commonly used on construction sites, but it poses several safety risks, particularly falls from heights due to improper use or unsafe practices. Although safety regulations, such as those defined by organizations like KOSHA and OSHA, exist, their enforcement on construction sites remains limited. Existing approaches have utilized computer vision technologies to identify hazards; however, these approaches are primarily limited to detecting safety-related objects and lack the ability to evaluate whether the overall setup complies with safety regulations. To overcome these challenges, this study proposes a framework that combines object detection with Vision-Language Model （VLM）-based semantic reasoning to automate the safety compliance evaluation of mobile scaffolding systems. An object detection model is utilized to detect safety-critical objects, including scaffold, guardrail, outrigger, vertical ladder, and workers with and without helmets. The results indicate that the model delivers strong performance in object detection, with a mAP@0.5 of 0.949, a precision of 0.963, and a recall of 0.930. The detection outputs are converted into a structured format that includes class labels, confidence scores, and bounding box coordinates. These structured representations are then integrated with a Vision- Language Model, which evaluates safety compliance using a carefully designed prompt based on KOSHA safety rules. In addition to determining whether a scene is compliant or non-compliant, the model generates clear and interpretable explanations to justify its decisions. The system is tested on a custom dataset of construction site images, achieving a reliable compliance reasoning accuracy （CRA） F1-score of 87.32%. The results demonstrate that combining object detection with semantic reasoning can enhance safety compliance. Overall, this research presents an intelligent framework for construction safety monitoring that not only identifies potential hazards but also explains them, thereby supporting more effective and informed safety management on construction sites. Keywords: Mobile scaffolding, Object detection, Vision-language model, Safety compliance, KOSHA.
more목차
CHAPTER 1: BACKGROUND AND INTRODUCTION 1
CHAPTER 2: LITERATURE REVIEW 5
2.1 SAFETY MONITORING ON CONSTRUCTION SITES 5
2.2 REGULATORY FRAMEWORK: KOSHA REQUIREMENTS 7
2.3 STATE-OF-THE-ART SCAFFOLD MONITORING STUDIES 9
2.4 COMPUTER VISION-BASED TECHNOLOGIES IN CONSTRUCTION SAFETY. 11
2.4.1 Object Detection Models 12
2.4.2 Vision Language Models 13
2.5 RESEARCH SCOPE AND OBJECTIVES 15
CHAPTER 3: RESEARCH METHOD 19
3.1. FRAMEWORK DESIGN BASED ON KOSHA REGULATIONS 21
3.2 VISUAL PERCEPTION: OBJECT DETECTION 23
3.2.1 Model Architecture and Design 24
3.2.2 Detection Objective and Loss Function 26
3.2.3 Scene Information Construction 26
3.3 VLM-BASED SEMANTIC REASONING 28
3.3.1 Input Representation 29
3.3.2 Compliance Prompt Structure 30
3.3.3 Compliance Evaluation and Output Generation 31
3.4 OVERVIEW OF DATASET 32
3.4.1 Data preprocessing. 32
3.4.2 Data labeling 35
3.5. TRAINING SETUP 36
3.5.1 YOLOv8n Training Configuration 36
3.5.2 Qwen3-VL8B Configuration 37
CHAPTER 4: EXPERIMENTS AND EVALUATION 39
4.1 EVALUATION METRICS. 39
4.1.1 Object detection evaluation metrics 39
4.1.2 VLM evaluation metrics 40
4.2 RESEARCH RESULTS 41
4.2.1 Object detection results 41
4.2.2 Semantic Reasoning Performance (CRA) 49
4.3 RESEARCH FINDINGS AND REFLECTION 53
CHAPTER 5: DISCUSSION AND CONCLUSION 55
5.1 DISCUSSION 55
5.1.1 Limitations of the Study 58
5.2 CONCLUSION 58
5.3 FUTURE WORK 60
REFERENCES 62
국문 요약 68
