A Conceptual Framework for a Lightweight System Integrated into Vehicles for Real-Time Road Surface Monitoring Using Vehicle-Mounted Vision Systems and Communication ()
1. Introduction
1.1. Overview of Road Surface Defects
Road surface defects such as potholes, cracks, and surface wear significantly contribute to traffic accidents and vehicle damage worldwide. These phenomena result from thermally induced volumetric changes in the pavement structure, triggered by the infiltration of rainwater into subsurface layers [1]. According to the World Health Organization (WHO), road traffic injuries caused an estimated 1.35 million deaths globally every year [2]. In Asia, Taiwan region and Chinese mainland also face considerable challenges, with rates of 12.1 and 17.4 per 100,000. Countries like Thailand and Vietnam report some of the highest traffic-related death rates, with Thailand at 30.3 and Vietnam at 20.3 deaths per 100,000 people [3], respectively. Countries with lower development status often face inadequate road infrastructure, which contributes to a higher likelihood of traffic accidents [4] [5]. Although not all accidents are directly caused by road defects, vehicle collisions can exacerbate existing damage, further deteriorating road conditions and compounding safety risks [6]. These statistics highlight the urgent need for effective road maintenance and monitoring systems to mitigate accident risks associated with poor road conditions.
1.2. Current Methods for Road Defect Detection
Traditionally, road surface monitoring has relied on manual visual inspections or citizen-reported issues through mobile apps or hotlines [7]. While these methods are cost-effective, they are often inconsistent, subjective, and labor-intensive. For example, in the United States, many state Departments of Transportation still rely on windshield surveys or on-foot inspections, which are prone to human error and limited in coverage [8]. Some cities have adopted specialized vehicles equipped with laser scanners, ground-penetrating radar, and high-definition cameras, but these systems are expensive and not scalable for widespread deployment [9]-[11]. Recent research has shown that computer vision algorithms can achieve comparable accuracy at a fraction of the cost [8].
1.3. Sensors Used in Vehicle-Based Environmental Sensing
Modern vehicles, especially those designed for autonomous driving or advanced driver-assistance systems (ADAS), are equipped with a variety of sensors to perceive their surroundings. These include:
LiDAR (Light Detection and Ranging): Provides high-resolution 3D mapping of the environment [12].
Ultrasonic Sensors: Useful for short-range detection, such as parking assistance, close objects detection [13].
Radar: Effective for detecting objects at longer distances and in poor visibility [14].
Light Sensors and IMUs: Help in adjusting to lighting conditions and tracking vehicle motion.
Cameras: Capture visual data for object recognition, lane detection, and surface analysis. Cameras are also commonly used for dashcam recording and surveillance purposes [15].
Sensor fusion-combining data from multiple sensors-enhances the reliability and accuracy of environmental perception. Among the above system. Camera or dashcam recordings and images can provide the majority of the data needed to identify road surface defects.
1.4. AI Algorithms for Road Defect Detection
Artificial intelligence, particularly deep learning, has revolutionized road defect detection. Object detection models such as YOLO (You Only Look Once), Faster R-CNN, and RetinaNet are widely used for identifying cracks, potholes, and other surface anomalies [16] [17]. In a recent study, an improved YOLOv8 model achieved a 87.2% in detection accuracy and a 23% reduction in computational complexity compared to the baseline model [18] [19]. These models are capable of real-time inference and can be deployed on edge devices like Raspberry Pi or smartphones, making them suitable for lightweight, vehicle-mounted systems [20].
This study introduces a novel framework that combines three key innovations: (1) a lightweight, edge-deployable system using consumer-grade hardware, Raspberry Pi and dashcams; (2) a collaborative sensing and alert mechanism that enhances road safety through GPS-based data sharing; and (3) a multi-modal detection approach that integrates both image-based recognition and vehicle dynamics for improved accuracy. These features collectively position the system as a scalable, cost-effective solution for real-time road surface monitoring and alert system. The remainder of this paper is structured as follows: Section 2 details the system framework, including hardware configuration, AI model selection, dataset utilization, and integration strategy. Section 3 discusses the system’s development, preliminary results, and future directions. Finally, Section 4 concludes with a summary of findings and the broader implications of the proposed approach.
1.5. Cloud-Based Networks
Collaborative, cloud-based networks for sharing road hazard information have been extensively studied within Intelligent Transportation Systems (ITS) and Vehicular Ad-hoc Networks (VANETs), leveraging V2X communication protocols such as DSRC and C-V2X for real-time safety applications [21]-[23]. These frameworks typically require specialized infrastructure and standardized communication layers, which can limit scalability and increase deployment costs. Our goal is to build upon these mature concepts and apply them in a practical, cost-effective manner by using consumer-grade hardware and a lightweight cloud architecture. Unlike traditional systems, our approach integrates reinforcement learning for continuous improvement and supports decentralized data sharing, enabling real-time alerts without reliance on expensive proprietary solutions. This design ensures adaptability and scalability for widespread adoption across diverse environments.
2. System Framework
2.1. Hardware Configuration
The proposed system utilizes a Raspberry Pi 4 paired with a built-in camera or dashcam, as shown in Figure 1, which supports AI processing in the sensor. This configuration enables real-time image capture and inference directly on the edge device, reducing latency and improving. Raspberry Pi 4 offers sufficient computational power for lightweight YOLO models such as YOLOv8, which are optimized for resource-constrained environments. The camera footage can be extracted from the rear-view camera or dashcam, leveraging existing vehicle infrastructure to minimize installation complexity.
Figure 1. Design paradigm of an intelligent road inspection system architecture.
2.2. AI Model Selection
The system employs the YOLO (You Only Look Once) family of models [24], specifically YOLOv8, due to its balance of speed, accuracy, and efficiency. YOLO is a one-stage object detection algorithm that processes images in a single pass, making it ideal for real-time applications. Recent enhancements to YOLOv8 in the integration of Spatial Pyramid Pooling, Convolutional Block Attention Modules (CBAM), and Selective Kernel Networks, have significantly improved its performance in detecting small and complex road defects. Compared to traditional models, YOLOv8 achieves higher precision and recall while maintaining low computational overhead, making it suitable for deployment on Raspberry Pi [24].
2.3. Dataset Utilization
For training and validation, the system uses publicly available road defect datasets. RDD2022 and N-RDD2024 are International datasets with thousands of annotated images from countries like China, India, and the USA, covering various defect types such as longitudinal cracks, alligator cracks, and potholes [25] [26]. These datasets provide diverse environmental conditions and defect types, enhancing the model’s generalization capabilities [27] [28].
To detect bouncing motion in vehicles, we focus on identifying subtle oscillations and vertical displacements captured in driving videos. Our approach involves extracting short video clips from various driving scenarios to train a motion detection model. We aim to leverage pretrained motion analysis models to enhance the accuracy and efficiency of bouncing motion recognition within our system [29] [30].
2.4. System Integration and Data Sharing
Our goal is to build upon mature concepts in Intelligent Transportation Systems (ITS) and Vehicular Ad-hoc Networks (VANETs) and apply them in a practical, cost-effective manner by using consumer-grade hardware and a lightweight cloud architecture. This approach ensures scalability and affordability while maintaining real-time responsiveness for road safety applications.
The system architecture is illustrated in Figure 2. It is designed around a cloud-based infrastructure that stores both open-source road defect images and those reported by vehicles in real time. Initially, the defect detection model is trained using publicly available labeled datasets. Following training, the model undergoes validation to ensure accuracy before deployment. Our objective is to deploy the model across as many vehicles as possible to build a comprehensive and scalable road defect database.
Figure 2. Schematic diagram of the detection system architecture.
Each vehicle is equipped with a Raspberry Pi unit that performs onboard image recognition. Images are extracted from dashcam footage and analyzed during driving. Upon detecting a defect, the system records the image along with GPS coordinates and uploads the data to the cloud.
To enhance detection reliability, the system also monitors vehicle dynamics-such as suspension bounce and vibration-which may indicate defects not captured visually. This multi-modal sensing approach improves both detection accuracy and robustness.
Once a defect is identified, the system activates visual and audio alerts to notify the driver. In addition to onboard detection, the system continuously updates GPS data and cross-references it with cloud-based information to provide timely warnings, ensuring a proactive and collaborative road safety network.
Additionally, the system supports reinforcement learning by uploading newly captured defect images to the training database, enabling continuous model improvement.
3. Result and Discussion
3.1. Validation Methodology
Initial training was conducted using a combination of labeled datasets from RDD2022 and personal dashcam images [31]. These datasets provided a diverse range of road defect images under various lighting and environmental conditions. The model selected for deployment was YOLOv8, chosen for its balance between speed and accuracy.
We conducted a structured validation process using publicly available datasets and custom dashcam footage. The dataset was divided into 70% for training, 20% for validation, and 10% for testing, ensuring balanced representation of various road defect types.
Refer to Table 1, The training dataset included: A total of 1,588 images were used for testing, covering diverse lighting and environmental conditions. The model’s performance was evaluated using standard object detection metrics, including precision, recall, and mean Average Precision (mAP). The results demonstrated strong detection capability across multiple defect categories, with precision reaching 94%, recall at 89%, and mAP of 92% on the test set. Figure 3 illustrates sample outputs of detected potholes and cracks.
Table 1. Training and testing dataset composition.
Defect Type |
Training Images |
Testing Images |
Longitudinal Crack |
3,969 |
560 |
Transverse Crack |
2,758 |
254 |
Alligator Crack |
3,496 |
454 |
Pothole |
1,114 |
320 |
Total |
11,337 |
1,588 |
Figure 3. Sample of the recognizing the pothole.
3.2. System Novelty and Innovation
The proposed system introduces several innovative features that distinguish it from conventional road surface monitoring solutions. These innovations are designed to enhance scalability, responsiveness, and accuracy while maintaining low deployment costs. The following subsections highlight three core novelties of the framework:
3.2.1. Lightweight, Edge-Based Deployment Using Consumer-Grade
Hardware
The system’s use of Raspberry Pi 4 and standard vehicle cameras (dashcam or rear-view) enables a cost-effective and scalable solution for road defect detection. Unlike traditional systems that rely on expensive, specialized equipment, this approach democratizes access to real-time monitoring by allowing deployment in everyday vehicles.
3.2.2. Collaborative Sensing and Real-Time Alert System
A key innovation is the integration of collaborative sensing through GPS-tagged defect data sharing. When a vehicle detects a defect, the system uploads the image and location to a central server. Other vehicles approaching the same location receive alerts, allowing drivers to take preventive action. This vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication enhances road safety and creates a dynamic, crowd-sourced road condition map.
3.2.3. Multi-Modal Defect Detection with Reinforcement Learning
Beyond visual detection, the system incorporates vehicle dynamics-such as suspension bounce or vibration-as indicators of road anomalies. This multi-modal approach improves detection accuracy, especially for defects that may not be visually prominent. Additionally, the system supports reinforcement learning by continuously uploading new defect images for retraining, allowing the model to adapt to evolving road conditions and improve over time.
3.3. Future Development
The primary novelty of this work lies in the absence of any commercial device that integrates multiple models into a unified system for real-time road defect detection and vehicle safety enhancement. This paper presents a conceptual framework that combines vision-based detection with motion analysis to improve accuracy and reliability. Our goal is to develop an external plug-and-play device that can be easily installed in vehicles without requiring modifications to existing systems, ensuring practical usability and commercial viability.
While the current model is trained on publicly available datasets, future development will focus on collecting proprietary data from various dashcams and rear-view cameras. This will help address variability in image quality and camera angles across different vehicles. Since camera installation positions differ significantly, segmentation results may vary, necessitating a more diverse dataset to improve model generalization. Dashcam videos, which typically offer higher resolution compared to other onboard cameras, will serve as the primary source for real-world footage, strengthening the model’s robustness and accuracy.
In addition to visual detection, we plan to collect a comprehensive dataset of vehicle bouncing motion videos and apply existing motion detection techniques, such as hand-shaking motion analysis, to develop a robust model for detecting suspension bounce. This multi-modal enhancement will complement visual detection and improve performance in scenarios where defects are visually subtle or obscured.
Our data-sharing architecture is designed to be general and commercially deployable, leveraging cloud-based infrastructure for collaborative sensing. Unlike traditional V2X systems that require costly infrastructure, our approach emphasizes decentralization and reinforcement learning for continuous improvement, enabling a dynamic and adaptive road safety network. By enabling real-time, distributed monitoring using existing vehicle infrastructure, the proposed framework reduces reliance on manual inspections and centralized systems. The collaborative data-sharing mechanism fosters a community-driven safety network, while the reinforcement learning loop ensures continuous improvement.
Ultimately, this system has the potential to reduce accident rates, optimize maintenance schedules, and enhance the driving experience across urban and rural environments.
4. Conclusions
This paper presents a lightweight, vehicle-integrated framework for real-time road surface monitoring using vision-based sensing and collaborative communication. By combining consumer-grade hardware like Raspberry Pi and dashcams with the YOLOv8 detection model, the system achieves fast and accurate defect recognition while remaining cost-effective and scalable.
GPS tagging and vehicle-to-vehicle alerts enhance safety by enabling proactive driver responses. The system also integrates vehicle dynamics and supports reinforcement learning, ensuring continuous improvement and adaptability. Preliminary testing with datasets from Hong Kong and open-source sources yielded a high success rate, confirming the system’s practical viability.
Future development will focus on collecting diverse dashcam footage to improve model generalization across different vehicles and camera setups. This will support broader deployment and further strengthen the system’s reliability in varied real-world conditions. Our goal is to develop an external plug-and-play device that can be easily installed in vehicles without requiring modifications to existing systems, ensuring practical usability and commercial viability.
NOTES
*These authors contributed equally to this work.