Real-Time Vehicle Classification and License Plate Recognition via Deformable Convolution-Based Yolo v8 Network

The rapid increase in vehicles has made license plate recognition (LPR) essential for transportation management, traffic control, and security. However, extracting alphanumeric characters remains challenging due to poor lighting, low resolution, varied fonts, and weather conditions.

Existing LPR methods based on YOLO (You Only Look Once) models include fog-dehazing and deep learning capabilities, which improve accuracy but struggle with text complexity, clutter, and high computational costs.

While models like Tiny-YOLOv3 achieve high accuracy, their real-time performance suffers due to processing demands, and traditional segmentation methods remain unreliable due to sensitivity to image quality and lighting.

This study introduces the DEN-YOLO (Deformable Convolution-Based YOLOv8) network for real-time vehicle classification and license plate recognition. It integrates advanced preprocessing techniques, including low-light enhancement using CLAHE (contrast-limited adaptive histogram equalization), to improve visibility by addressing poor contrast and blurriness.

The dark channel prior algorithm reduces fog and haze effects by estimating atmospheric light and transmission, ensuring clearer images. Additionally, deep learning-based super-resolution reconstruction enhances low-resolution images, reducing noise and blurring for better recognition of distant plates in CCTV footage.

The YOLOv8 model, adapted for both vehicle classification and license plate detection, optimizes resource use and improves performance. The enhanced deformable convolutional network refines recognition through a scalable architecture.

It incorporates deformable convolution and squeeze-and-excitation blocks to adjust the receptive field and improve feature selection dynamically. These enhancements enable accurate recognition across various scales, orientations, and environmental conditions.

Character segmentation and recognition are critical for precise LPR. After detecting plates, edge detection, thresholding, and morphological operations isolate characters, which convolutional neural networks (CNNs) convert into machine-readable text. The assembled characters support applications like vehicle tracking, toll collection, and law enforcement.

The dataset, which initially comprised 740 Tunisian images, was expanded with 610 additional images at resolutions of 320×240, 640×480, and 1280×700. It included varied plate colors (60% black, 20% blue, 20% white), fonts, and sizes. Data augmentation techniques such as deformation, random lighting, Gaussian blurring, and rotation generated 265,000 actual and 140,000 non-license plates, ensuring robustness in real-world scenarios.

Performance evaluation demonstrated the DEN-YOLO network’s effectiveness, achieving 98.9% accuracy with a training loss of 0.94. It outperformed YOLOv5, YOLOv6, and YOLOv7, achieving 98.94% accuracy—up to 8.02% higher than previous models. The model also reached 93.95% specificity, 94.92% recall, and the lowest execution time among the five tested algorithms, ensuring real-time efficiency.

The DEN-YOLO network offers a robust real-time vehicle and license plate detection solution, integrating advanced preprocessing techniques such as low-light enhancement, super-resolution, and defogging to improve image quality.

Enhancements to YOLOv8, including deformable convolution and squeeze-and-excitation blocks, further refine classification accuracy, ensuring reliability across diverse conditions. With superior accuracy over previous YOLO versions, the model is well-suited for traffic management and surveillance.

Future research will optimize DEN-YOLO’s real-time processing for high-traffic environments by reducing latency and adapting the system for deployment in complex environments. Expanding into diverse environments will enhance real-time decision-making for traffic management, toll collection, and law enforcement.