Computer Graphics Course Project

VisionDrive3D: Synthetic Driving Data for Detection & Segmentation

• Computer Graphics Course Project •

View Results GitHub Dataset 🤗
RGB input
Depth map
Instance seg.
2D bbox
3D bbox
Teaser (scene_00049): synchronized multimodal captures — RGB input, depth map (Inferno colormap, MiDaS / DepthAnything style: near = bright yellow, far = dark purple), instance segmentation mask, 2D bounding boxes, and projected 3D bbox wireframes. Click any image to enlarge.

Abstract

Abstract

We present VisionDrive3D, a synthetic driving scene renderer that generates RGB images with ground-truth bounding boxes, segmentation masks, and depth maps. We fine-tune YOLO detection and segmentation models on this synthetic data and evaluate transfer quality. Results show improved in-domain performance after fine-tuning compared with zero-shot baselines.

Dataset

Dataset

Resolution Classes Train Val Test Format
1280×720 car 800 100 100 COCO+YOLO+KITTI
Sample rendered scenes from the VisionDrive3D dataset (scene_00004, scene_00014, scene_00023, scene_00049).

Method

Method

OpenGL Renderer
→
GT Labels
→
Model
→
Evaluate
Depth Maps
↗

Stage 1. VisionDrive3D renders multi-car urban traffic scenes and exports aligned image, depth, and annotation outputs.

Stage 2. We train YOLO detection and segmentation models with deterministic train/val/test splits from synthetic images.

Stage 3. We compare zero-shot and fine-tuned checkpoints, then profile backbone quality-speed trade-offs on the held-out test split.

Results

Results

Cross-Dataset Pretrained 2D Detector Benchmark

COCO-pretrained weights evaluated zero-shot on the VisionDrive3D test split (100 images, car class only). FLOPs via thop (MACs×2).

Model Input Resolution mAP@50 mAP@50-95 FPS Params (M) FLOPs (G)

Zero-Shot Semantic Segmentation (Car Only)

Cityscapes-pretrained YOLO26-sem models evaluated on the same 100-image VisionDrive3D test split. Metrics evaluate car and background pixel classes.

Model Input Resolution Car IoU Car Dice mIoU FPS Params (M) FLOPs (G)

YOLO26 Fine-Tuning Comparison (Car Only)

COCO baseline and VisionDrive3D fine-tuned checkpoints evaluated on the same 100-image test split; inference uses confidence 0.001 and NMS disabled. Detection-only models do not report mask AP.

Model Task Weights mAPbox @50 mAPbox @50-95 Car IoU Car Dice mIoU

Qualitative

Qualitative

Conclusion

Conclusion

VisionDrive3D provides an end-to-end synthetic data workflow for object detection and segmentation. Fine-tuning on renderer-generated supervision improves target-task metrics, while model-family comparisons expose practical accuracy-speed trade-offs for deployment.

Citation

Citation

If you use the code or dataset from this project, please cite it as follows: 

@misc{visiondrive3d2026,
  title        = {VisionDrive3D: Synthetic Driving Data for Detection and Segmentation},
  author       = {Quoc Huy Nguyen, Chi Dai Le, Kim Khoi Le Nguyen},
  year         = {2026},
  howpublished = {\url{https://github.com/BenBunBo/VisionDrive3D}},
  note         = {Dataset: \url{https://huggingface.co/datasets/BenBunBo/VisionDrive3D}}
}