Скачать или смотреть Benchmarking Vision-Language Models on OCR in Dynamic Video Environments (Feb 2025)

Title: Benchmarking Vision-Language Models on Optical Character Recognition in Dynamic Video Environments
Link: http://arxiv.org/abs/2502.06445v1
Date: February 2025

Summary:
This paper presents a benchmarking study of Vision-Language Models (VLMs) and traditional Computer Vision OCR systems on Optical Character Recognition tasks in dynamic video environments. The study introduces a custom dataset of 1,477 manually annotated video frames from diverse domains. Three leading VLMs (Claude-3, Gemini-1.5, and GPT-4o) and two traditional OCR systems (EasyOCR and RapidOCR) were evaluated using metrics such as Word Error Rate, Character Error Rate, and Accuracy. The results show that VLMs outperform traditional OCR methods in dynamic video settings, with GPT-4o achieving the highest overall accuracy. The paper also discusses the challenges and limitations of using VLMs for OCR tasks, including content security policies that may impact reliability.

Key Topics:
Optical Character Recognition (OCR)
Vision-Language Models (VLMs)
Benchmarking
Video Processing
Dataset Creation
Performance Evaluation
Traditional OCR vs. VLMs

Chapters:
00:00:00 - Introduction to Video OCR
00:00:18 - Key Takeaway: New Benchmark
00:00:33 - Traditional OCR vs. VLMs
00:01:49 - The New Video OCR Dataset
00:02:53 - VLM Benchmark Results
00:03:47 - Limitations of VLMs
00:04:28 - VLM Errors in Handwriting
00:05:12 - Gemini's Financial Content Struggles
00:05:38 - Content Security Policies Explained
00:06:50 - Qualitative Analysis: Correction Errors
00:07:57 - Balancing Correction and Transcription
00:08:51 - VLMs Beyond OCR
00:09:36 - Ethical Considerations for VLMs
00:10:13 - Potential VLM Applications: Content
00:10:50 - Potential VLM Applications: Creation
00:11:33 - VLMs for Accessibility
00:13:09 - Final Takeaways