Скачать или смотреть Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks (Dec 202

Title: Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks (Dec 2025)
Link: http://arxiv.org/abs/2512.03262v1
Date: December 2025

Summary:
This paper investigates the security risks of "vibe coding," a paradigm where human engineers instruct LLM agents to complete coding tasks with minimal supervision. To assess these risks, the authors introduce SusVibes, a benchmark consisting of 200 repository-level feature requests derived from real-world open-source vulnerabilities. The study reveals a critical gap: while top-tier agents (like SWE-Agent with Claude 4 Sonnet) achieve 61% functional correctness, only 10.5% of the solutions are secure. Experiments show that standard mitigation strategies, such as adding vulnerability hints or security prompts, fail to improve security without significantly degrading functional performance.

Key Topics:
Vibe Coding
LLM Agents
Software Security
SusVibes Benchmark
Vulnerability Detection
Automated Software Engineering
Repository-level Tasks

Chapters:
00:00 - Is Vibe Coding Safe?
00:48 - The Shocking Security Gap
01:37 - Why Legacy Benchmarks Fail
02:53 - Designing the SusVibes Benchmark
04:08 - Functional But Insecure
05:10 - Comparing Model Security Capabilities
06:51 - Django Timing Attack Example
08:08 - Header Injection Vulnerability
09:08 - The Prompting Paradox
11:01 - Path to Secure Agents

Stock video credits:
Pressmaster - https://www.pexels.com/@pressmaster
Pixabay - https://www.pexels.com/@pixabay
Kelly - https://www.pexels.com/@kelly
Kindel Media - https://www.pexels.com/@kindelmedia
tunnel motions - https://www.pexels.com/@tunnelmotions
Engin Akyurt - https://www.pexels.com/@enginakyurt
Yaroslav Shuraev - https://www.pexels.com/@yaroslav-shuraev
StefWithAnF - https://www.pexels.com/@stefwithanf-1...
Colin Jones - https://www.pexels.com/@larchmedia
Oleg Gamulinskii - https://www.pexels.com/@oleg-gamulins...
Pavel Danilyuk - https://www.pexels.com/@pavel-danilyuk
Bedrijfsfilmspecialist.nl - https://www.pexels.com/@bedrijfsfilms...
Danil Shostak - https://www.pexels.com/@danil-shostak...
Ron Lach - https://www.pexels.com/@ron-lach
Pachon in Motion - https://www.pexels.com/@pachon-in-mot...
José Alfredo Munguía Lira - https://www.pexels.com/@rectorretro
Silviu Din - https://www.pexels.com/@silviu-din-16...
Trippy Lagoon - https://www.pexels.com/@trippy-lagoon...
Mikhail Nilov - https://www.pexels.com/@mikhail-nilov
Dan Cristian Pădureț - https://www.pexels.com/@paduret
cottonbro studio - https://www.pexels.com/@cottonbro
crazy motions - https://www.pexels.com/@crazy-motions...
@svetjekolem - https://www.pexels.com/@svetjekolem
Anete Lusina - https://www.pexels.com/@anete-lusina