Скачать или смотреть AI for developers: What made teams struggle with AI in 2024?

What if I told you ChatGPT and Copilot frustrated teams in 2024?

Pragmatic Engineer recently performed a survey on 200+ Engineering Managers, wherein almost 1/3rd of them were “UNIMPRESSED” with all these AI tools and plan to avoid them in 2025.

But where exactly did these AI tools go wrong?

✅ Hallucination is a big problem with AI
LLMs often rely on limited training data, which may become outdated, edited, or deleted over time. Yet, these tools continue suggesting functions, methods, or APIs that are no longer valid in real-world scenarios.

✅ Good for simple stuff, fails when it gets complicated
AI tools excel at speeding up straightforward tasks like simple refactoring, config file changes, or handling regular expressions. However, they fall short when navigating complex codebases, particularly those with uncommon coding styles or architecture patterns.

✅Doesn’t use context beyond file-level
Current LLM tools operate at the file level rather than the project level. While they handle small, self-contained tasks well, such scenarios are relatively rare. They struggle with tasks requiring a broader understanding of the codebase, such as types, interfaces, architectural patterns, or overall file structure.

✅Code reviews have become more time-consuming
After six months or more of use, about 60% of respondents became more cautious about AI tools. They reported spending extra time reviewing code to remove duplicate statements and fix non-existent or invalid functions generated by AI.

The consensus among many EMs is clear: while AI has potential, it is crucial to approach its integration thoughtfully. Rushing to adopt these tools without understanding their limitations could lead to more problems than benefit.

What’s your experience with AI tools? 🚀

#aitools #hypertest #chatgpt #copilot #ai