Скачать или смотреть Briefcast:Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs

This research paper investigates the reasoning capabilities of large language models (LLMs) by comparing their performance on deductive and inductive reasoning tasks. The authors argue that prior research has often blurred the line between these two types of reasoning. To address this, they introduce a novel framework called SolverLearner that focuses specifically on inductive reasoning, allowing LLMs to learn underlying functions from a limited number of in-context examples. The study finds that LLMs demonstrate remarkable inductive reasoning capabilities, achieving near-perfect performance on most tasks using SolverLearner. Surprisingly, LLMs struggle with deductive reasoning, particularly in "counterfactual" tasks that deviate from their pre-training data. These findings suggest that LLMs are better at learning patterns from data than at following instructions precisely, particularly in novel situations.

Paper: https://arxiv.org/abs/2408.00114