Assessing the Creativity of LLMs in Mathematical Problem Solving

Abstract

This study investigates the creative potential of Large Language Models (LLMs) in mathematical reasoning, an area previously under-explored. We propose a novel framework and benchmark, incorporating problems from middle school to Olympic-level competitions, to evaluate LLMs’ ability to generate novel solutions, employ multi-stage methods, and provide insightful reasoning. Our experiments reveal that while LLMs excel in standard mathematical tasks, their creative problem-solving abilities vary significantly. Notably, the Gemini-1.5-Pro model excelled in producing novel solutions across all tested LLMs. This research pioneers a new direction in assessing AI creativity, highlighting both the strengths and limitations of LLMs in mathematical innovation, and paves the way for future advancements in AI-driven mathematical discovery.