Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Graph of Thoughts (GoT) is a framework that models LLM reasoning as a directed graph, enabling complex problem-solving by combining, refining, and evaluating multiple thoughts. It extends Chain-of-Thought and Tree-of-Thought approaches, allowing for more flexible and iterative reasoning processes.
Background
- Graph of Thoughts (GoT) is a research framework that extends how large language models (LLMs) like GPT-4 solve complex problems. Instead of the standard "chain of thought" (a single linear sequence of reasoning steps), GoT models reasoning as a directed graph — where different "thoughts" can branch, merge, and be refined in parallel.
- This matters because many real-world problems (sorting, planning, document merging, etc.) don't have a single correct reasoning path. GoT allows an LLM to explore multiple approaches simultaneously, compare partial solutions, and combine the best parts — much like a human brainstorming session.
- The project is open-source from ETH Zurich's SPCL lab. It builds on earlier work like "Chain-of-Thought" (Wei et al., 2022) and "Tree of Thoughts" (Yao et al., 2023), which introduced branching but not merging of reasoning paths.
- Key capability: GoT can achieve up to 62% improvement in accuracy on sorting tasks compared to chain-of-thought, while also reducing computational cost by allowing the model to "prune" unpromising branches early.