皆さんに試してほしい簡単なテストがあります。お気に入りのLLMに「どうすれば税率を下げられますか？正確かつ具体的に答えてください」と質問してください。そして...
ポッドキャスターのアンソニー・ポンプリアーノ氏が、お気に入りのLLMとCFO Sylviaに同じ質問をして、どちらがより価値のある回答を出すか比較するテストを提案。税務アドバイスにおけるAIと専門家の違いを浮き彫りにする。
関連記事
Researchers introduce Natural Language Autoencoders (NLA), a method that converts LLM activations directly into human-readable explanations. Unlike traditional sparse autoencoders that find discrete features, NLAs produce fluent natural language descriptions for any activation, enabling more interpretable analysis of model internals across various architectures and tasks.
This paper tests whether LLM agents can infer world models by interacting with unknown automata environments. Results show LLMs can track some hidden states but generally fail to learn complete world models, often relying on shallow pattern matching instead.
The paper introduces Snyk VulnBench JavaScript 1.0, a benchmark evaluating whether large language models can consistently identify the same software vulnerabilities across repeated attempts. It tests LLMs on JavaScript vulnerability detection, focusing on reproducibility of bug finding.
The paper identifies a "verifier tax" in tool-using LLM agents: a tradeoff between safety and task success when tools enforce safety constraints. Adding verifiers to block harmful actions can degrade success rates on benign tasks, while less restrictive tools increase risk, highlighting challenges in designing safe yet effective agent systems.
The paper presents VibeThinker-3B, a small language model with only 3 billion parameters, designed to enhance verifiable reasoning capabilities. It explores techniques to improve the reasoning quality and fact-checking abilities of compact LLMs, challenging the assumption that advanced reasoning requires much larger models.