Lighthouse Attention

Nous Research introduces Lighthouse Attention, a novel attention mechanism designed to improve efficiency and performance in transformer models by selectively focusing on key information, reducing computational overhead while maintaining model quality.