data efficiency – Buonaiuto@Work

SpikingBrain: a revolutionary brain-inspired Chatgpt made in China

Massimo2025-09-180357 views

The Chinese SpikingBrain is a new family of brain-inspired large language models that reimagines how AI can process information more efficiently. SpikingBrain models adopt a biological principle: neurons remain idle until an event triggers them to fire. This event-driven design reduces unnecessary computation, cuts energy use, and enables faster responses. SpikingBrain achieves over 100× speedup in “time to first token” for sequences up to 4 million tokens. Energy consumption drops by 97% compared to traditional LLMs.

Meet in the Middle: A New Pre-Training Paradigm for Language Models to Enhance Text Infilling

Massimo2023-02-170445 views

This research introduces a novel pre-training paradigm for language models (LMs), termed “Meet in the Middle” (MIM), which optimizes data utilization by integrating both prefix and suffix contexts while preserving autoregressive properties. MIM employs a dual approach, training forward and backward LMs concurrently on a shared corpus, with an agreement regularizer to ensure consistency in token probability distributions. This method enhances data efficiency and model agreement, allowing for improved performance in text infilling tasks. Evaluation across various domains confirms MIM’s superiority over traditional models, showcasing its potential to redefine LM pre-training and application.