A state space model (SSM), in the context of modern AI, is a class of neural network architectures for sequence modeling that processes a sequence using a hidden state updated at each step according to equations borrowed from control theory and signal processing. Unlike the transformer, whose self-attention mechanism compares every token to every other token and thus scales quadratically with sequence length, state space models update a fixed-size hidden state at each position, giving them linear or near-linear scaling with context length.
The most widely discussed neural state space model is Mamba, introduced by Albert Gu and Tri Dao in December 2023, which added an input-dependent, or selective, mechanism to earlier linear state space models so the model could choose what to remember or forget at each step, addressing a key weakness of prior SSMs on language tasks.
History
State space models have a long history in control engineering, where they describe how a system's internal state evolves over time and produces observable outputs. Early neural adaptations, such as the S4 model published by Stanford researchers in 2021, showed that a carefully parameterized linear state space layer could handle very long sequences competitively with recurrent neural networks and convolutional approaches on benchmarks emphasizing long-range dependencies. These early SSMs underperformed transformers on language modeling until Mamba's 2023 selective mechanism closed much of the gap, prompting renewed interest in SSMs as a potential transformer alternative.
Architecture and mechanism
A basic SSM layer maintains a hidden state vector updated by a linear recurrence at each sequence position, conceptually similar to an LSTM but with a mathematical structure that permits efficient parallel computation during training through convolution or parallel-scan algorithms. Selective SSMs like Mamba make the recurrence's parameters a function of the current input, letting the model dynamically decide which information to propagate forward, a capability closer to attention's content-based lookup than a fixed linear filter. Hybrid architectures that interleave SSM layers with a smaller number of attention layers, seeking the linear-time efficiency of SSMs alongside some of attention's expressiveness, appeared from several labs by 2024 and 2025.
Comparison with transformers
The chief appeal of state space models is inference efficiency on long sequences: because the hidden state has fixed size, generating each new token does not require re-attending to the entire growing context, unlike standard transformer inference, which recomputes attention over prior tokens, partly mitigated in practice by key-value caching. This makes SSMs attractive for very long context windows and resource-constrained deployment. However, by 2025 pure SSMs had not clearly surpassed well-tuned transformers on most language benchmarks, and much of the field's activity shifted toward hybrid designs rather than replacing attention outright.
Reception and outlook
Mamba and its successors were received as one of the more credible architectural challenges to transformer dominance since the original 2017 Attention Is All You Need paper, generating significant academic interest and several open-source implementations. Researchers including Yann LeCun and others have noted that recurrent-style state, whether in an LSTM or an SSM, could offer advantages for tasks requiring reasoning over very long inputs, such as processing entire books, codebases, or genomic sequences. As of 2025, state space models remained a minority architecture in production large language models compared to transformers, but continued to influence hybrid model design at several major labs.