arxiv preprint – Spectral State Space Models

In this episode, we discuss Spectral State Space Models by Naman Agarwal, Daniel Suo, Xinyi Chen, Elad Hazan. The paper introduces a new type of state space model (SSM) for sequence prediction that utilizes spectral filtering to handle long-range dependencies in data. These spectral state space models (SSMs) are shown to be robust, as their performance is not affected by the dynamics’ spectrum or the problem’s size, and use fixed convolutional filters, bypassing the need for additional training while still achieving better results than traditional SSMs. The models’ effectiveness is demonstrated through experiments on synthetic data and real-world tasks that require long-term memory, thereby validating the theoretical advantages of spectral filtering in practical applications.