LSTM

Long Short-Term Memory is a recurrent network that can keep information across a long sequence. It uses gates and a cell state so useful signal does not die after a few dozen steps.

A plain RNN fails on long sequences because gradients vanish or explode when you backpropagate through time. An LSTM has three gates. The forget gate drops what the cell should not keep. The input gate stores new information. The output gate decides what to emit. The cell state is a highway. Gates protect it from the multiplicative decay that wrecks vanilla RNNs.

Important facts can last hundreds of steps. A hidden state still holds short-term context.

Hochreiter and Schmidhuber introduced LSTMs in 1997. They made sequence work practical: translation, speech recognition, text generation, before transformers took over language. The Gated Recurrent Unit uses two gates instead of three and often matches LSTM quality with less machinery.

Transformers now dominate natural language. LSTMs still show up in time series, online sequence work, and jobs where a step-by-step recurrent structure is the right fit. If you want to understand vanishing gradients, and why later models needed residual highways, start with the LSTM cell.

GRU papers often treat the two as cousins. If a sequence is short, either works. If you need a cell you can inspect gate by gate, the LSTM is the clearer diagram. LSTMs were published in 1997 to carry information across long sequences. Transformers later replaced them for most language work.