🔁

Recurrent Neural Networks Intermediate

Networks with memory for sequences. Visualise the hidden state evolving over time and implement the recurrence in JavaScript.

4 lessons 12 quiz questions ⚡ Live compiler

Lessons & quizzes Compiler Certificate

📚 Lessons & quizzes

Each lesson ends with its own short quiz. Answer them as you go — score 90% across all lessons to earn your certificate.

1 Networks with memory

A Recurrent Neural Network processes a sequence one step at a time, keeping a hidden state that carries information forward. Below the network is shown unrolled across four time steps — set each input and watch the state evolve.

2 The recurrence relation

At each step: hₜ = tanh(Wₓ·xₜ + Wₕ·hₜ₋₁ + b). The same weights are reused every step (weight sharing), and the previous state hₜ₋₁ feeds back in — that feedback is the “recurrent” part.

function rnnStep(Wx, Wh, x, hPrev) {
  // single hidden unit for clarity
  const pre = Wx * x + Wh * hPrev;
  return Math.tanh(pre);
}
let h = 0;
for (const x of [0.8, -0.5, 0.3, 0.6]) { h = rnnStep(0.9, 0.7, x, h); console.log(h); }

3 Vanishing gradients, LSTM and GRU

Plain RNNs struggle to remember long dependencies because gradients shrink as they flow back through many steps (the vanishing gradient problem). Gated variants — LSTM and GRU — add gates that protect and update the memory, handling much longer sequences.

4 Sequence tasks

RNNs power text generation, speech, time-series forecasting and translation. The final hidden state can summarise a whole sequence, or each step can emit an output (e.g. tagging every word). Many tasks are now handled by Transformers, but the recurrent idea remains foundational.

⚡ Recurrent Neural Networks Compiler

Code runs live in a sandboxed frame, rendered by your browser.

🎓 Certificate of Completion

🔒 Complete every lesson quiz above with 90%+ to unlock your downloadable certificate.