🌱 Introduction

β›“ ‍Basics of Sequence Models

β›“ Sequence Models In General

  • Sequences are data structures where each example could be seen as a series of data points, for example 🧐:


Input X

Output Y


πŸ’¬ Speech Recognition

Wave sequence

Text sequence


🎢 Music Generation

Nothing / Integer

Wave Sequence


πŸ’Œ Sentiment Classification

Text Sequence

Integer Rating (1➑5)


πŸ”  Machine Translation

Text Sequence

Text Sequence


πŸ“Ή Video Activity Recognition

Video Frames



  • Since we have labeled data X and Y so all of these tasks are addressed as Supervised Learning πŸ‘©β€πŸ«

  • Even in Sequence-to-Sequence tasks lengths of input and output can be different ❗

πŸ€” Why Do We Need Sequence Models?

  • Machine learning algorithms typically require the text input to be represented as a fixed-length vector πŸ™„

  • Thus, to model sequences, we need a specific learning framework able to:

    • βœ” Deal with variable-length sequences

    • βœ” Maintain sequence order

    • βœ” Keep track of long-term dependencies rather than cutting input data too short

    • βœ” Share parameters across the sequence (so not re-learn things across the sequence)

πŸ‘©β€πŸ’» My Codes