📚
General Concepts
General Concepts of Sequence Models
In the context of text processing (e.g: Natural Language Processing NLP)
Symbol | Description |
$$X^{}$$ | The tth word in the input sequence |
$$Y^{}$$ | The tth word in the output sequence |
$$X^{(i)}$$ | The tth word in the ith input sequence |
$$Y^{(i)}$$ | The tth word in the ith output sequence |
$$T^{(i)}_x$$ | The length of the ith input sequence |
$$T^{(i)}_y$$ | The length of the ith output sequence |
A way to represent words so we can treat with them easily
Let's say that we have a dictionary that consists of 10 words (🤭) and the words of the dictionary are:
- Car, Pen, Girl, Berry, Apple, Likes, The, And, Boy, Book.
Our $$X^{(i)}$$ is: The Girl Likes Apple And Berry
So we can represent this sequence like the following 👀
Car -0) ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉
Pen -1) | 0 | | 0 | | 0 | | 0 | | 0 | | 0 |
Girl -2) | 0 | | 1 | | 0 | | 0 | | 0 | | 0 |
Berry -3) | 0 | | 0 | | 0 | | 0 | | 0 | | 1 |
Apple -4) | 0 | | 0 | | 0 | | 1 | | 0 | | 0 |
Likes -5) | 0 | | 0 | | 1 | | 0 | | 0 | | 0 |
The -6) | 1 | | 0 | | 0 | | 0 | | 0 | | 0 |
And -7) | 0 | | 0 | | 0 | | 0 | | 1 | | 0 |
Boy -8) | 0 | | 0 | | 0 | | 0 | | 0 | | 0 |
Book -9) ⌊ 0 ⌋ ⌊ 0 ⌋ ⌊ 0 ⌋ ⌊ 0 ⌋ ⌊ 0 ⌋ ⌊ 0 ⌋
By representing sequences in this way we can feed out data to neural networks ✨
- If our dictionary consists of 10,000 words so each vector will be 10,000 dimensional 🤕
- This representation can not capture semantic features 💔
Last modified 2yr ago