This notebook demonstrates the linear property of transformers from the paper Attention is all you need [https://arxiv.org/abs/1706.03762]. The code shows how positoinal encodings can be represented by linear transformaitons of each other inside the transformer architecture.