Transformer

Definition

A neural network architecture based solely on an attention mechanism, which weighs the importance of each element of a sequence relative to the others. It was presented in 2017 in the paper “Attention Is All You Need”. Large language models are built on it, as are many robot policies, including ACT.

Checked on 28 September 2026 against the references below.

Related terms

• Large language model
• VLA (vision-language-action model)
• Action chunking

References

• Vaswani et al., “Attention Is All You Need”, 2017 (arXiv:1706.03762)
• Zhao et al., “A Survey of Large Language Models” (arXiv:2303.18223)