Spline-based Transformers

Опубликовано: 23 Сентябрь 2024
на канале: DisneyResearchHub

5,506

We introduce Spline-based Transformers, a novel class of Transformer models that eliminate the need for positional encoding. Inspired by workflows using splines in computer animation, our Spline-based Transformers embed an input sequence of elements as a smooth
trajectory in latent space. Overcoming drawbacks of positional encoding
such as sequence length extrapolation, Spline-based Transformers also
provide a novel way for users to interact with transformer latent spaces
by directly manipulating the latent control points to create new latent
trajectories and sequences. We demonstrate the superior performance of
our approach in comparison to conventional positional encoding on a
variety of datasets, ranging from synthetic 2D to large-scale real-world
datasets of images, 3D shapes, and animations.
Publication link: https://la.disneyresearch.com/publica...

На этой странице сайта вы можете посмотреть видео онлайн Spline-based Transformers длительностью часов минут секунд в хорошем качестве, которое загрузил пользователь DisneyResearchHub 23 Сентябрь 2024, поделитесь ссылкой с друзьями и знакомыми, на youtube это видео уже посмотрели 5,506 раз и оно понравилось like зрителям. Приятного просмотра!

3,390