World Model on Million-Length Video andLanguage with RingAttention
AI summary of the linked articleThe LargeWorldModel Organization lists two research projects on models that handle very long multimodal sequences. World Model on Million-Length Video and Language With Blockwise RingAttention, by Hao Liu, Wilson Yan, Matei Zaharia and Pieter Abbeel, is a vision-language model trained on million-length sequences of images, language and video. ElasticTok, by Wilson Yan, Matei Zaharia, Volodymyr Mnih, Pieter Abbeel, Aleksandra Faust and Hao Liu, is an adaptive tokenization model that encodes images and long video into variable-length sequences. |
 |
Recommended by 1 curator |
1 comment ruycer.ethSubmitter This paper was submitted in the AI channel on forecaster. I find it fascinating, as it has a potential to learn from video, and that opens up the possibility to learn from an endless source of good data. You can imagine learning from video from the world, and making a model of it. |