Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Pricing
Podcast Image

AI Breakdown

arxiv preprint - LITA: Language Instructed Temporal-Localization Assistant

29 Mar 2024

Description

In this episode, we discuss LITA: Language Instructed Temporal-Localization Assistant by De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, Jan Kautz. The paper introduces the Language Instructed Temporal-Localization Assistant (LITA), which tackles the issue of temporal localization in Large Language Models (LLMs) processing video content, where they struggle to identify "when" an event occurs in a video. LITA incorporates time tokens for better temporal representation, uses a SlowFast token architecture for finer temporal resolution, and emphasizes training on temporal localization data, introducing a new task with its dataset (ActivityNet-RTL). The implementation of LITA demonstrates strong performance improvements in temporal localization tasks and video-based text generation, with the code available on GitHub for public use.

Audio
Featured in this Episode

No persons identified in this episode.

Transcription

This episode hasn't been transcribed yet

Help us prioritize this episode for transcription by upvoting it.

0 upvotes
🗳️ Sign in to Upvote

Popular episodes get transcribed faster

Comments

There are no comments yet.

Please log in to write the first comment.