Process long videos, complex documents, and object localization with a single vision-language model
You can use a single model to find specific scenes in long videos, interpret graphs in complex documents, and pinpoint the location of objects on a screen. Eagle is a model series developed by NVIDIA that explores combining vision and language through a data-centric strategy. The recent addition of the LocateAnything generalization intelligence model allows for faster and more precise execution of various visual instruction understanding tasks.
The arrival of LocateAnything and recent updates
This model is a general vision-language intelligence model based on Eagle, supporting LoRA fine-tuning scripts for user-specific training. It also supports batch inference using the FlashAttention runtime, enabling efficient inference on GPUs that do not have Hopper or Blackwell architectures, such as the A100 or RTX 4090. These updates focus on expanding the model's accessibility and range of application.
Model lineup and the role of each version
The Eagle series is divided into several versions depending on research purposes and application areas.
- LocateAnything It increases processing speed by predicting bounding boxes in a single forward pass through parallel box decoding. It uses Qwen2.5-3B-Instruct and MoonViT-SO-400M as its backbone.
- Eagle 2.5 It can handle long contexts up to 128K tokens, making it suitable for analyzing long videos or documents. It uses Qwen2.5-7B-Instruct and SigLIP2.
- Eagle 2 It supports a 16K token context and is available in various model sizes (1B, 2B, 9B). It is used to verify the effectiveness of post-training data strategies.
Real-world applications and features
These models perform various tasks in both virtual and physical worlds. For example, LocateAnything demonstrates 3D object detection and spatial intelligence, providing examples of 3D object detection in real environments. Eagle 2.5 analyzes long video clips, dividing each scene into a title and detailed caption, and displays the start time in seconds. It also has the ability to interpret graphs within documents, extracting and explaining numerical information such as population change trends.
Things to check before use
The code for the Eagle models is released under the Apache 2.0 license, but the pre-trained model weights can only be used for non-commercial research purposes under the CC BY-NC 4.0 or NVIDIA license. Therefore, you must check the license terms before using them for commercial purposes. Additionally, some code includes modified components from other open-source projects such as LLaVA and InternVL, so you must also comply with their original licenses. Since the model is a research preview version, it is recommended to further verify stability and support scope before applying it to production environments.