arxiv:2605.00814

Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs

Published on May 1

· Submitted by

Authors:

Abstract

Large vision-language models suffer from visual signal dilution during long sequence generation, which is mitigated by a lightweight persistent visual memory module that maintains visual attention and improves reasoning performance.

AI-generated summary

While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a "Visual Signal Dilution" phenomenon, where the accumulation of textual history expands the attention partition function, causing visual attention to decay inversely with generated sequence length. To counteract this, we propose Persistent Visual Memory (PVM), a lightweight learnable module designed to ensure sustained, on-demand visual perception. Integrated as a parallel branch alongside the Feed-Forward Network (FFN) in LVLMs, PVM establishes a distance-agnostic retrieval pathway that directly provides visual embeddings for precise visual perception, thereby structurally mitigating the signal suppression inherent to deep generation. Extensive experiments on Qwen3-VL models demonstrate that PVM brings notable improvements with negligible parameter overhead, delivering consistent average accuracy gains across both 4B and 8B scales, particularly in complex reasoning tasks that demand persistent visual perception. Furthermore, in-depth analysis reveals that PVM can resist length-induced signal decay and accelerate internal prediction convergence.

View arXiv page View PDF GitHub 2 Add to collection

Community

chamber111

Paper submitter about 20 hours ago

Autoregressive LVLMs lose visual focus via "Visual Signal Dilution". We solve this with Persistent Visual Memory (PVM), a parallel branch shielding visual retrieval from growing text to ensure length-agnostic grounding and boost complex reasoning.

avahal

about 9 hours ago

the core idea of pvm is neat: a parallel visual memory path that retrieves from a fixed visual token bank and gates it back into the main stream, so perception doesn't fade as text length grows. i’ve seen similar vibes in retrieval-augmented models, but decoupling the visual attention normalization from the autoregressive context is the clever bit that could actually scale insights to long-horizon reasoning. the arxivlens breakdown helped me parse the exact placement (bottleneck adapter, layers 8,16,24) and the 512-d latent bottleneck, which is a nice design sweet spot (https://arxivlens.com/PaperView/Details/persistent-visual-memory-sustaining-perception-for-deep-generation-in-lvlms-4767-46a7743b). one question: would you expect diminishing returns as you scale the visual memory size or move to denser visual tokens, and how sensitive is it to the choice of the fixed visual token bank?