EMNLP 2026 Main Conference

GraSP-VL

Length as a Semantic Granularity Interface for Vision-Language Representations

Zesheng Li Chengchang Pan Honggang Qi

A frozen vision-language model already contains object, attribute, relation, and caption information. GraSP-VL reorganizes access to those signals so that representation length becomes a configurable semantic control.

One shared near-orthogonal transform is learned for both modalities. At inference, the transform is applied once and ordinary cosine retrieval uses the prefix required by the task. Full-dimensional geometry is preserved while shorter prefixes expose progressively finer semantic distinctions.

GraSP-VL moves semantic selectivity into the intended representation prefixes — Semantic distinctions emerge at their assigned prefix lengths.

01 / Method

A length-controlled interface over a frozen VLM

The transform is shared between image and text embeddings. A prefix is not a generic compression budget: it is an explicit semantic access contract fixed before training.

GraSP-VL overview showing frozen image and text embeddings, shared transformation, nested prefixes, and retrieval — Overview. The same transform reorganizes both modalities into nested prefixes with increasing semantic specificity. Download vector figure.

D/16ObjectWhat is present?

D/8AttributeWhat does it look like?

D/4Relation / eventWhat is happening?

D/2Full captionWhich details co-occur?

DPreservationOriginal VLM geometry

Training

Frozen image and text features are optimized with alignment, retention, selective-emergence, and geometry-preservation objectives. The shared Cayley parameterization keeps the learned map orthogonal.

Inference

Apply the transform to a fixed gallery once, cache the result, truncate to the requested prefix, and rank candidates with cosine similarity. No LLM call or additional retrieval architecture is needed.

Scope

Prefix-level scores measure the configured semantic interface. Full-dimensional classification and retrieval are preservation checks, not an extra semantic stage.

02 / Results

Semantic access improves without moving the full space

The camera-ready experiments use 20,147 COCO/Flickr30K examples with a fixed 16,785 / 1,861 / 1,501 train, validation, and test split.

53.01semantic staircase

89.76hard-negative selectivity

< 1e-6full-space cosine drift

20,147accepted weak-supervision examples

Comparison of direct prefixes and GraSP-VL prefixes across semantic negative types — GraSP-VL moves object, attribute, and relation sensitivity to the intended prefix boundaries.

Method comparison showing semantic staircase, hard-negative selectivity, caption retrieval, and full-space drift — The interface is distinct from generic compression, coordinate reordering, and unconstrained adaptation.

Full-space preservation. At prefix D, ImageNet-V2 Top-1/Top-5 remains 62.31/87.01. All six reported bidirectional recall metrics remain unchanged on the Flickr30K 1K and COCO 5K Karpathy test sets.

03 / Code

Reproduce the interface from local artifacts

The public repository keeps the model, losses, metrics, data utilities, training scripts, evaluation scripts, and examples together. Large or licensed artifacts stay outside Git.

Install

git clone https://github.com/LIZESHENG13/Grasp-VL.git
cd Grasp-VL
python -m pip install -e ".[dev]"

Smoke test

python -B examples/minimal_prefix_transform.py
python -m compileall -q grasp_vl scripts underdog examples

Repository map

grasp_vl/: Core model, losses, prefixes, and metrics
scripts/data/: Annotation validation and split utilities
scripts/train/: Main model and baseline training
scripts/evaluate/: Retrieval, transfer, and preservation checks
docs/: Project page, data schema, and reproducibility notes

Open the repository

04 / Data

Generated supervision, released with provenance

The public package contains the generated semantic views and typed negatives used to train the interface, without redistributing COCO or Flickr30K content.

20,147accepted records

3fixed splits

The release keeps G0-G2 and five generated negative types. Source captions, source distractors, G3, and the full negative are excluded; exact-string hashes are retained for provenance and reconstruction after users obtain the source datasets.

Download

train: 16,785 records
val: 1,861 records
test: 1,501 records

Open the dataset card

05 / Citation

Use GraSP-VL in your work

If the code or method is useful for your research, please cite the EMNLP 2026 paper.

@inproceedings{li2026graspvl,
  title = {GraSP-VL: Length as a Semantic Granularity Interface for Vision-Language Representations},
  author = {Li, Zesheng and Pan, Chengchang and Qi, Honggang},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
  year = {2026},
}

Supported by the National Natural Science Foundation of China (NSFC), Grant No. 62271466.