This is a Sentence Transformers script for training multi-vector embedding models to survive vector compression. It uses pooling-aware regularization so the encoder preserves critical information when merging similar token vectors. With a pool factor of 4, the method retains 97.8% of retrieval quality while reducing storage requirements by 4x.
It is worth testing if you are running ColBERT-style search systems and storage costs are becoming a bottleneck. The trade-off is that you must train the model specifically for your target compression ratio, as standard pre-trained models lose accuracy when compressed without this adjustment. It does not solve the fundamental latency issues of multi-vector scoring, only the memory footprint.
more: https://huggingface.co/blog/token-pooling
my other side: ivo.blogspot.com
Leave a Reply