Contextrast++: Robust multi-scale contextual contrastive learning for semantic segmentation

Semantic segmentation has made significant strides with deep learning; however, challenges remain in effectively capturing local and global contexts as well as addressing the long-tailed distribution problem. To tackle these issues, we present Contextrast++, a robust contrastive learning method for semantic segmentation that improves multi-scale feature integration and mitigates class imbalance issues. Our method consists of two key components: 1) contextual contrastive learning (CCL) and boundary-aware negative (BANE) sampling. CCL includes three subcomponents: adaptive fusion module, pixel-to-anchor (PA) loss, and anchor-to-anchor (AA) loss. The adaptive fusion module dynamically balances local and global feature integration, resulting in a more context-aware representation. While the PA loss leverages the fused multi-scale features to improve feature representation learning, the AA loss focuses on addressing long-tailed distribution problem by utilizing a memory bank that stores a fixed number of class-balanced representative anchors. Meanwhile, BANE sampling enhances segmentation precision by selecting hard negatives from misclassified boundary regions, which refines fine-grained details during contrastive learning. As verified in extensive experiments using public datasets, including ADE20K, Cityscapes, PASCAL-C, and CamVid, we demonstrate that Contextrast++ achieves a substantial semantic segmentation performance increase over existing contrastive learning-based, state-of-the-art approaches, while maintaining no additiontional computational overhead during inference.

Download paper here

Changki Sung, Hyungtae Lim, Wanhee Kim, Youngwoo Seo, and Hyun Myung, Contextrast++: Robust multi-scale contextual contrastive learning for semantic segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, (to appear), 2026.