ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data

Al Zadid Sultan Bin Habib; Tanpia Tasnim; Md. Ekramul Islam; Muntasir Tabasum

doi:10.1007/978-3-032-31397-3_1

Published ICPR 2026 Oral Paper RRPR Badge Self-Supervised Learning Feature-Level Contrast Remote Sensing

ZAYAN

ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data is a self-supervised, feature-centric framework that moves contrastive learning from the sample level to the feature level. ZAYAN learns robust, disentangled, and redundancy-reduced feature representations without labels during pretraining, then uses a Transformer for downstream classification.

Al Zadid Sultan Bin Habib¹, Tanpia Tasnim², Md. Ekramul Islam³, Muntasir Tabasum¹

¹ West Virginia University, USA · ² Green University of Bangladesh · ³ Stamford University Bangladesh

ICPR Oral Session Paper arXiv GitHub Code Question? PyPI RRPR Results Citation

International Conference on Pattern Recognition
ICPR 2026 · Lyon, France · Oral Presentation

View RRPR badge results ↗

What is ZAYAN?

ZAYAN stands for Zero-Anchor dYnamic feAture eNcoding. It is designed for tabular remote sensing and environmental data where features may be heterogeneous, redundant, noisy, and labels may be scarce.

Self-Supervised

Representation learning does not require class labels during ZAYAN-CL pretraining.

Feature-Level

ZAYAN contrasts augmented views of feature columns rather than using samples as the main learning unit.

Disentangled

A redundancy penalty encourages diverse, angularly separated, and less redundant feature embeddings.

Core idea: move contrastive learning from the sample level to the feature level, then preserve the learned feature geometry in the downstream Transformer.

From Instance-Level Contrast to Feature-Level Contrast

Conventional contrastive learning usually begins from a sample or image patch. ZAYAN instead treats each tabular feature column as a learning unit and builds positive pairs from two stochastic views of the same feature.

Conventional contrastive learning

sample / patch $x_i$

view 1 ↔ view 2

The learning anchor is typically a sample or patch.

→

ZAYAN feature-level contrast

feature $f_j$

augmented view 1 ↔ augmented view 2

Same-feature views are positives; other features provide negatives.

ZAYAN's central design shift from instance-level to feature-level contrastive learning.

ZAYAN Architecture

ZAYAN separates self-supervised feature representation learning from downstream supervised classification while preserving the geometry learned during pretraining.

ZAYAN-CL: Zero-Anchor Feature-Level Pretraining

1. Augment Create two stochastic views of each feature using noise, quantile warping with jitter, and masking.

↓

2. Contrast Pull two views of the same feature closer while pushing different feature embeddings apart.

↓

3. Disentangle Use redundancy reduction to encourage diverse and angularly separated feature embeddings.

Training objective

For each feature $f_j$, two augmented views are encoded as $z_j^{(1)}$ and $z_j^{(2)}$. The full ZAYAN-CL objective combines feature-level contrastive learning with redundancy reduction.

ZAYAN-CL objective \[ \min_{\theta}\; \mathcal{L}_{\mathrm{ZAYAN\text{-}CL}}(\theta) + \lambda\,\mathcal{R}(\theta) \]

Redundancy penalty \[ \mathcal{R}(\theta) = \|Z^\top Z-I\|_F^2 \]

No label dependence during pretraining ZAYAN-CL learns from the internal structure of feature columns and their stochastic views.

Positive and negative construction Two views of the same feature form a positive pair; other feature views provide negatives.

Geometric regularization The redundancy term suppresses pairwise cosine similarity and encourages a diverse feature embedding space.

ZAYAN-T: Transformer for Downstream Classification

After ZAYAN-CL, the learned feature embeddings are treated as Transformer tokens. ZAYAN-aware positional encoding is added, the sequence is processed by a Transformer encoder, and the token outputs are mean-pooled before classification.

Transformer encoding and aggregation \[ \tilde z_{ij}=z_{ij}+p_j, \qquad H_i=\mathrm{TransformerEncoder}(Z_i^{\mathrm{in}}), \qquad r_i=\frac{1}{m}\sum_{j=1}^{m} h_{ij} \]

Structure-preserving supervised objective \[ \mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{CE}} + \gamma\,\mathcal{L}_{\mathrm{preserve}} \]

Design principle: ZAYAN-T is task-aware, but it is trained to preserve the representation geometry established during self-supervised feature-level pretraining.

Experimental Setup

8datasets

31comparison baselines

5-foldcross-validation

150Optuna trials per tunable model / dataset

Benchmarks

Urban Land Cover

675 samples · 147 features · 9 classes

Wilt

4,839 samples · 5 features · binary

Crop Mapping

3,000-sample subset · 174 features · 7 classes

Forest Type Mapping

523 samples · 27 features · 4 classes

Census of Individual Trees

65,324 samples · 10 features · binary · synthetic balancing

RSI-CB256

5,631 images · ResNet features · PCA to 512-D · 4 classes

Pluvial Flood

144,401 samples · 9 features · 5 classes · 50% injected noise

Flood Risk in India

10,000 samples · 21 features · binary

The benchmark intentionally spans noisy, subsampled, class-balanced, PCA-compressed, and heterogeneous environmental settings rather than a single easy regime.

Key Results Against Strong Tabular Baselines

ZAYAN obtains the best or tied-best mean accuracy across all eight benchmarks, ranking first on seven datasets and tying KNN on Crop Mapping. Its overall average rank is 1.06.

Model	Urban	Wilt	Crop*	Forest	Tree+	RSI#	Pluvial$	Indian	Avg. Rank
TabR	83.21±1.66	98.78±0.17	98.62±0.68	90.25±1.94	64.78±0.42	99.32±0.18	89.88±0.62	49.59±1.88	9.25
TabM	82.07±0.98	98.80±0.20	98.92±0.96	88.72±0.98	65.34±0.24	99.36±0.10	89.64±0.98	51.49±0.92	6.94
TANDEM	83.29±1.53	98.37±0.39	99.18±0.12	90.05±2.07	66.96±0.12	99.72±0.14	90.72±0.32	50.26±1.55	6.06
TabICL	83.30±0.98	98.76±0.17	99.46±0.10	90.05±2.35	66.98±0.20	99.74±0.16	90.75±0.74	51.27±0.66	3.38
ZAYAN	84.80±7.10	99.69±0.40	99.66±0.34	97.21±0.45	67.00±0.12	99.77±0.15	93.61±0.47	51.55±0.43	1.06

* Crop: 3,000-sample subset. + Tree: synthetic balancing. # RSI: ResNet features reduced to 512-D by PCA. $ Pluvial: 50% noise added to training data.

Main result: ZAYAN achieves the best overall cross-dataset ranking, with average rank 1.06, ahead of TabICL (3.38), TANDEM (6.06), TabM (6.94), and TabR (9.25).

Ablation: What Makes ZAYAN Work?

Component and hyperparameter ablations on Urban Land Cover show that the full model benefits from contrastive learning, redundancy reduction, the preservation objective, and moderate stochastic augmentations.

Full ZAYAN

84.80±7.10%
5-fold CV accuracy.

Without redundancy penalty

78.45±0.72%
Clear drop when $\lambda=0$.

Without contrastive loss

75.98±0.88%
MSE-only representation learning is weaker.

Removing individual augmentations or loss components lowers mean accuracy, supporting the combined contrastive + redundancy + structure-preservation design rather than any single isolated component.

Robustness, Calibration, OOD & Deployment Diagnostics

Feature perturbations

Accuracy remains relatively stable under moderate feature shuffling and degrades more strongly under feature dropping.

Selective prediction

Higher confidence thresholds improve accuracy on retained predictions while reducing coverage.

Calibration / OOD

The study includes reliability, predictive confidence, entropy, synthetic OOD, and local sensitivity diagnostics.

0.151expected calibration error

1.000AUC in the Urban one-vs-rest triage diagnostic

5.36 msbatch-32 inference on the tested GPU

~51%kNN-Agree@5 under perturbation analysis

ICPR 2026 RRPR Badge

Open RRPR Results ↗

ZAYAN is associated with an ICPR 2026 RRPR badge. The badge image is linked directly to the official ICPR 2026 RRPR results page so visitors can verify the recognition from the source.

Official RRPR Results

ICPR 2026 Oral Presentation

ZAYAN is presented as an oral paper at ICPR 2026. Use the official session page below for the conference schedule and session details.

Oral Session 5E Official ICPR 2026 session page → ICPR 2026 Official conference website →

Code, Paper & Resources

GitHub Source code and project repository → PyPI Install the ZAYAN package → Springer Paper Published ICPR 2026 chapter → arXiv Open preprint → ICPR Oral Session Session 5E information → RRPR Badge Results Official reproducibility recognition page →

Install from PyPI

pip install zayan

Takeaway

ZAYAN shows that feature-level self-supervision can be a strong recipe for tabular sensing data. ZAYAN-CL learns zero-anchor, redundancy-reduced feature embeddings without labels, while ZAYAN-T performs task-aware Transformer classification while preserving the learned structure.

Feature-centric SSL

Contrast is defined over tabular features and their augmented views.

Strong cross-dataset rank

Best or tied-best mean accuracy across all eight evaluated benchmarks.

Beyond accuracy

Evaluation includes robustness, calibration, OOD behavior, and deployment-style diagnostics.

Limitations: the quadratic redundancy penalty can restrict scalability at very high feature dimensions, and the downstream Transformer may face memory constraints on wide or very large datasets.

Acknowledgements

This work was supported in part by the International Association for Pattern Recognition (IAPR) through ICPR 2026 registration support. The authors would like to thank Dr. Md Mahedi Hasan for his kindness in presenting our work as a proxy presenter.

Citation

If you use ZAYAN in your work, please cite the ICPR 2026 paper.

@inproceedings{habib2026zayan, title = {ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data}, author = {Habib, Al Zadid Sultan Bin and Tasnim, Tanpia and Islam, Md. Ekramul and Tabasum, Muntasir}, booktitle = {International Conference on Pattern Recognition (ICPR)}, year = {2026}, doi = {10.1007/978-3-032-31397-3_1}, url = {https://link.springer.com/chapter/10.1007/978-3-032-31397-3_1} }