Arctic Sea Ice Prediction With TerraMind

Author: Caroline Arnold, AI Consultant, Helmholtz-Zentrum Hereon, caroline.arnold@hereon.de

Arctic sea ice can be detected from spaceborne synthetic aperture radar (SAR) imagery. Knowing the extent of arctic sea ice is essential for marine traffic and ecosystem monitoring. Specialists can derive different sea ice parameters from the SAR imagery, including sea ice content, state of development, and floe size. Automated sea ice mapping has so far been mainly conducted with UNets that were trained on hand-labeled datasets (Stokholm 2022, Stokholm 2024).

Here we fine tune TerraMind to predict arctic sea ice maps. We apply TerraMind outside its comfort zone: We focus on the Arctic and use imagery with differently polarized Sentinel-1 channels. The task is distinct from the applications covered by the PANGAEA benchmark.

We explore the following aspects:

• Can TerraMind outperform the specialised networks in (Stokholm, 2024)?
• Can "Thinking in modalities" support automated sea ice prediction?
• Can TerraMind generate reasonable missing modalities in the Arctic?

Dataset

We use the ready-to-train version of the Arctic Sea Ice Dataset that was prepared for the AI4Arctic Sea Ice challenge (2022). It contains Sentinel-1 radar data in HH and HV polarization, as well as other channels and weather reanalysis data that we are not using here. The target variables are sea ice content (SIC), state of development (SOD), and floe size (FLOE).

The dataset is divided into train and predict data. Targets are available for all splits. We split a validation set with 40 ice charts off the train set, stratified for issuing center, year, and month of acquisition. We remove the ice-free charts to counter class imbalance and use the remaining 440 ice charts for training.

Model

We use the TerraTorch package to train the models. We train or fine-tune three different models:

1. TerraMind-base, pre-trained
2. Terramind-base-TiM with added modality "Digital Elevation Model", pre-trained
3. UNet following (Stokholm 2022), trained from scratch.

Each model architecture is trained for each of the three targets. We use the nersc_sar_primary and nersc_sar_secondary input features only. Our patch size is 768 in training. We sample the ice charts according to the number of valid pixels they contain, aiming for an average of 20 samples per chart per training epoch. We use median frequency weighted class weights to account for the severe class imbalance and use weighted cross entropy.

We trained all models on the same HPC system, using 1 node with 4 A100 GPUs. Terramind-base was fine-tuned within 5.5 hours, Terramind-TiM took 6.25 hours, and UNet 9 hours.

Results

Scores

In line with (Stokholm, 2024), we evaluate sea ice content based on the R2 score and both state of development and floe size based on the class-weighted F1 score. The combined score is obtained by weighting SIC and SOD scores with 2/5 and the FLOE score with 1/5. For comparison, we include the best-performing model from the AutoICE challenge (Stokholm, 2024). The best model per category is highlighted.

Note that Terramind-TiM is misaligned on sea ice concentration, investigation reveals that it often interprets closed sea ice as open water.

Terramind-base outperforms the other models also on the multi-class Jaccard index, measuring intersection over union. Thus, the fine-tuned foundation model can produce visually realistic sea ice charts.

Selected ice charts

We present a selected ice charts from the test set. Visually, the Terramind predictions look most realistic, the UNet shows artefacts.

Generated modalities

We explore the TerraMind any-to-any generation with a random patch from the Arctic Sea Ice dataset. The generated modalities strike us as unrealistic: The scenery is very green, which we would not expect from arctic sea ice, and the digital elevation model implies an average terrain elevation of 70 meters.

Conclusion

Terramind can be fine-tuned to create automated sea ice charts. It outperforms an unsophisticated UNet, but cannot compete with the specialised architecture that won the AutoICE challenge. It might be worth to investigate their data processing and training procedure to see if Terramind can be further improved.

Notably, the time to solution is short with Terramind. This holds true both for the model training time, which is cut in half by using the foundation model, but also for the developer's time. With minimal effort, we were able to achieve already satisfying results.

Terramind-TiM did not perform better than the default TerraMind. This can be attributed to the fact that the generated DEM modality is unrealistic and thus cannot help with automated sea ice mapping.

Acknowledgements

This work used resources of the Deutsches Klimarechenzentrum (DKRZ) granted by its Scientific Steering Committee (WLA) under project ID AIM. The training runs consumed a total of 78 kWh.

Reproducibility

• Ready-to-train Arctic Sea Ice dataset: DTU
• Code for fine-tuning TerraMind: Github

References

• Technical report: https://wandb.ai/crnld-helmholtz-zentrum-hereon/arctic-sea-ice/reports/Arctic-Sea-Ice-Prediction--VmlldzoxMzMzODYwOQ
• Stokholm et al, AI4SeaIce: Toward Solving Ambiguous SAR Textures in Convolutional Neural Networks for Automatic Sea Ice Concentration Charting, IEEE (2022), https://ieeexplore.ieee.org/abstract/document/9705586
• Stokholm et al, The AutoICE challenge, The Cryosphere (2024), https://tc.copernicus.org/articles/18/3471/2024/
• Jakubik et al, TerraMind: Large-Scale Generative Multimodality for Earth Observation, arxiv (2025), https://arxiv.org/abs/2504.11171