Speaker
Description
Modern sky surveys generate images of galaxies at a scale that renders manual classification as an unscalable task, motivating the development of automated deep learning pipelines. This study presents a systematic comparison of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) for automated galaxy morphology classification on a curated subset of the Galaxy Zoo 2 dataset. To address the severe class imbalance inherent in the Galaxy Zoo 2 dataset, the Hierarchical Data Learning with Weighted Sampling and Label Smoothing (HIWL) framework is adopted, which decomposes the seven-class classification problem into a two-stage hierarchical pipeline. Stage 1 performs coarse morphological grouping, with Stage 2 conducting fine-grained binary classification within each group. Weighted sampling and label smoothing are applied throughout to mitigate the effect of class imbalance and disagreement among volunteer annotations.
Linear probing is conducted on a candidate pool of ten architectures -ResNet-50, ResNet-101, ResNet-152, YOLOv8m-cls, YOLO11m-cls, ViT-Base/16, DINO, DINOv2, ViT-MAE, and RF-DETR - to obtain a baseline measure of representational transferability to galaxy morphology. Based on the linear probing performance, a subset of architectures is selected for full end-to-end fine-tuning, with additional architectures included to ensure more comprehensive evaluation across pretraining methods and architectural families. Layer-wise learning rate decay (LLRD) is applied to all ViT-based architectures during fine-tuning.
Findings reveal a consistent performance ceiling of approximately 88% overall accuracy and 81% Macro F1 across all fine-tuned architectures, suggesting the bottleneck is not architectural capacity but rather label noise and inter-annotator disagreement inherent to the Galaxy Zoo 2 crowdsourced annotations. This hypothesis is investigated through dataset filtering based on annotator agreement thresholds and an analysis using a new data partition, both of which provide indicative evidence of annotation-driven limits on performance.
Despite this ceiling, ViT-based models, particularly DINOv2, demonstrate a slight but consistent advantage over CNN-based counterparts, achieving the best overall performance of 88.56% accuracy and 81.07% Macro F1.