Improving SAM 2 for Agricultural Land Segmentation through Fine-Tuning, Point Prompt Augmentation, and Negative Prompt Calibration

Abstract

Background: The Segment Anything Model 2 (SAM 2) represents a state-of-the-art foundation model for object segmentation; however, its application to satellite-based agricultural mapping faces significant challenges. Standard SAM 2 architectures often struggle with the spectral ambiguity of fragmented tropical landscapes and the domain gap inherent in remote sensing imagery. Furthermore, the model’s interactive nature requires precise spatial guidance, making it sensitive to both the location and density of input prompts, which limits its scalability for automated large-scale monitoring. Objective: This study aims to (1) analyze the impact of domain-specific fine-tuning combined with automated Point Prompt Augmentation (PPA) and Negative Prompt Calibration (NPC) on segmentation accuracy; (2) evaluate the performance of four SAM 2 variants (Tiny, Small, Base+, and Large) to identify the optimal backbone for agricultural tasks; and (3) determine the optimal prompt density for both positive and negative points. Methods: The SAM 2 variants were fine-tuned using the LoveDA satellite dataset. Evaluation was conducted through an automated pipeline comparing two initialization strategies: Largest Agricultural Area (LAA) Centroid and random placement. The study implemented PPA to strategically increase positive prompt density and NPC to suppress "mask leakage" into irrigation infrastructure. Performance was quantified using mean Intersection over Union (mIoU) and Jaccard & F-measure (J&F) metrics. Results: The Small variant emerged as the superior backbone, achieving a peak mIoU of 0.7255 and J&F of 0.7734, representing a significant improvement over the pretrained baseline. The results indicate that the LAA Centroid strategy provides a more stable spatial anchor, while the integration of three positive and three negative points optimized the boundary alignment. The Small variant maintained a high computational efficiency with an average inference time of 2.62 minutes. Conclusion: Domain-specific fine-tuning, coupled with the proposed PPA and NPC frameworks, successfully mitigates the limitations of SAM 2 in agricultural remote sensing. This research provides a robust methodology for automated, high-precision land segmentation, bridging the gap between foundation models and specialized geographic information systems. Keywords: Agriculture Segmentation, Satellite Imagery, Segment Anything Model 2, Fine-tuning, Point Prompt Augmentation, Negative Prompt Calibration