City:  Stuttgart
Date:  Apr 11, 2024

Master Thesis - Semantic Segmentation with Segment Anything Model (SAM)

The Fraunhofer-Gesellschaft ( currently operates 76 institutes and research institutions throughout Germany and is the world’s leading applied research organization. Around 30 800 employees work with an annual research budget of 3.0 billion euros. 

In the Professional Service Robots - Outdoor research group we develop autonomous, mobile robots for a variety of outdoor applications, such as agriculture and forestry. The focus is on the development of an autonomous outdoor navigation solution as well as the hardware of the robots.
Environmental perception is a key component for interpreting the environment of autonomous robots. Camera-based solutions have been developed as an approach to visual perception.  To this end, semantic segmentation has been researched and improved over the years to effectively understand the scene so that robots can safely navigate their environment. More recently, the Segment Anything Model (SAM) has significantly influenced research in the field of semantic segmentation and is now widely used for annotation.


What you will do

The aim of this master thesis is to utilise SAM for semantic segmentation in agricultural environments. Even though SAM was trained on a large dataset, the training data does not cover all types of images. Due to the ever-changing scenarios, SAM does not adequately segment certain regions of interest with intricate details and fine structures, especially in an agricultural environment. The reason for this is that SAM's training dataset consists mainly of natural images with clear boundary information. Agricultural images, on the other hand, are more complex and are characterised by inconsistent lighting and shadows and have different types of complex variations in the scene, with soil, rocks, crops and other artefacts with unclear boundaries being more commonly observed. All this makes segmentation more difficult. In this work, SAM will therefore be extended to allow segmentation in an agricultural environment. Additionally (optionally), this work can be extended to develop a tool to help annotate the Fraunhofer IPA agricultural field dataset.


What you bring to the table

  • Background in Computer Science, Software Engineering, Electrical Engineering, Mechatronics or similar 
  • Profound knowledge of C++ /Python 
  • Experience in deep learning frameworks like Keras/Tensorflow/PyTorch
  • Experience with ROS
  • Analytical mindset and experience in algorithm development
  • Enthusiasm for mobile robotics
  • Fluent in English or German 


What you can expect

  • Cutting-edge technology in the field of outdoor mobile robotics 
  • Hands on with our robots on our own test fields in Stuttgart
  • Take on responsibility and freedom to implement your own ideas
  • Work with the best students in their discipline 
  • Familiar atmosphere including Cake Friday


We also offer the possibility of direct entry for excellent graduates.


We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability. 

With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future. 

If you are interested, please submit a short letter of motivation on your personal favourite topic listed above, CV, current grade transcript to 

Fraunhofer Institute for Manufacturing Engineering and Automation IPA 


Requisition Number: 71233                Application Deadline:


Job Segment: Test Engineer, Testing, Manufacturing Engineer, Electrical Engineering, Engineering, Bilingual, Technology