Master Thesis - Semantic Segmentation with Segment Anything Model (SAM)
In the Professional Service Robots - Outdoor research group we develop autonomous, mobile robots for a variety of outdoor applications, such as agriculture and forestry. The focus is on the development of an autonomous outdoor navigation solution as well as the hardware of the robots.
Environmental perception is a key component for interpreting the environment of autonomous robots. Camera-based solutions have been developed as an approach to visual perception. To this end, semantic segmentation has been researched and improved over the years to effectively understand the scene so that robots can safely navigate their environment. More recently, the Segment Anything Model (SAM) has significantly influenced research in the field of semantic segmentation and is now widely used for annotation.
What you will do
The aim of this master thesis is to utilise SAM for semantic segmentation in agricultural environments. Even though SAM was trained on a large dataset, the training data does not cover all types of images. Due to the ever-changing scenarios, SAM does not adequately segment certain regions of interest with intricate details and fine structures, especially in an agricultural environment. The reason for this is that SAM's training dataset consists mainly of natural images with clear boundary information. Agricultural images, on the other hand, are more complex and are characterised by inconsistent lighting and shadows and have different types of complex variations in the scene, with soil, rocks, crops and other artefacts with unclear boundaries being more commonly observed. All this makes segmentation more difficult. In this work, SAM will therefore be extended to allow segmentation in an agricultural environment. Additionally (optionally), this work can be extended to develop a tool to help annotate the Fraunhofer IPA agricultural field dataset.
What you bring to the table
What you can expect
We also offer the possibility of direct entry for excellent graduates.
We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability.
With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.
If you are interested, please submit a short letter of motivation on your personal favourite topic listed above, CV, current grade transcript to kevin.bregler@ipa.fraunhofer.de
Fraunhofer Institute for Manufacturing Engineering and Automation IPA
Requisition Number: 71233 Application Deadline:
Job Segment:
Test Engineer, Testing, Manufacturing Engineer, Electrical Engineering, Engineering, Bilingual, Technology