Estimation of the geographical coordinates of objects on the image with multi-task convolutional neural networks

Abstract

Determining GPS coordinates of the objects on the image is exceptionally complex problem. Images often contain enough information such as landmarks, cloud texture, grass type, road signs or architectural features that allow suggesting the location where the photo was taken. Previously, such issue was solved with image search methods. In contrast, the problem is stated as a classification task, subdividing the Earth's surface into geographical cells using a special type of space- filling curve. Thousands of differently scaled geographical cells, used to train the model. In this paper, several deep learning methods that follow the latter approach and take advantage of multitask learning are presented. Taking into account the content of the scene of the image, i.e. inside, outside, wild or urban setting, etc. is proposed. As a result, additional information with different spatial resolutions as well as more specific features for different environments are included in the learning process of the convolutional neural network. Reported metrics demonstrate the effectiveness of our out-of-the-box approach, while using a helper network to combine two datasets combined to spread scene labels on GPS dataset and receive more robust model. This model does not rely on search methods, which require an enormous amount of computational power, and implements a probabilistic approach.