Tracking climbers using stereo cameras

The rock climbing community in South Tyrol is an important touristic and cultural factor. Not only locals but also many tourists come here to take practice in both indoor and outdoor climbing. With the project SALSA, we are trying to analyze the activity of climbers. We want to identify, if a person is falling, climbing, and if the climber is taking a break. Furthermore, we want that our application can be used to measure the amount of time a climber needs to finish a certain route. To solve this problem, we used the stereo camera ZED 2, by the manufacturer Stereo Labs. With its body tracking, object detection, and depth-sensing features we were able to measure the relative coordinates and speed of a body in front of the camera. and so to track the movement of a climber. In this talk, I present the first prototype running on the Jetson Nano, a portable computer with a GPU necessary to process the images obtained through the camera.