The technologies regarding capturing motion and visualizing data enabled new possibilities describing complex systems to a broader audience. We used the latest methods of motion analysis through Machine Learning / Artificial Intelligence and visualization in Virtual and Augmented Reality (VR/AR) in order to analyze and explain the way the Hof Symphony Orchestra works. Therefore, we showed the diverse aspects from rehearsal to performance to a broad audience via interactive immersive Extended Reality experiences. In this paper we are describing the process and the implications for tracking motion and gestures in cultural places. 

Understanding acceleration forces and making progress in learning Skateboarding is a process of trial and error. In our paper we are describing our preliminary experiments for describing the complex interactions while pushing for speed in ramps and pump tracks. Therefore, we capture and visualize the body movement, the joint relations from hip to ankle and the resulting forces by joining iner- tial sensors on the skateboard and camera-based machine learning pose estimation of the athlete.

In our society, displays are becoming increasingly prevalent. While it is nearly inconceivable to imagine a daily life without screens, scientific research indicates that our mental well-being is nega- tively affected by the increasing use and time spent on screens. The "HomeGrid" project has been developed as a smart home concept, aiming to explore how traditional screens in private environments can be reduced while still effectively conveying information in an intuitive manner. To achieve this, two main approaches were em- ployed. Firstly, experimentation was conducted on the capture and communication of artificial light, and secondly, precise monitoring and visualization of indoor air quality were explored. These factors are fundamental indicators of both our mental and physical well- being, as high air quality, for example, enhances concentration, and light serves as a crucial regulator of our circadian rhythm.

The project Symotiv’s (http://symotiv.de) goal is to describe the complex function of a symphony through an interactive experience of the orchestra’s digital twin. We are capturing the musicians, their instruments and their sounds with machine learning based 3D human pose estimation and visualize them in Virtual Reality in a three-dimensional, virtual concert hall. Our audience steps into the role of the conductor and experience the scene and the sound from a perspective within the orchestra. This opens the possibility of experiencing and understanding what is happening from new spatial, but above all sonic perspectives.

In this paper we are describing our preliminary studies and proto- types for evaluating a surfer’s performance on a stationary wave. We are briefly describing the sport’s environment and development and the current state-of- the-art of machine learning based single camera tracking approaches we have evaluated and applied. The main part of the paper deals with the first implemen- tation, the tracking and the results of the evaluation of the movements of two surfers. We are closing with our lessons learned and our next steps.

The aim of the project Symotiv (http://symotiv.de) is to introduce people to classical music through interaction with a virtual orchestra. With the help of Virtual Reality, it does not only enable to immerse oneself in a three-dimensional, virtual concert hall, but also to experience images and sound from the perspective of a musician. This opened the possibility of experiencing and understanding what is happening from new spatial, but above all sonic perspectives. Therefore, we developed a workflow to capture the motions and sound of 50 musicians and the conductor by using a camera-based Machine Learning approach.

In this work we are describing our preliminary experimental evaluation of open-source machine learning based pose estimation solutions for tracking a surfer’s skeleton on a stationary wave. We are briefly describing the sport’s environment and development and the current state-of-the-art of machine learning based single camera 2D and 3D pose estimation solutions. The main part of the paper deals with the experiment’s setup and the interpretation of the resulting data of the movements of one surfer. We are closing with our lessons learned and our next steps.