Eugenio Diaz, BSEE/BSCE

In 2013, while working at the University of Central Florida Institute for Simulation and Training, I was tasked to come up with a methodology to characterize and measure the end-to-end latency of the virtual reality Dismounted Soldier Training System, or DSTS; and design and build the test equipment to measure the actual end-to-end latency, as well as sub-component latency when possible. The initial notional diagram is shown on Fig. 1, while due to resource and time constrains we simplified it as shown on Fig. 2.

In simplified terms, the methodology to measure the end-to-end latency consisted of two optical sensors, one to detect and register the moment starting of the physical movement (of the helmet mounted inertial sensor), and the other to detect the corresponding virtual motion on the screen. Both sensor were connected to an oscilloscope to measure the time between the two events. A sturdy motion platform was designed and built to mount the IMU sensor from the helmet, so that it’s physical impulse movement could be restricted to one axis and performed in a repeatable manner. The platform was initially designed with a motor and actuator to automate the movement, but this was later deemed necessary, since all we needed to detect was an edge signaling the start of movement. This sensor consisted of a Schmitt Trigger Photo-Interruper. A thin wire was attached to the motion platform to trigger the sensor. We were also able to Jerry Rig a foam board piece to the motion platform that allowed us to use a regular mouse for input instead of the IMU sensor, which gave us a idea of the IMU internal latency as compared to the mouse.

The HMD sensor was a bit more complicated, as it needed to detect pixel changes on the screen. I used a photo-transistor based detection circuit to detect a black bar edge on the screen. I tried to make it work through the lenses of the HMD as we were not allowed to dismantle it to access the raw display inside, however this proved very difficult, and after considering the HMD display specs, we decided a regular LCD monitor with similar specs would be suitable surrogate that would not affect the latency measurement in a significant manner. I designed an enclosure for the photo-transistor to limit it’s detection area to a very thin slit, and fine tuned the detection circuit to produce sharp edge detection on thin vertical bars on the screen. I coupled this with a graded virtual cylinder centered on the player in the virtual environment such that when the helmet IMU sensor was rotated, the player’s view will rotate accordingly in side the virtual cylinder. We developed and refined this technique until we were able to obtain repeatable and reliable measurements. This technique proved very useful since it could be implemented on different virtual environments and thus allowed us to measure the latency of of different game engines; we tested VBS3 and Unity with similar terrain complexity, and varying entity counts, fidelity settings, etc. I used the Open Data Kit to build a data collection Android app that allowed for the easy collection of measurements including parameter and photos, while storing them on the google engine back-end. We collected hundreds of measurements on varying environment and settings and performed a statistical analysis to determine standard deviations and correlations, etc. I was writing a formal paper to publish the results of this research, however it could not be finished and published due to a lack of funding, and no time/resource to finish it on my personal time. You can read the abstract with the results here.