Flight Deck and HUD/HMD

Augmented Reality (AR)

Augmented Reality (AR) systems allow users to overlay 2D and 3D virtual contents in the real world. This ability of AR systems can help pilots view the flight information along with the Out-the-Window (OTW) environment, or help technicians and engineers load documents, CAD models, or model-based animations to help perform assembly or maintenance tasks in the field. At the Ryerson MIMS Lab, we are exploring various AR applications including virtual flight training, advanced Head-Up Display (HUD) / Head-Mounted Display (HMD) system, flight instrument configuration, cabin interior configuration, and aircraft assembly and maintenance support.

Airbus A320-214 start-up and takeoff training guide on Microsoft Flight Simulator X

One of the researches being done at Ryerson MIMS lab includes exploration of Augmented Reality (AR) in development of engineering sectors using Microsoft Hololens. Hololens is one of the latest technologies that makes giant holograms on top of the real world while being able to control holographic spaces. The objective of MIMS lab's research is the development of an augmented reality immersive simulation for virtual flight deck design and evaluation (Holodeck), allowing specialists to prototype, design, develop, assess and operate user interfaces in an AR immersive environment. The holodeck will be cost effective to build and operate compared to the engineering mock-up and full flight simulator devices.

In addition, MIMS lab has developed "Flight Simulation Start-up Training guide for Airbus A320 using Microsoft HoloLens". Sample video is available via this link.

Dual-mode HMD system for DCS World Flight Simulator

This demo was created to show the connection capability between Microsoft’s HoloLens and a PC running a flight simulator. In the video the connection is made with DCS World, however this could have been any flight simulator that allows the export of live flight data. The 3D HMD symbology seen in the video is an attempt at converting traditional 2D based flight instrument data into a more volumetric 3D flight instrument data, as to better harness the capabilities of the next generation Holographic displays. The holographic elements were created and driven in Unity3D, and the visuals were remoted to the HoloLens, using the Holographic Remoting Tool.

Virtual Reality (VR)

Aircraft flight deck prototype visualization in VR

The flight deck seen in the video above is based on the CS300 aircraft by Bombardier (now the Airbus A220). It is not a 1:1 replication as the model was based on images found online. Most of the scaling and sizing of the flight deck was done in VR using Trimble’s Sketchup. Once the Model was complete enough the CAD model was imported into Unity3D to add interactive capabilities. The HMD used in this demo was the HTC Vive, and the controller used to do that hand and finger tracking was the Leap Motion Controller mounted on the headset. The addition of the Leap Motion controller provided the capability of natural interaction with the different elements in the flight deck, like the Control stick Throttle stick and various buttons, switches and knobs.

Oculus Virtual Reality Flight Simulation: UAV model

1. Creator - 3D Model

PRESAGIS Creator is the software to create optimized 3D models for real-time simulation, and allows content developers to work natively with OpenFlight data to output highly optimized models with advanced surface materials and multiple levels-of-detail. MIMS lab created a RAD UAV by using PRESAGIS Creator and applied it to PRESAGIS Stage.

2. VAPS XT - HMD design

Using PRESAGIS VAPS XT program, MIMS lab designed pilot's Head-Mounted display (HMD) for diplaying that presents flight data.

3. FlightSIM Integration - Oculus Simulation

Oculus simulation was implemented with UAV Ground Control Station (GCS) panels.

Go back to Research