Can I help in the air?

This is another idea that came from writing fiction. We are submitting this one to the Meta Community Forums Ideas Board. We’ll let you know if we get any nibbles or kudos as they call them on this board.  

Feature Idea: Emergency Flight Assistance Through AR

What if a passenger suddenly has to land the plane?

Imagine a passenger aboard a small aircraft.

The pilot suffers a medical emergency and becomes incapacitated. The passenger has no formal flight training.

The aircraft is still functioning. Its avionics are working. It may have an autopilot or even a certified emergency landing system. Air traffic control is available and capable of providing expert assistance.

The passenger's problem isn't necessarily a lack of information.

The problem is that they don't know how to translate that information into actions.

Could Meta Ray-Ban Display help bridge that gap?

A passenger might say:

“The pilot is unconscious. I don't know how to fly. Help me.”

Rather than attempting to teach the passenger to fly, the glasses could help connect them with the appropriate expert systems and translate those instructions into simple visual guidance in their field of view.

If the aircraft has emergency Autoland

Some modern aircraft are already equipped with certified emergency landing systems designed specifically for pilot incapacitation.

In an equipped aircraft, the glasses wouldn't need to fly the airplane.

They could help the passenger identify what already exists.

The display might recognize the relevant cockpit control and highlight it:

EMERGENCY AUTOLAND
↓
PRESS THIS BUTTON

Once activated:

AUTOLAND ACTIVE ✓
AIRCRAFT STABLE ✓
ATC NOTIFIED ✓

DO NOT TOUCH FLIGHT CONTROLS

The aircraft's certified systems remain responsible for navigation and landing.

The glasses simply make those systems understandable to the person who suddenly needs them.

What about aircraft without Autoland?

This may be an even broader use case.

Many general-aviation aircraft don't have autonomous landing capability, but may have autopilot, GPS navigation, radios and other useful systems.

Air traffic controllers can provide expert assistance during an emergency.

AR could make their instructions considerably easier for an inexperienced passenger to follow.

ATC says:

“Locate the autopilot heading control.”

Instead of searching an unfamiliar instrument panel, the passenger sees the appropriate control highlighted:

HEADING CONTROL
↓

ATC says:

“Set heading two seven zero.”

The display responds:

TURN THIS →

270°

270° ✓ STOP

The glasses aren't deciding how to fly the airplane.

ATC provides the expertise. The aircraft provides authoritative flight information. The glasses translate those instructions into the physical cockpit.

Use authoritative sources

For a safety-critical application, consumer AI shouldn't independently invent flight procedures.

A serious implementation could instead be designed around authoritative sources:

Aircraft avionics
Aircraft position, altitude, heading, system status and other flight information.

Certified aircraft automation
Autopilot, emergency descent, Autoland or other installed systems.

Air traffic control
Expert human guidance during the emergency.

Approved aircraft procedures
Checklists and procedures appropriate to the specific aircraft.

Meta Ray-Ban Display
Visual translation of that information into simple, understandable actions.

The glasses become the interface between expertise and the person who needs it.

Show less, not more

An inexperienced passenger doesn't need another cockpit full of information floating in front of them.

They need the opposite.

The emergency interface could deliberately reduce information to what matters now:

AIRCRAFT STABLE ✓

AUTOPILOT ENGAGED ✓

ATC CONNECTED ✓

Then one action:

PRESS THIS ↓

or:

TURN THIS →

STOP ✓

If no action is required:

DO NOT TOUCH FLIGHT CONTROLS

The system could reveal additional information only when necessary.

Know when not to help

An equally important capability would be recognizing when the system cannot safely provide an instruction.

If an authoritative procedure cannot be established, the glasses shouldn't improvise.

Instead:

SAFE PROCEDURE CANNOT BE VERIFIED

KEEP AIRCRAFT STABLE

CONNECTING TO AIR TRAFFIC CONTROL

Sometimes the most useful thing an AI system can know is that it isn't the expert.

Beyond an incapacitated pilot

The same capability could help in less dramatic situations.

A passenger could assist a trained pilot during an emergency by locating equipment or checklist items.

A pilot experiencing extreme workload could receive simplified checklist information.

A qualified pilot flying an unfamiliar aircraft could locate a particular control or procedure.

After an emergency landing, the system could transition to evacuation guidance.

And the underlying idea extends beyond aviation.

On a boat:

SAFE COURSE →

In an aircraft:

PRESS THIS ↓

During a medical emergency:

AED LOCATED →

During a building emergency:

SAFE EXIT →

Around unfamiliar equipment:

EMERGENCY SHUTOFF ↓

Emergency Expert Translation

The larger idea is simple:

What happens when the expert isn't available, but an ordinary person is?

AR glasses have an unusual advantage in that situation. They can potentially see what the person sees and place information directly onto the physical thing the person needs to interact with.

Instead of teaching someone an expert profession during an emergency:

Identify the situation.

Connect to authoritative expertise.

Translate that expertise into the person's field of view.

Show one safe action at a time.

The most useful emergency AR interface may not be one that shows us more information.

It may be one that shows us exactly what we need next.

Thank you for considering the idea.

I remain,

Abi Post

www.abipost.com