Autonomous multisensory enhancement of a visual neuroprosthesis for navigation: technical proof-of-concept with simulated prosthetic vision and single-subject case study of a visual prosthesis user

Objective. Visual impairments create significant challenges for navigation. This work explored the potential for an autonomous navigation aid with multisensory feedback to improve navigational performance for users of visual neuroprostheses. Approach. An autonomous navigation system was developed that maps the environment in real time and provides guidance using combinations of prosthetic vision, haptic, and auditory cues. Navigational performance was evaluated in 20 sighted participants using simulated prosthetic vision and in a single-subject case study of an Argus II visual neuroprosthesis user. Participants completed three tasks: navigate to destination, obstacle field traversal, and relative distance judgment. Multiple sensory feedback configurations incorporating visual, haptic, and auditory cues were compared. Performance metrics included collision rate, distance traveled, task completion time, navigation success rate, and accuracy of relative distance judgments. Main results. Performance differences across sensory configurations were most pronounced in navigation success and collision rates. Haptic plus audio feedback was highly effective for navigation tasks, enabling successful navigation in nearly all trials involving haptic guidance. Argus vision (AV) alone was inadequate for navigation. Depth vision (DV) provided modest improvements over AV but did not enhance performance beyond haptic and audio guidance when combined. Wide field-of-view DV yielded additional benefits, particularly for obstacle field traversal where its performance exceeded other modes. Adding AV to haptic and audio also provided no benefit and, in some cases, degraded performance. Performance trends for the Argus user were generally comparable to those of sighted participants across sensory modes, with the exception of the relative distance judgment task, in which the Argus user demonstrated better performance. Among sighted participants, increased field of view and resolution independently improved relative distance judgment accuracy. Significance. These findings demonstrate the potential of multimodal feedback systems to improve navigation for prosthetic vision users. (ClinicalTrials.gov NCT04359108).