Project CodeTalk

Visually impaired (VI) developers today cannot benefit from many of the productivity-enhancing features of IDEs like Visual Studio. The fundamental reason is that by definition visual IDEs present a diverse set of static and dynamic information about the software development process using many visual cues and tools, and by effective use of graphical user interface elements like windows and icons and menus. We have identified through user studies four broad classes of barriers to the effective use of such IDEs. These are i) discoverability, ii) navigability, iii) alertability and iv) glanceability.

CodeTalk is our evolving vehicle (implemented as a plugin to VisualStudio, but techniques broadly applicable to any visual IDE) that systematically addresses these barriers.

We have developed four tools that in combination are proving to be very powerful. These are:

1. Summarizing the code in a very accessible way (summary tree view, functions list, etc..)
2. Spatialized AUI: Converting many of the GUI elements of Visual studio into a spatialized audio user interface (AUI), thus enhancing the bandwidth between the system and the developer. This soundscape surrounding the user’s head utilize both speech and non-speech audio to complement the screen reader voice.
3. TalkPoints: these are programmer introduced statements in the code that speak out information appropriately. There are at present three types of Talkpoints, speech Talkpoints, tone Talkpoints and expression Talkpoints.
4. DrawHear (DrawVis): These are techniques to present sketches and diagrams like UML diagrams as audio. This is distinct from the audible accessibility tags specified by W3C for images.