Like most folks, Kate Williams uses her cellphone for digital pictures. She can’t resist snapping away during all the activities her five grandchildren enjoy. But choosing which pictures to share poses a challenge–not only because each image captures a special moment, but also because she is blind.

Although text-to-speech programs have opened up cellphone use to blind people, few tools exist to describe to Williams the images she has stored. “One of the reasons I don’t take more pictures is because of the difficulty of organizing them,” said Williams, who leads an employment training program at LightHouse for the Blind and Visually Impaired, a non-profit advocacy group in San Francisco. She relies on her sighted friends and family to help browse her pictures and share them with others.

But at the Baskin School of Engineering, computer science graduate student Dustin Adams plans to change that with a prototype mobile app, called VizSnap. His project is just one of the assistive technologies that UC Santa Cruz researchers, led by Professors Sri Kurniawan and Roberto Manduchi, are creating to help blind and visually impaired people with everyday tasks.

The winning shot

World health officials estimate that some 285 million people are blind or have low vision. Most blind people, such as Williams, still retain some light perception. She uses her sense of shadows to aim her camera at nearby blobs and hopes for the best. For Williams, taking pictures is more about recording a memory than getting the perfect shot. “Blind people like to take photos for all the same reasons that sighted people do,” Adams said. “They want to capture a happy moment in time, or share it with their friends and family.”

Since 2011, Adams has been part of UCSC’s Interactive Systems for Individuals with Special Needs Lab, founded by Kurniawan, a professor of computational media and computer engineering. Work on the photo-organizing app began in 2012, while Adams did a summer research internship at IBM, in Tokyo. There, he was introduced to the concept of blind photography–and the need for better tools–by his mentor Chieko Asakawa, a leader in accessibility research, who is also blind.

When Adams returned to UCSC, at the end of the summer, he continued the project with Kurniawan. Now, he’s preparing for a three-month user study of the prototype iPhone app. His award-winning design works by collecting ambient sound when the phone’s camera is being aimed, and then allows the user to record a voice message after taking a shot. Both sound files are linked to the image, and can be replayed on demand as a user scrolls through the camera roll. The app also enables users to play spoken information about the date, time, and GPS location associated with each picture.

“It would be great if that could be an app,” said Bryan Bashin, executive director of LightHouse, who believes that a photo-organizing tool would serve a very real need in the blind community.

Listening for cues

Choosing a project that benefits visually impaired people isn’t always easy for sighted engineers, said Manduchi, who runs UCSC’s Computer Vision Lab. Ten years ago, when he first started studying assistive technology, the professor of computer engineering dove into the development of a laser-guided “virtual cane” without first consulting blind users. His hand-held device, which detected distances between the user and objects in the environment, was an engineering success. But as a piece of assistive technology, it was fatally flawed. Manduchi discovered that people preferred their traditional canes because the low-tech design was cheap, light, durable, foldable, and never ran out of power.

“I learned a lot,” said Manduchi, with a laugh. “I learned that I needed to make more blind friends, and talk to orientation and mobility professionals.” One of Manduchi’s latest projects–which already has input from the visually impaired community–is an effort to improve public transit navigation. Still in its early stages, the pilot study puts wireless transmitters on some buses and bus stops on the UCSC campus. Using Wi-Fi signals, the system communicates with users’ mobile phones, and alerts people with a spoken message when they are near their bus stop, and again when their bus arrives. Users can also query for a list of stops as they proceed along the route, circumventing hard-to-hear announcements on bus speaker. The app even warns travelers when they should prepare to disembark.

Unlike other existing bus travel apps, Manduchi’s design doesn’t use GPS to track a rider’s progress. Instead, it picks up on pings between buses and stations. That’s important because GPS can be inaccurate or unavailable in some areas, Manduchi noted. And, using GPS continuously on a long trip can quickly drain a cellphone’s battery. Early trials of Manduchi’s Wi-Fi system have received positive reviews from blind users. But, he emphasized that much more testing is needed. If the system passes its engineering and usability tests, it will still require infrastructure investments to work in the real world, including outfitting bus fleets and stations with new transmitters.

A sense of style

In the meantime, Lourdes Morales, another graduate student in Kurniawan’s lab, hopes to address a more immediate need for many blind people: getting a job. According to the American Foundation for the Blind, only about 30 percent of working-age adults who are blind or have limited vision are employed in the United States.

Morales, who worked hard to reach graduate school despite her own severe visual impairment, was drawn to study technology that supports the professional advancement of blind people. “I liked the goal, to give blind people the same opportunities to develop in academic and professional environments,” she said.

For the past two years, Morales has been building a tool to help blind people create documents that are visually appealing for sighted readers. Most software programs that read text aloud will, by default, ignore formatting and style, such as capitalization, indentation, and font changes. Without that information, it’s hard for blind people to learn sighted style conventions, and find formatting mistakes in their own work. That can be a big deal–especially on resumes.

“Having a bullet point out of place on that resume indicates to a recruiter that this person may not be detail oriented,” said Kate Williams. She advises her students in the LightHouse employment training program to run their resumes by a sighted friend before submitting them to potential employers. The stakes are high, and even small errors can be costly.

One of the most popular screen readers, Job Access With Speech (JAWS), can be configured to read back information about a document’s format and style. But to hear all of the document’s features, the program takes users through a full playback of the text, interspersed with formatting notations. This mode can interfere with reading for meaning, or introduce tedious delays when users want to quickly assess a document’s overall appearance.

Furthermore, JAWS isn’t easy to learn. “It does require a fair amount of training,” said Tim Elder, president of the San Francisco chapter of the National Federation of the Blind.

Morales wants to help blind people do better. “When they rely on sighted people, they have to wait for them. And help might not always be available, or may cost money,” she said. “Their productivity is being affected.”

Her program works as an add-on to Microsoft Word, the most widely used word processor among the blind people Morales surveyed. By running the tool along with a screen reader, users can hear a report summarizing only the style and formatting features of a Word document, such as the number of times text is italicized, or the number of different font sizes that appear. Then, the tool offers users the chance to review each feature in more detail, reporting one-by-one the location of each instance of underlined text, for example. In each instance, users have the option to either undo special formatting, or move on.

Initial tests of the program have returned some positive feedback and suggestions for improvements, along with a few unexpected results. “I came into the project assuming that blind people just needed a tool to make formatting easier,” said Morales. But some users, after appreciating for the first time the wide variety of format and style choices available, wanted to learn more about the reasons sighted people apply such features as bold type or italics. In response, she is now working on modifications to the software that could help it double as an educational program for blind people.

Ultimately, Morales and her UCSC colleagues aim to create technologies that are tuned to the needs of the blind community. Whether in the realms of professional advancement, mobility, or personal enjoyment, the researchers hope that their tools can help blind people move forward in their lives with greater independence and confidence.

Related Links