Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


HoChih-Yuan2004 sf.pdf (5.3 MB)

ETD Abstract Container

Abstract Header

Multimodal Interfaces: Supporting Synchronous Distributed Multi-Agent Communication and Coordination in Complex Domains

Ho, Chih-Yuan
http://rave.ohiolink.edu/etdc/view?acc_num=osu1380545941

Abstract Details

2004, Master of Science, Ohio State University, Industrial and Systems Engineering.
Operators in many complex event-driven domains, such as the military, need to coordinate their goals and activities with numerous other co-located and distributed human and machine agents. One promising way to support effective synchronous coordination is through the introduction of adaptive multimodal interfaces that afford increased bandwidth, timesharing, complementarity, redundancy, and modality substitution. To inform the design of such interfaces, we explored people's natural tendencies for modality usage in the context of simulated battlefield operations. Three groups of three ROTC cadets/officers each completed a set of 30-minute scenarios over several 3-hour sessions. Participants were either co-located or distributed, and the scenarios differed with regard to the availability of radio communication (available, not available), the amount of coordination required by the scenario (low, high), and the tempo of operations (low, high). Throughout the scenarios, participants could communicate with each other via visual (text message, drawing/referring), auditory (two-way radio, face-to-face conversation), and tactile (tactile messages) channels. Also, computer-human interaction was available in the form of user-selected visual, auditory, and tactile alerts. Overall, our findings show that two-way radio was the primary medium for distributed communication because of participants' familiarity with this channel, its ease of use, and because it affords fast exchange of information. Participants used all other modalities as well but to a lesser extent. Text messages were used mostly to substitute for the auditory channel when radio communication was not available. Tactile signals were considered useful for attention capture and conveying pre-defined messages but should be simple and reserved for critical events. The cadets and officers did not always interact multimodally. Multimodal interaction occurred almost exclusively in the context of spatial tasks and thus often involved drawing or referring on a shared visual map. The majority of modality combinations occurred in sequence; they were employed either by individuals or jointly by group members to support disambiguation and common ground. Concurrent multimodal information presentation was observed only for participant-selected computer alerts to achieve redundancy gains. Modality switching was employed for the purpose of recovering from communication breakdowns. Overall, our findings improve our understanding of multimodal interaction and show that multimodal interfaces represent an effective means of communication and coordination that users can employ and exploit rather effectively in the absence of predefined rules or procedures. Modality usage strongly depends on the type of information to be exchanged, the user's tasks, scenario context, group dynamics, and interface management demands. This suggests that future multimodal interfaces should be easily adaptable and that interface management requirements need to be minimized as more modalities are introduced.
Nadine B. Sarter, Dr. (Advisor)
95 p.
Communication; Industrial Engineering

Recommended Citations

Citations

  • Ho, C.-Y. (2004). Multimodal Interfaces: Supporting Synchronous Distributed Multi-Agent Communication and Coordination in Complex Domains [Master's thesis, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1380545941

    APA Style (7th edition)

  • Ho, Chih-Yuan. Multimodal Interfaces: Supporting Synchronous Distributed Multi-Agent Communication and Coordination in Complex Domains. 2004. Ohio State University, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1380545941.

    MLA Style (8th edition)

  • Ho, Chih-Yuan. "Multimodal Interfaces: Supporting Synchronous Distributed Multi-Agent Communication and Coordination in Complex Domains." Master's thesis, Ohio State University, 2004. http://rave.ohiolink.edu/etdc/view?acc_num=osu1380545941

    Chicago Manual of Style (17th edition)

osu1380545941
192
© 2004, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.