With the incorporation of artificial intelligence into 21st-century machines, the collaboration between humans and machines has become quite complex for real-time applications. The role of a synthetic or artificial assistant in everyday tasks such as setting up reminders, managing calendars, and responding to search queries may not pose a significant risk. However, the penetration of such synthetic assistants in virtually every field has opened a path for a new area called Human Machine Teaming (HMT). When it comes to crucial tasks such as patient treatment and care, defense, and industrial production, the use of non-standardized HMT technologies may pose risk to human lives as well as billions of taxpayer dollars. A thorough literature survey revealed that there are no standardization or benchmarking methods have been established for HMTs. This dissertation hypothesizes that to standardize an HMT, there is an inevitable need to first develop task tailored intelligent systems, customized HMT simulation methods, and measurement techniques. To address these hypothesized needs, this dissertation presents new design methodologies, simulations, and experiment validations for HMTs. In this dissertation, the conducted research is presented and discussed in five phases with some exclusive objectives.
Phase I of the research study begins with an initial state-of-the-art literature survey. This includes analysis of all the available architectures and development methodologies as well as the establishment of a few conceptual basics that are essential for the HMT framework. Furthermore, the survey also discusses the different HMT components and human-machine systems (HMS) simulation methods available in the literature. Finally, the detailed objectives of the research needed to validate the stated hypotheses are discussed.
In Phase II, all the metrics available to measure HMTs are analyzed with the aim of constructing a matrix of metrics sorted based on different classifications and relationships to HMT, to achieve a final goal of constructing a common set of metrics for HMT benchmarking. The metrics are gathered through a keyword based systemic review from popular scholarly repositories and analyzed using metadata of metrics, such as measurement type, face value, dependency on adjacent metrics, and available standardized measuring methods. From there, they are categorized into different sets and models to measure HMT performance. This meta-analysis resulted in a color-coded chart of HMT metrics that are presented in this phase. More specifically, it is a matrix of metrics sorted based on different classifications and relationships to HMT. Furthermore, a set of common metrics is drawn based on the above study, and the selection criteria established are presented in this phase, which can be repeated for any similar future study. Finally, this phase presents models that can be used to measure different HMT performances through selecting common metrics sets.
Phase III discusses the development of intelligent systems that can be used as machines in HMTs. The tailored intelligent system can be called a synthetic agent (SA). This phase deals with SA in detail, particularly examining the backgrounds of SA and the continuous requirements of SA for this research. Furthermore, system design and detailed development of a voice-based synthetic assistant (VBSA) are also presented in this section. The VBSA constitutes a performance model of developed systems. The resultant voice-based synthetic assistant prototype is significant in constructing an HMT and is also effective in measuring an HMT’s different parameters, such as performance and efficiency. Finally, Phase III presents performance and operation analysis of the developed VBSA.
Phase IV of this research consists of human-in-the-loop (HITL) simulation and human factor user studies of generalized HMT architectures using controlled HMT scenarios, such as emergency care provider (ECP) treating patients and visual data processing that represents real-world applications. As part of this HMT simulation studies, the impact of each parameter related to machines and humans versus HMT is presented from the perspective of performance, rules, roles, and operation limitations. This phase also presents statistical analyses of measured performances with respect to participant groups. These statistical analyses are used as evidence to understand HMTs and components of HMT behavior.
Furthermore, Phase V presents guidelines for designing future HMTs and performing standardization studies in the pursuit of developing standardization techniques for benchmarking HMTs that can be used in critical situations. This phase concludes by rationally proving hypothesized research methods that include SA development, as metrics can be used to standardized future HMTs. Finally, future work is discussed in providing the guidelines for next-generation HMT research.