Foreword to Engineering Psychophysiology: Issues and Applications
Our purpose with this volume is to promote engineering psychophysiology as a discipline and to demonstrate its value to a new audience who we hope will consist of ergonomists, human factors psychologists, and engineers. We used a rather broad definition of what constitutes engineering, including all aspects of the fields known as human engineering, industrial engineering, and safety and systems engineering. We had two goals for this volume, which are reflected in the subtitle.
Our goal for the Issues section was to introduce the components critical for the successful application of psychophysiological methods to problems in engineering. In particular, the chapters in the Issues section are intended to provide an introduction for the reader who is unfamiliar with psychophysiology. These chapters are not comprehensive reviews, nor are they tutorials. Instead, their purpose is to provide the newcomer to the discipline with an overview of the basic theoretical, measurement, instrumentation, and experimental design questions inherent in the use of psychophysiological methods. The Boucsein and Backs chapter provides a very brief historical context of engineering psychophysiology as a discipline, introduces some theoretical constructs (e.g., arousal and stress/strain) and provides a taxonomic bibliography of recent research using the predominant measurement techniques in the field. The Gaillard and Kramer chapter reviews the major theoretical approaches (i.e., mental workload and stress/strain) that form the foundation for much of the research of the discipline. The Thayer and Friedman chapter critiques the classical approach to experimental design in psychophysiology and suggests ways in which research in engineering psychophysiology can and should move beyond the classical approach. The Luczak and Göbel chapter reviews bioelectrical signal processing and illustrates how psychophysiological research in complex tasks differs from typical laboratory psychophysiology. Finally, the Fahrenberg and Wientjes chapter reviews advantages and disadvantages of ambulatory assessment of psychophysiological measures. The newcomer is intended to use these chapters as guides to the more in depth treatments of each issue that can be found in the chapter references.
Our primary goal for the Applications section was to illustrate the many ways that psychophysiological methods are already being used in engineering applications. We also use a broad definition of application, one that includes laboratory and simulation research as well as field studies. However, what all chapters have in common is that they address questions that are relevant for applying psychophysiological methods in the field. Our intent is to stimulate investigators to use these methods in new problem areas; therefore, the content of these chapters varies widely. Some chapters review specific psychophysiological measures (e.g., Sirevaag & Stern). Other chapters review work performed on specific engineering problems (e.g., Boucsein; Luczak & Springer; Scerbo, Freeman, & Mikulka), including problems in industrial and safety and systems engineering (e.g., Bonnet & Arand; Caldwell & Caldwell; Gundel, Drescher, & Turowski; Lundberg & Johansson; G. Mulder, L. Mulder, Meijman, Veldman, & van Roon). Several chapters focus on the use of psychophysiological measures to assess mental workload (e.g., Freude & Ullsperger; Lenneman & Backs). Finally, there is a report on the status of the discipline in Japan (Yagi).
Our secondary goal for the Applications section was to have a broad sampling of measures and measurement techniques that have been used in engineering psychophysiology. In addition to the more typical psychophysiological measures of the central, autonomic, and peripheral nervous systems (i.e., electroencephalographic, cardiovascular, respiratory, electrodermal, occulomotor, and electromyographic), we also have included research using endocrine measures. Some chapters focus on a single response or response system, whereas the others use multiple measures. The latter approach is increasingly common in psychophysiological research and has much to recommend it (see the chapter by Thayer and Friedman).
We believe that the timing is right for this volume. The cost of instrumentation needed for psychophysiological research continues to fall. Simultaneously, the capability of the instrumentation continues to increase. Another important development is the rapid advancement of ambulatory technology that enables the transition of psychophysiological methods from the laboratory to the natural environment (see the chapter by Fahrenberg and Wientjes). As a result, the interest in psychophysiological methods has never been greater.
An indication of this increased interest is the recent founding of the Psychophysiology in Ergonomics (PIE) society that is also a technical group of the International Ergonomics Association and an interest group of the Society for Psychophysiological Research. This group has more than 180 members representing 15 countries. PIE has sponsored two international conferences and several special issues of major journals such as Ergonomics and Biological Psychology (more information on PIE can be obtained at: http://www.uni-wuppertal.de/FB3/psychologie/physio/pie.htm).
Finally, we would like recognize and thank some of the many people who made this project possible. First, of course, are the contributors; the volume would not have been possible without their goodwill throughout the revision process. Second, we would like to thank the reviewers, especially those contributors who also volunteered as reviewers: Mike Bonnet, Alex Gundel, Rob Henning, Dick Jennings, Art Kramer, Petra Netter, Raja Parasuraman, Bill Ray, John Stern, and Larry Walrath.
Mount Pleasant and Wuppertal, December 1998
Rick Backs and Wolf Boucsein
Autonomic responses elicited during information processing clearly qualify as "energetic" by any definition of the concept. Recent developments in autonomic theory hold the promise of advancing the utility of measures of autonomic activity for the assessment of mental workload. These developments will expand the capability to map psychological processes to autonomic physiological responses beyond what has previously been possible. The present chapter attempts to describe the relevant theoretical principles needed to appreciate this increased inferential capability and to briefly present some supporting cardiovascular evidence.
The diagnosticity of heart rate for mental workload
assessment can be improved with an autonomic space model
of sympathetic and parasympathetic nervous system
influences on the heart. Methods of deriving autonomic
components to estimate the underlying sympathetic and
parasympathetic activity needed to identify autonomic
modes of control for heart period were examined.
Alternative factor extraction and rotation methods were
compared using data from a laboratory study that measured
the electro- and impedance cardiogram simultaneously.
Principal components analysis with varimax rotation was
found to validly estimate the sympathetic and
parasympathetic activity when computed on multiple
psychophysiological measures obtained from raw EKG data.
Use of the method was illustrated with single and dual
mental arithmetic and second-order compensatory manual
tracking tasks. Different autonomic modes of control were
found for divided attention that were not evident in
heart period. Uncoupled sympathetic activation was
indicated for divided attention when the mental
arithmetic tasks were added to the tracking single tasks,
but uncoupled parasympathetic inhibition was indicated
for divided attention when the tracking tasks were added
to the mental arithmetic single tasks.
Central, autonomic, and metabolic physiological measures were observed concurrently along with performance and subjective measures to compare the effects of tracking task difficulty during selective and divided attention. Eighteen dextral males performed visual compensatory manual tracking as a primary task while attending to or ignoring secondary-task auditory oddball stimuli. The difficulty of the tracking task was varied factorially by requiring participants to track with acceleration (second-order) or velocity (first-order) control and high or low bandwidth sum-of sines disturbance. Tracking performance was affected by the difficulty manipulations but not by the attention manipulation. Event-related brain potential P300 amplitude to oddball target stimuli was sensitive to the division of attention and to tracking order-of-control, but not to tracking disturbance bandwidth when the oddball task was attended. Oxygen consumption, a measure of aerobic metabolism, was greater during acceleration than velocity tracking; however, cardiac measures were sensitive only to the division of attention. The results demonstrate that attention and the task difficulty manipulations have physiologically dissociable effects that were interpreted as supporting a cognitive/energetic model of attention.
Psychophysiological assessment of pilot mental workload using heart rate should be augmented with an autonomic space model of cardiovascular function. This model proposes that autonomic nervous system influences on the heart may change with psychological processing in ways that are not evident in heart rate. A method of mental workload assessment was proposed that used multiple psychophysiological measures of cardiovascular responsivity to derive the underlying sympathetic and parasympathetic information needed to represent the autonomic space for heart rate. Principal components analysis was used to extract Sympathetic and Parasympathetic components from heart period, residual heart period, respiratory sinus arrhythmia, and Traube Hering-Mayer wave in three experiments that manipulated perceptual/central processing and physical task demands. This initial evaluation of the method concluded that the autonomic components were valid, and that the components had greater diagnosticity, and for some manipulations greater sensitivity, than heart rate. These results support the contention that the Sympathetic and Parasympathetic components provided increased precision for mental workload assessment.
Mental workload assessment using heart rate can be
improved with an autonomic space model of sympathetic and
parasympathetic nervous system influences on the heart.
Principal components analysis of multiple
psychophysiological measures of cardiovascular activity
was used to derive the underlying sympathetic and
parasympathetic information needed to represent the
autonomic space for the heart. This method was applied to
cardiac data that were previously collected during a full
mission flight simulation. Volunteer commercial pilots
flew missions simulating normal flight and a high
workload flight in poor weather with an inoperative
autopilot, an engine stall, the loss of one hydraulic
system, and other distractors. The autonomic components
complemented the heart rate and heart rate variability
data and increased the number of mission phases that
could be differentiated from each other. Further, the
autonomic components indicated that heart rate was faster
during the high than low workload mission because of
increased sympathetic activation.
Backs, R. W., Lenneman, J.
K., & Sicard, J. L. (1999). The use of autonomic
components to improve cardiovascular assessment of mental
workload in flight simulation. The
International Journal of Aviation Psychology, 9,
33-47.
Principal components analysis of multiple
psychophysiological measures of cardiovascular activity
was used to derive the underlying sympathetic and
parasympathetic information needed to represent the
autonomic space for heart period. This method was applied
to cardiac data that were previously collected during a
full-mission flight simulation from volunteer commercial
pilots (Corwin et al., 1989). Pilots flew both low and
high workload mission scenarios in two separate test
sessions. The autonomic components provided increased
diagnosticity regarding the autonomic mode of control for
heart period change between scenarios and across flight
phases. Heart period was shorter (faster heart rate)
during the high than low workload scenario because of
uncoupled sympathetic activation. Heart period change
across flight phases was due primarily to coupled
reciprocal modes of autonomic control during the low
workload scenario. However, heart period change across
phases was due to multiple modes of control, including
coactivation, during the high workload scenario. Further,
heart period and the Sympathetic component score were
significantly reliable across test sessions.
Psychophysiology has contributed much to the development and understanding of the mental workload concept. Recent advances in theoretical models of autonomic nervous system functioning have the potential to increase the practical utility of psychophysiological measures in workload assessment. For example, “modes of autonomic control” can explain why psychophysiological measures sometimes dissociate from performance and subjective workload measures, and from each other when multiple measures are examined simultaneously. Further, these models may provide the neurophysiological structure to strengthen inferences about the relation between the psychophysiological measures and the underlying psychological processing of task demand manipulations.
27 university student participants (16 female) were trained on the TRACON video game and then practiced for 20 hrs. After practice, participants performed 3 mental workload conditions in a single session. Low, medium, and high mental workload scenarios were created by varying air traffic density: participants were required to handle 5, 10, or 15 aircraft within a scenario while cardiorespiratory, performance, and subjective mental workload data were collected. Heart rate change from baseline was faster during the high than during the low scenario. Pre-ejection period and high frequency heart rate variability indicated that the autonomic modes of control differed across scenarios: high and medium workload elicited significant reciprocally-coupled sympathetic activation and parasympathetic withdrawal, whereas low workload did not elicit significant change from baseline. Respiration rate change from baseline was faster during the high and medium scenarios than during the low scenario. Performance was significantly lower for the high than the low scenario. However, subjective workload increased significantly from the low to the medium to the high scenario.
Backs, R. W., Lenneman, J. K., & Sicard, J. L. (1997). Reliability of autonomic components for cardiovascular assessment of mental workload in flight simulation. In R. S. Jensen & L. A. Rakovan (Eds.), Proceedings of the Ninth International Symposium on Aviation Psychology (pp. 1536-1542). Columbus, OH: The Ohio State University.
Principal components analysis of multiple psychophysiological measures of cardiovascular activity was used to derive the underlying sympathetic and parasympathetic information needed to represent the autonomic space for heart period. This method was applied to cardiac data that were previously collected during a full-mission flight simulation from volunteer commercial pilots. Pilots flew both a low and a high workload mission in two separate test sessions. We report data for twelve male pilots who had artifact-free polygraph recordings in both test sessions during the following five flight phases: takeoff, top of climb, cruise, approach, and landing. Heart period and the Sympathetic, but not Parasympathetic, component scores were significantly reliable across test sessions. Further, the autonomic components provided increased diagnosticity regarding heart period change across flight phases. For example, heart rate was faster during the high than low workload mission because of uncoupled sympathetic activation.
Mental workload is the difference between available attentional resources and those required for performance. A theoretical cognitive-energetic approach was used in an attempt to account for association and dissociation between central and autonomic nervous system measures. Autonomic measures reflect the regulation of physiological state, which may affect resource availability. Although increases in task difficulty require more resources, physiological state will not change when sufficient resources are available for successful performance. P300 amplitude and heart period will associate only when the same physiological state cannot be maintained across levels of task difficulty. Participants performed 67 blocks of 90-trials per block of a visual Sternberg task across 6 sessions. Cognitive processing demand was manipulated by memory set size (2, 4, and 6 target items) and in the last session where participants also kept a single count of the total number of targets presented. Practice was the energetic manipulation. The central measure, P300 amplitude, decreased for both memory manipulations, and increased with practice. One peripheral measure, heart period decrease from baseline, was greater as memory set size increased, but did not differ with practice or with the addition of counting. The other peripheral measures, high and low frequency heart period variability, were suppressed from baseline, but were not affected by any manipulation. The cognitive and energetic constructs did not account for all patterns of dissociation observed. P300 amplitude was sensitive to the cognitive processing demand and to the change in demand due to practice, whereas heart period was sensitive to the memory set size.
Cardiovascular reactions to images from the International Affective Picture Set (Lang, Bradley, & Cuthbert, 1997) were examined for young and old adults. Pleasant/aroused, pleasant/calm, unpleasant/aroused, unpleasant/calm, and neutral conditions were created so that the valence and arousal norms for the nine images in each condition did not overlap. Nineteen young (median age 18 years) and nineteen old (median age 65 years) participants viewed all conditions, which were preceded and followed by resting baselines. Electro- and impedance cardiograms were recorded for the entire 3-mins. of the baselines and emotion conditions. Heart period, pre-ejection period, stroke volume, and cardiac output were obtained from ensemble averages of the cardiac cycles in each min. The neutral condition did not differ significantly from any of the others when all conditions were examined together. An age by valence by arousal analysis was conducted omitting the neutral condition. Heart period, stroke volume, and cardiac output increases from resting baseline were significantly greater for the young than for the old, but age did not interact with valence or arousal. Stroke volume and cardiac output increases were significantly greater for the unpleasant than for the pleasant conditions. There were no age differences when change from the neutral condition was examined instead of change from resting baseline. Further, stroke volume and cardiac output decreased in the pleasant conditions and increased in the unpleasant conditions compared to the neutral condition. An autonomic space analysis revealed no significant differences in pre-ejection period and respiratory sinus arrhythmia for the old adults, but revealed that the unpleasant/calm condition differed from the pleasant/calm and the unpleasant/aroused conditions for young adults.
Backs, R. W., Pohlman, R. L.,
Hunt, J. L., Pringle, D. D., Mann, B. L., & Isaacs,
L. D. (1994). Exercise and working memory effects on
heart rate. Psychophysiology,
31(S1), S23 S24.
We examined the relation between heart rate (in BPM) and
oxygen consumption (VO2 in ml/kg) in seven student
athletes under three conditions: exercise alone, memory
alone, and exercise and memory performed together. In an
auditory version of the QRST working memory task,
subjects maintained a running count of three target
letters (varied across trials) and reported their counts
for each at the end of a trial by pointing to an answer
board. One letter was presented every 4 s across 4 min
trials. Exercise sessions consisted of a graded exercise
tolerance test performed on a motor driven treadmill with
a minimum of one day between sessions. In the first
condition, subjects performed two sessions of exercise
alone to obtain a stable estimate of VO2 Max. Memory task
practice occurred in three separate sessions. In the
final conditions, subjects performed one trial of the
memory task alone and four trials of the exercise and
memory together. The four exercise and memory trials used
graded levels of exercise (20%, 40%, 60%, and 80% of VO2
Max) presented on ramped and counterbalanced schedules in
separate sessions. Mean memory error was less than 7% and
did not differ across exercise alone and graded exercise.
Individual subject regression analyses indicated that the
level of physical workload and the addition of the memory
task both had significant effects on heart rate. These
results support the contention that cognitive factors
contribute significantly to heart rate even at high
levels of physical workload. Further, no evidence of
metabolically exaggerated heart rate was found in the
memory task alone condition when the individual heart
rate-VO2 regression equations from either exercise alone
or exercise and memory together conditions were used for
prediction.
Fifteen male volunteers participated in a dual-task study in which the central processing load of visual memory and tracking tasks and the physical load of the tracking task were orthogonally manipulated to produce varying levels of task difficulty. Multiple modes of assessment were used to measure mental workload (MWL) across difficulty levels, including: performance, subjective, cardiovascular, and metabolic. To our knowledge, this study is the first to demonstrate metabolic change with manipulations of cognitive task difficulty; others have found only baseline-to-task changes. The relation of the metabolic demands of the task to central processing resource utilization provided support for a structural energetic model of attention that may help to explain measure dissociations. The results of the present study indicated that heart period was only sensitive to central manipulations of task difficulty that affected energetic resources. Performance and subjective MWL were sensitive to all cognitive components of the tasks. We suggest that cardiovascular measures will associate with other measures only when the manipulations of task difficulty require energetic adjustment, and would expect these measures to dissociate when energetic adjustment is not required.
Twelve subjects (six female) participated in an experiment designed to separate the effects of perceptual/central and physical demands on psychophysiological measures of peripheral nervous system activity. The difficulty of a single-axis continuous manual tracking task was varied in two ways: order of control was manipulated to vary perceptual/central processing demand, and disturbance amplitude was manipulated to vary physical demand. Physiological measures were sensitive to the imposition of a task, and were more sensitive to physical than to perceptual/central demands. A principal components analysis identified five factors (three of them physiological) that accounted for 83.1% of the observed variance. Perceptual/central processing demands specifically affected the component identified with sympathetic cardiovascular control, while physical demands were reflected in the component identified with parasympathetic cardiovascular control. This finding suggests that dissociations observed among cardiovascular measures in manual performance tasks are due to differential activation of the autonomic control systems.
Twelve subjects (six female) participated in a study designed to experimentally separate the effects of physical and cognitive effort on cardiac and pulmonary measures of workload. Cardiac (heart rate and variability), respiratory, and forearm muscle activity were measured while subjects performed a single-axis continuous manual tracking task. The central processing demands of the task were varied by changing the tracking system order over three levels: pure velocity, a combination of velocity and acceleration, and pure acceleration. The physical demands of the task were varied by requiring subjects to track either high or low amplitude sum-of-sines disturbance. RMS tracking error, subjective ratings, heart rate, and forearm muscle activity were sensitive to, but did not differentiate, the cognitive and physical manipulations; these measures increased as both tracking order and disturbance gain increased. However, the respiration measures dissociated under the physical and cognitive manipulations. Respiration rate increased with tracking order but not disturbance gain, while respiration amplitude increased with disturbance gain but not tracking order. Also, spectral power measures of respiratory activity and of cardiac activity at the respiration frequency (i.e., respiratory sinus arrythmia) dissociated for disturbance gain: respiratory power increased while cardiovascular power decreased as a function of increasing disturbance gain. These results verify the utility of a multiple-measures approach to the assessment of workload, and suggest the inclusion of respiration and cardiorespiratory interactions as workload measures.
Sensitive and diagnostic measures of information processing are needed to dynamically allocate task function in order to maintain mental workload at an optimal level and prevent performance failures. Central and peripheral nervous system physiological measures were examined currently along with performance and subjective mental workload during tasks that differed in their information processing demands. Eighteen participants (9 female) performed mental arithmetic and tracking tasks singly and together. EEG (7-10 Hz alpha), eye blink, performance, and subjective measures were examined while the difficulty of the arithmetic task and the physical demand of the tracking task were varied. EEG alpha power decreased and reaction time and subjective mental workload increased with mental arithmetic task difficulty, but no differences were observed for the physical manipulation. No performance differences were observed across single-to-dual tasks; however, EEG alpha power decreased and subjective mental workload increased in the dual tasks compared to the single tasks. EEG topography and eye blinks differed when going from arithmetic single tasks to the dual tasks as compared to going from tracking single tasks to the dual tasks. These between-task differences for the EEG and eye blink measures illustrate the importance of using multiple measures, and the dissociation among measures, to achieve diagnosticity. Further, because the psychophysiological change occurred before a performance decrement was observed, these results suggest the potential predictive utility of psychophysiological mental workload measures.
Psychophysiological measures of mental workload (MWL) must be sensitive across levels of physical activity when the physical demands associated with performance are not associated with MWL. Eighteen subjects performed mental arithmetic and second-order compensatory tracking tasks singly and together. Performance, subjective, and physiological measures were examined while the complexity of the arithmetic task (addition and subtraction problems requiring two or three operations) and the physical demand of the tracking task (four or eight pounds of force required for full range excursion of the cursor) were varied. The arithmetic task consisted of 36 problems which were presented at a fixed interval of 5 s. Subjects were instructed to respond as quickly as possible while maintaining 100% accuracy in the arithmetic task and to minimize the distance between the target and the cursor in the tracking task in both single and dual task conditions. Performance feedback was given at the end of each trial. Subjects performed 60 3 min practice trials over Days 1-3. All data reported are from one 3-min trial of each of the eight task conditions on Day 4. Reaction time and subjective MWL increased and tonic heart period was shorter during the three operator mental arithmetic condition than during the two operator condition. No performance differences were observed across single and dual tasks, but P300 amplitude at Cz and Pz was reduced, subjective MWL increased, blink duration, amplitude and rate decreased, and tonic heart period was shorter in the dual task conditions compared to the single task mental arithmetic. Although single task performance levels were maintained in the dual task conditions, both central and autonomic measures indicated that performance was not preserved without significant physiological cost. Further, the psychophysiological measures were sensitive across the physical manipulation of the tracking task.
Twenty-four participants (12 female) performed a continuous memory task during which metabolic, cardiorespiratory, performance, and subjective mental workload measures were taken. Task difficulty was varied using two manipulations in a within-subjects factorial design: memory load (one or three items) and temporal demand (interstimulus intervals of 2, 3, or 4 s). Males and females differed in initial metabolic rate, but did not differ in their response to the task. Memory load affected all measures, while temporal demand affected only respiration rate, performance, and subjective mental workload. Metabolic, Respiratory, Cardiovascular, and Subjective/Performance components were identified in a principal components analysis, and the Respiratory and Subjective/Performance components were affected by the task manipulations. When performance quality was examined, the Metabolic component revealed that poor performers had greater energy expenditure during the task than good performers, and the Cardiovascular component revealed that good and poor performers differed in their response to memory load and temporal demand. Cardiac and metabolic changes during mental work were not a function of overall mental effort, but were specific to the effort due to memory load and to the individual differences among participants in their ability to perform the task. However, respiration was sensitive to the mental effort associated with both memory load and temporal demand, but was not sensitive to individual differences.
Sixteen observers participated in a visual search experiment in which color coding, search type, and the amount of presearch information available to the observers were varied. Observers searched simulated symbolic tactical displays to find the number of target symbols (i.e., exhaustive search) or the quadrant of the display in which a single target symbol was located (i.e., self terminating search). Displays varied in the way in which the symbology was color coded: color was either relevant (i.e., redundant with symbol shape) or irrelevant (orthogonal to symbol shape), or the display was monochrome. Half of the observers were cued with regard to the coding scheme prior to display onset, while the other observers were not. There was no overall difference in search time or accuracy, number of eye fixations, or pupillary response between cued and noncued observers, but only because cued and noncued observers used the coding schemes differently. Redundancy gain was only evident for cued observers, who searched color relevant displays faster and with fewer fixations than color irrelevant or monochrome displays. Noncued observers' search pattern did not differ across color coding schemes, but they searched color irrelevant and monochrome displays faster than the cued observers. Differences between cued and noncued observers' search strategy are discussed with regard to their implications for design and evaluation of color multipurpose displays.
Eye movement and pupillary response measures (in addition to search time and accuracy) were collected as indices of visual workload during two experiments designed to evaluate the addition of colour coding to a symbolic tactical display. Displays also varied with regard to symbol density and the type of information participants were required to abstract from the display. These variables were factorially manipulated to examine the effects of colour coding in conditions of varying difficulty. In Experiment 1 (N = 8), search time and the number of eye fixations were affected by all variables and in a similar manner; fixation dwell time and the pupillary response dissociated from the other measures. Compared to monochrome displays, colour coding facilitated search (reduced search time, but not accuracy) during exhaustive search, but had no effect during self-terminating search. Experiment 2 (N = 8) was a replication of Experiment 1 with a pseudo-search control condition added to further examine the pupillary response measures, in particular to assess the effects of the physical parameters of the displays, and to verify the findings of Experiment 1. Pupillary response measures were sensitive to the information processing demands of the search task, not merely to the physical parameters of the display. Further, the search time, accuracy, and eye movement results from the active search condition generally replicated Experiment 1, but the fixation dwell time data did not. These between-study differences were interpreted as indicating the importance of participant search strategy.
Auditory workload in the cockpit has two sources, the communications channel and auditory caution/warning displays. The purpose of these displays is to alert the pilot that a caution or emergency condition exists and to transmit information regarding that condition without requiring him/her to direct attention to the instrument panel (Doll & Folds, 1986). These functions can be performed by either speech or nonspeech signals. While speech intelligibility has always been an area of concern for communications engineers, with the advent of speech displays it is now also a concern for the auditory display designer. Our emphasis here is on the intelligibility and workload associated with the use of speech displays.
Cardiovascular mental workload assessment can be
enhanced using an autonomic space model of cardiac
function. The model is sensitive to autonomic nervous
system influences on the heart that change with
psychological processing in ways that are not evident in
heart rate. A mental workload assessment method based
upon the autonomic space model is evaluated with
laboratory and in-flight data. The method uses principal
components analysis of multiple cardiovascular
psychophysiological measures to derive the underlying
sympathetic and parasympathetic information needed to
represent the autonomic space for heart rate. The method
was first applied to data from a series experiments that
used overlapping task manipulations. The autonomic
components were valid and had greater diagnosticity, and
for some manipulations greater sensitivity, than heart
rate alone. The results suggest that cognitive and motor
demands can be separated in autonomic space. The method
was then applied to cardiac data collected in-flight.
Fifteen general aviation pilots flew a single-engine
aircraft on a profile that included both Visual Flight
Rules (VFR) and Instrument Flight Rules (IFR) segments.
Heart rate and the Sympathetic and Parasympathetic
components from each flight segment were compared with
all other flight segments. Consistent with the laboratory
data, the autonomic components had greater sensitivity
and diagnosticity than heart rate. The autonomic
components differentiated more segments than heart rate
and varied in ways that support the contention that
autonomic components provides more precise mental
workload assessment.
Twenty male participants performed five types of tasks that varied in cognitive effort and active passive coping in an attempt to elicit different modes of autonomic control. Passive low-effort tasks were cold pressor and watching visual illusions. Active low-effort tasks were low (8 W) and high (25 W) intensity exercise on a bicycle ergometer. Passive high-effort tasks were low (1-item memory set) and high (3-item memory set) difficulty versions of the QRST working memory task. The active high-effort task was visuomanual compensatory tracking in which order-of-control (velocity or acceleration) and disturbance bandwidth (low or high) were factorially combined to form four tasks that varied in difficulty. Heart period significantly decreased from baseline-to-task during the cold pressor, both exercise intensities, high difficulty memory, velocity and acceleration tracking, and low and high disturbance tracking. Modes of autonomic control were determined using pre-ejection period (sympathetic activity) and respiratory sinus arrhythmia (parasympathetic activity) baseline-to-task changes. Coupled reciprocal sympathetic activation and parasympathetic withdrawal was found to be significantly greater for high than low intensity exercise. Uncoupled parasympathetic withdrawal that did not differ across difficulty manipulations was found for the tracking tasks. Results for the other tasks were less clear: cold pressor, visual illusions, and low difficulty memory tasks did not elicit consistent PEP or RSA change, whereas uncoupled parasympathetic withdrawal was found for the high difficulty memory task which had significantly greater sympathetic activity than the low difficulty version.
Twenty-seven university students (16 female) were trained on the Terminal Radar Approach Controller (TRACON) video game and then practiced for 20 hrs. After practice, participants performed low, medium, and high mental workload scenarios in a single session. Participants were required to handle 5, 10, or 15 aircraft within a scenario. We investigated whether measures of personality, divided attention capability, and cardiovascular functioning would improve prediction of performance beyond standard selection tests, and whether the predictors of performance would differ across workload scenarios. The predictors examined were trait anxiety, a profile score computed from the Profile of Mood States, divided attention performance from the monitoring and communications tasks of the Multi-Attribute Task Battery (MATB), and heart period, pre-ejection period, stroke volume, cardiac output, and high frequency heart rate variability obtained during resting baseline and during MATB performance. The selection tests did not account for significant TRACON performance variance in any workload scenario. However, significant gains in performance prediction were observed for each workload scenario, and the predictors differed across scenarios. Trait anxiety and divided attention performance significantly predicted TRACON performance in the low workload scenario. Cardiac output significantly predicted TRACON performance in the medium workload scenario. Heart period and pre-ejection period significantly predicted TRACON performance in the high workload scenario.
Backs, R. W., & da Silva, S. P. (2001, June). Young and old adults do not differ in long-tern affective picture recognition. Poster presented at the Thirteenth Annual Convention of the American Psychological Society, Toronto, Canada.
Long-term recognition memory for affective pictures was examined for groups of young and old adults. Pictures from the International Affective Picture Set (IAPS; Lang et al. 1997) were used to create discrete emotion conditions. Pleasant/aroused, pleasant/calm, unpleasant/aroused, unpleasant/calm, and neutral conditions were created so that the valence and arousal Self-Assessment Manikin (SAM) norms that accompany the IAPS did not overlap.
The Time 1 session consisted of three phases. In Phase 1, physiological data were collected during presentation of the nine pictures in each condition. Superlab was used to present color digital bit maps of the pictures on a 20 in. monitor. Phase 2 was a two-alternative forced-choice memory task performed about 20 min. later, where one picture from Phase 1 was paired with a new picture that matched the old in valence, arousal, and major content. Subjects chose one picture as old and provided a confidence rating (absolutely sure to guessing). In Phase 3, subjects provided SAM valence and arousal ratings for all pictures presented in Phases 1 and 2. The Time 2 session was a replication of Phases 2 and 3 except that a different set of new pictures (also matched to the old) was created from IAPS pictures that were not used at Time 1.
There were 23 (10 female) college students (mean age 20-years-old) and 23 (13 female) healthy community dwelling older adults (mean age 67-years-old) who participated in one 3-hr session at Time 1. At Time 2, 14 (8 female) college students and 14 (8 female) older adults returned for a 1-hr follow-up session. Mean time between sessions was 23 months. Memory (proportion correctly recognized) and mean confidence ratings from Phase 2 for the participants who returned at Time 2 were analyzed. Both groups recognized the old pictures at greater than .97 accuracy with absolute confidence at Time 1. Because there was essentially no variability at Time 1, decrement scores (Time 2 minus Time 1) were created to examine performance change across time.
Memory and confidence decrements differed significantly across time and emotion condition. However, neither decrement score differed between age groups or for the emotion condition by age group interaction. Post hoc tests showed that memory decrements were significantly smaller in the pleasant/aroused condition than in the neutral, unpleasant/aroused and unpleasant/calm conditions. However, confidence decrements were significantly smaller in the unpleasant/aroused condition than in all other conditions. Age group by arousal by valance analyses (omitting the neutral condition) were conducted on the Time 2 data to clarify the dissociation between memory and confidence. Memory was better for arousing than for calm pictures and did not differ across age groups. Confidence was higher for arousing than for calm pictures and for unpleasant than for pleasant pictures, but valence effects were limited to arousing pictures and differed across age group. Confidence for older adults did not differ between unpleasant and pleasant pictures. However, confidence for young adults was lower than the older adults' for pleasant pictures, but it was higher than the older adults' for unpleasant pictures.
Berntson, Cacioppo, and Quigley (1991) have formalized a “doctrine of autonomic space” that posits multiple modes of autonomic control that are multidimensionally determined as opposed to simply being reciprocally coupled. A mode of autonomic control refers to the pattern of sympathetic and parasympathetic nervous system activity responsible for an effector response such as heart rate. The present experiment used pre-ejection period and respiratory sinus arrhythmia (high frequency heart rate variability) to determine the modes of autonomic control for heart rate elicited during a visual single-axis compensatory manual tracking task. Three factors were predicted to affect the mode of autonomic control elicited during the tracking task: practice with the task, and the order-of-control and disturbance bandwidth parameters of the task. Order-of-control is the number of time integrations of the participant's input to the tracking task. Dual task studies using P300 methodology show that increasing the order-of-control from velocity (one integration) to acceleration (two integrations) control increases demand for perceptual/central processing resources, in addition to any influence upon motor processing, because of the greater need for operators to anticipate and predict the effects of their input (Backs, 1997). However, increasing the bandwidth of the sum-of-sines disturbance forcing function that is added to the tracking task affects demand for motor processing resources but not for perceptual/central processing resources (Backs, 1997). We predicted that the two tracking task parameter manipulations would elicit different autonomic modes of control reflecting the attentional resource demands (Backs, Ryan, & Wilson, 1994). The order-of-control manipulation was predicted to elicit reciprocally-coupled sympathetic activation, whereas the disturbance manipulation was predicted to elicit uncoupled parasympathetic withdrawal. It was also predicted that these differences would not be apparent before practice with the task, but would only occur after participants had sufficient practice to learn the different perceptual/central processing strategies needed for velocity and acceleration control (Lenneman & Backs, 2000).
Twenty university students (5 female) participated in a single three hour session. Participants completed a resting baseline, a two-minute trial of each of the four levels of the task, and another resting baseline during which cardiac and respiration data were collected. Next, participants completed 21 two-minute practice trials during which no physiological data were collected. Finally, participants repeated the physiological data collection procedure.
Performance as measured by RMS tracking error significantly improved after practice. The difference in RMS error between velocity and acceleration control was greater before than after practice, indicating that participants were developing separate perceptual/central processing strategies. The difference in RMS error for high and low disturbance did not change with practice, indicating merely refinement of motor skills. Heart rate difference from baseline significantly increased with task difficulty for order-of-control and significantly decreased after practice. Analysis of the autonomic control modes revealed the predicted result before practice, where uncoupled parasympathetic withdrawal was observed for both tracking task parameter manipulations. However, the predicted difference in autonomic control modes after practice did not occur; uncoupled parasympathetic withdrawal was again observed for both tracking task parameter manipulations. Figure 1 illustrates the uncoupled parasympathetic withdrawal before and after practice for the order-of-control manipulation. Respiratory sinus arrhythmia decreased significantly from baseline both before and after practice, but pre-ejection period did not. We suggest that the failure to find reciprocally-coupled sympathetic activation for the order-of-control manipulation after practice was because the tracking task did not demand enough perceptual/central processing resources to elicit sympathetic change. Instead, both task parameter manipulations were affecting primarily motor processing resources.
References
Backs, R. W. (1997). Psychophysiological aspects of selective and divided attention during continuous manual tracking. Acta Psychologica, 96, 167-191
Backs, R. W., Ryan, A. M., & Wilson, G. F. (1994). Psychophysiological measures of workload during continuous manual performance. Human Factors, 36, 514-531.
Berntson, G. G., Cacioppo, J. T., & Quigley, K. S. (1991). Autonomic determinism: The modes of autonomic control, the doctrine of autonomic space, and the laws of autonomic constraint. Psychological Review, 98, 459-487.
Lenneman, J. K., & Backs, R. W. (2000). The validity of factor analytically derived cardiac autonomic components for mental workload assessment (pp. 161-174). In Backs, R. W., & Boucsein, W. (Eds.), Engineering Psychophysiology: Issues and Applications. Mahwah, NJ: Lawrence Erlbaum.
Twenty-five college students (5 male) participated in two visual illusion tasks while impedance cardiographic measures were collected. One task consisted of viewing digitized images of the M. C. Escher print “Metamorphose” presented on a computer monitor. Successive images were presented every 0.52 s for 1 min., and participants were instructed to watch the tessellating patterns in the drawing. The other task was a replication of the visual illusion task used by Berntson, Cacioppo, and Fieldstone (1996, Biological Psychology, 1-17). Illusions were presented on a computer monitor every 20 s for 1 min. Illusions were accompanied by a brief question that participants were instructed to covertly answer (e.g., “Which line is longer?” for a Muller-Lyer illusion). Tasks occurred four times each and were presented in ABBA-BAAB order, where each task was preceded by a 1 min. resting baseline. Responses from each of the four task periods and the eight baseline periods were averaged and change scores from baseline were computed for heart period, pre-ejection period (PEP), and left ventricular ejection time (LVET). Heart period was slower than baseline for both tasks, but significantly so only for the replication task. PEP and LVET did not differ from baseline for either task. However, heart period and LVET were significantly longer in the replication task than in the Escher task when the two were compared to each other.
Nineteen college students (6 male) participated in a study examining cardiorespiratory change across practice with a single-axis compensatory tracking task. Tracking task difficulty was factorially manipulated by varying the system dynamics (velocity or acceleration order-of-control) and the bandwidth (0.06 Hz or 0.25 Hz break frequency) of the sum-of-sine's disturbance function added to the task. These two manipulations have been verified using P300 event-related brain potential methodology as having different information processing demands: Perceptual/central and response processing demands increase from velocity to acceleration tracking; whereas only response processing demands increase from low to high bandwidth disturbance. Participants performed 21 2-min. trials of the tracking task in a single 3-hr session. Heart period, high frequency heart rate variability (HF-HRV), and respiration rate and amplitude were collected during one trial of each of the four tracking tasks before practice (the first trial of each), early in practice (after 8 more trials), and late in practice (after 9 more trials). Cardiorespiratory data collection trials were preceded and followed by 2-min. resting baselines. Heart period change from baseline was significantly shorter and HF-HRV was significantly more suppressed from baseline during tracking with the acceleration than with the velocity system. Respiration rate, but not amplitude, change from baseline was significantly faster during tracking with the acceleration than with the velocity system. None of the cardiorespiratory changes from baseline differed significantly across practice. The present results suggest that the cardiorespiratory changes observed across tracking system order was a stable indication of perceptual/central processing demand of the task.