Cognitive Systems Engineering Strategies

1. Introduction
Cognitive Task Analysis (CTA) methods should be integrated into systems engineering since complex systems typically support highly cognitive activities. New system endeavors are geared towards augmenting the users’ macrocognitive functions such as: problem detection, coordination (i.e. – coordinating teams), and planning. Systems fail if they cannot support these cognitive functions, therefore it is important that the CTA practitioners learn what the functions are and relay them back to the systems engineers. Identifying and accommodating cognitive functions not only prevents systems from failing, it also enhances usability by improving the interactions between users and the system.

The field of Cognitive Systems Engineering (CSE) emerged as a solution to the problem of how to effectively integrate CTA methods into systems engineering. CSE combines the capabilities of technologists (i.e. – software engineers and computer scientists) and CTA researchers (i.e. – cognitive scientists and human factors psychologists). Almost all of the CSE strategies depend on CTA since it is important to understand the cognitive work the systems are trying to support. In this paper, I will look at some of the more popular CSE strategies being used today.

2. Situation Awareness (SA)-Oriented Design
Situation Awareness (SA) has been defined as: "the perception of the elements in the environment within a volume of space and time, the comprehension of their meaning, the projection of their status into the near future, and the prediction of how various actions will affect the fulfillment of one's goals" [1]. The SA-Oriented Design strategy is based on the idea that SA is fundamental in driving human decision-making in complex, dynamic environments [2]. Decision-making and performance can be dramatically improved if designs that enhance the users’ situational awareness are leveraged. SA-Oriented Design has been used to develop and evaluate system design concepts in order to improve human decision-making and performance. SA-Oriented Design is comprised of three main components: SA Requirements Analysis, SA-Oriented Design Principles, and SA Measurement and Validation.

For the SA Requirements Analysis, a form of cognitive task analysis called Goal Directed Task Analysis (GDTA) is used. GDTA focuses on uncovering the situation awareness requirements associated with a given job function. The first step in GDTA is to identify the major goals of a job and the sub-goals needed to reach these goals. Next, the major decisions that need to be made for each sub-goal are identified. Finally, the SA requirements needed for making these decisions and carrying out each sub-goal are identified. Since decision-making is shaped by goals, this requirements analysis is based on goals or objectives, not tasks (as in traditional task analysis).

A set of fifty SA-Oriented Design Principles was developed to help in creating a system design that meets the SA requirements. The principles are based on the mechanisms and processes involved in acquiring and maintaining SA. The principles provide design guidelines for: displaying goal-oriented information, dealing with extraneous information, dealing with system complexity, how to convey confidence levels in information, etc [2].

It is important to measure the proposed design concepts in order to determine if the design actually helps SA, or hinders it. Prototyping is an important tool that can be used when measuring the effectiveness of the design; it can rapidly expose the design to subject matter experts. Simulations of new technologies within the operating environment can also prove to be invaluable when measuring design effectiveness on SA.

3. Decision-Centered Design (DCD)
The DCD strategy was created to leverage CTA in the development of new technologies, including complex software systems [3]. This strategy uses CTA methods to specify the primary cognitive requirements for the design process. Importance is placed on the key decisions users have to make when performing their jobs (hence the name Decision-Centered Design). The idea behind DCD is that CTA can help the design team come up solutions that support the users’ when making key decisions.

DCD’s primary focus is on the difficult and challenging decisions the system must support (the “tough cases”). The focus on these tough cases ensures the system to be robust. The system and its users will be able to bounce back from extraordinary circumstances. If the system is successful in supporting tough cases, chances are it will also be successful in supporting the more routine / menial cases. Focusing on tough cases also ensures that the development process is efficient. There are rarely enough resources to cover all of the cognitive decisions in a complex system; covering only the difficult decisions is more efficient as it yields better coverage.

DCD consists of five stages: preparation, knowledge elicitation, analysis & representation, application design, and evaluation. During preparation, the practitioner is to understand the domain and the nature of the work the users are doing. The practitioner also identifies the appropriate CTA methods to use. In knowledge elicitation, the CTA methods are used to perform an examination of the difficult decisions and complex cognitive tasks. The practitioner then identifies the decision requirements by analyzing the recorded data. The analysis is shared with the systems engineers working on the design, and the design is developed iteratively via an open feedback loop with the users. Finally, performance of the system is measured and, if needed, improvements are made to the prototype.

4. Applied Cognitive Work Analysis (ACWA)
The ACWA methodology was developed by the Cognitive Systems Engineering Center in Pittsburgh, PA as the Cognitive Task Design (CTD) portion of a high quality, affordable systems engineering process [4]. It is touted as being at the same maturity level as the best software engineering methodologies. ACWA was specifically developed to satisfy two critical challenges to making CTD part of system engineering efforts. The first challenge is that CTD must become a high quality engineering methodology (on par with the efforts for software engineering processes made by Carnegie Mellon’s Software Engineering Institute). The second challenge is that the results of CTD must seamlessly integrate with the software and system engineering processes used to construct powerful decision support systems. [4]

ACWA contains a step-by-step approach to CTD. The design steps include domain analysis, decision requirements analysis, supporting information analysis, design requirements analysis, and finally design. Each step produces an artifact and the collection of artifacts provides a link from the cognitive task analysis to the system design. The artifacts serve two purposes: they progressively record the results of the design thinking for subsequent steps in the process, and they provide an opportunity to evaluate the quality and completeness of the design effort.

References
[1] Mica R. Endsley, Towards a Theory of Situation Awareness in Dynamic Systems, Human Factors, 37(1), 32-64, 1995

[2] Mica R. Endsley, Cheryl A. Bolstad, Debra G. Jones, and Jennifer M. Riley, Situation Awareness Oriented Design: From User’s Cognitive Requirements to Creating Effective Supporting Technologies, October 2003

[3] Hutton, R. J.B., T. E. Miller, and M. L. Thordson, Decision-Centered Design: Leveraging Cognitive Task Analysis in Design, Handbook of Cognitive Task Design, 383-416, October 2003

[4] Elm et al, Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances, 2002

List of Acronyms
ACWA - Applied Cognitive Work Analysis
CSE - Cognitive Systems Engineering
CTA - Cognitive Task Analysis
CTD - Cognitive Task Design
DCD - Decision-Centered Design
GDTA - Goal Directed Task Analysis
SA - Situation Awareness