1. Introduction
Cognitive Task Analysis (CTA) methods should be integrated into systems engineering since complex systems typically support highly cognitive activities. New system endeavors are geared towards augmenting the users’ macrocognitive functions such as: problem detection, coordination (i.e. – coordinating teams), and planning. Systems fail if they cannot support these cognitive functions, therefore it is important that the CTA practitioners learn what the functions are and relay them back to the systems engineers. Identifying and accommodating cognitive functions not only prevents systems from failing, it also enhances usability by improving the interactions between users and the system.
The field of Cognitive Systems Engineering (CSE) emerged as a solution to the problem of how to effectively integrate CTA methods into systems engineering. CSE combines the capabilities of technologists (i.e. – software engineers and computer scientists) and CTA researchers (i.e. – cognitive scientists and human factors psychologists). Almost all of the CSE strategies depend on CTA since it is important to understand the cognitive work the systems are trying to support. In this paper, I will look at some of the more popular CSE strategies being used today.
2. Situation Awareness (SA)-Oriented Design
Situation Awareness (SA) has been defined as: "the perception of the elements in the environment within a volume of space and time, the comprehension of their meaning, the projection of their status into the near future, and the prediction of how various actions will affect the fulfillment of one's goals" [1]. The SA-Oriented Design strategy is based on the idea that SA is fundamental in driving human decision-making in complex, dynamic environments [2]. Decision-making and performance can be dramatically improved if designs that enhance the users’ situational awareness are leveraged. SA-Oriented Design has been used to develop and evaluate system design concepts in order to improve human decision-making and performance. SA-Oriented Design is comprised of three main components: SA Requirements Analysis, SA-Oriented Design Principles, and SA Measurement and Validation.
For the SA Requirements Analysis, a form of cognitive task analysis called Goal Directed Task Analysis (GDTA) is used. GDTA focuses on uncovering the situation awareness requirements associated with a given job function. The first step in GDTA is to identify the major goals of a job and the sub-goals needed to reach these goals. Next, the major decisions that need to be made for each sub-goal are identified. Finally, the SA requirements needed for making these decisions and carrying out each sub-goal are identified. Since decision-making is shaped by goals, this requirements analysis is based on goals or objectives, not tasks (as in traditional task analysis).
A set of fifty SA-Oriented Design Principles was developed to help in creating a system design that meets the SA requirements. The principles are based on the mechanisms and processes involved in acquiring and maintaining SA. The principles provide design guidelines for: displaying goal-oriented information, dealing with extraneous information, dealing with system complexity, how to convey confidence levels in information, etc [2].
It is important to measure the proposed design concepts in order to determine if the design actually helps SA, or hinders it. Prototyping is an important tool that can be used when measuring the effectiveness of the design; it can rapidly expose the design to subject matter experts. Simulations of new technologies within the operating environment can also prove to be invaluable when measuring design effectiveness on SA.
3. Decision-Centered Design (DCD)
The DCD strategy was created to leverage CTA in the development of new technologies, including complex software systems [3]. This strategy uses CTA methods to specify the primary cognitive requirements for the design process. Importance is placed on the key decisions users have to make when performing their jobs (hence the name Decision-Centered Design). The idea behind DCD is that CTA can help the design team come up solutions that support the users’ when making key decisions.
DCD’s primary focus is on the difficult and challenging decisions the system must support (the “tough cases”). The focus on these tough cases ensures the system to be robust. The system and its users will be able to bounce back from extraordinary circumstances. If the system is successful in supporting tough cases, chances are it will also be successful in supporting the more routine / menial cases. Focusing on tough cases also ensures that the development process is efficient. There are rarely enough resources to cover all of the cognitive decisions in a complex system; covering only the difficult decisions is more efficient as it yields better coverage.
DCD consists of five stages: preparation, knowledge elicitation, analysis & representation, application design, and evaluation. During preparation, the practitioner is to understand the domain and the nature of the work the users are doing. The practitioner also identifies the appropriate CTA methods to use. In knowledge elicitation, the CTA methods are used to perform an examination of the difficult decisions and complex cognitive tasks. The practitioner then identifies the decision requirements by analyzing the recorded data. The analysis is shared with the systems engineers working on the design, and the design is developed iteratively via an open feedback loop with the users. Finally, performance of the system is measured and, if needed, improvements are made to the prototype.
4. Applied Cognitive Work Analysis (ACWA)
The ACWA methodology was developed by the Cognitive Systems Engineering Center in Pittsburgh, PA as the Cognitive Task Design (CTD) portion of a high quality, affordable systems engineering process [4]. It is touted as being at the same maturity level as the best software engineering methodologies. ACWA was specifically developed to satisfy two critical challenges to making CTD part of system engineering efforts. The first challenge is that CTD must become a high quality engineering methodology (on par with the efforts for software engineering processes made by Carnegie Mellon’s Software Engineering Institute). The second challenge is that the results of CTD must seamlessly integrate with the software and system engineering processes used to construct powerful decision support systems. [4]
ACWA contains a step-by-step approach to CTD. The design steps include domain analysis, decision requirements analysis, supporting information analysis, design requirements analysis, and finally design. Each step produces an artifact and the collection of artifacts provides a link from the cognitive task analysis to the system design. The artifacts serve two purposes: they progressively record the results of the design thinking for subsequent steps in the process, and they provide an opportunity to evaluate the quality and completeness of the design effort.
References
[1] Mica R. Endsley, Towards a Theory of Situation Awareness in Dynamic Systems, Human Factors, 37(1), 32-64, 1995
[2] Mica R. Endsley, Cheryl A. Bolstad, Debra G. Jones, and Jennifer M. Riley, Situation Awareness Oriented Design: From User’s Cognitive Requirements to Creating Effective Supporting Technologies, October 2003
[3] Hutton, R. J.B., T. E. Miller, and M. L. Thordson, Decision-Centered Design: Leveraging Cognitive Task Analysis in Design, Handbook of Cognitive Task Design, 383-416, October 2003
[4] Elm et al, Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances, 2002
List of Acronyms
ACWA - Applied Cognitive Work Analysis
CSE - Cognitive Systems Engineering
CTA - Cognitive Task Analysis
CTD - Cognitive Task Design
DCD - Decision-Centered Design
GDTA - Goal Directed Task Analysis
SA - Situation Awareness
Showing posts with label Cognitive Task Analysis. Show all posts
Showing posts with label Cognitive Task Analysis. Show all posts
Wednesday, August 10, 2011
Thursday, February 24, 2011
A Summary of "Understanding How Consumers Make Decisions: Using Cognitive Task Analysis for Market Research" from "Working Minds"
Citation
Crandall, Beth; Klein, Gary; Hoffman, Robert R. "Understanding How Consumers Make Decisions: Using Cognitive Task Analysis for Market Research." Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, Mass.: MIT Press, 2006. 215-228.
Summary / Assessment
In this chapter of Working Minds, the authors demonstrate how the application of cognitive task analysis (CTA) techniques to market research can help clarify various cognitive questions such as: “How do consumers decide whether to purchase a product?” “How do consumers make sense of what the product does or how it works?” -and- “When consumers have developed incorrect or inadequate mental models, what would help them shift to better models so they are more satisfied with the product?” The use of CTA techniques augments the traditional approach of doing market research by enabling us to discover the way consumers think about using and buying products, and how consumers make decisions. (The traditional approach is relegated to studying consumer beliefs, attitudes, and preferences.)
The traditional way of performing market research is to ask consumers how they make their purchasing decisions. This technique is incomplete because consumers are sometimes unable to explicitly express their cognitive processes when choosing a particular product. We must ask the appropriate questions and observe the consumer “in the wild” to infer their decision strategy. CTA techniques also provide a means to elicit and document the internal mental models consumers build about products. Their mental models include how they perceive the product (behaviorally, emotionally, and reflectively), and what makes the product effective or ineffective. In learning about product effectiveness, we also learn about how the product is being used and in what context it is being used.
The authors define the following three basic strategies for applying CTA methods to market research: Concurrent Observations and Interviews, Simulations and Props, and Retrospective Interviews. Concurrent Observations and Interviews are used while the consumers are making product decisions and while the consumers are interacting with products. They are used to record what users are thinking about in situ (not to record remembrances or hypothetical cases). Simulations and Props are used when it isn’t possible to observe consumers in action. We can use this technique to “probe consumer cognition”, to see what consumers were picking up on, and what they were considering or ignoring. Retrospective Interviews can be leveraged when trying to understand reflective issues such as brand loyalty and what prompts users to switch brands. (For example, the consumer’s history with the product is identified and examined in such interviews.)
Just as it is difficult for Consumers to express their cognitive processes when choosing or using a particular product, Users have difficulties when expressing the complex cognitive processes involved in their profession. Herein lies the nexus between studying consumer behavior and designing effective software products. Users may express what they think they need (new technologies, certain design scheme, etc.), but in reality it is hard for users to express -or translate- their true needs into exact specifications. Professionals such as business analysts or requirements engineers can leverage CTA techniques to elicit details about cognitive processes and internal mental models, which can then be incorporated into to functionally accurate and subjectively pleasing designs.
For further reading, I would suggest the entire text of “Cognitive Minds”. It provides a really good explanation of CTA theories and methods. It also provides good practical examples of the application of CTA techniques.
Crandall, Beth; Klein, Gary; Hoffman, Robert R. "Understanding How Consumers Make Decisions: Using Cognitive Task Analysis for Market Research." Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, Mass.: MIT Press, 2006. 215-228.
Summary / Assessment
In this chapter of Working Minds, the authors demonstrate how the application of cognitive task analysis (CTA) techniques to market research can help clarify various cognitive questions such as: “How do consumers decide whether to purchase a product?” “How do consumers make sense of what the product does or how it works?” -and- “When consumers have developed incorrect or inadequate mental models, what would help them shift to better models so they are more satisfied with the product?” The use of CTA techniques augments the traditional approach of doing market research by enabling us to discover the way consumers think about using and buying products, and how consumers make decisions. (The traditional approach is relegated to studying consumer beliefs, attitudes, and preferences.)
The traditional way of performing market research is to ask consumers how they make their purchasing decisions. This technique is incomplete because consumers are sometimes unable to explicitly express their cognitive processes when choosing a particular product. We must ask the appropriate questions and observe the consumer “in the wild” to infer their decision strategy. CTA techniques also provide a means to elicit and document the internal mental models consumers build about products. Their mental models include how they perceive the product (behaviorally, emotionally, and reflectively), and what makes the product effective or ineffective. In learning about product effectiveness, we also learn about how the product is being used and in what context it is being used.
The authors define the following three basic strategies for applying CTA methods to market research: Concurrent Observations and Interviews, Simulations and Props, and Retrospective Interviews. Concurrent Observations and Interviews are used while the consumers are making product decisions and while the consumers are interacting with products. They are used to record what users are thinking about in situ (not to record remembrances or hypothetical cases). Simulations and Props are used when it isn’t possible to observe consumers in action. We can use this technique to “probe consumer cognition”, to see what consumers were picking up on, and what they were considering or ignoring. Retrospective Interviews can be leveraged when trying to understand reflective issues such as brand loyalty and what prompts users to switch brands. (For example, the consumer’s history with the product is identified and examined in such interviews.)
Just as it is difficult for Consumers to express their cognitive processes when choosing or using a particular product, Users have difficulties when expressing the complex cognitive processes involved in their profession. Herein lies the nexus between studying consumer behavior and designing effective software products. Users may express what they think they need (new technologies, certain design scheme, etc.), but in reality it is hard for users to express -or translate- their true needs into exact specifications. Professionals such as business analysts or requirements engineers can leverage CTA techniques to elicit details about cognitive processes and internal mental models, which can then be incorporated into to functionally accurate and subjectively pleasing designs.
For further reading, I would suggest the entire text of “Cognitive Minds”. It provides a really good explanation of CTA theories and methods. It also provides good practical examples of the application of CTA techniques.
Subscribe to:
Posts (Atom)