Leveraging Cognitive Architectures in Usability Engineering

Introduction
In chapter 5 of "The Human-Computer Interaction Handbook", Byrne discusses how cognitive architectures can be applied in usability engineering (Byrne 95).  He mentions that traditional engineering disciplines are grounded in quantitative theory.  Engineers in these disciplines can augment their designs based on predictions derived from such theory.  In contrast, usability engineers do not have these quantitative tools available and therefore every design must be subjected to its own usability test (Byrne 95).  Computational models, based on cognitive architectures, have the potential to give usability engineers quantitative tools similar to those available in traditional disciplines.  With these tools, usability engineers could potentially quantify the effects of changing attributes of a system (e.g. – changing aspects of a user interface).  In this paper, I discuss additional implications and applications of cognitive architectures in usability engineering.

Testing Economy
There are clear, positive implications in leveraging cognitive architectures in usability engineering.  Most apparent is an economy in testing.

Software testing can be costly, especially if the software is intended for a large population of users.  For example, if the software is to be used by a global audience, languages and other aspects of the target cultural ecosystems need to be considered.  Testing would need to be duplicated to support the variance in users.  Additionally, testing can be costly for large systems containing many functional points.  To meet the goal of building a large and functionally accurate system, multiple usability tests are performed that iteratively shape the software being developed.  Usability tests must be altered and additional usability tests need to be created depending on how much the software changes between iterations.  Therefore in highly-iterative development, the total cost of usability testing is multiplied by a factor of the number of iterations the software has gone through.  Another point of consideration is the administrative costs of performing usability testing.  This includes items such as: engineering test cases, writing test plans, distributing test plans, setting up security access for testers, and the coordination and tabulation of test results.

Leveraging a cognitive architecture can help mediate these costs if we can create realistic cognitive agents that model the user base.  The costs of utilizing people to perform testing will be reduced since cognitive agents could test in their place.  For global applications where there is a disparate user base, user differences could be simulated.  For example, varying cultural dimensions could be modeled within the agents.  Costs from iterative development could also be avoided as agents that were constructed for initial tests could simply be reused for subsequent testing. Overhead involved with administering tests will be lessened since there would not be a need for the coordination and distribution of testing among large groups of users.

Social Networking System Application and Usability
In his paper, Dr. Ron Sun explains how CLARION, a cognitive architecture, can be applied to modeling social networks (Sun, 2006).  CLARION is particularly well suited to model social networks since aspects of its various subsystems allow the creation of realistic social cognitive agents. CLARION includes a motivational subsystem that models needs, desires, and motivations within agents. More specifically, it can be used to model the physical and social motivations of the agent as the agent interacts with its environment (Sun, 2006).  Additionally, the agent can understand other agents’ motivational structures, thereby promoting cooperation in a social setting. CLARION also includes a meta-cognitive subsystem that orchestrates the interaction of other subsystems within the architecture. This allows an agent to reflect on and modify its own behaviors, an ability that makes social interaction possible (Tomasello 1999). This “self-monitoring” allows agents to more effectively interact with each other by providing a means for the agent to alter behaviors that may prevent social interaction.

With the ability to effectively model human social aspects, we can use cognitive architectures to perform usability analysis on systems that function within a large social setting (for example: a big city population).  Traditional usability analysis on the effects such systems might not be possible.  The physical deployment of systems to a large community of people is met with several obstacles.  First and foremost is cost involved in usability testing.  As mentioned in the section above, there are overhead costs such as coordination of testing and distributing test plans.  Additionally, a system interface would have to be set up for each person in the community to simulate its effects precisely.  Thus, we would incur major expenses without first understanding potential benefits.  Secondly, the actual coordination of usability testing in a large community would not be feasible. This is because recruiting the number of individuals required isn’t practical.  Finally, there are temporal issues since a social network matures slowly in real-time.  Using a cognitive architecture, we can construct a model of the social network that would enable us to avoid these pitfalls.

Along with mitigating the difficulties in usability analysis, there are other benefits to using cognitive architectures. Parameters of the simulated social network can quickly be changed to model real-life scenarios.  (Parameters include community size, agent type distribution, and epoch length.)  The beliefs, goals, and knowledge of the simulated people (cognitive agents) can also be modified.  Finally, since the system is not deployed to actual users, coordination and deployment of changes to users does not need to occur.  These benefits allow the social model to be adjusted rapidly and without recourse when managing shifting user requirements.  Ultimately, being able to effectively manage change leads to a more usable software system.


References
Byrne, M. D. (2007). "Cognitive architecture." In Sears, A. & Jacko, J. (Eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Second Edition (Human Factors and Ergonomics). (pp. 93-114). Lawrence Erlbaum.

Sun, Ron (2006). “The CLARION cognitive architecture: Extending cognitive modeling to social simulation.” In: Ron Sun (ed.), Cognition and Multi-Agent Interaction. (pp. 1-26) Cambridge University Press, New York.

Tomasello, Michael (1999). The Cultural Origins of Human Cognition. Harvard University Press.