Cognitive Systems Engineering Strategies

1. Introduction
Cognitive Task Analysis (CTA) methods should be integrated into systems engineering since complex systems typically support highly cognitive activities. New system endeavors are geared towards augmenting the users’ macrocognitive functions such as: problem detection, coordination (i.e. – coordinating teams), and planning. Systems fail if they cannot support these cognitive functions, therefore it is important that the CTA practitioners learn what the functions are and relay them back to the systems engineers. Identifying and accommodating cognitive functions not only prevents systems from failing, it also enhances usability by improving the interactions between users and the system.

The field of Cognitive Systems Engineering (CSE) emerged as a solution to the problem of how to effectively integrate CTA methods into systems engineering. CSE combines the capabilities of technologists (i.e. – software engineers and computer scientists) and CTA researchers (i.e. – cognitive scientists and human factors psychologists). Almost all of the CSE strategies depend on CTA since it is important to understand the cognitive work the systems are trying to support. In this paper, I will look at some of the more popular CSE strategies being used today.

2. Situation Awareness (SA)-Oriented Design
Situation Awareness (SA) has been defined as: "the perception of the elements in the environment within a volume of space and time, the comprehension of their meaning, the projection of their status into the near future, and the prediction of how various actions will affect the fulfillment of one's goals" [1]. The SA-Oriented Design strategy is based on the idea that SA is fundamental in driving human decision-making in complex, dynamic environments [2]. Decision-making and performance can be dramatically improved if designs that enhance the users’ situational awareness are leveraged. SA-Oriented Design has been used to develop and evaluate system design concepts in order to improve human decision-making and performance. SA-Oriented Design is comprised of three main components: SA Requirements Analysis, SA-Oriented Design Principles, and SA Measurement and Validation.

For the SA Requirements Analysis, a form of cognitive task analysis called Goal Directed Task Analysis (GDTA) is used. GDTA focuses on uncovering the situation awareness requirements associated with a given job function. The first step in GDTA is to identify the major goals of a job and the sub-goals needed to reach these goals. Next, the major decisions that need to be made for each sub-goal are identified. Finally, the SA requirements needed for making these decisions and carrying out each sub-goal are identified. Since decision-making is shaped by goals, this requirements analysis is based on goals or objectives, not tasks (as in traditional task analysis).

A set of fifty SA-Oriented Design Principles was developed to help in creating a system design that meets the SA requirements. The principles are based on the mechanisms and processes involved in acquiring and maintaining SA. The principles provide design guidelines for: displaying goal-oriented information, dealing with extraneous information, dealing with system complexity, how to convey confidence levels in information, etc [2].

It is important to measure the proposed design concepts in order to determine if the design actually helps SA, or hinders it. Prototyping is an important tool that can be used when measuring the effectiveness of the design; it can rapidly expose the design to subject matter experts. Simulations of new technologies within the operating environment can also prove to be invaluable when measuring design effectiveness on SA.

3. Decision-Centered Design (DCD)
The DCD strategy was created to leverage CTA in the development of new technologies, including complex software systems [3]. This strategy uses CTA methods to specify the primary cognitive requirements for the design process. Importance is placed on the key decisions users have to make when performing their jobs (hence the name Decision-Centered Design). The idea behind DCD is that CTA can help the design team come up solutions that support the users’ when making key decisions.

DCD’s primary focus is on the difficult and challenging decisions the system must support (the “tough cases”). The focus on these tough cases ensures the system to be robust. The system and its users will be able to bounce back from extraordinary circumstances. If the system is successful in supporting tough cases, chances are it will also be successful in supporting the more routine / menial cases. Focusing on tough cases also ensures that the development process is efficient. There are rarely enough resources to cover all of the cognitive decisions in a complex system; covering only the difficult decisions is more efficient as it yields better coverage.

DCD consists of five stages: preparation, knowledge elicitation, analysis & representation, application design, and evaluation. During preparation, the practitioner is to understand the domain and the nature of the work the users are doing. The practitioner also identifies the appropriate CTA methods to use. In knowledge elicitation, the CTA methods are used to perform an examination of the difficult decisions and complex cognitive tasks. The practitioner then identifies the decision requirements by analyzing the recorded data. The analysis is shared with the systems engineers working on the design, and the design is developed iteratively via an open feedback loop with the users. Finally, performance of the system is measured and, if needed, improvements are made to the prototype.

4. Applied Cognitive Work Analysis (ACWA)
The ACWA methodology was developed by the Cognitive Systems Engineering Center in Pittsburgh, PA as the Cognitive Task Design (CTD) portion of a high quality, affordable systems engineering process [4]. It is touted as being at the same maturity level as the best software engineering methodologies. ACWA was specifically developed to satisfy two critical challenges to making CTD part of system engineering efforts. The first challenge is that CTD must become a high quality engineering methodology (on par with the efforts for software engineering processes made by Carnegie Mellon’s Software Engineering Institute). The second challenge is that the results of CTD must seamlessly integrate with the software and system engineering processes used to construct powerful decision support systems. [4]

ACWA contains a step-by-step approach to CTD. The design steps include domain analysis, decision requirements analysis, supporting information analysis, design requirements analysis, and finally design. Each step produces an artifact and the collection of artifacts provides a link from the cognitive task analysis to the system design. The artifacts serve two purposes: they progressively record the results of the design thinking for subsequent steps in the process, and they provide an opportunity to evaluate the quality and completeness of the design effort.

References
[1] Mica R. Endsley, Towards a Theory of Situation Awareness in Dynamic Systems, Human Factors, 37(1), 32-64, 1995

[2] Mica R. Endsley, Cheryl A. Bolstad, Debra G. Jones, and Jennifer M. Riley, Situation Awareness Oriented Design: From User’s Cognitive Requirements to Creating Effective Supporting Technologies, October 2003

[3] Hutton, R. J.B., T. E. Miller, and M. L. Thordson, Decision-Centered Design: Leveraging Cognitive Task Analysis in Design, Handbook of Cognitive Task Design, 383-416, October 2003

[4] Elm et al, Applied Cognitive Work Analysis: A Pragmatic Methodology for Designing Revolutionary Cognitive Affordances, 2002

List of Acronyms
ACWA - Applied Cognitive Work Analysis
CSE - Cognitive Systems Engineering
CTA - Cognitive Task Analysis
CTD - Cognitive Task Design
DCD - Decision-Centered Design
GDTA - Goal Directed Task Analysis
SA - Situation Awareness

The "PersonaBadge" Social Facilitator and Simulations of its Affects on Social Capital Acquisition

This paper presents a social facilitator and its effects on social capital acquisition in large emergent social networks. Firstly, the software and hardware specifications for the social facilitator are explicated. Next, social network simulations and the cognitive architecture used to construct the simulations are described. Finally, the facilitator is tested within the social simulation and results of the simulation are analyzed.

Direct Link to PDF
http://romalley2000.home.comcast.net/documents/persona_doc/documents/omallr2_design_report.pdf

PersonaBadge Prototype Videos
http://www.youtube.com/user/HCICrossroads

Embedded PDF

A Summary of "CLARION" from "A Tutorial on CLARION 5.0"

Citation Sun, Ron. “A Tutorial on CLARION 5.0.” Department of Cognitive Science. Rensselaer Polytechnic Institute. 6 Oct. 2009. http://www.sts.rpi.edu/~rsun/sun.tutorial.pdf

Summary / Assessment
CLARION (short for Connectionist Learning with Adoptive Rule Induction ON-Line) is a cognitive architecture used to simulate human cognition and behavior. Dr Sun has led the development of CLARION at RPI. In the tutorial cited above, Dr. Sun provides an introduction to CLARION and its structure.

One important aspect of CLARION, and something that sets it apart form other cognitive architectures, is the method in which it models human knowledge. In CLARION, knowledge is split up into implicit knowledge, and explicit knowledge. The two types of knowledge are handled differently in the architecture, just as in real-life. For example, implicit knowledge is not directly accessible, but it can be used during computation (just as tacit knowledge is not easily passed on in real-life, but is used by people when solving problems). More specifically, in CLARION, implicit knowledge is modeled as a backpropagation neural network, which accurately represents the distributed and subsymbolic nature of implicit knowledge. On the other hand, explicit knowledge is represented in a symbolic (localist) way. Explicit knowledge is given a direct meaning making it more accessible and interpretable. Explicit knowledge is further divided into “rules” and “chunks”. Rules govern how an agent interacts with its environment (for example: If the stove is hot, don’t touch the stove). Chunks are combinations of implicit dimension/value pairs that are tied together to form a mental concept. For example: table-1: (size, large) (color, white) (number-of legs, four) where “table-1” is the mental concept and the dimension/value pairs are contained in parenthesis.

A layered concept is used when discussing the overall structure of CLARION. Explicit knowledge forms the top layer, while implicit knowledge forms the bottom layer. The architecture is further divided up into subsystems that handle a specific aspect of human cognition. Each subsystem has a module in the explicit layer (top layer), and a module in the implicit layer (bottom layer).

CLARION contains four subsystems: the Action-Centered Subsystem, the Non-Action-Centered Subsystem, the Motivational Subsystem, and the Meta-Cognitive Subsystem. The Action-Centered subsystem represents the part of the mind that controls an agent’s physical and mental actions. (For example, manipulating objects in the real world and adjusting goals.) The ACS uses input from the environment, and other internal information to determine the best action to take. The Non-Action-Centered Subsystem contains what can be considered as general knowledge or “semantic memory”. This type of knowledge includes ideas, objects, and facts. In the NACS, the upper layer contains connections (or associative rules) that connect declarative knowledge chunks, and the bottom layer contains implicit declarative knowledge in the form of dimension/value pair networks. The Motivational Subsystem represents the part of cognition that supplies reasons behind an agent’s actions. The MS describes the “drives” of the agent which in turn determines the agent’s goals. (For example, a drive may be to quench thirst, so the current goal structure would be centered on obtaining a source of water.) The Meta-Cognitive Subsystem is the main controller of cognition. It regulates the communication between all other subsystems. For example: it adds goals to the current goal structure, in response to the drives formulated by the Motivational Subsystem. It also transfers information learned in the ACS to the NACS.

As far as implications for design and HCI: I think that we can learn a great deal about human cognition from utilizing cognitive architectures, and we could leverage them to effectively simulate how our HCI designs will function in the real-world.

Electrode Positions for the Emotiv EPOC Research Neuroheadset

Channel names based on the International 10-20 locations are: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4.

A Summary of "Simulating Organizational Decision-Making Using a Cognitively Realistic Agent Model"

Citation
R. Sun and I. Naveh, Simulating organizational decision-making using a cognitively realistic agent model. Journal of Artificial Societies and Social Simulation, Vol.7, No.3. 2004.

Summary / Assessment
In their paper, Sun and Naveh argue that social simulation needs cognitive science concepts. Social interaction is basically the result of individual cognition; therefore, the more accurately one can model individual agents and their cognitive processes, the more accurate and dependable the social model will become. The authors also argue that utilizing a realistic cognitive model helps us to better understand individual agent cognition, more specifically the socio-cultural aspects of cognition and how individual agents learn from each another. The paper mentions how the CLARION cognitive architecture is well suited to model cognitively accurate agents within a social simulation. The authors describe CLARION’s dual representation of human learning, and go into detail on how the lower layer (the implicit layer) interacts with the upper layer (the explicit layer). They cite this dual-representation as being fundamental in constructing realistic cognitive agents.

The paper moves on to explain an organizational task that is to be studied. The task is to determine if a blip on a radar screen is a hostile aircraft, a flock of geese, or a civilian aircraft (i.e. – whether the blip is an enemy, neutral, or friendly). Each blip has nine attributes that carry a weighted value. If the sum of the attributes is less than 17 it is friendly, if greater than 19 it is hostile, otherwise it is neutral. The organization is presented with 100 problems (100 blips). An organization’s performance is a measure of how accurately it can classify blips.

Originations are set up in one of two ways, as a team structure or as a hierarchy. In a team structure, every agent’s calculations are equally weighted and the final result is determined democratically. In a hierarchy, agents’ decisions are passed on to a supervisor agent, and the supervisor makes the final decision. Additionally, information is disbursed within the organization in one of two ways. In a distributed fashion where no single agent has access to all required pieces of information, or in a blocked fashion where each agent has access to all pieces of required information. (In both cases, all pieces of information are accessible to the agents as whole.)

Earlier studies, performed by Carley et al in 1998, revealed that humans typically perform better in a team situation, especially when information was provided in a distributed manner (no single agent has access to all pieces of required information). The worst performance occurred in hierarchical team structures that used blocked data access (every agent has access to all pieces of required information). In the same study, Carley used four cognitive architectures, other than CLARION, to simulate the same task. Results from these simulations were compared with human data. The comparison showed that the simulations did not provide results that were inline with human capabilities. It is argued that this is due to shortcomings in the architectures to accurately model human intelligence and learning.

Sun and Naveh simulated the same task with the CLARION cognitive architecture. Results from this simulation were compared with the actual human data and it was found that CLARION was able to provide results that were inline with human results. It is argued that the similarity is due to the fact that CLARION accurately models human learning in its dual representation structure. The authors then augmented the simulation by extending runtime, by varying cognitive parameters (such as learning rate, and temperature of randomness), and by introducing differences in the cognitive agents (such as introducing weak learners). The overall result is that when using CLARION, one can create cognitively realistic social simulations yielding results that are in accord with psychological literature.

A Summary of "Collective Intelligence: Mankind’s Emerging World in Cyberspace"

Citation
Levy, Pierre. "Introduction." Collective Intelligence: Mankind’s Emerging World in Cyberspace. Cambridge, Massachusetts: Perseus Books, 1999. 1-19.

Summary / Assessment
In the introduction to his book, Levy first discusses collective intelligence as it relates to economy. He writes: “the prosperity of a nation, geographical region, business, or individual depends on their ability to navigate the knowledge space.” The more we can form intelligent, highly capable communities, the more we can ensure our success in a highly competitive environment. As an example: businesses transform themselves to promote information exchange between departments. This results into what Levy calls “innovation networks”. Departments can easily interact with one another transferring knowledge, personnel, and skills. This allows companies to be more receptive to ever changing demand for skills (such as scientific, technical, social, and aesthetic skills). An organization that is inflexible to changing skills, will eventually collapse.

Levy defines an anthropological space as: “a system of proximity (space) unique to the world of humanity (anthropological), and thus dependant on human technologies, significations, language, culture conventions, representations, and emotions”. Four spaces are defined: earth, territorial space, commodity space, and knowledge space. The Earth Space defines our identity in terms of our “bond with the cosmos”, as well as our affiliation or alliance with other humans (ex: our name is a symbol representing out place in an ancestral line). In the Territorial Space, the meaning of identity shifts. Identity, in this space, is linked with the ownership of property and group affiliations (ex: our home address identifies our geographic location as well as affiliations with certain groups of individuals (our neighborhood)). In the Commodity Space, identity is defined by one’s participation in the process of moving commodities (goods). This includes involvement in the production of goods and involvement in the exchange of goods. In the fourth space, the Knowledge Space, one’s identity is defined by knowledge and the capacity to rapidly acquire knowledge. Levy identifies three aspects of the Knowledge Space: Speed – information/knowledge can rapidly be acquired. Mass – It is impossible to restrict the movement of knowledge; therefore a larger mass of individuals now has access to information/knowledge. Tools – tools have been created that enable individuals to acquire, manage, and filter information as needed.

Levy defines collective intelligence as “a form of universally distributed intelligence, constantly enhanced, coordinated in real-time, and resulting in the effective mobilization of skills.” Knowledge is enhanced at the level of the individual as we try to better ourselves through the acquisition of new skills and abilities. Intelligence is coordinated in real-time through digital mediums and emergent technologies. Skills are mobilized first by acknowledging an individual’s skills, and then by recognizing the individual’s contributions to the collective.

Levy also adds to his definition of collective intelligence by writing, “the basis and goal of collective intelligence is the mutual recognition and enrichment of individuals, rather than the cult of fetishized or hypostatized communities”. In other words, the goal of collective intelligence should be that individuals are compensated or acknowledged based on the quantity and more importantly, the quality of their contribution to the collective. Furthermore, acknowledgement should be made at the level of the individual, rather than at the level of the collective.

A Summary of "The Motivational and Meta-Cognitive Subsystems" from "A Tutorial on CLARION 5.0"

Citation
Sun, Ron. "Chapter 4: The Motivational and Meta-Cognitive Subsystems." "A Tutorial on CLARION 5.0." Department of Cognitive Science. Rensselaer Polytechnic Institute. 6 Oct. 2009. 
http://www.sts.rpi.edu/~rsun/sun.tutorial.pdf

Summary / Assessment
In this chapter, the Motivational Subsystem (MS) and the Meta-Cognitive Subsystem (MCS) of the CLARION architecture are described. The MS is concerned with an agent’s drives and their interactions (i.e. – why an agent does what it does and why it chooses any particular action over another). The MCS controls and regulates cognitive processes. The MCS accomplishes this, for example, by setting goals for the agent and by managing ongoing processes of learning and interactions with the surrounding environment.

Dr. Sun mentions that motivational and meta-level processes are required for an agent to meet the following criteria when performing actions: sustainability, purposefulness, focus, and adaptivity. Sustainability refers to an agent attending to basic needs for survival (i.e. – hunger, thirst, and avoiding danger). Purposefulness refers to an agent selecting activities that will accomplish goals, as opposed to selecting activities completely randomly. Focus refers to an agent’s need to focus its activities on fulfilling a specific purpose. Adaptivity refers to the need of an agent to adapt (i.e. – to learn) to improve its sustainability, purposefulness, and focus.

When modeling a cognitive agent, it is important to include the following considerations concerning drives. Proportional Activation: Activation of drives should be proportional to offsets or deficits within the agent (such as the degree of the lack of nourishment). Opportunism: Opportunities must be factored in when choosing between alternative actions (ex: availability of water may lead an agent to choose drinking water over gathering food, provided that the food deficit is not too high). Contiguity of Actions: A tendency to continue the current action sequence to avoid the overhead of switching to a different action sequence (i.e. – avoid “thrashing”). Persistence: Actions to satisfy a drive should persist beyond minimum satisfaction. Interruption When Necessary: Actions for a higher priority drive should be interrupted when a more urgent drive arises. Combination of Preferences: Preferences resulting from different drives could be combined to generate a higher-order preference. Performing a “compromise candidate” action may not be the best for any single drive, but is best in terms of the combined preference.

Specific drives are then discussed. Drives are segmented into three categories: Low-Level Drives, High-Level Drives, and Derived Secondary Drives. Low-Level drives include physiological needs such as: get-food, get-water, avoid-danger, get-sleep, and reproduce. Low-Level drives also include “saturation drives” such as: avoid-water-saturation, and avoid-food-saturation. High-Level drives include “needs” such as: belongingness, esteem, and self-actualization (and others from Maslow’s needs hierarchy). Derived Secondary Drives include: gradually acquired drives through conditioning (i.e. – associating a secondary goal to a primary drive), and externally set drives (i.e. – drives resulting from the desire to please superiors in a work environment).

Meta-cognition refers to one’s knowledge of one’s own cognitive process. It also refers to the monitoring and orchestration of cognitive processes in the service of some concrete goal or objective. These concepts are operationalized within CLARION’s MCS through the following processes: 1) Behavioral Aims: which set goals and their reinforcements, 2) Information Filtering: which determines the selection of input values from the environment, 3) Information Acquisition: which selects learning methods, 4) Information Utilization: which refers to reasoning, 5) Outcome Selection: or determining the appropriate outputs, 6) Cognitive Modes: or the selection of explicit processing, implicit processing, or combination thereof, and 7)Parameter Settings: such as parameters for learning capability (i.e. – intelligence level).

A Summary of "The Key Characteristics of Produsage" from "Blogs, Wikipedia, Second Life, and Beyond - From Production to Produsage"

Citation
Bruns, Axel. "The Key Characteristics of Produsage." Blogs, Wikipedia, Second Life, and Beyond - From Production to Produsage. New York, NY.: Peter Lang Publishing, Inc, 2008-2009. 9-36.

Summary / Assessment
In the second chapter of “Blogs, Wikipedia, Second Life, and Beyond”, Axel Bruns discusses the idea of Produsage. He begins the chapter by giving background information on the traditional model of production. The traditional model contains three clearly defined, separate tasks: the producer, the distributor, and the consumer. One side effect of the traditional model is consumers are not active in product development. Highly competitive producers shield product development thereby making consumer involvement impossible. The traditional model gradually changed to involve consumers in a limited manner through focus groups and general market research. The consumer became more than just an end user of fixed products. Through feedback, the consumer was able to alter products based on his/her needs. (The term “prosumer” is used to describe such consumers.)

Bruns then discusses how the Internet shifts information consumption to information usage, and how it challenges the traditional stance on information production. He demonstrates this shift through the following points: With the internet, information access is on a pull-basis, rather than a traditional push-basis. Access to producing and distributing information is readily available and is not as limited as it once was. With technology, users can communicate and engage directly with one another, bypassing the traditional model. Digital information can be shared quickly and can be remixed to create new artifacts.

By means of the Internet, producers are able to move from the traditional hierarchical model to the distributed and communal network model. The term “hive-mind” is used to describe such a model. Here, users are intercreative and there is a “collective intelligence” that emerges. Four attributes of the collective model are identified: 1) Problem solving is probabilistic, not directed. Users can take a more holistic view of the system, leading to contributions in areas outside of those they may have been bounded to within a more traditional approach. 2) Equipotentiallity, not hierarchy. Equipotentiallity is the assumption that each participant can make constructive contributions to the system. 3) Granular tasks. Tasks should not be too complex to require significant administrative overhead. The size of each task must support probabilistic problem solving and Equipotentiallity of contributors. 4) Content is shared, not owned. Sharing content is fundamental to collaboration and supports the three previous points.

With a communal/network model, the idea of Produsage is possible. The cogently terse definition of Produsage given by the author is “the collaborative and continuous building and extending of existing content in the pursuit of further improvement.” The four principles of Produsage are: 1) Open Participation, Communal Evaluation. The basic idea here is that all users are able to participate in achieving the goal, and the more users participating, the more probability there is of identifying the most appropriate/correct solution. 2) Fluid Hierarchy, AD Hoc Meritocracy. “Ad-hocracies” are fluidly built based on the ideas of Equipotentiallity and communal evaluation. Users who actively contribute relevant material to the group have a higher standing, conversely, the user’s standing within the group can diminish if the quality of their contributions decline. 3) Unfinished Artifacts, Continuing process. The idea is that Produsage does not work towards the completion of products, but rather through iteration, creates successfully better products (i.e. - artifacts). 4) Common Property, Individual Rewards. Instead of focusing on monetary rewards, participation in Produsage is motivated by contributing to a communal purpose. These non-monetary rewards grow a community by encouraging individuals to make continual contributions to the overall goal.

A Summary of "Understanding How Consumers Make Decisions: Using Cognitive Task Analysis for Market Research" from "Working Minds"

Citation
Crandall, Beth; Klein, Gary; Hoffman, Robert R. "Understanding How Consumers Make Decisions: Using Cognitive Task Analysis for Market Research." Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, Mass.: MIT Press, 2006. 215-228.

Summary / Assessment
In this chapter of Working Minds, the authors demonstrate how the application of cognitive task analysis (CTA) techniques to market research can help clarify various cognitive questions such as: “How do consumers decide whether to purchase a product?” “How do consumers make sense of what the product does or how it works?” -and- “When consumers have developed incorrect or inadequate mental models, what would help them shift to better models so they are more satisfied with the product?” The use of CTA techniques augments the traditional approach of doing market research by enabling us to discover the way consumers think about using and buying products, and how consumers make decisions. (The traditional approach is relegated to studying consumer beliefs, attitudes, and preferences.)

The traditional way of performing market research is to ask consumers how they make their purchasing decisions. This technique is incomplete because consumers are sometimes unable to explicitly express their cognitive processes when choosing a particular product. We must ask the appropriate questions and observe the consumer “in the wild” to infer their decision strategy. CTA techniques also provide a means to elicit and document the internal mental models consumers build about products. Their mental models include how they perceive the product (behaviorally, emotionally, and reflectively), and what makes the product effective or ineffective. In learning about product effectiveness, we also learn about how the product is being used and in what context it is being used.

The authors define the following three basic strategies for applying CTA methods to market research: Concurrent Observations and Interviews, Simulations and Props, and Retrospective Interviews. Concurrent Observations and Interviews are used while the consumers are making product decisions and while the consumers are interacting with products. They are used to record what users are thinking about in situ (not to record remembrances or hypothetical cases). Simulations and Props are used when it isn’t possible to observe consumers in action. We can use this technique to “probe consumer cognition”, to see what consumers were picking up on, and what they were considering or ignoring. Retrospective Interviews can be leveraged when trying to understand reflective issues such as brand loyalty and what prompts users to switch brands. (For example, the consumer’s history with the product is identified and examined in such interviews.)

Just as it is difficult for Consumers to express their cognitive processes when choosing or using a particular product, Users have difficulties when expressing the complex cognitive processes involved in their profession. Herein lies the nexus between studying consumer behavior and designing effective software products. Users may express what they think they need (new technologies, certain design scheme, etc.), but in reality it is hard for users to express -or translate- their true needs into exact specifications. Professionals such as business analysts or requirements engineers can leverage CTA techniques to elicit details about cognitive processes and internal mental models, which can then be incorporated into to functionally accurate and subjectively pleasing designs.

For further reading, I would suggest the entire text of “Cognitive Minds”. It provides a really good explanation of CTA theories and methods. It also provides good practical examples of the application of CTA techniques.

A Summary of "Chapter 3: Natural Interaction" from "The Design of Future Things"

Citation
Norman, Donald A. "Chapter 3: Natural Interaction." The Design of Future Things. New York: Basic Books, 2007. 57-90.

Summary / Assessment
In this chapter of the “Design of Future Things”, Donald Norman discusses ideas surrounding the incorporation of natural communication within designs. He draws a distinction between communication and signaling. He writes that “interactive” devices of today signal their users, rather than provide an effective means of natural communication. A dishwasher beeps when the dishes are done. A microwave beeps when food is ready. Such signals may be useful in isolation, but a cacophony of these types of signals may prove to be distracting, un-interpretable, and potentially dangerous. We should use more natural communication and sounds in our designs. Natural sounds (i.e. - sounds we encounter everyday, not sounds generated by an electronic device) can provide the location of an object, reveal their composition, and reveal their activity. The primary example of natural sound/interaction given by Norman is the whistling tea kettle.

This natural communication is referred to as implicit communication. Implicit communication also includes communication afforded by the natural side effects of people’s activities. The messy research laboratory provides the implicit signal that it is being used. Footprints in the sand implicitly tell us that someone has passed by earlier. The presence of sticky notes or underlined passages in a book tells us that the book has been read. These “non-purposeful” clues can inform us of what is happening or what has happened, provide awareness of the environment, and let us know if we should take action or continue on with what we are doing.

Affordances are “the range of activities an animal or person can perform upon an object in the world.” For example: a chair affords sitting or hiding-behind for an adult, but not for an elephant. Affordances are not attributes of an object, but rather, relationships between agents and objects. Affordances exist whether or not they have been discovered; the design challenge is to make affordances apparent to users. If affordances are easily apparent, they guide users’ behavior, and they make object interaction intuitive and natural.

Interaction with autonomous, intelligent devices is particularly challenging because communication has to go both ways (person-to-machine and machine-to-person). Norman offers horseback riding as a good example of interaction between intelligent agents. An important aspect of horseback riding is “tight-reign” and “loose-reign” control. In tight-reign control, power shifts from the horse to the rider; in loose-reign control, power shifts from the rider to the horse. Loose-reign control allows the horse to be more autonomous; however the rider can still provide some oversight through natural interaction of verbal commands, and heel kicks. This idea of allowing the natural variance of independence and interaction is powerful and can be incorporated into designing human-machine interactions.

In the remainder of the chapter, Norman makes a few other points germane to the design of human-machine interaction.

• Be Predictable. - Intelligent machines of the future should not attempt to read user’s minds or predict users’ next actions. There are two issues in doing this: firstly, predictions could be wrong, and secondly, it makes the machine’s actions unpredictable. Unpredictability leads to the user guessing at what the machine is trying to do.
• Don’t distance users from implicit communication. – Today’s automobile isolates its users from certain implicit communication, thereby reducing situational awareness. The user relies more and more on the technology in the automobile (such as automatic lane-keeping). This distancing and reliance can potentially make the automobile more dangerous to operate.
• The best designs compensate human intelligence, rather than supersede it. – For example: power-assisted devices can augment human capabilities, but they also can limit human capabilities where needed.

A Framework for Online Charitable Giving

This paper presents a framework that can be utilized by charitable organizations to increase online giving. The framework offers a systematic approach to thinking about online giving, and provides tools to construct robust mechanisms for eliciting and collecting donations. The framework consists of three components: a persuasive design component, an emotional design component, and a donation-usability component.

Direct Link to PDF
http://romalley2000.home.comcast.net/documents/OMalley_Charitable_Giving_Framework.pdf

Embedded PDF

Making jqGrid work with a .NET Web Service pt. 2

This is a follow up to my post: Making jqGrid work with a .NET Web Service.   I was asked to attach a full working example of the code I wrote in that post.

The attached Visual Studio project demonstrates how to use the GetJqgridJson method with a simple jqGrid and webservice.  (The GetJqgridJson method converts a DataTable to a JSON string suitable for jqGrid consumption.)

Download the Code
Note: To get the example to run, you will need to change the url parameter in the AJAX call on the default.aspx page.  Make sure the url parameter points to the correct location and port on your machine.

A Summary of "Beyond the Interface: Users’ Perceptions of Interaction and Audience on Websites"

Citation
Light, A.; Wakeman, I. "Beyond the Interface: Users’ Perceptions of Interaction and Audience on Websites.” Interacting with Computers 13 (2001) 325-351.

Summary / Assessment
In “Beyond the Interface”, the authors present results of a study to measure user behavior and thought processes during website interaction. The authors’ intent is to describe how the process of interacting with websites brings about two levels of awareness in the user: (1) awareness of the interface, and (2) awareness of the social context beyond the interface.

The study is grounded in several HCI theories that place focus on using the Web as a medium for communication, rather than using it as a tool for problem solving. Perhaps one of the most important theories suggests user interactions follow similar rules to the “turn taking of spoken engagement”. When these rules are violated, users have a hard time understanding the behavior of the software. Another theory leveraged in the study deals with the idea of “ritual constraints”. “Ritual constraints” are social rules inherent in the nature of human interaction. Here, the user is cognizant of the implication of his or her acts, specifically, how the acts are interpreted by others in the social setting.

Data was gathered from twenty participants on various computer-related activities such as downloading software, using search engines, and subscribing to magazines. Accounts were collected using a retrospective interviewing technique (Vermersch’s Explicitation Interviewing Technique). Behavior and thoughts of the users were recorded in audio for analysis. The goal of the analysis was to discover patterns in the users’ accounts and differences between the accounts.

The study revealed the main point regarding user awareness. Users are aware of the two levels of interaction, one with the interface, and one with the recipients behind the interface. The second awareness seemed to present itself when users were actually entering data/interacting with the website, as apposed to simply navigating the website. The corollary is the importance for developers to adopt a communication metaphor when creating interactive components on a website. The study revealed another interesting point regarding the user’s perception of the recipients of inputted data. Recipient identity is constructed from a combination of the user’s expectations of a commercial brand, the user’s experience with interactive portions of the site, and the user’s purpose in visiting the site.

The paper concludes with recommendations on how to apply the findings of the study. The findings suggest that interactive websites would benefit from: proper identification of the recipient of the user’s data (especially if the data is personal in nature), confirmation pages after the entry of data, statements about how the recipients will use the submitted data, and an explanation of why the recipient is collecting data that does not appear to assist in the completion of the user’s task.

I think the ideas expounded upon in this paper are of great importance to the HCI professional. They help us to understand not only what users think, but how users think when engaged in discourse with interactive web applications. We can use this information when designing web applications, or any other artifact, to help us predict how users will internalize the artifact (i.e. - what mental models the users will build).

Leveraging Social Psychology Theories in Software Requirements Elicitation

I wrote this paper for my Theory and Research in HCI master's course at RPI.  The paper discusses exemplary social psychology theories that can be leveraged in the elicitation of software system requirements. Firstly, social psychology theories such as the covariation model and halo effect are presented, each theory is then explained in terms of social context, and finally, links between the theories and requirements engineering are explicated.

Direct Link to PDF
http://romalley2000.home.comcast.net/documents/OMalley_Social_Psych_Requirements.pdf

Embedded PDF

Leveraging Cognitive Architectures in Usability Engineering

Introduction
In chapter 5 of "The Human-Computer Interaction Handbook", Byrne discusses how cognitive architectures can be applied in usability engineering (Byrne 95).  He mentions that traditional engineering disciplines are grounded in quantitative theory.  Engineers in these disciplines can augment their designs based on predictions derived from such theory.  In contrast, usability engineers do not have these quantitative tools available and therefore every design must be subjected to its own usability test (Byrne 95).  Computational models, based on cognitive architectures, have the potential to give usability engineers quantitative tools similar to those available in traditional disciplines.  With these tools, usability engineers could potentially quantify the effects of changing attributes of a system (e.g. – changing aspects of a user interface).  In this paper, I discuss additional implications and applications of cognitive architectures in usability engineering.

Testing Economy
There are clear, positive implications in leveraging cognitive architectures in usability engineering.  Most apparent is an economy in testing.

Software testing can be costly, especially if the software is intended for a large population of users.  For example, if the software is to be used by a global audience, languages and other aspects of the target cultural ecosystems need to be considered.  Testing would need to be duplicated to support the variance in users.  Additionally, testing can be costly for large systems containing many functional points.  To meet the goal of building a large and functionally accurate system, multiple usability tests are performed that iteratively shape the software being developed.  Usability tests must be altered and additional usability tests need to be created depending on how much the software changes between iterations.  Therefore in highly-iterative development, the total cost of usability testing is multiplied by a factor of the number of iterations the software has gone through.  Another point of consideration is the administrative costs of performing usability testing.  This includes items such as: engineering test cases, writing test plans, distributing test plans, setting up security access for testers, and the coordination and tabulation of test results.

Leveraging a cognitive architecture can help mediate these costs if we can create realistic cognitive agents that model the user base.  The costs of utilizing people to perform testing will be reduced since cognitive agents could test in their place.  For global applications where there is a disparate user base, user differences could be simulated.  For example, varying cultural dimensions could be modeled within the agents.  Costs from iterative development could also be avoided as agents that were constructed for initial tests could simply be reused for subsequent testing. Overhead involved with administering tests will be lessened since there would not be a need for the coordination and distribution of testing among large groups of users.

Social Networking System Application and Usability
In his paper, Dr. Ron Sun explains how CLARION, a cognitive architecture, can be applied to modeling social networks (Sun, 2006).  CLARION is particularly well suited to model social networks since aspects of its various subsystems allow the creation of realistic social cognitive agents. CLARION includes a motivational subsystem that models needs, desires, and motivations within agents. More specifically, it can be used to model the physical and social motivations of the agent as the agent interacts with its environment (Sun, 2006).  Additionally, the agent can understand other agents’ motivational structures, thereby promoting cooperation in a social setting. CLARION also includes a meta-cognitive subsystem that orchestrates the interaction of other subsystems within the architecture. This allows an agent to reflect on and modify its own behaviors, an ability that makes social interaction possible (Tomasello 1999). This “self-monitoring” allows agents to more effectively interact with each other by providing a means for the agent to alter behaviors that may prevent social interaction.

With the ability to effectively model human social aspects, we can use cognitive architectures to perform usability analysis on systems that function within a large social setting (for example: a big city population).  Traditional usability analysis on the effects such systems might not be possible.  The physical deployment of systems to a large community of people is met with several obstacles.  First and foremost is cost involved in usability testing.  As mentioned in the section above, there are overhead costs such as coordination of testing and distributing test plans.  Additionally, a system interface would have to be set up for each person in the community to simulate its effects precisely.  Thus, we would incur major expenses without first understanding potential benefits.  Secondly, the actual coordination of usability testing in a large community would not be feasible. This is because recruiting the number of individuals required isn’t practical.  Finally, there are temporal issues since a social network matures slowly in real-time.  Using a cognitive architecture, we can construct a model of the social network that would enable us to avoid these pitfalls.

Along with mitigating the difficulties in usability analysis, there are other benefits to using cognitive architectures. Parameters of the simulated social network can quickly be changed to model real-life scenarios.  (Parameters include community size, agent type distribution, and epoch length.)  The beliefs, goals, and knowledge of the simulated people (cognitive agents) can also be modified.  Finally, since the system is not deployed to actual users, coordination and deployment of changes to users does not need to occur.  These benefits allow the social model to be adjusted rapidly and without recourse when managing shifting user requirements.  Ultimately, being able to effectively manage change leads to a more usable software system.


References
Byrne, M. D. (2007). "Cognitive architecture." In Sears, A. & Jacko, J. (Eds.). The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Second Edition (Human Factors and Ergonomics). (pp. 93-114). Lawrence Erlbaum.

Sun, Ron (2006). “The CLARION cognitive architecture: Extending cognitive modeling to social simulation.” In: Ron Sun (ed.), Cognition and Multi-Agent Interaction. (pp. 1-26) Cambridge University Press, New York.

Tomasello, Michael (1999). The Cultural Origins of Human Cognition. Harvard University Press.

Tools for Aligning Mental Models with Conceptual Models

Introduction
As HCI professionals, a major goal is for our users to construct mental models that are closely aligned to our conceptual design models.   If the two models match up, the users will understand the system and how to use it.   We typically don't speak directly with the users, so our only means of communicating the system is through what Norman refers to as the “System Image”.   The system image is comprised of the user interface as well as any other artifacts related to the system (e.g. - user manuals and training materials).  Therefore, in order to meet our goal, we must construct an appropriate and functionally accurate system image for our users.

To construct the system image, it is important to (1) understand our users and (2) understand the tasks our users need to perform.  The reading material from this week describes methods for understanding users.  We can leverage ideas from Light and Wakeman to understand how users perceive certain aspects of the system interface in regards to awareness.  We can also leverage the differential psychology theories mentioned in Dillon and Watson to understand our users' cognitive abilities.   In understanding users' tasks, task analysis and requirements elicitation methods can be utilized.  Furthermore, a lot of work has been done on the application of cognitive task analysis to draw out tacit knowledge that users may hold about their business processes.

Beyond the Interface
In “Beyond the Interface” from the Human-Computer Interaction Handbook, Light and Wakeman present results of a study to measure user behavior and thought processes during website interaction.  The authors’ intent is to describe how the process of interacting with websites brings about two levels of awareness in the user: (1) awareness of the interface, and (2) awareness of the social context beyond the interface.  The study is grounded in HCI theories that place focus on using the Web as a medium for communication, rather than using it as a tool for problem solving.  Perhaps one of the most important theories suggests user interactions follow similar rules to the “turn taking of spoken engagement”.  When these rules are violated, users have a hard time understanding the behavior of the software.   Another theory leveraged in the study deals with the idea of “ritual constraints”.  “Ritual constraints” are social rules inherent in the nature of human interaction.  Here, the user is cognizant of the implication of his or her acts, specifically, how the acts are interpreted by others in the social setting.

The study revealed the main point regarding user awareness.   Users are aware of the two levels of interaction, one with the interface, and one with the recipients behind the interface.  The second awareness seemed to present itself when users were actually entering data/interacting with the website, as apposed to simply navigating the website.  The corollary is the importance for developers to adopt a communication metaphor when creating interactive components on a website.  The study revealed another interesting point regarding the user’s perception of the recipients of inputted data.  Recipient identity is constructed from a combination of the user’s expectations of a commercial brand, the user’s experience with interactive portions of the site, and the user’s purpose in visiting the site.

The paper concludes with recommendations on how to apply the findings of the study.   The findings suggest that interactive websites would benefit from: proper identification of the recipient of the user’s data (especially if the data is personal in nature), confirmation pages after the entry of data, statements about how the recipients will use the submitted data, and an explanation of why the recipient is collecting data that does not appear to assist in the completion of the user’s task.

User Analysis in HCI
In their paper from the Human-Computer Interaction Handbook, Dillon and Watson assert that user analysis in HCI can benefit from the application of differential psychology research.   User analysis is normally relegated to understanding users in general terms such as educational background and technical expertise.  (Nielsen's three dimensional analysis of users is cited.)  The authors argue that such general user analysis is highly context sensitive and it does not offer solutions that can satisfy unique users/user groups.  The aim of differential psychology is to understand user behavior and specific aptitudes of users such as memory, perceptual speed and, deductive reasoning.

Dillon and Watson go on to describe studies of how work done in differential and experimental psychology was leveraged directly in the design of user interfaces.  One study focused on logical reasoning, the other on visual ability.  They concluded that appropriate design of the interfaces/training material can reduce discrepancies in cognitive abilities amongst users.

Application
I think it's apparent how we can apply the ideas presented in both papers mentioned above. They demonstrate not only how users think, but what users think when engaged in discourse with interactive web applications.  Also demonstrated is how this knowledge can be applied directly to user interface design.  I think the idea of the “conversation” with the system is a huge factor when measuring how closely the user's mental model matches up to the conceptual design model.  I believe its the best way we can describe software systems.  After all, “conversations” define software use cases, and software use cases define a system.

A Regular Expression for Multiple Forms of Scientific Notation

I needed to create a regular expression to match numbers formatted in scientific notation.  It turns out there are many forms used around the world, so coming up with a regex to match all possible variations was a bit of a challenge.  Well, here it goes... 

I found the following “acceptable” forms of scientific notation:
123,000,000,000 can be written as:
1.23E+11
1.23e+11
1.23X10^11
1.23x10^11
1.23X10*11
1.23x10*11

0.000001 can be written as:
1.0E-6
1.0e-6
1.0X10^-6
1.0x10^-6
1.0X10*-6
1.0x10*-6

Additionally…

• A positive or a negative sign can be appended to the beginning to represent a positive or a negative number respectively. For example:
• +1.0E-6
• -1.0E-6
• The decimal point and the standalone 0 following the decimal point may be omitted. For example:
• 1E-6
• The positive sign following the E, e, ^, or * may be omitted. For example:
• 1.23E11

The regular expression I derived to match all forms mentioned above is:
^[+|-]?\d\.?\d{0,}[E|e|X|x](10)?[\^\*]?[+|-]?\d+$
 

Values Engendered by Revision Control

Introduction
This paper presents a Value-Sensitive-Design (VSD) conceptual investigation of revision control. It focuses on revision control as it is employed when constructing software applications. Firstly, the sociotechnical problem space is explicated by 1) defining revision control and 2) explaining how organizations can implement revision control through the use of specialized tools. Next, implicated values are identified and defined in terms of interactions with version control tools. Finally, stakeholders are identified and the effect of implicated values on stakeholders is analyzed

Sociotechnical Problem Space
Revision control (also known as version control) is used to manage changes in documents, source code, or any other type of file stored on a computer. Revision control is typically used in software engineering when many developers are making contributions/changes to source code files. As changes to a source code file are committed, a new version of the file is created. Versions of a file are identified either by date or by a sequence number (i.e. – “version 1”, version 2”, etc.). Each version of the file is stored for accountability and stored revisions can be restored, compared, and merged.

The fundamental issue that version control sidesteps is the race condition of multiple developers reading and writing to the same files. For instance, Developer A and Developer B download a copy of a source code file from a server to their local PC. Both developers begin editing their copies of the file. Developer A completes his/her edits and publishes his/her copy of the file to the server. Developer B then completes his/her edits and publishes his/her copy of the file, thereby overwriting Developer A’s file and ultimately erasing all of Developer A’s work.

There are two paradigms that can be used to solve the race condition issue: file locking and copy-modify-merge (Collins-Sussman).

File locking is a simple concept that permits only one developer to modify a file at any given time. To work on a file, Developer A must “check out” the file from the repository and store a copy of the file to his/her local PC. While the file is checked out, Developer B (or any developer for that matter) cannot make edits to the file. Developer B may begin to make edits only after Developer A has “checked in” the file back into the file repository. File locking works but has its drawbacks. File locking can cause administrative problems. For example: a developer may forget to check in a file effectively locking the file out and preventing any other developer from doing work. File locking also causes unnecessary serialization. For example: two developers may want to make edits to different parts of a file that don’t overlap. No problems would arise if both developers could modify the file, and then merge the changes together. File-locking prevents concurrent updates by multiple developers so work has to be done in-turn.

In the copy-modify-merge paradigm, each developer makes a local “mirror copy” of the entire project repository. Developers can work simultaneously and independently from one another on their local copies. Once updates are complete, the developers can push their local copies to the project repository where all changes are merged together into a final version. For example: Developer A and Developer B make changes to the same file within their own copies of the project repository. Developer A saves his/her changes to the global repository first. When Developer B attempts to save his/her changes, the developer is informed that their copy is out of date (i.e. – other changes were committed while he/she was working on the file). Developer B can then request that Developer A’s changes be merged into his/her copy. Once the changes are merged, and if there are no conflicts (i.e. – no changes overlap), Developer B’s copy is then saved into the repository. If there are conflicts, Developer B must resolve them before saving the final copy to the project repository.

A development organization may implement revision control through the use of specialized tools dedicated to source code management. There are several open-source and commercial tools available, each with their advantages and drawbacks. Subversion, an open-source software package, is a well-known and widely used tool (Tigris). Subversion (“SVN”) uses a client-server model. Source code files are stored on the SVN server (aka “repository”) and can be accessed by any PC’s running the SVN client. This allows many developers to work on source code files from different locations/PC’s. Some key features of SVN are: utilization of copy-modify-merge (and file-locking if needed), full directory versioning, atomic commits to the repository, and versioned metadata for each file/directory.

Values Defined
The use of a good revision control methodology engenders several values within a development organization. This section identifies and defines some of these values.

By leveraging revision control, an organization fosters collaboration between its developers. Gray defines collaboration as “a process of joint decision making among key stakeholders of a problem domain about the future of that domain” (Gray, p.11). Source control permits developers to work in teams where each individual can contribute to the overall goal of delivering a quality software product. Each individual makes decisions on which piece of code will work best to reach that goal. The future of the domain, or software release, is defined by the collaborative effort of developers within the workspace.

Revision control usage also engenders accountability. In their book, Friedman et al write: “accountability refers to the properties that ensure the actions of a person, people, or institution may be traced uniquely to the person, people, or institution” (Friedman). Upon change commit (i.e. - submitting a change to the repository), revision control tools record the responsible developer and place a timestamp on the new version of the file. Moreover, the developer can enter comments to describe changes that he/she has made. For these reasons, revision control tools provide a good mechanism for accountability as a complete audit trail of change is recorded.

Another value brought about by revision control is work efficiency. This is especially true when the copy-modify-merge paradigm is utilized. The major advantage of this paradigm is that it allows developers to work individually and concurrently, thereby maximizing available development time. Compare this to the file-lock paradigm where developers can be locked out a file at any given time. Additionally, copy-modify-merge minimizes the coordination effort and expense between developers.

Along with the values stated above, revision control also: enhances communication between developers, prevents loss of work through backups, enables better coordination of efforts, manages code merges, and provides code stability by allowing organizations to rollback to previous versions of the code (O'Sullivan).

Stakeholders
The most apparent direct stakeholders are the software developers. Revision control benefits developers by providing them with a more stable work environment. Without revision control, it is very easy to experience loss of work. Race conditions can occur if multiple developers are sharing the same copy of files. The danger of overwriting updates is real, and it increases exponentially as the project size and organization size increase. Moreover, a complete loss of data can be avoided as copies of code files are constantly being generated and backed-up.

Another benefit for developers is comprehensibility of the system code lifecycle. Developers can review the ancestry of files and by reading other developer’s comments they can elicit the reasoning behind code changes. This information helps ensure that they stay the course of the current branch of development.

In a hierarchical organization, the indirect stakeholders are members of management (ex. - IT Team Leaders). IT Team Leaders are rated on how well their teams meet project timeline and budgetary expectations. Development teams have a better chance at hitting targets with a revision control strategy, as pitfalls that cause delays and unexpected costs can be avoided. Consequently, benefits of meeting targets get cascaded up to higher levels of management within the organization.

End users of the constructed software product are also indirect stakeholders. All of the benefits garnered from revision control are ultimately parlayed into building a more usable and functionally accurate software product that is intended for end user consumption.


References
Collins-Sussman, Ben. "Version Control with Subversion". Tigris.org. 10/20/2009 http://svnbook.red-bean.com/en/1.4/index.html.

Friedman, B., Kahn, P. H., Jr., & Borning, A. (2006). Value Sensitive Design and information systems. In P. Zhang & D. Galletta (eds.), Human-Computer Interaction in Management Information Systems: Foundations, (pp. 348-372). Armonk, New York: M.E. Sharpe.

Gray, Barbara. Collaborating: Finding Common Ground for Multiparty Problems. San Francisco: Jossey-Bass, 1989.

O'Sullivan, Bryan. "Making Sense of Revision-control Systems". ACM. 10/20/2009 http://queue.acm.org/detail.cfm?id=1595636.

Tigris. "Subversion Home Page". Tigris.org. 10/19/2009 http://subversion.tigris.org/.

Terry Winograd's Shift from AI to HCI

Introduction
In a more recent paper [1], Terry Winograd discusses the gulf between Artificial Intelligence and Human-Computer Interaction. He mentions that AI is primarily concerned with replicating the human / human mind whereas HCI is primarily concerned with augmenting human capabilities. One question is wether or not we should use AI as a metaphor when constructing human interfaces to computers. Using the AI metaphor, the goal is for the user to attribute human characteristics to the interface, and communicate with it just as if it were a human. There is also a divide in how researches attempt to understand people. The first approach, what Winograd refers to as, the “rationalistic” approach, attempts to model humans as cognitive machines within the workings of the computer. In contrast, the second approach, the “design” approach, focuses on modeling the interactions between a person and the enveloping environment.

During his career, Winograd shifted interests and crossed the gulf between AI and HCI. In his paper, he mentions that he started his career in the AI field, then rejected the AI approach, and subsequently ended up moving to the field of HCI. He writes “I have seen this as a battle between two competing philosophies of what is most effective to do with computers”. This paper looks at some of the work Winograd has done, and illustrates his shift between the two areas.

Winograd's Ph.D. Thesis and SHRDLU
In his Ph.D. thesis entitled “Procedures as a Representation for Data in a Computer Program for Understanding Natural Language” [2], Winograd describes a software system called SHRDLU that is capable of carrying on a English conversation with its user. The system contains a simulation of a robotic arm that can rearrange colored blocks within its environment. The user enters into discourse with the system and can instruct the arm to pick up, drop, and move objects. A “heuristic understander” is used by the software to infer what each command sentence means. Linguistic information about the sentence, information from other parts of the discourse, and general information are used to interpret the commands. Furthermore, the software asks for clarification from the user if it cannot understand what the inputted sentence means.

The thesis examines the issue of talking with computers. Winograd underscores the idea that it is hard for computers and human to communicate since computers communicate in their own terms; The means of communication is not natural for the human user. More importantly, computers aren't able to use reasoning in an attempt to understand ambiguity in natural language. Computers are typically only supplied with syntactical rules and do not use semantic knowledge to understand meaning. To solve this problem, Winograd suggests giving computers the ability to use more knowledge. Computers need to have knowledge of the subject they are discussing and they must be able to assemble facts in such a way so that they can understand a sentence and respond to it. In SHRDLU knowledge is represented in a structured manner and uses a language that facilitates teaching the system about new subject domains.

SHRDLU is a rationalistic attempt to model how the human mind works. It seeks to replicate human understanding of natural language. Although this work is grounded in AI, there a clear implications for work in HCI. Interfaces that communicate naturally with their users are very familiar and have little to no learning curve. Donald Norman provides several examples of natural interfaces in his book “The Design of Future Things” [3]. One example that stands out is a tea kettle whistle. The tea kettle whistle offers natural communication that water is boiling. The user does not need to translate the sound of a tea kettle whistle from system terms into something he/she understands; it already naturally offers the affordance that the water is ready.

Thinking machines: Can there be? Are We?
In “Thinking Machines” [4], Winograd aligns his prospects for artificial intelligence with those of AI critics. The critics argue that a thinking machine is a contradiction in terms. “Computers with their cold logic, can never be creative or insightful or possess real judgement”. He asserts that the philosophy that has guided artificial intelligence research lacks depth and is a “patchwork” of rationalism and logical empiricism. The technology used in conducting artificial intelligence research is not to blame, it is the under-netting and basic tenets that require scrutiny.

Winograd supports his argument by identifying some fundamental problems inherent in AI. He discusses gaps of anticipation where in a any realistically sized domain, it is near impossible to think of all situations, and combinations of events from those situations. The hope is that the body of knowledge built into the cognitive agent will be broad enough to contain the relevant general knowledge needed for success. In most cases, the body of knowledge contributed by the human element is required; since it cannot be modeled exhaustively within the system. He also writes on the blindness of representation. This is in regards to language and interpretation of language. As expounded upon in his Ph.D. thesis, natural language processing goes far beyond grammatical and syntactic rules. The ambiguity of natural language requires a deep understanding of the subject matter as well as the context. When we de-contextualize symbols (representations) they become ambiguous and can be interpreted in varying ways. Finally, he discusses the idea of domain restriction. Since there is a chance of ambiguity in representations, AI programs must be relegated to very restricted domains. Most domains, or at least the domains the AI hopes to model, are not restricted (e.g. - medicine, engineering, law). The corollary is that AI systems can give expected results only in simplified domains.

Thinking Machines offers some interesting and compelling arguments against the “rationalist approach”. It supports the idea that augmenting human capabilities is far more feasible than attempting to model human intelligence. This is inline with the “design approach” (i.e. - placing focus on modeling interactions between the person and his/her surrounding environment.)

Stanford Human-Computer Interaction Group
Winograd currently heads up Stanford's Human-Computer Interaction Group [5]. They are working on some interesting projects grounded in design. One such project, d.tools is a hardware and software toolkit that allows designers to rapidly prototype physical interaction design. Designers can use the physical components (controllers, output devices) and the accompanying software (called the d.tools editor) to form prototypes and study their behavior. Another project, named Blueprint, integrates program examples into the development environment. Program examples are brought into the IDE through an built-in web search. The main idea behind Blueprint is that it helps to facilitate the prototyping and ideation process by allowing programmers to quickly build and compare competing designs. (more information on these projects can be found on their website (link is below))


References
[1] Terry Winograd, Shifting viewpoints: Artificial intelligence and human–computer interaction, Artificial Intelligence 170 (2006) 1256–1258.
http://hci.stanford.edu/winograd/papers/ai-hci.pdf

[2] Winograd, Terry (1971), "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language," MAC-TR-84, MIT Project MAC, 1971.
http://hci.stanford.edu/~winograd/shrdlu/

[3] Norman, Donald (2001), The Design of Future Things, New York: Basic Books, 2007.

[4] Winograd, Terry (1991), "Thinking machines: Can there be? Are We?," in James Sheehan and Morton Sosna, eds., The Boundaries of Humanity: Humans, Animals, Machines, Berkeley: University of California Press, 1991 pp. 198-223. Reprinted in D. Partridge and Y. Wilks, The Foundations of Artificial Intelligence, Cambridge: Cambridge Univ. Press, 1990, pp. 167-189.
http://hci.stanford.edu/winograd/papers/thinking-machines.html

[5] The Stanford Human-Computer Interaction Group
http://hci.stanford.edu/

Making jqGrid work with a .NET Web Service

(Also see Part 2 of this blog entry for a full working example of the code.)

I tore my hair out trying to get jqGrid to display results from a .NET web service. I ended up creating a method (in c#) to convert a datatable to a JSON string appropriate for consumption by jqGrid. Here is what I did...

First, I wrote the following method to convert a datatable to a jqGrid-appropriate JSON string:

public static string GetJqgridJson(DataTable dt) {
            ///<summary>
            ///     This method converts a DataTable to a JSON string
suitable for jqGrid consumption.
            /// </summary>
            ///<param name="dt">
            ///     The DataTable to be processed.
            /// </param>
            ///<returns>
            ///     jqGrid-JSON string if dt parameter has rows, null if dt 
parameter has rows
            /// </returns>

            StringBuilder jsonString = new StringBuilder();
            if(dt != null && dt.Rows.Count > 0) {
                jsonString.Append("{");
                jsonString.Append("\"total\": \"1\",");  //set as 1 for example
                jsonString.Append("\"page\": \"1\",");   //set as 1 for example
                jsonString.Append("\"records\": \"" + dt.Rows.Count.ToString() + "\",");
                jsonString.Append("\"rows\":[");

                for(int i = 0; i < dt.Rows.Count; i++) {
                    jsonString.Append("{\"cell\" :[");
                    for(int j = 0; j < dt.Columns.Count; j++) {
                        if(j < dt.Columns.Count - 1) {
                            jsonString.Append("\"" + dt.Rows[i][j].ToString()
.Replace("\"", "\\\"") + "\",");
                        } else if(j == dt.Columns.Count - 1) {
                            jsonString.Append("\"" + dt.Rows[i][j].ToString()
.Replace("\"", "\\\"") + "\"");
                        }
                    }

                    if(i == dt.Rows.Count - 1) {
                        jsonString.Append("]} ");
                    } else {
                        jsonString.Append("]}, ");
                    }
                }

                jsonString.Append("]}");
                return jsonString.ToString();
            } else {
                //Return null if datatable has no rows.
                return null;
            }
        }

Next, I created a .Net WebMethod that uses the GetJqgridJson method. My WebMethod fetches data as a DataTable, uses the GetJqgridJson method to convert the DataTable to a JSON string, then returns the JSON string to the browser. (I wont include my WebMethod code here for sake of brevity.)

When you're creating your WebMethod, just be sure to include the following directive above your method to ensure you are sending a JSON and not an XML string back to the browser:

[ScriptMethod(ResponseFormat = ResponseFormat.Json)]

Finally, I wrote my JavaScript code. Here is some sample code that can be used to set up a jqGrid and fetch the JSON string via AJAX. (Note: The following JavaScript code will not work out of the box. You'll need to update the portions surrounded with "**" to make it work within your project.)

$("**YOUR JQUERY SELECTOR HERE**").jqGrid({
                            datatype: 'jsonstring',
                            jsonReader: { root: "rows",
                                page: "page",
                                total: "total",
                                records: "records",
                                repeatitems: true,
                                cell: "cell"
                            },
                            gridview:true,
                            colNames: [**YOUR COLUMN NAMES HERE**],
                            colModel: [**YOUR COLMODEL HERE**],
                            viewrecords: true,
                            caption: '',
                            loadonce: false,
                            loadui: "enabled",
                            beforeSelectRow: function(rowid, e) {return false;},
                            hoverrows:false,
                            height:200,
                            scroll:1
                });
                $("**YOUR JQUERY SELECTOR HERE**").clearGridData();
                $.ajax({
                    type: "POST",
                    url: "**YOUR WEBMETHOD URL HERE**",
                    data: JSON.stringify(webservice_object),
                    contentType: "application/json; charset=utf-8",
                    dataType: "json",
                    success: function (msg) {
                        var data = msg.d;
                        $("**YOUR JQUERY SELECTOR HERE**").jqGrid('setGridParam', 
                                        {datatype: 'jsonstring', datastr: data});
                        $("**YOUR JQUERY SELECTOR HERE**").trigger('reloadGrid');
                    }
            });

C18 Brain Model from 3B Scientific

I recently purchased 3B Scientific's C18 Brain Model for use in academic research. It is a 5-part midsagitally sectioned brain taken from the cast of an actual human brain.

Pluses

• Taken from the cast of an actual human brain so it's very accurate.
  
• Nicely detailed and very clearly labeled. (A user manual with structure names is included.)
  
• Constructed out of rubber material making it unbreakable.
  
• The sections are held together by metal connecting pins that snap securely into place. I have seen other cheaper models that use screws for connectors making disassembling/assembling tough.
• Comes with a plastic display stand.
• Construction is very nice and has a professional feel. Other budget brain models don't have the same visceral build quality.
  

Minuses
Really none, but...

• One nitpick I have is the manual only contains names of the structures; It does not contain pictures or additional descriptive information. This isn't a big deal since I could just look it up the on the web, but it would be really nice to have this information right in the manual. (I plan on putting a pdf together that contains pictures and descriptions. I'll post it on this blog shortly.)
• Most important structures are visible however one structure that I'm currently researching, the amygdala, is not visible. As you can see from the 2nd photo below, the amygdala falls in about the same location as the connecting pin for the temporal lobe and encephalic trunk pieces. 

(Click images to enlarge)

Assembled, with manual.

5 pieces (left lobe stays intact).

I would highly recommend this model based on it's realism, build quality, and value. It's definitely something that could be used in the academic and professional arenas.