Week 11: Evaluation and Agile Development
Cognitive walkthroughs: a method for theory-based evaluation of user interfaces
Topic: a new methodology (cognitive walkthroughs) for performing theory-based evaluations of the user interface designs early in the design cycle.
Main focus of cognitive walkthroughs: ease of learning.
Cognitive walkthrough inputs:
- Detailed design description of the user interface
- A task scenario
- Explicit assumptions about the user population and the context of use
- A sequence of actions with which a user could successfully complete the task using the design under evaluation
Goal structure management comes in four forms:
- Generating the goal structure
- Generating goals for actions
- Interpreting feedback
- “And-then” goal structure
What distinguishes cognitive walkthroughs from other evaluation methods?
- Role of simulation
- Focus on mental operations
- Use of task context
- Links to the interface
- Role of theory
Assumptions:
- User’s initial goals for a task will be incomplete in that they do not specify a complete task decomposition that includes both representations of subtasks and the action sequence necessary to carry out each subtask
- Prompts, button labels and menu items interact with the user’s background knowledge and existing goals to create more explicit goals, which define components of the complete task and establish goals to perform specific actions
- The lowest-level goal is to execute some physical action
- When an action has been taken, the goal structure must be revised
- In many tasks and in interactive dialogs, goals are not posted individually but as part of a structure that represents a goal and an associated sequence of subgoals that must be accomplished in a fixed order
The cognitive walkthrough is a precisely specified procedure for simulating a user’s cognitive processes as the user interacts with an interface in an effort to accomplish a specific task.
A cognitive walkthrough has two phases, preparation and evaluation.
How do design and evaluation interrelate in HCI research?
Topic: relationship between designing for usability and evaluating usability to build overarching theory of HCI.
Design methods have evolved over decades:
- First generation, or product oriented, design methods focused on systems theory and software engineering
- Second generation, or process oriented, design methods developed in the 1970’s, focused on user participation, communication and democracy in the design process
- Third generation, or use oriented design methods, focus on the actual use situation and assess the quality in use of the designed system
Evaluations can be done four basic ways: automatically, empirically, formally, and informally
In HCI research, design and evaluation are typically treated as separate activities. In addition, students are typically taught these skills separately and industry typically hires people to be either designers or evaluators.
Overall, the HCI community sees seven clusters of authors with seven corresponding viewpoints within the topic of usability… Topics do not split between design and evaluation, but rather split according to philosophies of how systems should be designed and evaluated
Towards a framework for integrating agile development and user-centered design
Topic: integration between UCD (user-centred design) and agile development. Additionally, authors will highlight five principles for integrating UCD and agile development
Three project teams in one organisation were observed for around 2-4 hours per week on site by one individual for a period of 6 months. The organisation hosting these projects was a large media company with a tradition of employing a user-centred approach to development.
A fundamental problem of communication exists between the developers and designers within each team and the subject of power within the project is a tricky one. Designers within a project defend their discipline in response to decisions made by the developers, and vice versa.
Five Principles For Integrating UCD And Agile Development
- User involvement - user should be supported by each member
- Collaboration and culture - more effective communication between developers and designers
- Prototyping - designers must provide prototypes and user feedback to developers
- Project life-cycle - basic customer needs to be explored before code release
- Project management - agile and UCD must work well with project management framework