Evaluation

Today’s lecture was about evaluation. Evaluation is about gathering data, about the usability or design of a product. We are interested in a particular set of users, related to a particular set of tasks. We are not interested in every user. And we need to evaluate which situation it will be used in and in what location?

Why are we evaluating?

· to understand the real world

· to compare designs

· to engineer the product towards a target

· to check conformance to a standard

· to ask different questions at different points of the process.

Real world

· what does the user actually want?

· Remember the designer is not the end user, what we want is not what they want!

· What problems do users experience? What is wrong with the current system?

· What tasks do users perform

· How do users perform tasks

· What problems do they experience

Comparing designs

· Commercial competition? What do the rivals do, what is good? What can we improve?

· Which design is best and what do we mean by best? What makes it the best? Speed, appeal?

· Answer specific design questions

Engineering towards a target

· Aiming for specific performance measures

· Improving areas of an existing design

· Being better than a competitor

Conformance to a standard

· ISO standards

· Accessibility standards

· Health and safety standards

· In-house standards

· Design guidelines

When to evaluate

· Formulative evaluation

o Evaluation as part of the design process

o Predesign proof and viability

o Initial design with user

o Interactive design

o Acceptance testing

· Predict usability of system or aspects thereof

· Check design teams understanding of user requirements

· Test out ideas quickly and informally

· Summative evaluation

o Evaluation of a essentially finished system

o Interactive design

o Acceptance testing

· Identifying user difficulties so products can be finely tuned

· Upgrading product

Factors

· Kind of information required

· Nature of system or specification under evaluation

· Stage in the life cycle being evaluated

· Whether or not statistical validity is required

· Resources available

Qualitative evaluation

· Descriptive

· Typically non-numerical

· May be subjective

· Introspection

· Direct observation

· Query via questionnaires and interviews

· Continuous evaluation through feedback and field studies

Quantitative evaluation

· Precise measurement

· Numerical values

· Bound on how to correct our statements are

· User performance

· Controlled experiments

· Statistical analysis

Measurable quantities

· Time taken to complete task

· Number of tasks completed within a given time

· Ratio between successful interactions to errors

· Time spent recovering from errors

· Number of errors

· Number of commands or features actually used by the user

· Number of features user can remember after the test

This was basically everything said in the lecture. Not long until the Christmas break!w0o0o0o0o0o0o0o0o0o!!!!!Bye for now!