Module 6: Focus Groups, Observational Research, and Beta Testing
Overview of Module 6/Week 6 of Class
Overview of Topics
During the class, we have reviewed different approaches to researching users and understanding their behaviors, expectations, and preferences. In this module/week, we will continue examining approaches to researching usability as well as doing initial assessments of a product's usability. Specifically, we will focus on three approaches to researching such factors: Focus groups, observational research, and beta testing. These approaches examine users and assess usability from different perspective and are often used at different points in the product development and product assessment processes.
During the class, we have reviewed different approaches to researching users and understanding their behaviors, expectations, and preferences. In this module/week, we will continue examining approaches to researching usability as well as doing initial assessments of a product's usability. Specifically, we will focus on three approaches to researching such factors: Focus groups, observational research, and beta testing. These approaches examine users and assess usability from different perspective and are often used at different points in the product development and product assessment processes.
Focus Groups
In some instances, it is beneficial to test sample designs with individual users to determine how they perceive and use a product. In other instances, it can be beneficial to test such items with a group of individuals from the intended audience for the item. In such cases, focus groups are an effective mechanism for collecting group comments on a design.
In focus groups, a small cohort of individuals from the intended audience is assembled in one place and asked a series of questions on the design of a product. In some cases, the members of that group might be asked to use the product in advance of that meeting. In other cases, group members might be asked to review and use the product in the group setting and comment on it during or after such use. In both cases, the objective is to determine if the dynamics of the group can reveal information about what design factors are effective (to keep), problematic (need revisions), or needed (features to be added).
The central notion is group dynamics might help participants better remember, comment on, critique, and discuss design elements in a way that speaking with individuals alone might not (e.g., the comments of one group member might prompt another group member to remember or comment on something s/he might not otherwise have -- Person 1: "Didn't you also find this feature problematic?" Person 2: "Now that you mention it, I did find that feature problematic.")
Ideally, focus groups would be used in addition to other data collection techniques -- like interviews with individual users or user tests with individuals -- to determine what aspects of a product's design might require revision.
In some instances, it is beneficial to test sample designs with individual users to determine how they perceive and use a product. In other instances, it can be beneficial to test such items with a group of individuals from the intended audience for the item. In such cases, focus groups are an effective mechanism for collecting group comments on a design.
In focus groups, a small cohort of individuals from the intended audience is assembled in one place and asked a series of questions on the design of a product. In some cases, the members of that group might be asked to use the product in advance of that meeting. In other cases, group members might be asked to review and use the product in the group setting and comment on it during or after such use. In both cases, the objective is to determine if the dynamics of the group can reveal information about what design factors are effective (to keep), problematic (need revisions), or needed (features to be added).
The central notion is group dynamics might help participants better remember, comment on, critique, and discuss design elements in a way that speaking with individuals alone might not (e.g., the comments of one group member might prompt another group member to remember or comment on something s/he might not otherwise have -- Person 1: "Didn't you also find this feature problematic?" Person 2: "Now that you mention it, I did find that feature problematic.")
Ideally, focus groups would be used in addition to other data collection techniques -- like interviews with individual users or user tests with individuals -- to determine what aspects of a product's design might require revision.
Observational Research
What individuals say they do and what they actually do can differ. For this reason, usability researchers often need to collect information in addition to responses provided by interviews and focus groups. This kind of research focuses on using observations to compare if words and actions align in relation to actual user behaviors in setting. Individuals who know they are being observed, however, often act differently than they would if doing a process more naturally (i.e., unobserved). Similarly, individuals performing a task in a testing context or any location other than the one where they usually perform that activity often do so in different ways -- ones that don't often align with how they actually use or do something. For these reasons, observational research often involves two categories or kinds:
What individuals say they do and what they actually do can differ. For this reason, usability researchers often need to collect information in addition to responses provided by interviews and focus groups. This kind of research focuses on using observations to compare if words and actions align in relation to actual user behaviors in setting. Individuals who know they are being observed, however, often act differently than they would if doing a process more naturally (i.e., unobserved). Similarly, individuals performing a task in a testing context or any location other than the one where they usually perform that activity often do so in different ways -- ones that don't often align with how they actually use or do something. For these reasons, observational research often involves two categories or kinds:
- Controlled Observations where individuals are aware they are being tested or observed or tasks are performed in a location other than the one where users generally use or do something.
- Naturalistic Observations where individuals are unaware they are being observed, are observed performing activities in the actual location where they would perform such tasks, or both.
Beta Testing
In previous modules, we examined user testing -- a process use to determine how well a completed product works. Unfortunately, by the time user testing is done, the ability to change the design of the product is limited, for that product is (essentially) completed. Beta testing is an approach used to avoid such limitations.
With beta testing, a designer creates a model of what a completed product might look like (e.g., a test version of what a final web page might look like). The designer then tests this draft version with members of the intended audience. The objective is to determine how effectively individuals can use that draft item to perform certain activities. The process involves using an approach similar to user testing (e.g., ask an individual to perform a process and observe what they do) to identify where improvements might be made in the final version of the product's design.
While user testing can identify problems in the final design of a product (e.g., "The wording of this link needs to be changed.") beta testing often allows for more comprehensive revisions (e.g., "Not only do we need to change the wording of this link, we should also add a new link for . . ."). In this way, beta testing allows designers to revise product designs as a product is being developed vs. address the design of a final, completed product.
In previous modules, we examined user testing -- a process use to determine how well a completed product works. Unfortunately, by the time user testing is done, the ability to change the design of the product is limited, for that product is (essentially) completed. Beta testing is an approach used to avoid such limitations.
With beta testing, a designer creates a model of what a completed product might look like (e.g., a test version of what a final web page might look like). The designer then tests this draft version with members of the intended audience. The objective is to determine how effectively individuals can use that draft item to perform certain activities. The process involves using an approach similar to user testing (e.g., ask an individual to perform a process and observe what they do) to identify where improvements might be made in the final version of the product's design.
While user testing can identify problems in the final design of a product (e.g., "The wording of this link needs to be changed.") beta testing often allows for more comprehensive revisions (e.g., "Not only do we need to change the wording of this link, we should also add a new link for . . ."). In this way, beta testing allows designers to revise product designs as a product is being developed vs. address the design of a final, completed product.
Other Module 6/Week 6 Materials
To access other materials for Module 6/Week 6, click on the related link below