The yellow box shows how the step of measuring individual differences is easily added to a typical study setup.
Here we will provide a description of the tests that we often use in our various studies to evaluate the impact of individual differences on HCI tasks. We often use the terminology 'individual differences' and 'user characteristics' interchangeably to refer to the set of measures presented here (and by no means is this an exhaustive list of terminologies or measures that are out there). Currently, the set of tests we utilize consists of three cognitive measures: Perceptual speed, Verbal working memory, Visual working memory, and one personality: Locus of control.
If there are plans to utilize these characteristics for new studies in HCI or Psychology, please let us know! We would love to start compiling a record of all the papers and studies that evaluate these measures, especially when using the implementations provided here.
Here is the reference to our project/papers that have used these tests: ATUAV Advanvced tools for User Adaptive Visualizations.
Email contact: Dereck Toker, PhD student -> dtoker [ at ] cs.ubc.ca
Perceptual speed is a measure of speed when performing simple perceptual tasks (e.g., finding figures or making comparison between letters, numbers, objects, pictures, or patterns) such that everyone could score perfect if there were no limitation on time [Salthouse 2000].
We use the Identical Figures Test (P-3) [Ekstrom 1976], which is a paper-based test that measures a participant’s speed in matching a figure on the left side of each row to one of five possible figures appearing on the right side. The test consists of two 90 second trials of 48 items each, however we found many participants were completing the test with a perfect score, therefore we modified the test so that it was done in one 90 second trial over all 96 items. The final test score is the number of correctly answered rows, ranging from 0 to 96.
References:
Download test: Perceptual Speed, Identical Figures (P-3)
Verbal working memory is part of the working memory responsible for temporary maintenance and manipulation of verbal information [Baddely 1986].
We use the Operation-word span test (OSPAN) for verbal working memory to measure this user characteristic. This test accounts for both the storage and the processing components of this trait and has shown the highest correlation with reading comprehension when compared to other tests [Turner 1989]. The test is computer-based, and shows a series of arithmetic equations that the participant must classify as True or False. Once a selection is made for a given equation, one word appears on the screen for 800ms and then the next arithmetic equation appears. At the end of a series, participants are required to recall the words in correct order. This process is repeated several times with each series varying in length from 2 to 6 words. The insertion of secondary tasks between memory items (here, solving math examples) means that subjects are required to recall information that is periodically unattended and vulnerable to proactive interference [Kane 2002]. The final test score is a measure of the average word span in working memory, ranging from 0 to 6.
References:
Download test: Verbal Working Memory, (OSPAN)
The OSPAN test is a Java program authored by Taiyu Lin at ALTRC, Massey University, New Zealand
Visual working memory is part of the working memory responsible for temporary storage and manipulation of visual and spatial information [Logie 1995].
Several tests are available for measuring visual working memory, but they each measure slightly different constructs (e.g., the ability to remember sequences of visual elements or the ability to integrate information from long-term memory). The test most appropriate for the nature of the tasks in this study is the one proposed and tested by Fukuda & Vogel [Fukuda 2009] which measures the ability to recall the location of basic colored squares over short intervals and most closely matched the task structure in our study. We were not specifically interested in looking at sequential or long-term visual memory since each of our tasks was static and did not require any visual information to be remembered from one task to the next. The Fukuda & Vogel test is computer based, and consists of 120 trials. In each trial an array of either 4, 6, or 8 different colored squares flash on the screen for 500ms. After 200ms, one of the colored squares re-appears in its original location, and participants must select whether the square has the same or a different color than when it was first flashed. The final score for the test measures the percentage of correct answers for each array size (K4, K6 & K8). In our studies we use only K6 as our measure of visual working memory.
References:
Download test: Visual Working Memory
The Visual Working Memory Test runs in Matlab and requires the Psychophysics Toolbox
Locus of control is a personality measure of the degree to which individuals perceive outcomes as either a result from their own behavior, or from forces that are external to themselves. Assessment is carried out using a 29-item questionnaire. Final scores range from 0 to 23 (lower = external, higher = internal)
Reference:
Link to test: Locus of Control
Also, here is the scoring guide for the test: http://www.wilderdom.com/psychology/loc/RotterLOC29Scoring.html
Here are the descriptive statistics from our last two studies we ran: Intervention Study and Valuechart study.
And here are histograms for those curious.
