User Characteristic Tests

Here we will provide a description of the tests that we often use in our various studies to evaluate the impact of individual differences on HCI tasks.

User CharacteristicDefinition, ReferenceTest SourceTest DurationImpact on visualization processing (our work) Impact on XAIPrediction from gaze + interaction data (our work)
Perceptual Speed
(Identical Pictures Test)		 Speed in comparing figures or symbols, scanning to find figures or symbols, or carrying out other very simple tasks involving visual perception.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.		 Download the Paper Test. There is no online version of this test. 1.5 min [2, 4, 7,
8, 9, 10,
12, 13, 14]		 [18] [1, 5, 6, 11]
Visual Working Memory
 Measures the quantity of visual information (e.g., shapes and colours) that can be temporarily maintained and manipulated in working memory.

Vogel, E.K., Woodman, G.F., and Luck, S.J. 2001. Storage of features, conjunctions, and objects in visual working memory. J. Exp. Psychol. Hum. Percept. Perform. 27, 1, 92–114.		 Download a folder with the Matlab Test. Follow instructions from the readme.txt file. You will need Matlab to run this test. 6 mins [2, 4, 7,
8, 9, 10,
12, 13]		 [18] [1, 5, 6, 11]
Visual Scanning
 The speed in finding the shortest paths in a wide or complicated spatial field. Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests.

Educational Testing Service Princeton, NJ.
Download the Paper Test. There is no online version of this test. 8 mins [2] N/A [1]
Verbal Working Memory
(OSPAN)		 Measures the quantity of verbal information (e.g., words) that can be temporarily maintained and manipulated in working memory.

Turner, M.L. and Engle, R.W. 1989. Is working memory capacity task dependent? Journal of Memory and Language. 28, 2, 127–154.		 Download a folder with the source code to run this test. Follow instructions from readme.doc file. 10 mins [2, 4, 7,
8, 9, 10,
12, 13, 15]		 N/A [1, 5, 6, 11 , 19]
Visualization Literacy
 The ability to confidently use a given data visualization to translate questions specified in the data domain into visual queries in the visual domain, as well as interpreting visual patterns in the visual domain as properties in the data domain.

Boy, J. et al. 2014. A Principled Way of Assessing Visualization Literacy. IEEE Transactions on Visualization and Computer Graphics. 20, 12, 1963–1972.		 Try the
Online Test
or download the
source code  to run this test. Follow instructions from readme.txt file.		 10 mins [2] N/A [1, 19, 20]
Disembedding
(Hidden Patterns Test)		 The ability to hold a given visual percept or configuration in mind so as to disembed it from other well defined perceptual material.

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.		 Download the Paper Test. There is no online version of this test. 3 mins [14, 15] N/A N/A
Spatial Memory
(Shape Memory Test)		 The ability to remember the configuration, location, and orientation of figural material (e.g., multiple objects on a map).

Ekstrom, R.B. et al. 1976. Manual for kit of factor referenced cognitive tests. Educational Testing Service Princeton, NJ.		 Download the Paper Test. There is no online version of this test. 9 mins [2, 14] N/A [1]
Need for Cognition
 Measures the extent to which individuals are inclined towards effortful cognitive activities.

Cacioppo, J.T. et al. 1984. The Efficient Assessment of Need for Cognition. Journal of Personality Assessment. 48, 3, 306–307.		 Try the
Online Test
or download the
source code to run this test. Follow instructions from readme.txt file.
 3 mins [2, 14] [16] [18] N/A
Locus of Control
 Measure of the degree to which individuals perceive outcomes as either a result from their own behavior, or from forces that are external to themselves.

Rotter, Julian B. 1966. Generalized Expectancies for Internal Versus External Control of Reinforcement. Psych. Monographs: General and Applied 80.1, 1-28.		 Try the
Online Test
or download the
source code to run this test. Follow instructions from readme.txt file.
 5 mins [2, 4, 7,
9, 10]		 N/A [5]
Big-Five Personality Traits
(TIPI)		 Measures personality in terms of Agreeableness, Conscientiousness, Extraversion, Emotional Stability, and Openness.

Gosling, S. D., Rentfrow, P. J., and Swann, W. B., Jr. 2003. A Very Brief Measure of the Big Five Personality Domains. Journal of Research in Personality, 37, 504-528.		 Try the
Online Test
or download the
Paper Test		 1 min [3] [17] [18] N/A
Reading Proficiency
(X_Lex)		 Measures vocabulary size and reading comprehension ability in English.

Paul Meara. 1992. EFL vocabulary tests, second edition 2010. Lognostics, Swansea: Wales.		 Download the Paper Test. There is no online version of this test. 5 min [15] [18] coming soon
Verbal IQ
(NAART)		 Overall verbal intellectual abilities that measures acquired knowledge, verbal reasoning, and attention to verbal materials.

Esther Strauss, Elisabeth M. S. Sherman, Otfried Spreen, and Otfried Spreen. 2006. A compendium of neuropsychological tests: administration, norms, and commentary. Oxford University Press, Oxford; New York.		 Download the Paper Test. There is no online version of this test. 5 min N/A N/A N/A
Five-Dimensional Curiosity Scale
 Measures the five dimensions of curiosity i.e., joyous exploration, deprivation sensitivity, stress tolerance, social curiosity, and thrill seeking.

Kashdan, Todd B., Melissa C. Stiksma, David J. Disabato, Patrick E. McKnight, John Bekier, Joel Kaji, and Rachel Lazarus. "The five-dimensional curiosity scale: Capturing the bandwidth of curiosity and identifying four unique subgroups of curious people." Journal of Research in Personality 73 (2018): 130-149.		 Try the
Online Test
or download the
source code to run this test. Follow instructions from readme.txt file.
 5 min N/A [18] N/A

References

Only journal papers and strictly refereed conference papers published by our group are listed.

  1. Conati, C., Lallé, S., Rahman, M.A., and Toker, D. 2017. Further Results on Predicting Cognitive Abilities for Adaptive Visualizations. Proc. of the 26th International Joint Conference on Artificial Intelligence, AAAI, 1568-1574. [link]
  2. Lallé, S., Conati, C., and Carenini, G. 2017. Impact of Individual Differences on User Experience with a Visualization Interface for Public Engagement. Proc. of the 2nd International Workshop on Human Aspects in Adaptive and Personalized Interactive Environments (in conjunction with UMAP 2017), ACM. [link]
  3. Lallé, S., Mudrick, N.V., Taub, M., Grafsgaard, J.F., Conati, C., and Azevedo, R. 2016. Impact of Individual Differences on Affective Reactions to Pedagogical Agents Scaffolding. Proc. of the 16th International Conference on Intelligent Virtual Agents, Springer, 269–282. [Best paper award - link]
  4. Conati, C., Carenini, G., Toker, D., and Lallé, S. 2015. Towards User-Adaptive Information Visualization. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 4100–4106. [link]
  5. Gingerich, M., and Conati, C. 2015. Constructing Models of User and Task Characteristics from Eye Gaze Data for User-Adaptive Information Highlighting. Proc. of the 29th AAAI Conference on Artificial Intelligence, AAAI Press, 1728–1734. [link]
  6. Steichen, B., Conati, C., and Carenini, G. 2014. Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data. ACM Transactions on Interactive Intelligent Systems 4, 2, Article 11. [ACM TiiS Best Paper Award, link]
  7. Conati, C., Carenini, G., Steichen, B., and Toker, D. 2014. Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making. Computer Graphics Forum (EuroVis 2014) 33, 3, 371–380. [link]
  8. Steichen, B., Michael M.A. Wu, Toker, D., Conati, C., and Carenini, G. 2014. Te,Te,Hi,Hi: Eye Gaze Sequence Analysis for Informing User-Adaptive Information Visualizations. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 183-194. [Best paper award, link]
  9. Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J.T. 2014. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 1835–1844. [link]
  10. Toker, D., and Conati, C. 2014. Eye Tracking to Understand User Differences in Visualization Processing with Highlighting Interventions. Proc. of the 22nd International Conference on User Modeling, Adaptation, and Personalization, Springer, 219–230. [link]
  11. Steichen, B., Carenini, G., and Conati, C. 2013. User-adaptive Information Visualization: Using Eye Gaze Data to Infer Visualization Tasks and User Cognitive Abilities. Proc. of the 18th International Conference on Intelligent User Interfaces, ACM, 317–328. [Best Paper nominee, link]
  12. Toker, D., Conati, C., Steichen, B., and Carenini, G. 2013. Individual User Characteristics and Information Visualization: Connecting the Dots Through Eye Tracking. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 295–304. [link]
  13. Toker, D., Conati, C., Carenini, G., and Haraty, M. 2012. Towards Adaptive Information Visualization: On the Influence of User Characteristics. Proc. of the 20th International Conference on User Modeling, Adaptation, and Personalization, Springer, 274–285. [link]
  14. Conati, C., and Maclaren, H. 2008. Exploring the Role of Individual Differences in Information Visualization. Proc. of the Working Conference on Advanced Visual Interfaces, ACM, 199–206. [link]
  15. Toker, D., Conati, C., Carenini, G. 2018. User-adaptive Support for Processing Magazine Style Narrative Visualizations: Identifying User Characteristics that Matter. Proc. of the 23rd International Conference on Intelligent User Interfaces, ACM, (to appear). [link]
  16. Millecamp, M., Htun, N.N., Conati, C. and Verbert, K. 2019. To explain or not to explain: the effects of personal characteristics when explaining music recommendations. Proceedings of the 24th International Conference on Intelligent User Interfaces, 397-407. [link]
  17. Kouki, P., Schaffer, J., Pujara, J., O'Donovan, J. and Getoor, L. 2019. Personalized explanations for hybrid recommender systems. Proceedings of the 24th International Conference on Intelligent User Interfaces, 379-390. [link]
  18. Conati, C., Barral, O., Putnam, V. and Rieger, L. 2021. Toward personalized XAI: A case study in intelligent tutoring systems. Artificial intelligence, 298, p.103503. [link]
  19. Barral, B., Lallé, S., Guz, G., Iranpur, A., and Conati, C. 2020. Eye-Tracking to Predict User Cognitive Abilities and Performance for User-Adaptive Narrative Visualizations. In Proceedings of the 22nd International Conference on Multimodal Interaction, 298, pp. 163–173, ACM. [link]
  20. Lallé, S., Toker, D., and Conati, C. 2021. Gaze-Driven Adaptive Interventions for Magazine-Style Narrative Visualizations. IEEE Transactions on Visualization and Computer Graphics, 27, p.2941-2952. [link]