We are delighted to welcome Willemijn Elkhuizen who shares her CHI 2024 paper called On the Role of Materials Experience for Novel Interactions with Digital Representations of Historical Pop-up and Movable Books.
Hi, what’s your name?
Hi, my name is Willemijn Elkhuizen
Citation:
Willemijn Elkhuizen, Jeff Love, Stefano Parisi, and Elvin Karana. 2024. On the Role of Materials Experience for Novel Interactions with Digital Representations of Historical Pop-up and Movable Books. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24), May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 18 pages.https://doi.org/10.1145/3613904.3642142
TL;DR (if you had to describe this work in one sentence, what would you tell someone?):
Because we often cannot interact (directly) with cultural heritage artifacts, we miss opportunities to experience and learn from their materiality; taking the case of historical pop-up and movable books, we characterize their material experience, and explore how these characterisations can inspire novel interactions with their digital representations.
What problem is this research addressing, and why does it matter?
With the rapid digitalisation of cultural heritage (CH), more and more of our encounters with cultural heritage are and will be in the digital realm, which is currently often limited to seeing an artifact’s image, or in some cases a 3D digital model. These representations lack many experiential material aspects. Also in museum contexts, we are almost never allowed to handle artifacts, or able to use all our senses to experience them, which also deprives us of all kinds of interpretations and affective reactions. Also, our (non-)expertise, past experiences, and the change of material over time, affect our material experiences. Although HCI researchers and designers have introduced numerous forms of digital augmentation in the cultural heritage context, exactly what and how artifacts’ materiality is considered in their designs, remains underlit. In this paper, we aim to provide a new perspective on the design of (digitally-mediated) CH, aiming to bring CH HCI beyond the technology-driven, or ‘as-realistic-as-possible’ experience paradigm.
How did you approach this problem?
To explore this perspective, we took the collection of historical pop-up and movable books of the National Library of the Netherlands (KB) as a case study. Such books have rich performative potential, which is not well or easily captured with conventional digitalisation approaches (e.g. photography, or 3D scanning).
We conducted a series of observations and interviews, where participants (27 adults, and 9 children with their guardians) had open-ended explorative interactions with 1) historical and contemporary pop-up and movable books (image 1), and 2) a VR experience featuring pop-up and movable books, the latter created prior by the KB (image 2). In the interview we asked people about their overall experience, and questions relating to the books’ material experience, on the following levels: performative (i.e. what interactions the books invited), perceptual (i.e. how they were sensed), interpretive (i.e. what associations the books elicited), and affective (i.e. what emotions the books elicited).
What were your key findings?
We found rich performative interaction with the pop-up books, and to a lesser extent with the VR experience (see image 3).
Furthermore, we clustered the material experiences (across the different levels), and related them to five key material qualities, namely paper’s fold-ability, slide-ability, age-ability, tear-ability, and print-ability. For example, actions like repeated opening-and-closing of books (performative), positive surprise (affective), dynamic visual effects (sensorial), and the associations with gift unwrapping or origami (interpretive) could all be associated with the fold-ability quality of paper, which is exploited in pop-up books.
Next, we used these qualities to inspire two extended reality prototypes, with a historical pop-up and movable book: ‘Tip and Top boven de Wolken’, 1964, by Voljech Kubasta. The first is a mixed reality experience (image 1), and the other is VR experience (see paper for details). In the mixed reality prototype, we aimed to enable the diverse performative interaction with the book, as well as enhancing some qualities on specific pages, such as its fold-ability (color of depiction change with opening and closing the page), age-ability (visuals slowly degrade with repeated use), and tear-ability (a part of the page audibly and visually tears loose from the page).
What is the main message you’d like people to take away?
It is important that we consider the materiality of our heritage collections, and how that contributes to our experience of them, when designing digitally-mediated CH experiences. In this case study we show that zooming in on the experiential material aspects of heritage artifacts, leads to rich data, usable for inspiration. Yet, what remains to be further explored, is how we can support designers (further) in utilizing this rich data, also when involving diverse stakeholders. Furthermore, we have yet to deeply investigate how the proposed novel interactions (inspired by the experiential material characterisation), in return shape the material experience (of the digital representation), but also our experience and understanding of the original.
What led or inspired you to carry out this research?
In my PhD, I worked on creating 3D printed reproductions of paintings, with the aim of closely matching their perceptual appearance as much as possible, which had a strong focus on the perceptual aspects of digital(ly-created) representations. In my collaboration with Elvin Karana, and reading her paper on the material experience framework, I became inspired to also look beyond this perceptual perspective, and explore how I could apply her framework in a different context, namely in relation to experiencing cultural heritage artifacts.
And, moreover, the research-in-residence position with the National Library of The Netherlands (KB), led to the concrete formulation of this case study, as I found their collection of (historical) pop-up books to be very well suited to explore this topic.
What kind of skills or background would you say one needs to perform this type of research?
I think somebody with a multidisciplinary background (such as can be found in design), or a multi-disciplinary team, would be best suited to carry out this type of research.
Setting up such a study requires experience with qualitative research methodologies, but also a keen interest to prototype with and design for digital technologies.
I believe this work also requires a genuine interest in cultural heritage, (museum) experiences, and the human side of HCI (as not to see CH as ‘just’ a context to showcase the capabilities of a new technology).
Any further reading you recommend?
- Elisa Giaccardi and Elvin Karana. 2015. Foundations of Materials Experience: An Approach for HCI. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2447–2456. https://doi.org/10.1145/2702123.2702337
- Eva Hornecker and Luigina Ciolfi. 2019. Human-computer interactions in Museums. Springer. https://doi.org/10.1007/978-3-031-02225-8
Your biography
I am an assistant professor at Industrial Design Engineering, TU Delft. My research is situated at the intersection of digital technologies, cultural heritage, and material perception/experience. My research focuses on how we can utilize digital technologies to support our experience of cultural heritage, and currently take a material experience as a lens. Prior, I collaborated in two Dutch heritage documentation and conservation projects, with the high-resolution 3D imaging of Johannes Vermeer’s ‘The Girl with a Pearl Earring’ (project: Girl in the Spotlight), and Rembrandt’s ‘The Night Watch’ (project: Operation Nightwatch).
Besides that, I coordinate the minor Advanced Prototyping, where I teach students (with diverse technical backgrounds) about prototyping, incl. using technologies like 3D scanning, 3D modeling, digital fabrication (incl. 3D printing), and virtual reality.
I have a background in design, as I hold a MSc and PhD degree in Industrial Design Engineering (both from TU Delft).
If you have an affinity with cultural heritage and digital technologies, feel free to reach out for collaborations, etc.