Berks professor tells Malaysian WWII survivors’ stories in VR, digital exhibit

WYOMISSING, Pa. — “As soon as the British left, the Japanese arrived. Nicky saw them on bicycles, adorned with their flags… as they made their way to Taiping town... These Japanese soldiers set up a camp… and would raid homes looking for food, goods and women. This first wave of soldiers… was known as ‘suicide squads,’ as they were frontline military. They were brutal.”  

This is the story of Cheryl Nicholas’ father, who was a child during the Japanese invasion of Malaysia during World War II. Stories such as this inspired Nicholas, associate professor of communication arts and sciences and global studies at Penn State Berks, to document other real-life stories of Malaysian survivors of WWII.  

This past spring, Nicholas and Heidi Mau, a professor from Albright College, led a team of students on the creation of a virtual reality (VR)/digital exhibition at the Perak Museum in Taiping, Perak, Malaysia, which opened on Aug. 15 to a crowd of more than 3,000 attendees, including government ministers. The exhibition runs through Oct. 22 — extended from Sept. 30 — and gives viewers a snapshot of that moment in history. 

“The exhibition was inspired by my father and grandfather, who told me about their experiences during the Japanese occupation when I was very young and did not quite appreciate them in the way I do today,” Nicholas said. “I conducted oral history interviews with over 20 people starting in 2013… and now, 10 years later, here I am with this exhibit, and with only five of the WWII survivors [who participated in the oral histories still alive]. Importantly, these stories provide a historical context for ‘everyday living’ during that time. These are stories not shared in history books as they were the stories of ordinary people, shared mainly within families. And as the storytellers passed on, so too did their stories.” 

The team behind the exhibition included project leader Nicholas and concept and production associate Mau. Penn State’s Center for Immersive Experiences handled the VR build and design, led by Patrick Dudas, research and development engineer; along with Alex Fatemi, programmer and analyst, and Bart Masters, lead programmer for XR. 

The group also comprised a Penn State Berks student research team, which included Chantel Bennett, Kimberly Nicholas, Aaliss Osidele, Avery Rivera and Jaymi Smith. Bennet, Osidele, Rivera and Smith were all students enrolled in Nicholas’ "Intercultural Communication" course when they became interested in the project. Kimberly Nicholas was invited to join the team because she is of Malaysian descent and could help with language translation during the students’ research. 

Two student members of the team visited Malaysia during spring break of 2023 to take panoramic LiDAR360 images and collect historical data. Since that time, the student team researched WWII Malaysian culture and history and helped write and select the stories featured in the exhibition booklet as well as the VR experience. Four students went to Malaysia in August to set up the exhibition and train museum staff to use the VR systems.  

“When I first learned about this project, I was very excited," said Rivera, a senior global studies major from Reading. "I really wanted to get more acquainted with Malaysian history and culture, especially since it’s a country I knew very little about. I also learned more about the Muslim culture. I hope one day I get to return to Malaysia and experience more of what I saw during this project.” 

Bennet, a senior communication arts and sciences major from Washington D.C., said this was an amazing opportunity to have a valuable learning in a country she didn’t know much about.

“When I was growing up, my mom would always tell me stories about my family tree," Bennet said. "When I would ask questions, sometimes she wouldn’t know how to answer. There have been too many times in history when important stories have been lost or ignored and I wanted to be a part of a project that focused on preserving stories such as these.” 

The exhibition focuses on the lived experiences of Malaysian survivors of various races and genders. The digital component utilizes mobile virtual reality and 360-point movement technologies navigable using Oculus Meta Quest 2. Museum guests use an Oculus system to participate in a virtual walkthrough of four spaces — a front yard, living room, kitchen and backyard — of a Malaysian home set in the 1940s, during the time of the occupation.  

Guests can also watch a pre-recorded 360-point tour of the house. The house design, like the stories it carries, is a blend of diverse cultural categories. Some of the furniture and objects displayed in the house belong to the WWII survivors from this project. Museum guests can move around the rooms and click on highlighted items which will open a three to five minute video montage of WWII stories from the survivors. Visitors can also view the VR world and the short video vignettes on a large monitor and two iPads set up as part of the exhibition. 

Nicholas explained that she started collecting oral histories, thinking she would include them in a book project. But when survivors’ families started asking about the stories, Nicholas said she reconsidered her original idea as she wanted the exhibition to be more accessible to people in her hometown, especially the remaining survivors and the survivors’ families. Working with her partner and spouse, Heidi Mau, the virtual exhibition idea took form. Nicholas credited Mau with the concept of the virtual exhibition and the living spaces.  
 
Along with the VR exhibition, the team created a short booklet featuring the oral histories. The booklet also provides background information about the event and the exhibition team. 

“This exhibition, using technology in this way is incredibly innovative and certainly unique for a Malaysian museum,” Nicholas said.  

This project was made possible with funding from the Penn State Student Engagement Network Travel Grants; Institute for Computational and Data Sciences Seed Grant; Alumni Fellow Award; and the Mrs. Harold McI. Grout and Mrs. C. Glenn Kauffman Endowment for the Arts. 

For more information on the ‘Malaysian WWII Oral Histories: A Digital Experience’ exhibition, contact Nicholas at [email protected].