Studying Chinese in Beijing, by way of Rensselaer

by Emily Donohue on December 13, 2013

If you wanted to learn to speak Chinese – really speak Chinese at a level that allowed you to converse with native speakers at a high level and conduct business in China – you’d have to go to China, right?

Not necessarily, thanks to the work of some Rensselaer Polytechnic Institute professors.

They’re working to create the next generation of language education – a hybrid of immersive classroom experiences and virtual reality adventures designed to grab students’ attention and keep it as they master Chinese.

Teaching a foreign language, especially one as difficult for native English speakers as Chinese, is a particularly thorny problem. Listening, speaking, reading, and writing are all crucial components, but so is the cultural context in which the language is spoken or written.

Particularly thorny problems are exactly the kind that professors Lee Sheldon and Ben Chang like to tackle. Together, they run the Emergent Reality Lab (ERL) in the Rensselaer Technology Park.

Their concept is to combine elements of gaming, like extended narratives and ongoing quests utilizing mixed-reality technology, with the best practices of language education to create the world’s most advanced Chinese language pedagogy. Helping them are two experts in Chinese language education recently added to the HASS faculty, Jianling Yue and Yalun Zhou.

“We think about language as always being situated in a culture and in a physical location. It’s connected to place and to people; it’s not usually just an isolated thing. So experiencing a language in that way, in that context is going to be a valuable thing,” Chang said. “With the additional elements of game play and narrative, there’s this heightened motivation and interest and engagement level with the learning.”

In late November, the first class of students taking Chinese at Rensselaer since efforts began to re-establish the Chinese language minor, stepped into the Mandarin Project’s hangar-like space in the ERL for a test-drive.

They took seats at café tables surrounded on three sides by massive screens projecting the image of a Beijing teahouse. Facing the students at the front of the room was a woman – a virtual woman – who would serve as their teacher.

The students donned 3-D glasses, the same kind you’d find at a movie theater, and one student was designated the guide and controlled the experience by wearing a hat studded with ping pong-like balls and holding a video game controller.

The lights dimmed and the students traveled to Beijing.




(Photos by Ray Felix)

Video: Students test drive Mandarin Project technology
(Video by Rensselaer videographer Eleanor Goldsmith)

As Sheldon and Chang looked on, with what seemed like equal parts anxiety and excitement, the students were led through a series of questions by the virtual instructor. She taught them about traditional Chinese tea ceremonies and they had to use their knowledge of Chinese – still fledgling at this early stage of their studies – to interpret her questions and select the correct answer. By doing so, they moved on to the next part of the lesson.

That desire to move forward through the narrative is one of the keys of the Mandarin Project. Before his career in video games, and then academia, began, Sheldon worked as writer for television shows including Star Trek: The Next Generation, Charlie’s Angels, and Cagney and Lacey.

His success as a television writer has informed his work with the Mandarin Project. “You don’t complete every episode or every week or every story on a daytime soap all at the same time, you overlap them,” he said. Eventually, students studying Chinese through the Mandarin Project will have a similar experience: “This is an ongoing story, they want to learn what happens next and the only way to do that is to learn the next week’s vocabulary.”

The goal is for students to “leave the end of each class with a cliffhanger so they want to take the next level of Chinese,” he added.

Inextricable from the storytelling of Mandarin, is the mixed-reality setting in which the story unfolds. That is Chang’s specialty.

Mandarin utilizes CAVE — a virtual reality technology that originated at the University of Illinois at Chicago — to create an environment that can transport a group of students together, rather than one student individually.

“We don’t want to take students completely away from their physical bodies and their classmates and the people around them,” Chang said. “We’re transporting them as a group to this physical place.”

The technology used to create the mixed-reality setting for Mandarin isn’t new, but it is being used in an entirely new way at Rensselaer.

Chang calls it “an incredibly potent technology and an incredibly potent creative medium that we’ve only begun to scratch the surface of.” The applications in education and innumerable other fields have just barely been tapped, he added.

One of the things limiting exploration of this type of mixed-reality technology is expense, he said.

“Something unique about Rensselaer getting involved with this technology is being able to build a system like this in what’s, relatively speaking, a kind of DIY, low-cost method using a lot of homegrown things and consumer hardware… and then being able to commit a valuable amout of time and resources and energy and creativity into doing something that is far more experimental and ‘blue sky’ than a lot of the typical applications for the technology,” Chang said.

The team behind Mandarin is larger than Sheldon, Chang, Yue, and Zhou and includes graduate and undergraduate students and experts from a variety of fields. Chang compared the creation of Mandarin to a movie production with a large team of specialists coming together to word toward one goal.

Everyone – from the students to the professors – is learning on the job, he added.

In the fall of 2013, the first class of students took the first level of Chinese language offered at Rensselaer in several years – that class of students is the class that test drove Mandarin in November. During the spring semester, two more level one classes will be offered as well as a level two course. The rollout of Mandarin will be slow and deliberate, but it is likely to begin to be incorporated into the more advanced Chinese courses in coming semesters.

Ben Chang, left, and Lee Sheldon address the students who test drove Mandarin in November