Logo CMGT

"Join one of the compelling projects either trough the minor Immersive Media or the Saxion Smart Solutions learning community of Immersive Technologies and Storytelling"

About the Learning Community: Immersive Technologies and Storytelling

Within the learning community of Immersive Technologies and Storytelling we would like to invite you to work together with Saxion research groups and external partners on new and challenging questions in the areas of storytelling, computer graphics, gaming, virtual and augmented reality, computer vision, interaction- and game technology.

At the start of this module you will get together with various partners to discuss the proposed research topics listed below to see where you fit in given your motivation and personal learning goals. Together with the client you will define a definitive research project and form groups based on the expertise available.

During the five months semester you will work together with all the students participating in the learning community and the minor immersive media to exchange knowledge, inspiration and ideas. As a base of operation you will work full-time in the Saxion XR lab where all hardware and expertise is available to finish the projects.

How to apply

The learning community Immersive Technologies and Storytelling is part of the Saxion Smart Solutions Semester. The learning community is open for all Creative Media and Game Technology students who are eligable for the Smart Solutions module. The accesibility for students from other study programs is limited based on the projects available.

About the Minor Immersive Media

Within the minor Media you will work 5 months full-time with new and interesting types of technologies to design and create solutions to real problems found in the (creative) industry. These solutions will normally take the form of a product or prototype in the areas of virtual/augmented reality, animation, (embodied or digital) gaming, visual effects, etc. You will work in teams with other students from the minor and smart solutions under the guidance of a coach. At the start of this module you will get together with various partners to discuss the proposed projects listed below to see where you fit in given your motivation and personal learning goals. Together with the client you will define a definitive project goals and form groups based on the expertise available. During the minor you will work full-time in the Saxion XR lab.

The result of the minor should be a relevant and interesting professional portfolio item, new skills, new knowledge and new contacts within the creative industry.

How to apply

The minor Immersive Media is organised by the Saxion Creative Media and Game Technologies study program in Enschede. The minor is open for all students, either from Saxion or other universities. Partaking in the minor requires you to be in full-time available in Enschede for the duration of the minor.

For more information and to subscribe Saxion students can go to the Saxion website.
Other students can subscribe to the minor through www.kiesopmaat.nl.

Projects / research topics

Here you will find a list of project in which you can participate. Given your preference you will work in one of the project groups. You will choose your project in the first two weeks of the minor.

  1. Artificial EDM Creator
    PLANETART
  2. Bullying game
    Social Work, Ambient Intelligence & University of Melbourne
  3. FIFA data analysis and visualization
    UTwente eSport Lab
  4. Historical Storytelling Animation Concept
    Regional Development, Nymus3D & SchoolTV
  5. Neolithic Burial Mounds Experience
    Regional Development
  6. Procedural world creation & real data
    Bodem & ondergrond, Smart Cities
  7. VR Live Performance Arts
    Symfonia, Metropool, KITT Engineering
  8. Heracles: soccer field AR game
    Heracles, Ambient Intelligence
  9. Digital Humans for Virtual Reality Training
    COP maatschappelijke participatie
  10. Procedural visualization of brand values
    The Rebellion
  11. Photovoice
    Community Building Application (Social Work)
  12. African Cultural Insight Animation
    Tiny Giants

Artificial EDM Creator

PLANETART

When it comes to music, there are as many different tastes as there are people. Especially given the endless combinations of beat, melody, rhythm and harmony, DJ's are in the challenging position of finding combinations that are pleasantly listened too by a broad audience. When it comes to live music, this is an even more difficult task, that requires not only the skills to make the music but also an understanding and feeling for one's audience. This problem prompts PLANETART to ask the following question. Can an intelligent system, in combination with (bio)-sensors and computer vision, generate personalized EDM (Electronic Dance Music). Ideally such a system would generate video as well.

Research Goals

Research and create a prototype intelligent EDM/video creator that utilizes (body) sensors, computer vision, machine learning and adaptive audio to generate personalized audio and video experience. Here the system must adjust to its audience over time, leading to an optimum engagement.

  1. Research and create a biofeedback system that can read out information about music listeners and evaluate if music is enjoyable.
  2. Research and create an intelligent adaptive audio system that can generate music based on the feedback system.
  3. Build a portable and engaging installation that can be exhibited at various events.

Guidance

During the process you will be guided by experts from Saxion and PLANETART. You will work in the Saxion XR lab and/or the WARP Technopolis where you could possibly work together with DJ's like Bas Amro and Joep Wiertz.

Topics

Sound, adaptive audio, (bio)-sensors, artificial intelligence.

PLANETART

https://www.planetart.nl

Bullying game

Social Work, Ambient Intelligence & University of Melbourne

The role of digital platforms, specifically social media, in facilitating online and offline bullying and violence between young people is receiving attention from both governments and industry. Social media can play a major role in setting norms within communities and groups that facilitate violent behaviour. Evidence points to social norms being the main predictor for bullying and youth violence. However, there is limited evidence as to how different online content, especially global versus local misinformation, and polarizing information impacts these norms and their associated role in youth violence. Given this background, the Saxion research groups of Social Work and Ambient Intelligence together with the University of Melbourne would like to ask your help by building a gamified intervention on this topics.

Research Goals

Research and create a prototype bullying game that seeks to mitigate cyber bullying while at the same time collects data about people's behaviour. While designing and building the game you should look into:

  1. What causes bullying and which strategies could be used to mitigate the behaviour.
  2. How can a game be used to collect valuable information about the influence of information and polarization on bullying, while at the same time aligning with privacy law (e.g. GDPR)
  3. How can these insights be used for fun and engaging game (gameplay is king!).
  4. Which platform and technologies should be used for the creation of a protype.
  5. Which art style is fitting for a bullying game.

Guidance

During the process you will be guided by experts from Saxion, KITT Engineering, Metropool and Symfonia. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Cyber bullying, game design, game art, game development.

Links:

FIFA data analysis and visualization

UTwente eSport Lab

One of the prominent developments currently in the game industry that is rapidly evolving eSports scene. Within this branch professional gamers focus on games such as FIFA20, CSGO, League of Legends and Rocket League to compete in professional global competitions which will even include the 2024 Olympics. Similar to professional (traditional) sports teams such as soccer or speed skating, eSports teams are composed of athletes, sport psychologists, cognitive scientists, nutritionists, and health specialists. Another important group of professionals forming the backbone in these sports teams are the are data scientists that analyze, interpret and visualize in-game data and performance.

All this effort is aimed to optimize performance of the eSporters. Here minor differences in the setup, training or (psychological, physiological or performance) can have major impact on in-game performance. For FIFA20 (and FIFA 21 when the assignment starts) the University of Twente, the eDivisie, Team Gullit, and SciSports are teaming up with the aim to develop tools for eSporters to give them that edge in FIFA by providing, interpreting and visualizing their in-game data. We furthermore work closely together with professional FIFA players.

Research Goals

In the current project I hope to challenge you to come up with new and innovative ways to:

  1. Provide players and/or coaches with data driven in-game feedback in (remote) training situations (dashboards, training tools) and furthermore capture and visualize player behavior such as controller or keyboard use.
  2. Provide players and/or coaches with relevant feedback during competitive gameplay with regard to in-game data.
  3. Develop new ways to provide the audience with player information to increase viewer engagement. (*nice to have).
  4. Create a tool that can be used for data driven research with FIFA20/21.
  5. Create an AI FIFA training bot (*as a super cool nice to have).

We aim to combine knowledge from all parties during this project. Soccer data analysis from scisports and Tripple (team Gullit), real professional FIFA players from soccer clubs, and research skills and facilities from the eSportslab of the University of Twente.

Guidance

During the process you will be guided by experts from the UT-IEBIS, eDivisie, Team Gullit and Scisports. You will work in the Saxion XR Lab or the UTwente eSports Lab.

Topics

Computer vision (to scrape in-game elements), data visualization, data interpretation, tool development, eSports or FIFA.

Links:

Historical Storytelling Animation Concept

Regional Development, Nymus3D & SchoolTV

In the complex European political and cultural landscape one thing that connects us is a shared history. Within this rich context there are many, often regional stories that had profound effects on people, countries and Europe as a whole. Given this background the Saxion Research group of Regional Development together with Nymus3D (and SchoolTV) would like to challenge you to come up with an interesting and compelling media concept that focus on animation and historic storytelling.

Research Goals

Design and create an animation/video storytelling concept consisting of multiple short stories around important, but lesser known historical events. Here you will have to tailor the concept and stories to the target audience and take into account historical accuracy and relevance. In this assignment you will have to:

  1. Come up with a commercially feasible media format around historical storytelling.
  2. Research historic events that can be converted into short animations preferably around a central theme.
  3. Create a compelling visual style and a trailer for one of the short stories.

Guidance

During the process you will be guided by experts from the Saxion, the research group of Regional Development, Nymus3D and SchoolTV. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Storytelling, animation, video, media concept, history

Links:

Neolithic Burial Mounds Experience

Regional Development

Regional tourism is an important source of income for small businesses and municipalities throughout the Netherlands. Especially given the overload of tourists in hotspots like Amsterdam, drawing these visitors to these other, less known locations is one of the research lines for the Saxion research group of Regional Development.

One of the projects the research group is involved in is the realization of a new to build tourist center in the east part of the Dutch region of the Veluwe (https://en.wikipedia.org/wiki/Veluwe). Here an extensive collection of burial mounds can be found dating back to 3000 B.C. For this project, the research group is looking at how to combine technology and storytelling to draw in more visitors. Not only to the tourist center but also the region as a whole. For this purpose, they would like your help.

Research Goals

Design and create an interactive (visual) experience that allows visitors of a new to build tourist center, to experience the culture and stories behind the neolithic burial mounds. Here you will have to:

  1. Research the historical background of the burial mounds and the people who created them. Within this context you will have to find interesting stories that can be used to engage the target audience.
  2. Research different technologies that can be utilized for deep immersive storytelling.
  3. Research how the target group can be engaged.
  4. Look into how your solution can be used to draw more visitors to the tourist center and the Veluwe region as a whole.

Guidance

During the process you will be guided by experts from the Saxion and the research group of Regional Development. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Storytelling, animation, video, media concept, history

Procedural world creation & real data

Bodem & ondergrond, Smart Cities

The procedural creation of complex interactive worlds is a practice much used in the game industry where whole dynamic cities are created, sometimes based on real data. In the world of urban development such practices are more and more used to help in developing multi-stakeholder policies. Given these practices, the research group of Soil and Subsoil, and the research group of Smart Cities would like to know to what extend these technologies can be used to model the real-world (above ground and the subsoil) in real-time based on real data. Here to aim is to create a visual compelling prototype 3D application that can be used by various organizations to communicate with civilians about cities and subsoil.

Research Goals

Research and create a prototype application that can procedurally combine different real-world data sources into a realistic, visual compelling and interactive 3D visualization of both the area above ground and the subsoil. To build this prototype you will have to research:

  1. Which data sources are available and how this data can be preprocessed for procedural content creation.
  2. Given the client which features should be in the application on how can they be technically realized. This also includes interactions and possible simulations.
  3. How to utilize proceduralism to create a realistic as possible representation of the real world.
  4. How can such representations help in developing policies on subsurface use in urban areas?

Guidance

During the process you will be guided by experts from the Saxion. Data will be made available through the Twentse municipalities, NAM and TNO and other open sources. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Proceduralism, 3D, game engines, city building.

Links:

Example

VR Live Performance Arts

Symfonia, Metropool, KITT Enginering

Given the recent developments with the Coronavirus and its impact on society, one area that has been deeply affected is the cultural sector. Due to the impossibilities to organize events, revenue is falling and the future of many organizations is uncertain. However in every crisis there is also an opportunity to innovate. Here three regional organizations, the Metropool, Symfonia and KITT Engineering would like to request your help to explore the use of VR to bring live music to people's homes.

Research Goals

Research and create a prototype live performance arts VR experience. Here you should take into account:

  1. What kind of experience would draw a potential audience to VR for a live music performance.
  2. How should a VR live performance be made. Here you will have to take into account the possibilities and limitations of the medium and combine it with other technologies such as motion capture. Here innovative solutions focusing on a complete VR experience are prefered.
  3. Which business models could be used to bring in revenue for the involved cultural partners (Metropool and Symfonia).

Guidance

During the process you will be guided by experts from Saxion, KITT Engineering, Metropool and Symfonia. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Live performance arts, VR, business models, motion capture, computer arts.

Links:

Heracles: soccer field AR game

Heracles, Ambient Intelligence

Why idly wait for the soccer match to begin or during the break, when you can play a game right on the soccer field with AR on your phone? The soccer club Heracles is one of the most technologically advanced clubs and they also want to bring this to their visitors. Aside from optimizing the matchday experience for visitors, such an AR game could also provide new advertising opportunities for club sponsors.

Your challenge is to design and implement an AR game to be played on the soccer field of Heracles, that is appealing to their fan base, can be played while waiting in their seats, and that ideally can include sponsor advertisement integrated within the game itself. No, it does not have to include cars ;)

Research Goals

Research and create a prototype mobile AR game that can be played on the soccer field of Erve Asito (the stadium of Heracles). To build this prototype you will have to research:

  1. How to detect the soccer field and place virtual objects on it in the intended size and orientation, that are persistent at least for the duration of the game
  2. What kind of game types / topics / art style / concepts appeal to the Heracles fan base (it could be fun to actually have some co-creation sessions with fans!)
  3. Bonus: Ways to integrate sponsor advertisements into the game (no pop-up ads)
  4. How the game you developed is actually received by the fan base (do they like it? What can be improved?)

Guidance

During the process you will be guided by experts from Saxion. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Augmented reality, mobile, game, soccer, advertisement.

Links:

Digital Humans for Virtual Reality Training

COP Maatschappelijke participatie

Creating believable virtual humans is a difficult tasks that requires specialist skills and knowledge. Especially for the purpose of virtual reality training applications these avatars should be of very high quality, able to convey emotion and bridge the uncanny valley.

In a previous Immersive Technology and Storytelling project a group of students build a prototype VR aggression training application. Here they used motion capture and an iPhone to drive a virtual human which was created through photogrammetry and various 3D modelling software. With this new project we want to expand upon this and then especially the creation of digital faces in combination with the capturing and conveying of emotions. In this process we would like to ask your help.

Research Goals

Within the research we would like you to look at the complete pipeline for capturing and driving of virtual humans, for live and prerecorded VR experiences. When a prototype is build you should test it. Here we would like to know if your prototype is believable to humans and to which extend it is usable for training situations. To be more specific we ask you to look into:

  1. How to robustly scan humans for digitization. Here you should focus on the face mostly, with photogrammetry as a starting point.
  2. How to drive a digital face using an iPhone and FACS (Facial Action Coding System)
  3. How to create believable skin shaders as realistic as possible that can be run real-time.
  4. How does training with a virtual human compare to training with a real human.

Guidance

During the process you will be guided by experts from Saxion. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Virtual humans, photogrammetry, rigging, technical artist, user research.

Links:

Procedural visualization of brand values

The Rebellion

Looking at the process of designing a brand we’re facing a black box. The input consists, amongst others, of brand values, target audience, trends and the output is noticeable by colors, typefaces, shapes etc. But what happens between input and output is for most of us some sort of magic. The inside of this black box must be revealed to find out if it would be possible to procedurally visualize brand values.

We are not looking for a new generator that enables us to generate house styles. We don’t want you to just enter tons of parameters. No, we want you to help us to develop a cognitive computing application that processes information more like a human than like a computer. As in the design process the value of the human touch is key, we want to explore the possibilities to go beyond an application that just combines building blocks. We want the outcome to go beyond these existing generators:

Research Goals

The starting point is to analyze the creative process. How do brand designers approach the visualization of a brand? What value do designers add to come to a striking, touching design that stands out from the crowd. And what makes one design likable, while the other fails epically? In other words, can you open the black box? The challenge is to find in this black box the data the engineer needs to develop a procedural design application with a “human touch”, combining emotion with efficiency.

We want artists, designers and engineers that are interested in diving into the creative process. And that would like to work on computational creativity.

Guidance

During the process you will be guided by experts from Saxion. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

computational creativity

Links:

Photovoice

Community Building Application (Social Work)

When discussing societal issues, one of the problems encountered is that there is no consensus among people on the exact nature of the issues. This problem is present on a national but also local levels where citizens of the same municipality with different background and from different communities disagree on an issue like poverty, good health, access for disabilities, cleanliness of neighborhoods, etc. To help solve these problems, a prototype application called Photovoice has been developed. The idea behind this application is that civilians can express their concerns by posting photos in a map of the municipality.

Based on the receptions of the initial prototype by the municipality of Steinfurt (Germany), we would like to explore further development of this concept. Here the focus is on the one hand on how it can encourage social cohesion and open dialogue between citizens and government. Simultaneously we are interested in exploring additional technical features and how this could contribute to better user experience.

Research Goals

Within the Photovoice Phase 2 we would like you to research how to current prototype application can be further developed such that it can be used as a successful tool in creating more social cohesion, empathy and dialogue. Here you will have to specifically look into:

  1. How the application can be brought to market and create a plan how the municipality of Steinfurt could implement it in its day-to-day workflow.
  2. How to incorporate multi-user functionality where civil servants, decision-makers and civilians from different backgrounds can communicate and act upon published content.
  3. Explore how to incorporate a realistic (interactive) 3D map of Steinfurt into the application such that user can recognize the area where content is uploaded. See an example screenshot below for reference.
  4. Explore different display options for interaction with the map. Here you can think of big projections (e.g. in the city hall), desktop, mobile, VR and AR.
  5. Test the various prototypes with the target groups in terms of influence on social cohesions, empathy and dialogue. As a result of the tests, adjust the application if necessary

Guidance

During the process you will be guided by experts from Saxion. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

User research, social work, application development, interface design.

Logo CMGT

African Cultural Insight Animation

Tiny Giants

Africa is the next growing continent in the world. With many colorful cultures derived from there and the endless amounts of storytelling within, it would only be a shame if they are untold. Therefore Tiny Giants, together with Wajenzi Productions, is promoting the awareness of African culture within Western communities and families, primarily children.

We are interested in the challenge of communicating foreign stories and cultural values to western children. Based on personal experiences, we truly believe that there is plenty to learn there.

As a pilot project, we would like to produce an animation that tells a specific story about African culture. As a means to create social inclusion, we are curious about the various ways possible to create such an animation, as many things will have to be considered, including social behavior, the optimal visual style, the way in which characters are expressed, and more! Addition to that, we are curious about the implementation of real-time motion capture technology, as a way to bring out the best in character expression and acting.

Research Goals

Research and create an animation to inform people about African culture. Here you will have to look into:

  1. What are the optimal visual styles when stimulating and teaching values about African culture towards western children?
  2. What are the benefits of using motion capture technology to record character movement and acting, and what are the social limitations there? An example of this would be the translation of real-acting to caricature behaviour.
  3. How can we incorporate custom-produced music based on African culture as a form of communication within the animation, instead of using actual voice narration?
  4. What are the psychological elements and implications when using specific types of visual decisions and what works best when communicating to younger age groups?

Guidance

During the process you will be guided by experts from Tiny Giants. You will work in the Saxion XR Lab in the Saxion Alphons Ariëns building (Ariënsplein).

Topics

Animation, motion capture, storytelling, digital arts

Links: