Experiential Design: Task 3: Project Prototype

5.10.2023 - 10.11.2023 (Week 6 - Week 11)
Tan Yi-Tyng (0353327)
Bachelor of Design (Hons) in Creative Media
Experiential Design
Task 3: Creative Experience Design Project Prototype

INSTRUCTIONS

TASK 3

Experience Design Project Prototype

Once our proposal is approved, we will then work on the prototype of our project. The prototype will enable us to discover certain limitations that we might not have learned about before, and we'll have to creatively think about how to overcome these limitations to materialize our proposed ideas. The objective of this task is for us to test out the key functionality of our project. The output may not necessarily be a finished visually designed project. We will be assessed on our prototype's functionality and our ability to creatively think of alternatives to achieve the desired outcome.

Requirements for the final submission:

1. Submit the Blog post and Google Drive link to MyTimes.

2. Provide a video presentation and walkthrough of our AR project prototype.

3. Explain our AR app and elaborate on what has been done and what is yet to be done.

4. Include our progress in building or collecting assets.

DESIGN PROCESS

1. Finalize the App Interface Design

Since the last project, I have already constructed the interface design for some of the pages in Figma. At this stage, I made adjustments to the existing design, such as button placement or color. I added new pages, including a homepage, to enhance user navigation. After completing the interface design, I began importing the visuals into Unity and started building the app.

Fig.1.1 -References for the Interface Design

Fig. 1.2 - Process of designing the UI in Figma

Fig. 1.3 - Process of designing the UI in Figma

2. Redesign and Finalize the World Map

Following feedback from Mr. Razif, who suggested making adjustments to the previous world map design for better image target scanning, I decided to redesign it. To improve scanning, landmarks on the world map should have a strong contrast with the background and be represented by icons rather than text. I redesigned the world map with more contrasting colors, using black icons to represent landmarks. This not only makes them stand out but also adds a sense of mystery, encouraging users to explore by scanning and revealing a realistic live view 3D rendering of the landmark.

However, for this prototype stage of the AR app, I haven't printed it out yet; instead, I use my iPad for scanning when necessary. I'm cautious about potential changes to the world map design. Once I confirm the design and finish building the app, I'll proceed to print it out.

Fig. 2.1 - Previous World Map Design

Fig. 2.2 - Process of redesigning the map in Illustrator

Fig. 2.3 - New World Map Design

3. Upload the Image Target to Vuforia

In Adobe Illustrator, I duplicated the icons of the landmarks and placed them on another artboard with a white background. For this prototype, I experimented with one landmark first, the Eiffel Tower. I uploaded the image of the Eiffel Tower to Vuforia Engine, initially receiving a two-star rating for the image target. After reading how to achieve a higher rating, I learned that the edges or shapes of the image need to be unique and detailed, avoiding too many round edges. I recreated the Eiffel Tower icon in Illustrator, reuploaded it to Vuforia Engine, and finally achieved a four-star rating.

Fig. 3.1 - Upload the image target to Vuforia

4. AR Scan in Unity

Moving onto the next phase, which involves building the asset in Unity. I began by creating an image target and importing the database from Vuforia Engine. Initially, I used a 3D cube to test whether it worked or not, and fortunately, it worked successfully when I scanned the map, causing the 3D cube to pop out. (*Sorry for my mistake that I forgot to capture some of the process.)

Following this, I went online to search for free 3D model assets of the Eiffel Tower. After downloading the model in either FBX or OBJ file format, I imported it into Unity to replace the cube.

Fig 4.1 - Download the 3D model from online

Upon importing the 3D model into Unity, I noticed that the object didn't have the texture and color. To solve this issue, I searched for tutorials on YouTube to learn how to apply textures in image form onto the 3D object.

Fig. 4.2 - Import the 3D model into Unity but with no texture.

Fig. 4.3 - Youtube Tutorial on Apply texture on 3D object

Fig. 4.4 - Texture applied on the 3D object of the Eiffel Tower

I also created an animation for the preview of the information. When the user scans the landmark on the map, it will not only show the 3D view of the landmark but also display an animation of the preview info panel floating up from the bottom. This animation guides the user to click the 'Read More' button for more detailed information about the landmark in the next scene.

Fig. 4.5 - Animation of preview info panel

Fig. 4.6 - The preview on scanning the map:
3D object pop out & preview info panel float up

5. Panorama 360 view

For the next step, which involves experimenting with the panorama 360-view feature, I turned to YouTube to learn how to create this functionality. The process of creating this feature was quite smooth and easy to follow. However, the only issue I faced was the difficulty in finding a free source of high resolution 360-view images online. Most of the available images required payment, and the only free image I found was in low resolution. Mr Razif mentioned that it's acceptable to use that image for the prototype stage, but I'll need to replace it with a clearer image when working on the final project.

Fig. 5.1 - Process of creating the 360 view feature

Fig. 5.2 - YouTube Tutorial of create 360 view

6. Image slider on the info page

For the Info page of the landmark, I decided to create an image slider for users to view a series of images showcasing different views of the Eiffel Tower, such as during the daytime and at night with appealing lighting.

Fig. 6.1 - Process of creating the image slider

Fig. 6.2 - YouTube tutorial of the Image slider.

7. Audio Play and Stop in Info Page

To enhance accessibility, I implemented a feature with an audio introduction that explains information about the Eiffel Tower. Users can click the button in the top right corner to play the audio, and they can easily stop it by clicking the same button again. As I felt like this feature would be easier to create by using script, I asked ChatGPT to help generate the script, and it worked perfectly!

For the audio, I wrote out the speaking script and used a website to convert it into audio. It took me quite some time to find the desired audio, as most of them sounded too much like an AI robot speaking.

Fig.7.1 - Script for the Audio

Fig. 7.2 - Convert text to audio online

8. Video Player on Image Target

This feature was not initially part of my plan. While browsing the Unity tutorials on YouTube, I came across this interesting concept and am now considering integrating it into my AR app. Originally, I planned to use image targets in the form of a world map (which I have already done).

Fig. 8.1 - Youtube Tutorial of Video player on image target

The new idea involves using a marker card with information about landmarks on one side, following the initial plan. On the other side of the card, there is an image representing the first frame of a video. When users scan this side of the card, the image seamlessly transitions into a video playing on top of the card.

However, the implementation process has been challenging. I encountered several issues and spent a large amount of time attempting to solve them by following multiple YouTube tutorials. Currently, the functionality somewhat works fine, but there is a minor issue where the audio automatically plays without initiating the scanning process. Fixing this issue might take some time, and I'm considering whether to proceed with this idea.

Fig. 8.3 - Recording of how the video player on image target looked like.

9. Switching scene with button

Lastly is the stage to link all the scenes together so users can switch between scenes by just clicking the button.

Fig. 9.1. - YouTube tutorial of scene change

Fig. 9.2 - Script for the scene change

Fig. 9.3 - Setting on the inspector section in Unity

Fig. 9.4 - Setting on the inspector section in Unity

FINAL SUBMISSION

Task 3: Creative Experience Design Project Prototype

Google Drive link:
https://drive.google.com/drive/folders/18IxBeF732E_CMxQbssRAAB9py42cce1K?usp=sharing

YouTube link:
https://youtu.be/WHC2rkFsCvo?si=Fh8EL0CBtza81PdC

FEEDBACK

WEEK 10 - Task 3 (Prototype)

4.11.2023

Change the first scene model pop out model to the accurate one.
You probably want to choose different icon, because AR icon normally inside is just a cube in a perspective view, or you can just put AR inside that box, the icon for panoramic view, in the middle you can put 360
You can use animation to create horizontal scrolling, divided into one whole section and when you click this it will be animate to the left and move horizontally the left section.

WEEK 11 - Task 3 (Prototype)

8.11.2023

For the video player on image target, remember to adjust size of the new texture material according to your video size.

REFLECTION

My feelings throughout the completion of this project have been akin to a roller coaster ride. Initially, I felt lost and uncertain about how to start building my prototype. However, after watching tutorials and receiving guidance from classmates, I became more familiar with Unity, and the process didn't seem as challenging as I had initially thought.

Driven by a combination of ambition and perfectionism, I pushed myself to incorporate as many features as possible. Unfortunately, as I added more features in Unity, a lot of errors began to surface. The situation escalated when I encountered difficulties exporting the app to my phone. At that moment I was overwhelmed as I spent nearly 2 whole days fixing this problem and it was very time consuming.

Fortunately, Mr. Razif came to help me resolve the export issue. Although I managed to make it work, some parts of the project were affected, and certain features that used to function properly were now malfunctioning. This whole experience taught me the importance of staying optimistic and seeking help from classmates and even ChatGPT (HAHA). There is always a solution to be found.

Overall, the most important thing I keep reminding myself during this process is to BACK UP the file. I felt like this unity software was so unstable and it would suddenly pop out an unknown error and might cause the whole project crash, so having a backup file is more safe.

QUICK LINKS

Task 1: Trending Experience

Task 2: Project Proposal

Task 3: Project Prototype

Final Project: Completed Experience

TAN YI-TYNG (0353327)