Experiential Design - Final Project: Completed Experience

10.11.2023 - 17.12.2023 (Week 11 - Week 16)
Tan Yi-Tyng (0353327)
Bachelor of Design (Hons) in Creative Media
Experiential Design
Final Project: Completed Experience

INSTRUCTIONS

FINAL PROJECT

Completed Experience

We need to synthesize the knowledge gained in tasks 1, 2, and 3 for application in this final project. We will integrate visual assets and refine the prototype into a complete, working, and functional product experience.

Requirements for the final submission:

1. Unity Project files and folders

2. App build file (APK for Android, Build Folder for Ios)

3. Video Walkthrough/Presentation of our App

4. Blog post link

DESIGN PROCESS

After completing the prototype of our AR app, we are now in the process of continuing its development, focusing on refinement and adding the final touches. This step is crucial as it prepares us for the final project, where our goal is to create a fully comprehensive and polished application. This involve refining the user interface, enhancing functionality, and ensuring a seamless user experience.

1. AR Ground Scan

As mentioned in the previous prototype, I did not successfully create the AR Ground Scan effect. Fortunately, I sought help from Mr. Razif, who provided suggestions on trying different ways to resolve the issue. It might have been due to a file version issue or a bug, but after a few hours of experimentation, it finally worked. A big thanks to my classmates who provided me with the script. I have successfully implemented the functionality where the 3D model can be scaled using pinch gestures, allowing users to have a more engaging way to interact with the landmark.

Fig. 1.1 - Ar Ground Scan

Fig. 1.2 - Script for Pinch to scale the 3D models

Fig. 1.3 - Result of AR ground scan with pinch to scale the landmark

And once I learned how to create the AR Ground Scan effect, I proceeded to work on and build for other landmarks. I decided to focus on only five landmarks, and during the initial planning phase, I listed several landmarks without finalizing my choices in the prototype phase. However, I could only decide based on the availability of related sources online. I chose landmarks that had the most accessible sources online, as my app relies on free online resources.

Since I could only access free sources online, my options were limited, and some 3D models of landmarks required payment or were challenging to find. For example, locating satisfactory sources for the Sydney Opera House and the Colosseum proved difficult. As a result, I ended up finding 3D models of the Statue of Liberty, the Leaning Tower of Pisa, the Arc de Triomphe, and the Petronas Twin Towers. I proceeded to build the AR Ground Scan for each scene using these selected landmarks.

Fig. 1.4 - Finding 3d model online

Fig. 1.5 - Build Ar Ground Scan for other landmarks

2. Panorama 360 view

For the 360 view, I successfully built one scene for the Eiffel Tower, and technically, building the effect presented no major problems. However, the primary challenge arose from the limited availability of downloadable 360 view images online. I spent a significant amount of time researching and searching for free sources of 360 view images, and I discovered a website (Link) that provided multiple options for the panorama view of landmarks. Unfortunately, access to these images required payment. To address this, I found an alternative website (Link) that allowed me to remove watermarks and obtain free resources. The drawback, though, was a reduction in image quality and some blurriness. Nevertheless, the crucial achievement was that I learned how to build the 360 view effect in Unity, successfully meeting the goals of the module.

Fig. 2.1 - 360 view sources online

Fig. 2.2 - Tutorial on 360 view image in Unity

Fig. 2.3 - Result of 360 view of the landmark

For some of the landmarks where I couldn't find 360 view images, I came up with the idea of replacing them with videos for a 360 view. Fortunately, I found a tutorial on YouTube that worked well and provided a slightly clearer view than the images. It's interesting that a panoramic view through video adds a more realistic touch, allowing me to immerse myself more in the environment of the landmark.

Fig. 2.4 - 360 view video source on Youtube

Fig. 2.5 - Download the 350 view video

Fig. 2.6 - Tutorial on create 360 view video in Unity

Fig. 2.7 - Result of 360 view video of the landmark

3. Video Player on Image Target

As mentioned in the previous task (prototype), I experimented with creating the feature where users can scan the marker card, and the video will play on top of it. However, when I followed the guide from a YouTube tutorial, it did not work as expected. Subsequently, I discovered a tutorial in the playlist that Mr. Razif posted online. I carefully followed the steps, and finally, it worked. I was delighted with the result, considering the significant amount of time I spent on trial and error.

However, one issue I faced was that the image did not target properly, and the video started playing automatically. After receiving feedback from Mr. Razif, he reminded me to set the "On Click Target Found" and "Lost," and this adjustment resolved the problem. It finally worked well, and I greatly appreciated the guidance in overcoming this hurdle.

Fig. 3.1 - Add component (Video Player)

Fig. 3.2 - Set the target found and lost

Fig. 3.3 - Result of video player on image target

For the landmark videos, I spent some time browsing through YouTube to find the best shots of the landmarks from different angles, aiming to provide users with the opportunity to view the landmark and immerse themselves in the environment. However, I encountered an issue where some videos were excessively long, potentially testing the patience of users as they need to watch the video on the image target.

To address this concern, I turned to Premiere Pro to edit the videos. My goal was to shorten the timeframe by combining selected shots, focusing on the most visually appealing angles. Additionally, I replaced the original background music with another track and added a film dissolve effect at the end of the video. This cinematic touch enhances the overall user experience, making the video more engaging and visually appealing.

Fig. 3.4 - Video editing in Premiere Pro

Below is the link to the edited video of each landmark for video player image target:

https://drive.google.com/drive/folders/10i6wJxAE7bf9c2fI7aAmaByLJNh0lxkX?usp=drive_link

4. Card Design

In the previous task, I mentioned that I had not yet decided whether to choose a card or a world map as the image target. Since I have successfully created the video player to work with the image target, I have decided to select a card as the target, even though I had previously designed the world map. I chose a marker card because it has designated designs on both sides. One side features an icon of the landmark, and when users scan it, a 3D model of the landmark pops up, allowing users to read information about the landmark. The other side of the card displays a visual image of the landmark, and when scanned, it instantly plays the associated video on top of it. I believe this approach will be more interesting and engaging for users, providing a unique and immersive AR experience and a more captivating way to learn about landmarks.

Fig. 4.1 - The world map that was designed previously but was not selected now

It might be curious why I started designing and printing the card halfway through the process rather than at the beginning. This is because, in the initial stages, I had not yet decided on which five landmarks to include in the app. There was uncertainty regarding the availability of sources, such as 3D models or 360-view images for certain landmarks, making it challenging to finalize my selection. For example, I initially planned to include the Sydney Opera House, but halfway through, I faced limitations in finding relevant source material. Therefore, I needed to ensure that all the content and assets were ready and confirmed for use before proceeding to design and print the card.

Fig. 4.2 -Process of designing cards in Illustrator

After completing the design of the card, I try upload all the images to Vuforia Engine and luckily all of them got 4 start and above. Then I downloaded the database and import to my Unity file and try to scan it if it worked. Make sure all the card can be scanning well and worked and I could then proceed to print the card.

Fig. 4.3 - Upload the image target to Vuforia

Fig. 4.4 - Final Outcome of the printed cards (Front)

Fig. 4.5 - Final Outcome of the printed cards (Back)

Fig. 4.5 - Final Outcome of the printed cards (Front and Back)

Unfortunately, the outcome of the printed cards was not as I expected due to some color changes, even though I set the file in CMYK mode when sending it. The original card color was in a brown theme that matched the color theme of the app, however the background color changed from light brown to white, and the brown changed to pink. Nevertheless, since the cards can still be scanned for the image target, I proceeded to work on other tasks.

Fig. 4.3 - Final Card Design (PDF)

To enhance visual presentation, I also created a mockup for the cards.

5. Working on the Info Page

Moving on to the next phase, which involves continuing to work on the info page for all landmarks. I have created a carousel on the introduction page featuring images of the landmark, including different viewing angles, day and night views, and close-up details of the landmark. This allows users to gain a clearer understanding of the design and structure of the landmark.

Fig. 5.1 - Carousel

In the previous prototype, I faced challenges getting the animation to work on the button between the About section and the Construction & History section. Mr. Razif taught me how to create the animation by adjusting the position of the entire section and setting the button on click to trigger the animation. I'm satisfied with the result of the transition between the two sections, but I wanted to further emphasize the button with color and a line when users are on the section.

However, I encountered confusion and experimented a few times without success. To resolve this, I turned to YouTube to find a relevant tutorial. Finally, I found a tutorial video. While it wasn't an exact match, but it provided me with the knowledge of the technique allowing me to implement the desired button emphasis in the page.

Fig. 5.1- Design of the Info Page

To enhance accessibility, I implemented a feature with an audio introduction that explains information about the Landmark. Users can click the button in the top right corner to play the audio, and they can easily stop it by clicking the same button again. For the audio, I wrote out the speaking script and used a website to convert it into audio. It took me quite some time to find the desired audio, as most of them sounded too much like an AI robot speaking.

Fig. 5.2 - Convert text to audio

Fig. 5.3 - Create audio voice over in Unity

Then, I proceeded to continue working on the information pages for other landmarks. This process also took quite a bit of time as I dedicated efforts to researching information about each landmark. Summarizing the content was essential, emphasizing only the most important points for a concise presentation. Additionally, for the construction and history pages, I try my best in sourcing old images or images from the building process to provide users with a deeper understanding of the landmarks' information and background.

Fig. 5.4 - Design the info page in Figma

Fig. 5.5 - Continue work on others landmarks info page

Link to the file about contents of the landmarks:

https://docs.google.com/document/d/1yHoeXC_esu6Tnd-WGAFXDxl3Ko4hh2MLtCb0DLuGmu8/edit?usp=sharing

Fig. 5.6 - Contents of the Landmarks (PDF)

6. Refinement on the Homepage

Although I have completed the design and build of the homepage for the prototype, now that I have selected the 5 new landmarks, I need to replace the content with accurate information by changing the image and text.

Fig. 6.1 - Refinement of the homepage
(right: before, left: after)

App interface design in Figma file:
https://www.figma.com/file/OQqn5bayAaZgIxWrn6OoSA/Experiential-Design-App?type=design&node-id=0%3A1&mode=design&t=cQOT5bGBWKbAJHeO-1

Fig. 6.2 - Figma File

7. Arrange the scenes & Build to device

I arranged the files neatly, organizing the sources or materials for each landmark, including their scenes, images, or videos, in dedicated folders. This way, it is easier for me to locate specific files.

Fig. 7.1 - Assets file arrangement (scene)

After completing the building of all the scenes, I double-checked all the designs, ensuring that all scenes are linked through user-clickable buttons and that the arrangement is satisfactory. Next, I proceeded to the final step of checking that all scenes and arrangements are accurate in the build settings. Finally, I started to build the app on my device.

Fig. 7.1 - Build settings

FINAL SUBMISSION

AR World Explore Application

Google Drive Link:
https://drive.google.com/drive/folders/1d0lC6U52UXKKmVjziwu857wG7ZcCytHc?usp=drive_link

Video Walkthrough Link:

https://youtu.be/q-F1MSV01Qk?si=mq-gWD5RZS4VkDp1

Fig. 8.1 - Video walkthrough AR World Explore app

Fig. 8.2 - Cards with front and back design (Image Target)

Fig. 8.3 - Mock up of the AR World Explore app

Fig. 8.4 - Mock up of the cards

Fig. 8.5 - Mock up of the cards

FEEDBACK

23.11.2023- Week 13

Both using the world map and a card as an image target is okay, it depends on which one you want to choose.
For the video player, you need to uncheck the 'play on awake' option and set the target found and lost to play or stop the video.
For the info page, use horizontal and vertical scroll simultaneously. You can maintain the scrolling for the vertical scrolling, and for the horizontal scroll, you can use animation to move the entire section and set the buttons clickable to activate the animation.

REFLECTION

What a relief that I finally completed this entire project. I'm very proud of myself for creating an AR app from scratch. In the beginning, I felt stressed, keep questioning my ability to achieve the proposed outcome in the proposal. I can say that hard work paid off. I spent large amount of time watching multiple YouTube tutorials, experimenting, trying and making errors numerous times, and I'm very satisfied with the outcome. As a slow learner, I always need more time than others to familiarize myself with new things, and learning Unity software and using C# was completely new to me. Consequently, I had to invest more time than others, sacrificing my sleep, and sometimes spending nearly four days just solving one issue or bug. I truly appreciate Mr. Razif for helping me solve all the issues and providing suggestions that saved me a lot of time and made my work on the project more effective. Overall, it has been a very interesting journey of learning and being exposed to the area of study of creating AR apps. I have gained a lot of knowledge, not only in technical skills but I have also learned a lot of new knowledge when researching the landmark's history and construction process.

QUICK LINKS

Task 1: Trending Experience

Task 2: Project Proposal

Task 3: Project Prototype

Final Project: Completed Experience

TAN YI-TYNG (0353327)