Night Within Interactive Experience

research

The first stage of the project was to understand and research current interactive projects and experiences that explore ideas and questions about similar topics.
‍
By having prior knowledge with developing similar experiences, I was able to dig deeper into more advanced development practices, and further expand my understanding on more complex interactive prototypes.

Ideation

After the initial research, I spend a largest portion of my time on sketching, building, and testing interactive prototypes as well as complex mechanics, assets, and micro-interactions.

Most of these ideas were quick prototypes that allowed me to evaluate the experience and build on top of the feedbacks gathered from playtesting.

end result

Almost all ideas were validated or completely disproved during the early stages of testing. All final designs, interactions, and details were results of thoughtrough development and and constant usability testing.

During the development of the project, several unanticipated problems surfaced that required constant debugging and creative problem solving.

*IMPORTANT NOTE: This video and "play-trough" of the project represent a shortened and edited version of the experience. This in progress demo does not "fully" represents the experience.

"Do androids dream of electric sheep?"
novel by philip k. dick

From the beginning, the project was highly influenced and inspired by Philip K. Dick's Novel, and Ridley Scott's movie adaptation. As a Blade Runner fan, I was often looking for inspiration for the visuals, atmospheric and sound design . While there is no direct connection between the movie and story, there are some "easter eggs" hidden in the experience that hints at my fascination with the movie and novel.

*Player Pawn Blueprint — This blueprint controls the player and character.

*Tower Screen BP— This blueprint controls the display and UI on the tower elements.

fully interactive experience without a single line of code —Well, almost.

The entirety of the project was designed, developed, and built using Unreal Engine 4's built-in Blueprint System. While "visual scripting" had it's own difficulties and challenges, this system allowed me to create fully working prototypes without a single line of code—to be exact, without writing C++ scripts.

Building prorotypes with UE4's blueprint system was often quite a bit faster process than writing native C++ code, however, this is not always the case.

how does it work and what is the objective of the experience?

The base of the project is build around a simple experience and idea, failure. While failure is is not a particularly great experience, it is crucial to progress further. Every failure in the experience is a lesson and opportunity that allows the player to complete the demo. In this cold, desolate, and dangerous world, perhaps the biggest challenge is embracing failure.

limited reach and time available with every try—at firts.

The player is equipped with a "power source" that only allows the character to experience a limited time in this world before the energy is completely depleted. When that happens, the player needs to start from the beginning. It is impossible to compete the experience with a single try.

every failure will result in more possibilities.

When the player's energy source is completelty depleted, the system will place a "memory tower" (recharging station) at the location where the player "failed." Every time the player starts over, a memory tower will mark the location of the locations of previous tries, and guides the player for the rest of the experience.

taking advantage of failure is a core element of the experience.

When encountering an memory tower, the player is given the opportunity to recharge it's power source, which in turn allows the player to once again progress further in the experience. Every failure will allow the player to get closer to the ending moments of the experience.

*Project Playground — This image shows and example test level where all prototypes and new ideas were tried and tested.

Project progression and iterative prototyping—project playground.

With some prior experience in building such projects, it was crucial that prototypes were tested and built in a "grey box" manner prior to jumping into details and implementing more features (trying to avoid being a "feature creep"). There are several "Playground" levels and maps that's sole purpose was to play-test mechanics, functionality, and controls before it was added to the "final / next version" levels and maps.

deploying prototypes for gameplay, visual, and performance evaluation.

Since the project was designed for a handheld device and experience (iPad), it was crucial that each prototype was deployed and tested on the target platform before making any further development. Early on in the project it became a "blocker" issue that the project progressed too far before it was tested on the target device, which resulted in unexpected bugs and "unnatural" interactions.

To "combat" this, I made sure to build tools and progress the project sytsemically to allow for quick testing and meaningful (each prototype gave insights and progressed the project further) prototyping.

*Image on the left shows an earlier version, as the right shows a later version of the project.

*Tower Blueprint — This blueprint controls the tower component needed to be activated by the player throughout the experience.

*Tower Blueprint Breakdown— This graph shows the breakdown of components and elements within the tower blueprint.

DEVELOPING prototypes with efficiency and modularity in mind.

One of the goals of the project was to develop prototypes with efficiency and scalability in mind. Almost all assets and blueprints were broken down into small and modular components to make assets as reusable as possible. Since I had to build several prototypes for testing and research, my goal was to make sure that components are modular enough to be reused in other areas of the project, if needed. This allowed me to quickly test ideas by combining smaller components into something "new."

Every function, event, actor, shader, texture, and assets was designed to make sure that the project is easily scalable. It was essential to the project that blueprints (logic) running behind the front end interface and experience are just as "elegant" and thoughtful as the design and the art itself.

Level building and design—keeping everyithing simple.

Designing levels and environments was a fairly challenging part of the project, since it not only had to make sense for the gameplay and story, but work efficiently and be optimized for better performance. Assets and levels needed to be optimized for large environments as well as designed to be meaningful for the story itself. One of the main and introductory level of the game was kept fairly simple with assets and foliage, while trying to present an exciting and mysterious environment. Less was often more.

Simple solutions and rapid-prototyping.

Often I had very litmited time and resources to work on the porject, so I had to come up with simle and cheap solutions to complete the project. Assets and object were often first built in a "grey box" manner with little to no detail, and later on upgraded during the later development of the project.

One of many challenges was recording sound effects for the project. I recorded several foley sounds (breathing, cloths, objects, ground and rocks, etc) and as an extra, some adudio form Google Translate. I took these "raw" (unedited) sound effects and transformed them using Garage Band. This was a fast process to achieve interesting sound effects that added an important layer of immersion and experience to the project. Sound was ofter crucial to the immersive ness of the project and to achive a higher, more believable fidelity.

how to tell a story and create narrative?

Storytelling was a crucial, but challenging part of the design process. The project went through several iterations and changes to the story, as well as how to deliver an experience that the user can think about.

I was interested in exploring speculative storytelling ideas where I leave the person experiencing the project come up with their own story and theory based on the interactions and environment. There are no dialogues, textual storytelling or any form of explanation. The experience was built around visual storytelling, short animations, cutscenes, and world exploration. The player is faced with an experience that presents visual clues, but the definition of the story is left to the user to synthesize themselves.

*Character design in Blender — The entire player character was designed to fit the base skeleton exported from UE4.

*Character Imported in UE4 — Character imported, re-targeted, and fixed in UE4.

ASset design and development.

Almost all assets present in the experience is custom designed and built in Blender.

One of the more difficult challenge was designing a custom character and implementing it with UE4' mannequin skeleton. It took some trial and error to get the pipeline work properly, but once I was able to figure out efficient ways to import rigs and assets, it made further development and updates simpler and significantly faster.

EXploring micro-interactions and small details.

The project was mainly built around interactions and small experiences. There are a number of micro-interaction in the project that are very small details, but if it was missing, it would break the immersion. For example, camera movement and shake when walking and running, sound effects when the character is in motion, and audio and visual feedbacks when interacting with controls.

This part of the project was the longest and perhaps most time consuming. All micro-interactions required constant testing after every little change. A large portion of these test were often fine tuning the interaction and timing, to make sure that the experience feels high quality and enjoyable. It was important to make sure that whatever interaction happens in the project, the player feels like it meant to happen and feels natural.

*EXPLANATIONS: (1) Player approaching and interacting with the Master Terminal. (2) Player encountering a field mine that instantly depletes the power source. (3) Player encountering a memory tower that recharges the character's power reserve. (4) Player reached a section that can only be competed with placing a memory tower. (5) Player getting through the laser barriers. (6) Player reaching the activation tower and interacting with the control panel.

Sound effects, music, and level cinematics—can you hear it?

Sound effect and music was unexpectedly important part of the project. Audio features and effects where often half of the experience. Without proper sound feedback, interactions and experiences felt unnatural and less immersive. I spent a significant amount of time adding sound effects and audio to the experience to make sure that each play-through feels exciting and immersiv (and hopefully sounds good).

In-game cinematic and intros also became an important part of the project. It was evident early on that I needed some sort of storytelling and introduction to the game and objectives. I ended up creating and designing several short in-game cinematic that meant to further immerse the player and highlight objectives and player goals.

Building the illusion of smart AI for in-game ENEMY.

During the creation of the "Drone" actor, I had explored several different avenues for building a seemingly intelligent enemy. Using UE4's "AI Behavior Tree" seemed to be a great place to start at first, but I figured out ways to simplify that system using only Events and simple functions within the Actor Blueprints.

Since I was designing a floating & flying drone, it made it easier to create an actor, that did not necessarily need collision avoidance due to the design of the environment. I was able to built a simple system that randomly "picks" locations in a dynamic area, where the drone would continuously move to the updated locations in a fluent, but naturally random pace.

This created the sense of "patrolling," and created thension in the experience. The player has no way of knowing where the drone would move next. Unpredictability became an interesting element of the entire project.

*Drone Encounter — This demo shows the events following the detection of the player by an enemy drone patrolling the area. Each "hit" will lower the player's battery until is completely depleted.

importance of OPTIMIZATION and contsant debugging.

Designing and developing an interactive experience for tabled had it's challenges and difficulties. While iPad had great performance and allowed to build even more complex scenes and designs, it was important to make sure that everything is as light on performance as possible. It was important for the project to create a smooth and seamless experience with keeping the performance close to "60 FPS" (frame per second).

For example, at the beginning of the project, materials and shaders were built quite uneficciently, and resulted in very expensive shaders. This often created sudden "frame drops" when the player looked at a specific location in the level. This was a jarring experience, and pushed me to create more efficient shaders as well as functions, assets and blueprints.

invisible, yet crucial details BEHIND the scene.

There are several development practices as solution that makes this experience run smoothly (as much as possible). For example, one important development practice was to eliminate refreshing functions and events every frame ("Event Tick").

While this is a fast way to prototype functions and event, this could be a very expensive event that potentially lowers the frame-rate, and creates a choppy experience. It was crucial to build the project where actors, assets, and functions only update when it is necessary for them to do so—this in fact made the project a bit more difficult.

testing every mechanic, interaction, event, and experience.

Considering that this project was a fairly "complex" interactive experience, every little change and idea needed to be carefully tested, ofter numerous times. Every interaction and detail that made it into the experience was played and tested several time, both in the engine and editor as well as packaged into the iPad.

Although I was constantly building and testing the project on my own, I have worked and showed the experience to other players to collect feedback and understand their experience with the project.

During one of these testing, one player mentioned that the controls were a bit slow and hard to navigate. To address this feedback, I placed settings for touch controls and HUD elements where players could change the sensitivity and behaviors of the character.

*Testing employ and experience on iPad Pro and evaluating feedback for further iterations and updates.

*Widget Designer View— Interface design for "Settings" section.

*Graph View— This represents the base functionality for the "Settings" page using blueprint systems.

designing simple menus and in-game interfaces.

It was important to make sure that every little detail within the project was thoughtful and fully considered. I designed several versions of in-game interfaces, loading screens, and other UI elements.

Almost all menus and interfaces have parallax effect built in, that reacts to the orientation and tilting of the device. It was important to make the interface feel alive, responsive, and dynamic when held by the player.

BUIlding tools and ways to debug on the fly.

It became a very useful practice during development to build "developer" tools that allowed me to test and debug the experience while playing. This proven to be extremely useful since I was able to test different scenarios and reset parts of the game, if needed to be, without needing to restart or completly recompile the app.

Including several other tools, I built a simple "developer interface" that allowed me to modify and change aspects of the in-game objects and actors, so I can evaluate and understand where the project breaks and fails. (Without these tools, I would had to restart the project every time I wanted to make a change or test a scenario)

what went wrong during the development of the project?

One of the best, but most challenging part of the project was constant problem solving. Every little change and addition to the project created problems and complexity that was often unpredictable. Layering complexity often resulted in a sea of errors and unexpected bugs.

I had to develop ways in which I could build incrementally and make sure everything works well and efficiently together.

My expectations to approach a "large" project as such come to the realization that this is more work than one person can handle in a short amount of time. I had to sacrifice ideas, sections of the project, experiences, and mechanics due to complexity and time limit.