AI Collaborator

working with artificial intelligence and machine learning to write and create stories.

Project Overview

Script-r is an AI collaborator tool designed to help with the creation and generation of stories and screenplays. Our goal was to design new ways for crafting a story using AI as collaborator. We want to inspire writers and artists to tell stories in collaboration with an AI.
‍
This project was developed and built in collaboration withDavis Brown.

WHERE IS THE AI?

The AI is used as another "writer," and collaborator who is giving ideas on how a story could turn out. Using AI and machine learning, writers could generate an "infinite" version of a story through quick iterations. The more one uses this tool, the better it gets at coming up with ideas for a story.

For our demos and work, we used third party applications and software to generates text documents and stories based on the given parameters.

TOOLS USED

The project was mostly developed and built using Sketch, Photoshop, Keynote, Runway ML, and Protopie.

My contributions

My contribution was project research, building interface designs, interactive prototypes, and interaction & experience design. My main section was creating and developing the graph editor and app UI.

Research

A significant portion of the project was spent with researching the ArtCenter archives and other libraries. It was key importance that we fully understood how the archival system and user experience works in other institutions and libraries.

We visited libraries and were able to first hand witness the process of the archiving process and contents.

IDEATION

After the initial research, we spent several week with ideation on user experience and building several functioning prototypes. We were looking for startegies whihc could help to connect the activist with the users.

We were exploring ways that could make the discovery of archival contents simple and exciting.

Findings

One of the greatest finding and realization was the amount of contents archives in general hold. At ArtCenter, we were surprised the about the quality of the meticulously documented and persevered contents.

However, none of these contents were easy to find and access.

I have a

n idea...

Ai powered tool to help generate innocative ideas and exciting stroies.

Script-r is an AI collaborator app designed to quickly ideate short stories and screenplays. Our goal is to design new ways of crafting a story using AI as collaborator. We want to inspire writers and artists to tell stories in collaboration with an AI.

HOW DOES script-r WORK?

Swcipt-r was designed to help with generating ideas and stories through a fast and iterative process. The app uses a dataset of scripts, books, and literature to help generate content. These datasets could be uploaded by the user as specific examples, or by using the app, it will build it's own dataset based on the ideas that the user liked.

Script-r is not a tool designed to spend hours and months on, but to generate new ideas in an instant. The AI is another writer in the room that could help with generating interesting and unique stories.

create a story through simple ideas, and let the ai connect the dots.

Script-r was designed to be a quick and fun way to create ideas for a story. To crate this, we took inspirations form visual node editors. Nodes could be defined with simple ideas by the writer / user, and with the help of AI form these ideas into a sort story. This makes the process of generating and iterating between ideas fast and exciting.

build a story through nodes, or with visual inspirations.

The second part of the tool was to create a way in which story creators could visually define their ideas, and let the AI figure out how to create a story around it. Besides node based story editor, this was another powerful way to define a story, or just design the visual feel and experience. We used ML tool to create text content based on imigasen in order to reserach and create this feature.

Better at writing the story? the AI can help with that, and more.

We wanted to make sure to leave a way in which creatures can simply write out their ideas, and let the app generate the appropriate nodes for it. This made the app backward compatible, meaning that users could edit the story in every which way they like (node editor, visual editor, and text editor), and let the AI update the story and final output.

define NODEs with simple ideas, the AI will make the most of it.

Simplicity starts with creating node consist of small ideas, sentences, or even just a few words. These definitions give enough insight and starting point for the AI to generate a paragraph, section, or a short "story." Creator could focus on big and small ideas, and let the app to try to figure out the rest.

STYLING NODE HELPS VISUALly differentiating BETWEEN IDEAS.

Node editors and visual graphs could become very dense and confusing, so we wanted to make sure to add styling options such as colors and object size to highlight important nodes and ideas. This makes the editor easy to understand and visually differentiate between ideas for characters, places, plots and so on.

adjust stories by playing with the interpolation intensity.

We spent a significant amount of time figuring out new ways in wish creators could visually adjust and modify the story. One of our directions was to use "story arcs" as a new way of modifying the story and the generated output. The interpolation tool is a simple and visual way that helps creators adjust the section, or the entire story based on the intensity editor. Playing round with the story arc will immediately change and modify the story to fit the tone and appropriate outcome.

single button to iterate a section, or the entire story.

At the heart of the app and experience, there is a single button designed to help generate and iterate on ideas. The iterate button is where the user can intentionally activate AI and machine learning to create a story based on established deas. A single button made iteration and engaging with the app simple and exciting.

fully control over what to iterate on.

Iterating a story could be a confusing and complicating process, and it is possible to lose some of the ideas that the AI comes up with. We designed a simple away where users could select specific sections of the story to savee, lock, or specifically iterate on. By using the simple text selection that people are naturally used to, we came up with simple features and tools to help lock and unlock parts of the story to prevent further iterations—maybe the AI came up with something awesome.

SIMPLISTIC interface and tools designeD for easE of use.

Script-r was designed to be simple and userfriendly tool that lets creators and writers to focus on story creation, and not get lost in the mists of complicated tools and interfaces. We spent a long time researching and testing what are the most important and necessary tools we need available to make this experience fun and seamless. We ended up with a simple side tool interface that consists of every tool and feature one would need to start creating stories both large and small.

light mode is great, but you have to have dark mode.

We wanted to make sure Script-r is a handy tool and app that allows users and creators to visually modify the interface. One of these features was to create a "dark more" to help with the reduction of eye-strain when looking at it for a long periods of time. Also, dark mode is just cool.

What type of datasets does script-r uses?

Script-r was designed based on the direction that it could use available and published scripts, books, and other forms of written documents as a base dataset. It could learn the vocabulary of literature and writings, and build out a base dataset that users could build on top of.

Our research prototypes and systems used GPT-2 large transformer-based natural language processing model which is trained on websites and information gathered on web contents.

Disclaimer: These images are example scripts and only used for educational research. These scripts belong to their rightful owners.

how does the system works and helps generating stories?

The app is built around the system of using user's "vocabulary" data to feed the AI with.
Through usage, the AI will acquire more vocabulary and able to further refine ideas and language. This dataset could be pre-existing and app default, or user could decides to upload and add to the project. The added content could be words, phrases, sentences and stories, paragraphs or images. Using this information in combination with the base dataset of scripts and books, the AI will continuously try to come up with new ideas and stories. These new ideas then could be added to the base vocabulary to further enhance and refine the too.

This way the AI will get more and more refined as the user iterates on the story. The finished story could be exported as a text document or a visual imagery based storyboard.

Flow of how the system works from the front-end interface to the output data.

Researching machine learning and available tools.

Throughout our research, we spent a significant amount of time with extending our understanding of machine learning and artificial intelligence. We explored several different tools and available softwares, some of which were just perfect for this project. Two of the main third-party applications and tools we used was "Talk to Transformer," and "RunwayML." We used these tools to understand how do they achieve text generation from a certain dataset. We also took adventage of these tools to generate our own stories for Script-r.

These tools proven to be a great examples as proof of concept where the type of machine learning and functionality potentially could be built into Script-r, and make it function as such.

Tools used for the project: Talk to Transformer website on the left, RunwayML on the right. These images are examples only for educational purpose.

researching example projects and asking questions.

During the early research phase of the project, we wanted to find example projects and other works that uses machine leaning and AI as a proof of concept. We found several exciting projects and examples that uses AI and similar technologies to generate stories and full scripts.

Benjamin the AI

Open AI

Animations and Screenplays by Disney

—
How does the role of the writer change when scrips and stories are generated by a trained AI?

—
What are new ways in which we can craft a story while retaining some traditional elements?

—
Who will have the rights to a story when it’s generated by an AI?

—
Can an AI keep the essence of the story, but adjust the tone based on the story arc?

—
Can a story be crafted where the writer is just curating AI content?

—
Can an AI just simply run out of new and unique ideas?

NON-LINEAR STORYTELLING
—
Remove constraints of start and end to the story. This will break the creation process out of a linear flow and enable multiple storylines to be told in sequence.

RAPID AND QUICK IDEATION
‍—
Design a system that allows for quick ideation and iterations with the help of the AI. This help the writer spur creative thinking and new ideas previously untold.

INDUSTRY STANDARDS‍
‍—
The app should be able to export stories and scripts to other industry standard applications and tools like Final Draft with correct script syntax.

how to use story arcs as a form of interface to modify the story?

Early on into the project we got really interested in how story arcs define and help visualize stories. We looked up several different examples and "templates" of different story arcs, and came up with the idea to turn these into a form of interface that would allow users to completely modify the story with the help of the AI through modifying the story arc.

This idea gave birth to the Interpolation Feature and interface that helps fine tune and refine the story though a visual graph.

Tools used for the project: Talk to Transformer website on the left, RunwayML on the right. These images are examples only for educational purpose.

Where does script-r fit into the story creation process?

We spent a significant amount of time discussing and researching where would Script-r fit into the screen and story writing pipeline. We found several professional tools such as "Final Draft 11" that convinced us that we should create a tool that capitalizes on the simplicity and quickness of creating a story idea.

As such, we designed Scriptr to be a tool addition to the story writing pipeline. This drove the design decision to make it into an iPad app where writers could quickly generate story ideas, and then take the ideas into a full desktop application that would help further develop and finish it.

Tools used for the project: Talk to Transformer website on the left, RunwayML on the right. These images are examples only for educational purpose.

Early project design, research, and insights.

During the course of the project, we developed several different approaches and designs that we contantly tried to gather feedback on. Through this process we arrived with out final solutions to develop three parts to the project: Editor, Board, and Script page. Each has its own unique features and functionality in developing the story. We gathered and tried to synthesize all the feedbacks into key insights.

Our final design is built around the idea of collaboration with an AI. There are several design solutions that are facked with functionality. One of these solutions is the coloring and theme of the interface. Every button and interface element is dark colored on a white base, everything except buttons that and interactions that represents the presence of the AI. Every interface element that is presented in blue color, is an interaction with the AI.

Another important, but crucial design decision was the font types for the story and for the interface. Every UI element that are not directly related to the story uses the app and stystem default font, whereas every story related content is represented in Courier New. This makes it very distinguishable form other elements within the interface and blends into more industry standard script formats.

Prototyping interactions and user journey.

We created a number of small and simple prototypes that captures not only some of the interactions in the app, but showcases the use case and user journey.

Demo video below shows an example of one of these prototypes that goes through the stops of creating a short story.

what were the main challanges and learning outcome?

One of the most challenging part of the project was figuring out interesting ways in which a story can be crafted. We really wanted to take a none-linear approach as well as trying to find interesting interactive and visual ways to modify a story. This took a lot of prototyping and ideation. Once we had an idea, we had to find ways to validate it, so it became crucial to talk to people who are experienced in this field. We interviewed some film makers and scrip writer to try to understand how can we make this experience and process better.

Another challenge was working with tools and understanding the principals behind ML and how an AI would function. It took a lot of research and prototyping with ML tools to find the right directions and to even validate our ideas. We needed solid proof of concepts to showcase that this idea has potential. Working withs such tools and researching such technologies was the most exciting and rewarding learning experience of this project.

Developing the interface was a difficult, but exciting challenge. We wanted to make sure to include tools that are crucial for a astray writer, but do it in a way that it's new and simple. Since we wanted to make this app to be an addition to already exciting tools, this gave us some freedom to cut down the tools to only the essentials. This took some prototyping and designing before we were able to finalize the final sets of tools and features.