Process
At the start of the project, we tried to integrate a webcam to discern emotions and generate corresponding handwriting visuals. However, technical glitches during emotion transitions and the challenge of determining combined happiness and anger percentages led us to discard the webcam approach.
Opting instead for a projection, we meticulously refined visuals, introducing asemic writing and experimenting with mood-related visual elements. In parallel, we explored the integration of our collected handwriting audio into a soundtrack, progressing through calm, anxious, angry, and energetic phases.
Recognizing the potential for the soundtrack to overshadow the raw handwriting audio, we decided to focus on a final outcome - an immersive exhibition experience. This experience entails a 30-second edited projection accompanied by interactivity, allowing viewers to trigger different visuals by pressing buttons on a laptop, all complemented by the raw handwriting audio.
The immersive exhibition is further enriched by an accompanying publication, a study booklet elucidating our analysis of different handwriting moods and the determination of our visuals. The subjective interpretation of visuals and colours is augmented by universally recognized meanings associated with colours, coming in a comprehensive and engaging artistic exploration.
Proposal, background
In a digital era where handwriting wanes, our project delves into the interplay of human emotion and artistry, using handwritten text as a medium. Handwriting, a communication cornerstone, mirrors innate feelings from joy to sorrow. Our exploration blends scientific inquiry with creative expression, recognizing emotions' influence on handwriting—negative moods manifest in quicker, narrower writing. We collect handwritten samples spanning anger, sadness, and happiness, transforming raw expressions into visuals. Utilizing projection, our immersive exhibition aims to connect viewers with the visuals.
We also revive asemic writing, conveying emotional nuances without traditional context. Despite technical challenges, tutorials on p5.js and touchdesigner fueled our journey. Though the web camera idea was abandoned, our project evolved dynamically, transitioning from real-time data to generative aesthetics. This experimental journey, enhanced by numerous trials, led to a rich tapestry of emotion, even if some aspects didn't make the final cut.
Proposed objective
The primary aim of this study is to investigate the correlation between an individual's mood and their handwriting characteristics, encompassing structure and speed. Additionally, our objective is to integrate this analysis into creative coding, presenting it as an interactive and immersive exhibition experience. We aspire to visually convey these emotions through creative coding, providing a distinctive representation of an individual's emotional state. Recognizing the intricacies of constructing an AI model for emotion detection from handwriting, we've opted to collect auditory data from handwriting and experiment with p5.js for a more accessible approach.
This choice arises from the absence of existing prototypes and the technical challenges associated with AI-based emotion recognition from handwriting inputs. Consequently, our study charts a course for novel insights at the intersection of emotion, handwriting, and creative coding.
Proposed approach
This study aims to delve into the intriguing relationship between a writer's emotional state and the structure and speed of their handwriting. To accomplish this, we have incorporated simulations that elicit specific emotions, namely anger and calm, in the individuals tasked with writing the text. By manipulating these emotional states in a controlled environment, we intend to capture and analyse the variations in handwriting style and writing speed that manifest as a direct response to these emotions.
Furthermore, this investigation also encompasses an auditory dimension as we record the sound produced during the act of writing which will provide valuable insights into the physicality of the writing process. These steps will aid us in creating visuals according to the moods, which will be displayed later on during our projection in the exhibition-like space.
Data collected
We initiated our study by inviting individuals to freely jot down anything on a piece of paper. Subsequently, we conducted a detailed analysis of their moods based on the strokes, spacing, and consistency present in their handwriting. The specific choice of words held no significance, as our focus was on the intrinsic qualities of their handwriting.
In our experimental exploration, we intentionally created mood-specific environments to investigate the correlation between individuals' emotions and their handwriting. During the collection phase, we gathered visual and auditory samples, including distinct pen and iPad sounds. Utilizing three mediums—pen and paper, marker and paper, iPad and Apple Pencil—we aimed to detect variations in sound and consistency. However, we narrowed our focus to pen and paper recordings due to their superior accuracy in portraying emotional nuances, while the Apple Pencil, producing a bass-like sound, proved less effective in identifying specific emotions in handwriting.
This revelation served as inspiration for us to convey our visuals using an abstract handwriting format known as Asemic writing. Asemic writing proved particularly valuable in instances where individuals struggled to articulate their feelings verbally, as it transcends language, presenting itself as expressive scribbles. The creation of Asemic writing visuals was informed by the various handwriting factors we considered during the analysis of individuals' emotions. For example, the handwriting of a person experiencing anger exhibited sharp, chaotic, and somewhat inconsistent traits, leading us to craft Asemic writing for anger characterized by thin strokes, chaos, and pointed ends.
Click here to view the data we collected
Artefact
The project's scope seemed expansive, prompting feedback to integrate the visual video into a larger concept. This led us to explore projection mapping on three surfaces, ultimately favoring a 45-degree bent port with two oversized paper backdrops, reminiscent of handwriting on paper.
Our exhibition space aimed to portray four distinct emotions: Angry, Calm, Energetic, and Anxious. Anger was expressed through a red, black, and white palette, while calm featured pastel colors and slow-moving wave asemic writing. Anxiety was depicted through restless patterns and intermittent striped spheres. For energy, we visualized increasing shapes and elements in both visuals and asemic writing.
Opting for larger surfaces aimed to provide an immersive experience, enveloping viewers in the visuals to experience the emotions firsthand. We complemented this with a publication offering insights into the concept behind the space, providing a deeper understanding for viewers.
"Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep." - Scott Adams
Conclusion
Throughout 'Inkscapes,' our exploration expanded from handwriting analysis to experimenting with p5.js and TouchDesigner visuals, resulting in a publication and exhibition projection. Initially targeting real-time emotion detection, technical challenges led to excluding the webcam, transitioning to 'InkScapes - Generative Aesthetics.'
Our project faced some confusion in the transition from collected handwriting data to asemic characters, prompting the question: "Why should the audience care?" In addressing this query, we refined our narrative approach - Departing from traditional writing, our project embraces asemic writing as a responsive medium that interprets writing nuances and intensity, providing a space to express emotions.
Post-feedback, we acknowledge opportunities for improvement, suggesting dissecting strokes for asemic characters and implementing more analytical distinctions in our visuals.