MoCap Studio A1
Individual coursework for Motion Capture Studio.
Abstract visual experimentation with mocap animation.
Instructor: Karen Kriss
⇲ Artist Research I
“There is a lot of invisible information in movement… unveiling these forces and render them into digital visuals beyond physical restrictions requires to identify core aspects, extract and amplify them and avoid elements that are already sufficiently represented in reality.”
As Tobias Gremmler explained in an interview, he articulated his philosophy of extracting the essence of movement and expressing it through digital forms that go beyond the limitations of physical representation. His work often explores not just the human body, but also the motion inherent in subtler processes and natural phenomena.
One example is his work Visualizations of Drawing Chinese Calligraphy, where he deconstructs and reinterprets the process of writing Chinese characters. Through a variety of dynamic visual techniques—including particles, fluid dynamics, flowing ribbons, and trails—he reveals the rhythm, pressure, and directionality of each stroke.
Rather than simply simulating reality, the visuals amplify the intangible qualities of writing, transforming the act into a flowing, immersive performance. The result is both aesthetically captivating and conceptually rich, offering a new way to perceive motion not just as a sequence of positions, but as an expressive force in itself.
Gremmler’s work informs my own exploration by demonstrating how abstract visualization can preserve and even enhance the essence of a performed action. By selecting different visual elements and techniques, it becomes possible to extract and highlight various aspects of motion—whether its contour, dynamic, energy, or sense of expansion in time and space.
⇲ Artist Research II
Janusz Jurek has developed a unique illustration technique that reflects a strong personal style and falls within the category of generative art—not based on machine learning models, but rather on procedural programming. His method relies on new technologies and algorithmic generation, yet distinguishes itself from mainstream computer-generated art by incorporating strong elements of traditional drawing.
The creative process begins with writing custom code that enables the computer to generate complex spatial forms. Then, using his skills in drawing and sculpting, Jurek reshapes these digital structures into forms inspired by the human body. The result is a striking form of abstract visual expression. His images often outline human contours while simultaneously evoking a sense of raw vitality and explosive energy. They possess both a clear sense of direction and dynamic flow, making the visuals feel alive and in motion.
Although Jurek’s works are presented as static 2D illustrations, I believe his approach could be effectively extended into time involved, dynamic 3D visualization media. The underlying forms and energies he captures lend themselves naturally to motion, making them a strong conceptual and aesthetic reference for motion capture-based experimental animation.
⇲ Artist Research III
In his VR animation INK TRAILS, Goro Fujita creates a childish and charming world, composed entirely of continuously flowing and morphing hand-drawn lines. The animation tells its story through motion itself, as the linework never rests—constantly evolving to reflect emotion, rhythm, and change.
Fujita’s artistic style is marked by a playful tension between simplicity and complexity: behind the clean silhouettes lies a layer of energetic chaos; beneath the controlled character animation, there is a subtle sense of randomness and unpredictability. This layered motion adds inner life to the characters and their environment, making the entire scene feel organic, breathing, and ever-evolving. Created using Quill, a VR illustration and animation tool, his work demonstrates how abstract visual elements—such as line, flow, and transformation—can serve not only as background design, but as an active narrative force.
⇲ Artist Research IV
"磨太太的w" is an artist who specializes in stylized rendering using Blender. In the post BLENDER Stylization Exercise, a walking human figure is rendered in a striking visual style. It is composed of high-contrast black-and-white visuals, with sharp lines that almost resemble industrial design sketches. These straight, sketch-like lines continuously trace the contours of the moving figure, creating a sense of both structure and instability. The addition of glitch effects further enhances the aesthetic, giving the animation a raw, edgy atmosphere. It’s visually bold and full of character.
These two artists’ work inspires me to consider how continuous motion and procedural visual changes can be used in a motion capture project—not merely to simulate physical movement, but to express internal change, emotion, and narrative through evolving abstract forms.
⇲ Experiment #1
Perhaps influenced by previous student work, but also driven by my own interests, I decided to explore a domain where motion capture technology is often applied—games. Rather than creating a purely cinematic or non-interactive animation, I chose to approach this project as a “mini game demo”, an interactive experience that integrates motion capture in a playable form.
With that in mind, I set Unreal Engine as the core platform for developing this motion capture project. This direction naturally came with a range of challenges.
– First, since the project needed to run in real time, the visual quality could not compete with that of effects rendered over several hours. Instead, everything had to be optimized for performance.
– Second, Unreal Engine has limitations when it comes to highly stylized rendering, especially compared to other DCC tools like Blender, which offer more flexibility in non-photorealistic shading and abstract visual effects.
Achieving the kind of expressive, hand-drawn or glitch-inspired aesthetics I envisioned required additional experimentation and workarounds within the constraints.
⇲ Motion Data
To start with, I needed to find suitable captured motion data. Among the many great resources provided by our instructor, I stumbled upon a set of “drunk” walk and idle animations on Mixamo. When slowed down a bit, these motions took on a different tone—appearing more like lost, aimless stumbling, filled with a sense of emotional disorientation.
For the character model and skeleton, I chose a simple human base mesh from a Blender demo file, as I planned to apply stylized treatments later. I used the Auto-Rig Pro add-on in Blender to rig the mesh and exported it with a UE-compatible skeleton. Once imported into UE, the downloaded animation sequences was retargeted to this skeletal mesh, allowing me to use the Mixamo “drunk” walk and idle motions directly on the base mesh character.
⇲ Character Control
To enable control of the character, the UE mannequin’s Character Blueprint was repurposed. The transition between the idle, walk, and run animations was handled through a simple State Machine and Blendspace set up within the Animation Blueprint.
⇲ 'Pencil Shader'
For the stylization process, the goal was to create wild, energetic lines that would protrude from the model’s surface, appearing dynamically both on the surface and around its silhouette. Initially, instancing-based methods were considered—an approach I had experimented with in a previous project, using purely material tricks, textures, and Blueprints to render multiple copies of the same mesh. However, that method proved to be quite complex and prone to both visual and technical issues.
Instead, attention was shifted toward post-processing as a more efficient solution.
↳ same logic partially implemented in UE
Though I’m not a programmer, with AI helping me rewrite the code in HLSL, I managed to recreate the outline portion of this shader in Unreal Engine within a custom post-process material.
↳ main material
In the material for the character, a hand-drawn texture I created was used as an opacity mask, combined with a Panner node to generate motion across the character’s surface (somewhat similar Goro Fujita’s style). With this addition, the character’s stylization was finalized, achieving a dynamic mix of structure, movement, and chaotic energy.
↳ drunk walk in UE
⇲ Trail with Fluid Simulation
Although Unreal Engine doesn’t offer something as convenient as MASH in Maya, I still wanted to add an extra layer of motion trails to enhance the character’s movement. However, simple trail effects didn’t feel expressive enough, so I decided to incorporate fluid simulation into the system.
Thanks to FluidNinja LIVE, a fast and responsive fluid simulation plugin developed for UE, I was able to track multiple bones on the character’s skeleton and apply an interactive, fluid-simulated motion trails to them. But for this part of the experiment, I intend to further refine it.
↳ with fluid sim
⇲ Facial Motion Capture
In addition to the above, I also conducted some basic facial motion capture experiments. I created a simple MetaHuman character using presets, and used the Live Link Face app to capture a short facial performance, which I looped within my scene. This experiment was mainly to test the full workflow and prepare for future practices.
⇲ Experiment #2~5
Experiment #2~5 continued to explore abstract visualizations of the same “drunk walk” motion from Mixamo. All of these were created in Blender, using a combination of Geometry Nodes, custom materials, and some add-ons.
⇢The End
⇢Tom's Journal
⇢Jun 2025
MoCap Studio A1
Individual coursework for Motion Capture Studio.
Abstract visual experimentation with mocap animation.
Instructor: Karen Kriss
⇲ Artist Research I
“There is a lot of invisible information in movement… unveiling these forces and render them into digital visuals beyond physical restrictions requires to identify core aspects, extract and amplify them and avoid elements that are already sufficiently represented in reality.”
As Tobias Gremmler explained in an interview, he articulated his philosophy of extracting the essence of movement and expressing it through digital forms that go beyond the limitations of physical representation. His work often explores not just the human body, but also the motion inherent in subtler processes and natural phenomena.
One example is his work Visualizations of Drawing Chinese Calligraphy, where he deconstructs and reinterprets the process of writing Chinese characters. Through a variety of dynamic visual techniques—including particles, fluid dynamics, flowing ribbons, and trails—he reveals the rhythm, pressure, and directionality of each stroke.
Rather than simply simulating reality, the visuals amplify the intangible qualities of writing, transforming the act into a flowing, immersive performance. The result is both aesthetically captivating and conceptually rich, offering a new way to perceive motion not just as a sequence of positions, but as an expressive force in itself.
Gremmler’s work informs my own exploration by demonstrating how abstract visualization can preserve and even enhance the essence of a performed action. By selecting different visual elements and techniques, it becomes possible to extract and highlight various aspects of motion—whether its contour, dynamic, energy, or sense of expansion in time and space.
⇲ Artist Research II
Janusz Jurek has developed a unique illustration technique that reflects a strong personal style and falls within the category of generative art—not based on machine learning models, but rather on procedural programming. His method relies on new technologies and algorithmic generation, yet distinguishes itself from mainstream computer-generated art by incorporating strong elements of traditional drawing.
The creative process begins with writing custom code that enables the computer to generate complex spatial forms. Then, using his skills in drawing and sculpting, Jurek reshapes these digital structures into forms inspired by the human body. The result is a striking form of abstract visual expression. His images often outline human contours while simultaneously evoking a sense of raw vitality and explosive energy. They possess both a clear sense of direction and dynamic flow, making the visuals feel alive and in motion.
Although Jurek’s works are presented as static 2D illustrations, I believe his approach could be effectively extended into time involved, dynamic 3D visualization media. The underlying forms and energies he captures lend themselves naturally to motion, making them a strong conceptual and aesthetic reference for motion capture-based experimental animation.
⇲ Artist Research III
In his VR animation INK TRAILS, Goro Fujita creates a childish and charming world, composed entirely of continuously flowing and morphing hand-drawn lines. The animation tells its story through motion itself, as the linework never rests—constantly evolving to reflect emotion, rhythm, and change.
Fujita’s artistic style is marked by a playful tension between simplicity and complexity: behind the clean silhouettes lies a layer of energetic chaos; beneath the controlled character animation, there is a subtle sense of randomness and unpredictability. This layered motion adds inner life to the characters and their environment, making the entire scene feel organic, breathing, and ever-evolving. Created using Quill, a VR illustration and animation tool, his work demonstrates how abstract visual elements—such as line, flow, and transformation—can serve not only as background design, but as an active narrative force.
⇲ Artist Research IV
"磨太太的w" is an artist who specializes in stylized rendering using Blender. In the post BLENDER Stylization Exercise, a walking human figure is rendered in a striking visual style. It is composed of high-contrast black-and-white visuals, with sharp lines that almost resemble industrial design sketches. These straight, sketch-like lines continuously trace the contours of the moving figure, creating a sense of both structure and instability. The addition of glitch effects further enhances the aesthetic, giving the animation a raw, edgy atmosphere. It’s visually bold and full of character.
These two artists’ work inspires me to consider how continuous motion and procedural visual changes can be used in a motion capture project—not merely to simulate physical movement, but to express internal change, emotion, and narrative through evolving abstract forms.
⇲ Experiment #1
Perhaps influenced by previous student work, but also driven by my own interests, I decided to explore a domain where motion capture technology is often applied—games. Rather than creating a purely cinematic or non-interactive animation, I chose to approach this project as a “mini game demo”, an interactive experience that integrates motion capture in a playable form.
With that in mind, I set Unreal Engine as the core platform for developing this motion capture project. This direction naturally came with a range of challenges.
– First, since the project needed to run in real time, the visual quality could not compete with that of effects rendered over several hours. Instead, everything had to be optimized for performance.
– Second, Unreal Engine has limitations when it comes to highly stylized rendering, especially compared to other DCC tools like Blender, which offer more flexibility in non-photorealistic shading and abstract visual effects.
Achieving the kind of expressive, hand-drawn or glitch-inspired aesthetics I envisioned required additional experimentation and workarounds within the constraints.
⇲ Motion Data
To start with, I needed to find suitable captured motion data. Among the many great resources provided by our instructor, I stumbled upon a set of “drunk” walk and idle animations on Mixamo. When slowed down a bit, these motions took on a different tone—appearing more like lost, aimless stumbling, filled with a sense of emotional disorientation.
For the character model and skeleton, I chose a simple human base mesh from a Blender demo file, as I planned to apply stylized treatments later. I used the Auto-Rig Pro add-on in Blender to rig the mesh and exported it with a UE-compatible skeleton. Once imported into UE, the downloaded animation sequences was retargeted to this skeletal mesh, allowing me to use the Mixamo “drunk” walk and idle motions directly on the base mesh character.
⇲ Character Control
To enable control of the character, the UE mannequin’s Character Blueprint was repurposed. The transition between the idle, walk, and run animations was handled through a simple State Machine and Blendspace set up within the Animation Blueprint.
⇲ 'Pencil Shader'
For the stylization process, the goal was to create wild, energetic lines that would protrude from the model’s surface, appearing dynamically both on the surface and around its silhouette. Initially, instancing-based methods were considered—an approach I had experimented with in a previous project, using purely material tricks, textures, and Blueprints to render multiple copies of the same mesh. However, that method proved to be quite complex and prone to both visual and technical issues.
Instead, attention was shifted toward post-processing as a more efficient solution.
↳ same logic partially implemented in UE
Though I’m not a programmer, with AI helping me rewrite the code in HLSL, I managed to recreate the outline portion of this shader in Unreal Engine within a custom post-process material.
↳ main material
In the material for the character, a hand-drawn texture I created was used as an opacity mask, combined with a Panner node to generate motion across the character’s surface (somewhat similar Goro Fujita’s style). With this addition, the character’s stylization was finalized, achieving a dynamic mix of structure, movement, and chaotic energy.
↳ drunk walk in UE
⇲ Trail with Fluid Simulation
Although Unreal Engine doesn’t offer something as convenient as MASH in Maya, I still wanted to add an extra layer of motion trails to enhance the character’s movement. However, simple trail effects didn’t feel expressive enough, so I decided to incorporate fluid simulation into the system.
Thanks to FluidNinja LIVE, a fast and responsive fluid simulation plugin developed for UE, I was able to track multiple bones on the character’s skeleton and apply an interactive, fluid-simulated motion trails to them. But for this part of the experiment, I intend to further refine it.
↳ with fluid sim
⇲ Facial Motion Capture
In addition to the above, I also conducted some basic facial motion capture experiments. I created a simple MetaHuman character using presets, and used the Live Link Face app to capture a short facial performance, which I looped within my scene. This experiment was mainly to test the full workflow and prepare for future practices.
⇲ Experiment #2~5
Experiment #2~5 continued to explore abstract visualizations of the same “drunk walk” motion from Mixamo. All of these were created in Blender, using a combination of Geometry Nodes, custom materials, and some add-ons.
⇢The End
⇢Tom's Journal
⇢Jun 2025
⇢swipe or scroll⇢