In my practical research, I will use Meshyai as my 3D AI generator to explore how an AI-generated 3D model can fit into a traditional animation workflow.
I created the following 3D creature using ZBrush, which I will use as concept art for Meshy AI. I expect to achieve similar features from Meshy AI.
In Meshyai, you can recreate 3D models through various methods, including Text to 3D, Image to 3D, AI Texturing, and Text to Voxel.
Step 01 – Text to 3D
I will modify the existing text prompt from Meshyai to fit my 3D model requirements.
Meshy AI promt as follows
Design a humanoid monster with a twisted, muscular frame, standing tall at 8 feet, covered in jagged, stone-like skin. Its face is a blend of human and beast, with hollow, glowing eyes and sharp, fang-like teeth. The arms are unnaturally long, ending in razor-sharp claws, and its spine protrudes, giving it a hunched, menacing posture. The creature exudes dark energy, with veins of glowing red running through its skin. Ultra detailed, best 8k texture, perfect mesh, #animation#
https://www.meshy.ai/discover?search=monster
Customised prompt
Design a 4 legs 2 hands humanoid monster with a twisted, muscular frame, standing tall at 8 feet, covered in jagged, stone-like skin. Its face is a blend of human and beast, with hollow, red eyes. The arms are unnaturally long, ending in razor-sharp claws, and its spine protrudes, giving it a hunched, menacing posture. The creature exudes dark energy, with veins of glowing red running through its skin. Ultra detailed, best 8k texture, perfect mesh, #animation#
1- Add your prompt to the prompt area. 2- Click generate button
I customized the prompt several times and ended with the following result.
Final promt –
Design a 6 legs monster with a twisted, muscular frame, standing tall at 8 feet, covered in jagged, stone-like skin. Its face is a blend of human and beast, with hollow, red eyes. The arms are unnaturally long, ending in razor-sharp claws, and its spine protrudes, giving it a hunched, menacing posture. The creature exudes dark energy, with veins of glowing red running through its skin. Ultra detailed, best 8k texture, perfect mesh, #animation#
Final textured 3D model as follows
Step 02 – Image to 3D
I uploaded my concept 3D model to Meshy and generated it. I received the following result.
Step 03 – AI Texturing
In this method, you can upload an OBJ file with a maximum file size of 50 MB.
In the prompt section, I included the following text:
skin covered in jagged, stone-like skin. Its face is a blend of human and beast, with hollow, red eyes. The arms are unnaturally long, ending in razor-sharp claws, and its spine protrudes, giving it a hunched, menacing posture. The creature exudes dark energy, with veins of glowing red running through its skin. Ultra detailed, best 8k texture, perfect mesh, #animation#
The results are as follows:
AI texturing based on concept art was not available at this time. ( 11/11/2024)
I used deep ai to create a concept art of festive rabbit
https://deepai.org/machine-learning-model/3d-cartoon-generator
I will use the above AI-generated art as a concept and ask Meshy AI to develop a similar 3D model.
Method 01 – First I will use text to 3D model in meshy AI
I am using the following text prompt to create my character
The object is a cartoonish, symmetrical, soft-textured bunny with large ears, a fluffy tail, and a striped red scarf, standing upright with open paws
Here is what Meshy AI generated according to the prompted text.
Method 02 – I will use Image to 3D as my next step
Here is what Meshy AI produced based on the image-to-3D option.
Meshyai has a built-in AI-generated texture editing tool, and at the time (15th Nov. 2024) I was doing this research, the feature was not available.
Animating the created 3D model by using meshy AI
Meshy AI has a feature that animates our models, bringing them to life. I am going to explore this AI capability.
Link to my final animated festive bunny – https://www.meshy.ai/3d-models/Bunny-Cheer-01936230-fed7-704f-88ca-de942380fa29?animation=true&fbclid=IwY2xjawGxBl5leHRuA2FlbQIxMAABHYFODgcvlgAcDWRIiOSaC9ku6lTo9aJHhrn7ByOyzkYJCo8vbZbbtNpRNQ_aem_vA1VeKCYswi1UmSnKXNflw
Meshy AI exports the FBX file along with the animation. I successfully imported that file into MAYA and made modifications.
Maya Playblast file as follows
In my research, I found a community influencer, Jon Draper ( https://www.youtube.com/watch?v=7aYy9H7jQuQ / https://aianimation.com/ ), who shows how he used meshy AI for his animation pipeline.
He used Meshyai to generate 3D models, textures, and animation. After creating the models, he exported his 3D character from Meshy as FBX and imported it into Blender. He combined walking and running animations in a Blender and exported them as animated “glb” or “gltf” file formats. Next, he imported the “glb” file into Adobe After Effects, did the final composition, and exported it as an mp4 video. He also demonstrated how to add facial animation to the character by using another AI tool called LivePortrait: https://github.com/KwaiVGI/LivePortrait. He replaced his reference voice with a different one using Eleven Labs (AI Voiceover generation): https://elevenlabs.io/. He upscaled his final video file using Krea (AI video enhancer/upscale) https://www.krea.ai.
The following video will explain how you animate AI-generated 3D modal in an Unreal Engine.
AI 3D Models – Improve The Details – Tutorial