Introduction
The following StableDazFusion process has been written as a procedure that breaks down into several stages.
The aim of the StableDazFusion process (SDFp) is to provide a guide towards the means of posing and rendering assets in Daz3D and then batch rendering in Stable Diffusion (Automatic 1111) for photorealism or artistic results.
The function of the SDFp assumes the following:
- Daz Studio is installed locally
- Stabel Diffusion / Automatic 1111 and Python are installed locally
- The user has access to a image manipulation tool such as Photoshop or PaintPro
- The client machine has a graphics card with at least 4GB of Vram. 11GB or higher is advised
- You have lots of disc space. The checkpoint models are usually about 4GB each
Setting up Stable Diffusion Automatic1111
The following youtube videos give good instructions for setting up Stable Diffusion Automatic111, which I currently feel is the best interface for carrying out this process
Also another favorite:
Once Stable Diffusion Automatic1111 is running you are ready for the next step.
Choose a Checkpoint Model
The standard model may not deliver the style or results you are looking for.
I suggest that you try reading through the models listed at civitai.com
Setting up the Scene in Daz3D
- Create a new scene in daz
- Add a new figure and create a memorized pose for each keyframe (20-50 frames)
- Import HDR file as scene backdrop (saves a lot of rendering time and it will be re-drawn in Automatic1111 anyway)
- Add props on frame 1
- Create keyframes on each frame for props to be used/handled
- Add lighting to frame one and create keyframes per froma for each light
- Add cameras for each batch of renders
- Pre render in DazStudio using Character
- Render settings can be light (1200 steps)
- Render 706 x 1024
- No canvases needed
- Consideration for Depth map (this might help out where it is difficult to see foreshortening or separating the figure from the background)
- Render to PNG (batch rendering currently only accepts pngs)
Once you have some renders to work with you can move on to the next step
IMG2IMG in Automatic1111
Launch Automatic1111 and ensure you have the model you wish to work with installed
Create your prompt: Tip: Use the Interrogate Clip button to get started.
Create your Negative prompt: Really important.Say in here anything you don’t want it to be like a sketch or painting. You have to trial out a few test images and look at the results. If you see somethng that you do not like then add it here.
For example if I put in the prompt ‘looking at camera’ and the AI keeps creating cameras in the picture then add the word camera to your negative prompts.
go to the IMG2IMG tab and import an image you want to work with.
Here are some standard settings that I commonly use:
You will note that the samples are fairly high. IMG2IMG seems to get a sharper result with higher samples
Make the output image bigger that the imported render. The AI will add more detail.
Run a test run of a batch size of about 4. It won’t take too long and you’ll be able to see at least one image that looks promising.
CFG Scale: I set this to about 14-15 . This seems high but I want the AI to stick to the prompt and also respect the original render
Denoising strength: To counteract the high CFG Scale I need to boost up the denoising strength to around 0.25 to 0.26. Much higher and things get weird.
I don’t set a particular seed but when I find an image that I like I usually to a batch of images using the same seed to see what variations I get.
The results you get from this may need further work such as tidying up hands or fixing the exposure.
Some Tips
For character likeness and beautiful renders:
- Use VAE
- Use a good model (I currently use Protogen)
- Make the pose of the character easy for the AI to interpret i.e. not upside down or obscured by objects
- Do not set restore faces