iClone 7
Not Reproducible
Issue 6592
Iray fails when working with subdivision & displacement on the same model - Iray crashes in iClone and Iray standalone the GPUs don't work
I provided this info and more in another FT post and tech support stated it was fixed in V 7.7 - I don't know what was fixed, even the update notes did not state this Iray problem was fixed.

This issue has been ongoing since Iray came out. It seems to be a mesh resolution limit that is not being accounted for in the VRam when using tessellation.

I figured out a solution which I think should be a tip for the manual talking about texture tessellation and displacement when using organic models.

It seems I'm able to get displacement just having the model use smooth mesh by subdivision and use a displacement texture but don't use tessellation (keep tessellation at 0 or keep it to the bare minimum) This would be a recommendation for organic models. I've tried subdivision on its own and works fast in Iray.. I tried texture tessellation on its and its very slow trying to get the same level of detail and I'm using an RTX 2080 Ti! but using both with high tessellation settings likely makes the mesh size extreme.

So the tip may be this "when using Iray use smooth mesh by subdivision, set tessellation to 0 or very minimal or the model resolution will become too big for Iray to handle. Iray may need to allow for larger mesh or project triangle size. I has taken me a looooong time of trial and error to figure this out for organic models that need more detail and it should be a tip for both content creators and a tip in the manual for general users.

Ultimately there should be some way to see what the additional triangle count is when using tessellation -maybe artificially change/override the current model count doubling it with each tessellation level ... maybe the bug or lack of visual cues is that either Iray has a max triangle limit or triangle count isn't accounted for in the video Memory stats and gives a false impression to the user that the Video card isn't being fully utilized when in fact it is.

The information below is how I arrived for the solution above.

Tessellation and subdivision works fine by themselves although refreshing with tessellation itself is very slow and could be the problem. When using subdivision with tessellation my particular project triangle count is 214376

I tried baking the displacement into a new model which came to 48 MB (856744 project triangles) using 3.1 GB vram and imported it into iClone and loaded Iray.. it crashed but perhaps it was because I wanted to see if additional displacement using the same displacement texture would all me to push the quality a bit further (It didn't, I'd have to use smooth mesh by displacement again to prevent tearing. I opened up the project again, deleted the displacement textures, set iClone to min mode and made sure nothing else was enabled )unchecked smooth mesh, displacement, all reflection settings. I used the same 8K textures It was blazingly fast! no crashes. Still need to get subdivision working with tessellation.

Also in minimal mode with veiwport preview Iray doesn't refresh with any new angles but works properly when adjusting camera angles.
OS: Windows 10
  •  3
  •  1549
Submitted byAscensi
It was reported by Reallusion that Smooth mesh by subdivision is not compatible with Iray at this time BUT!  to save on space, the stand Alone Renderer could be made to turn it on before rendering each frame rather than saving subdivided models as the rendering files.
Feedback Tracker Admin
Hi Ascensi,

Can you provide project file to help us identify the problem?

Feedback Tracker Admin
Hi Ascensi,

Since displacement and subdivision are functions that exhaust GPU performance, so Iray may not have enough resources to render.

However, in theory, it should be possible to improve the performance after exporting mi and rendering it using Standalone.

The result of your test is 200,000 faces when turning on the smooth mesh, and the result of baking is 800,000 faces. However, the result of baking is faster. This is strange.

We expected to turn on MI after outputting smooth. Close the iClone AP and go to Standalone to render. The speed should not be much  different in theory.

Can we ask you for the Project, and Bake's mesh? It will help us check the problem.

Thank you very much for putting in so much time to generate this issue and share it with us.