Mintmaster
Veteran
Of course, but we're not doing infinite iterations to get that infinite detail. We're just making pretty picturesI see what you mean. This fractal however has infinite detail, the more accurate you try to render edges the more of them you will find. As the current rendering is only a relative smooth approximation it may be worth investigation.
Yes, the isosurface method is possible, but you can still do adaptive supersampling with this unbounded volumes raytracing method.The problem can be considered rendering of an isosurface whose surface is at a certain distance 'D' from the fractal. To make it more accurate, a more accurate intersection with the isosurface would be needed. This could be done by an iterative approach similar to high quality isosurface volume rendering. Also for getting more accurate normals this would be needed. Once this is done right, adaptive supersampling near edges could be done as you describe.
Now that I've thought about it further, you don't even need DX10.1 or AA support (duh, why use hardware AA if there's no polygons ). Just optionally supersample in the shader itself if either of those conditions are met, and average the samples, maybe with gamma correction.