Cal3d in XVR

[marco]

>    I have thought about a similar thing: how I would integrate Cal3D in XVR but encountered some problems.


essentially the problem is the same, as the parts of piavca that are important to the integration are the bits connected to cal3d. We are working on this so if you are happy waiting for a couple of months you can have our work to base what you are doing
 

>    I think there are several ways to integrate it:

     

>    (1) its quite straight forward to just create a Cal3D dll that renders characters into XVR using OpenGL,

>    but in that way XVR wouldn´t be “aware” of the rendered character I think. Franco sent me an example external dll to render opengl in XVR.


Doing it this way means that you cannot use any more advanced XVR rendering features. I wouldn't be in favour of doing this.

>    Using this option I think it wouldn´t be possible for example to use the XVR physics engine in combination with the characters.

>    I don´t know if the XVR physics engine is an external dll also ??

Not directly answering this question, but in case you didn't know, cal3d has a mini spring based physics engine for handling hair. I guess you probably want something more sophisticated for your cloth simulations.

>    (2) Another way may be to write a loader dll that reads Cal3D characters and creates XVR meshes.

>    Another dll would then load the mesh from XVR for character animation in the  Cal3D.

>    In this case you would need some way to represent vertex bone weights which I think are not available in XVR ?

 This is the way we are doing it. We are going to use Cal3D to handle the mesh manipulation (morphs/skinning) but nor the rendering. In fact the renderer isn't a core part of Cal3D, if you note the renderer is in the example view not the main project. How Cal3d works is that you pass it a buffer which it fills with the updated vertex values. My plan would be to get a pointer to the internal cal3d vertex array representation and pass cal3d (assuming that the representation can be casted to an array of floats). I've done something similar with OpenSG, I could send you example code if you want it now, but as I said we have a masters student who will start working on this next week so if you're not is a massive hurry you might as well wait until we have some example code in XVR, to avoid duplicating work.

>    Also Marco I remember you showed me an animation file of Cal3D in XML format. When I look at the .cmf or .caf  or files which I think describe character

>    and character motion they look binary to me. How can I view them in XML format ?


there are 2 formats cmf/caf is binary, but they also have an xml based xmf/xaf, to save them out just use the normal 3DS Max exporter but name the output file xmf instead of cmf.
 

    thanks,

    Bernhard

     

[Support@VRMedia]

This is the way we are doing it. We are going to use Cal3D to handle the mesh manipulation (morphs/skinning) but nor the rendering. In fact the renderer isn't a core part of Cal3D, if you note the renderer is in the example view not the main project. How Cal3d works is that you pass it a buffer which it fills with the updated vertex values. My plan would be to get a pointer to the internal cal3d vertex array representation and pass cal3d (assuming that the representation can be casted to an array of floats). I've done something similar with OpenSG, I could send you example code if you want it now, but as I said we have a masters student who will start working on this next week so if you're not is a massive hurry you might as well wait until we have some example code in XVR, to avoid duplicating work.

This is for sure an option, but there is something that should be remebered: performances.

- if you want to drive a single PC installation (and this means 90% of the installations) there are not much troubles in changing all the vertexes position of a mesh frame after frame (applying morphing and blending), even if this is sub-optimal, given that it generates a constant traffic over the PCI-Express bus (or AGP).

- if you want to drive a cluster based Installation things are very different, as every single display primitive will have to be sent for every frame to all the nodes in the cluster (this is an advanced topic that we will expose in detail at the workshop in Pisa)

The best option right now is probably to develop a brand new renderer for character able to perform skinning inside a vertex program: in this way it's possible to minimise the node-to-node traffic.

Note: I suspect this is also why the avatars are so slow on the UCL PRISM.

Franco

[bernardo]

Franco/ Marco,

There is an example renderer in Cal3D that does skinning on hardware. "miniviewer_gl_vp"
I guess using that only the bone changes would have to be transmitted over network on a cluster.
But then in order to do collision detection with other geometry in the scene for say a
physics engine at least the master would also have to know the values of the vertex
positions unless all the physics is done on GPU too.

Bernhard

[Support@VRMedia]

Hi all,

hardware assisted blending: very interesting. Surely the best option.

Let me explain how the XVR network driver works, so that you can have an idea of what will be going on.

To allow XVR to work on a cluster of PCs, we are developing a virtual OpenGL driver that (exactly like Chromium) intercept OpenGL calls and broadcast them around the network. It does a couple of other things that Chromium can't provide (because it gets additional info from the XVR kernel), but replicating OpenGL commands is what it does 99% of the time.

The good thing about this approach is that if you write OpenGL code in an external DLL, these calls will be intercepted as well, so you will get CAVE-enabled code for free even for your own graphics code.

To achieve the best possible performances, the number of primitives that you send over the network should therefore be limited. As an example, it's MUCH faster to send a single glCallList than sending thousands of glVertex for each frame, and this is the reason why blending using hardware will make things much more performant.

Beware, there are limitations: we don't currently support ALL the opengl commands, as some of them need a very complex heuristic in order to be packetized and transmitted (glVertex arrays are the typical nightmare), but there is a HUGE quantity of tricks that you can do by using display lists anyway.

So, it's very simple to figure out how much bandwidth you are going to use in the underlying XVR network driver: it’s proportional to the payload of all the OpenGL commands that you will send frame by frame (in reality we compress/decompress the packets, but this is just a detail)

You will see more details of our network driver implementation at the workshop. Meanwhile I would say that a clever combination of Display lists and Vertex programs are the best way to go to have complex avatars in a networked PC cluster (in general, not necessarily XVR driven)

Best,

Franco

Ah, and concerning collision detection:

- I would be very surprised if avatar-to-avatar or avata-to-world collision detection is performed triangle by triangle. I've seen that in real life scenarios a much coarser collision detection algorithm might be sufficient. Therefore, I believe that collision will be rather independent from graphics anyway, or use a simplified avatar mesh that will not be transmitted anyway.

[marco]

My reason for wanting to update the XVR meshes is similar to Bernhard's. If you have a completely separate renderer just for characters then it makes if very difficult for the rest of XVR to interact with the characters, which I think will cause problems. Physics is one case where this is true, I expect itegrating trackers is another. Also we would not be able to use all of XVRs rendering features for the characters (e.g. shadows). It would be possible to use the Cal3d renderer, but I'm not sure I see the benefit as that still just writes the values of each vertex every frame and the renderer itself lacks a lot of features.

Doing everything in hardware is of course the fastest method and we have another student working on doing that. Does XVR have any shader support? It still leaves the problem of how to get the mesh data in such a way that it can be used by things like a physics engine or exploit other rendering features of XVR. One option is reading data back from the graphics card though I heard that this is slow and difficult (though I heard it a long time ago it might have changed). The other is to see whether it is possible to itegrate teh shader code with the XVR renderer in some way, again this depend on how XVR handles shaders, I'd appreciate some info

marco

Marco

[Support@VRMedia]

Hi Marco,

first some info on Vertex & Pixel Shaders. Yes, XVR supports OpenGL GLSL vertex and fragment shaders. Have a look in the Help documentation to the class called CVmShaderProgram, to see what you can do.
Actually, using vertex and pixel shaders is very simple in XVR, as it offer a simple sintax to pass parameters to the shaders. Our opinion is that GLSL is the future of OpenGL programming, so we are trying hard to fully support it, and even the XVR network driver that will run the CAVE is already supporting them. I'm committed to posting a couple of project examples in the XVR WIKI that we are setting up.

This said, I can see the point of your consideration about Avatars rendering. Writing a DLL renderer (dedicated to characters)  is obviously a very flexible technique, and it's something that any "hard coder" may be happy to do, but it's not everyone taste.
What I can propose is this. We can outline a new class, written entirely in XVR scripting language, that will be dedicated to rendering characters inside the environment and that will make use of Vertex shaders too, in order to be 1) fast and 2) network friendly. We will make this class skeleton public so that anyone will be able to modify it and add whatever feature he/she likes. This will of course use all the features of our renderer. Performance is not a problem, as most of the hard work will be done on the GPU anyway, thanks to Displaylists and vertex shaders.
What do you think of this option?

[marco]

that may well be a good option, I'll read up on the shader support to try to understand how it might work

Marco

[D.Taica]

Is there anyone on TC who wants to participate in Season 14 CAL-O? If you do participate, please note you will have to scrim a ton, and have full access to ventrilo. There will be a starting 5 and 5 backup players probably.