Jo Plaete

Visual Effects – 3D – Interactive Design

Notes on a Massive Crowd Simulation Pipeline

with 41 comments

As promised, in this post I will talk about how to use Massive Crowd Simulation software in a project pipeline based upon the experience I built up doing the Path To War group project where I had to solve several crowd shots. I will not go in depth into how massive works, agent brain building or explain every step I took in detail. Rather I will pick out some specific issues and techniques I came along using the tool. This document assumes you know what Massive is, what it does and its internal workflow. Watch the Film.

Pipeline
Before I address some more specific issues I’d like to sketch out the overall pipeline we’ve set up to pull off the project as after all one of the biggest challenges in using Massive Crowd Software was to get it integrated into our university pipeline and set up a decent workflow with other 3d tools. We had quite a lot of software involved on the project with Softimage XSI at the core, Massive Prime for crowd simulation, E-On Vue for the natural environment creation/vegetation and Apple Shake as 2d compositing tool. Next to those there was also Autodesk Maya and Motion builder for the motion capture pipeline and Z-Brush for the character artists as sculpting/texturing tool. Because of the various software we were rendering out of three different renderers being Mental Ray for xsi shots, Renderman for crowds and the Vue internal renderer for the environments. The overall pipeline is shown schematically in figure 1.
Pipeline
Figure 1. Path To War overall pipeline.

In general, the core of the pipeline was XSI, this is where the previs was done and the cameras and terrains got placed to be then transferred over to other tools. Also all the modeling, texturing and animation of the close up hero shots were done using XSI. However, the initial idea of having straight a way communication between XSI and Massive did not work out as well as planned. Multiple prove of concepts of bringing over rig and animation data had various positive but also negative results and proved in the end that unfortunately we could not rely on this. Also as the project was on a very tight (8 week) deadline (including learning the tool, setting up the pipeline and finishing off), there was no time to develop custom tools to solve certain issues. Therefore we brought in Autodesk Maya where Massive was initially written for (pipeline wise) and used this as a platform in between Massive and other software.

Models and Textures
A fairly straight-forward part of the pipeline was the transfer of the character/props models and textures as this worked perfectly fine by getting OBJ files with uv information for the accompanying textures out of XSI straight into Massive. To prevent crashing the engine or overloading the renderer with a big amount of agents we had the models in 3 levels of detail. For long shots we had a low polygon version that contained around 4000 polygons. More close up/medium shots had a slightly more detailed model and the hero shots (rendered out of XSI) had the full high polygon model with z-brush displacement maps, etc. It was important though to keep the models fairly low poly for Massive but also to have the possibility to switch to a higher level of detail as agents came closer to the camera. This could be accomplished using the level of detail functionality inside Massive.

Motion Capture
As the project progressed we quickly found out we definitely needed to bring in motion capture to fill up the Massive motion trees with realistic animation and let our animators concentrate on the hero acting shots in XSI. This brings me to an interesting bit which is how the actual motion capture pipeline worked. After a few tests with our xsi-based rigs I decided to leave them for solely XSI purposes and build a new rig that would fit better into the Maya pipeline. After various testing and looking at the motion capture pipeline it seemed most stable to reverse engineer the process and build the actual crowd rig in Massive itself and export it back out to Maya (.ma) from which it could then go back and forth to Motion Builder to apply the motion capture animation. Bringing that animation back into Maya and export it as a .ma for Massive made it very convenient to import actions for the agents. Once imported into Massive, I used the action editor to prepare the actions for use in the brain. Something I’d like to point out that kept me busy for a bit is to remove the global translation and rotation curves from the action in the action editor to make the brain able to drive those based upon its calculations.
Motion Tree

Further, to get some hand animated actions into massive, we’ve built some FK shadow rigs in XSI which mimicked the Massive rigs we had in Maya. This way we were able to FK-wise transfer XSI animation data on the XSI rig through the shadow rigs into Maya and from there into Massive. Which we didn’t really use in the end as we chose to render all the hero characters out of XSI and composite them in with the crowds.

Camera
Because the terrain got rendered out of Vue, hero characters out of XSI and the crowd characters out of Massive, it was very important to get all geometry and cameras lined up. The best way to do this appeared to be to take the camera from XSI to Maya via the FBX format and then import it into Massive out of a .ma file (figure 3). Using an xsi-vue plug-in we were then able to take that same XSI camera into Vue and render out the background plates from the exact same perspective. The terrain geometry was imported into Massive using the OBJ format to have the agent interact with it but only the agents were rendered out from Massive. To keep the simulation as light as possible it was good to cut up the terrain for different shots and only import the parts needed for certain shots.

Camera
Figure 3. Camera pipeline xsi->Maya->Massive.

Rendering
For the rendering I took advantage of the Massive passes based set up approach towards Renderman compliant renderers. We exported the RIB files out of Massive and rendered them out with Prman. We did have to do some post processing on the RIB files though to get them to render out as we wanted by adding attributes and changing them towards our linux based renderfarm. Several python scripts were developed to remove windows path names and change the massive.dll reference into a .so one.

The passes produced out of Massive – Prman were a diffuse pass (figure 4), specular pass, ambient occlusion and a variation pass. Originally, we tried depth map shadows but it was hard to get a decent pass out so an ambient occlusion pass was needed to stick the agents to the ground when composited onto the background plates. Since the ray traced ambient occlusion on a lot of agents (+300) crashed the renderer, a script had to be developed to split up the AO RIB files which were then rendered out to multiple AO files per frame (figure 5) and combined back together in compositing.

Render Passes

Figure 4. Passes.

Another pass we’ve called the variation pass (figure 6) was pretty interesting and helped the final output a lot. Since we didn’t have much time to create a big amount of character variations, we used this pass to pull more variation in the crowd in compositing. Technically it’s just a constant shader which get passed a random argument value between 0.2 and 0.8 defined by a massive agent variable. Make sure to pass on the random value from the agent and so on a RIB level as randomizing inside the shader would give you ‘noise’ agents which is not what you want. This way we had a pass of all agents having a random grayscale value which the compositers could use to pull more variation into the crowds. In the end a lot depended on the compositors to make it all work together since the terrains were rendered with the Vue renderer, the hero characters with Mental Ray out of XSI and the crowds with Prman. So out of massive we had only the crowd agents which were then composited into the terrain which was rendered out in separate layers to achieve the best integration. Another thing to take into account when rendering out using this approach is not to render the terrain out of Massive but do use it as a masking object to occlude agents that are behind the terrain.

Variation pass
Figure 6. Variation pass.

Randomized Agent Setup
When doing a crowd simulation one of the most important things is to take care not to give away similarity or patterns in your crowd. Next to the additional variation pass at render/compositing stage we tried to address this problem as much as possible by making variations of the character models and textures. We ended up having 2 different Viking base models which both got 4 modeling sub variations and 4 texture variations plus also 4 different weapon props. This was all combined into 1 agent which had some randomized agent variables tied into a massive option node.
Looking at the movie you might join my opinion that the Viking crowd looks a bit more dynamic and randomized then the Egyptian one. This is solely due to the fact that we had 2 base models in that Viking crowd and only in the Egyptian crowd. Though there were 2 model variations, 4 texture variations and different props assigned to those Egyptian warriors, this variation could not live up to having 2 different source models also having this sub variation.

Of course, randomizing your animation is vital as well. Maybe even more then visual pattern appearances, movement patterns are spot very easily and give away your crowd quickly. So it’s good to keep this in mind when preparing/directing your motion capture session to go for as much various motion as possible. For example capturing multiple different run cycles for the same run motion and randomize those in between your agents is a good way to go. Next to that putting a slightly random variable onto your action playback speed gives really nice results too. As we hadn’t much time to prepare our motion capture session we came out of it with good but slightly not enough data so we went in and manually changed some of the basic run loops slightly and randomized that playback rate in the agents brains to get a more natural feel to the overall movement.

Directing the Agents Movement and Flow
The most interesting interaction that had to be taken care off inside of Massive was the terrain. We had trees on the terrain rendered out of Vue which had to be avoided by the agents but obviously were not apparent inside massive. Something simple but which helped a lot for this was the feature in Massive to put an image as your viewports background. So I brought in the terrain and camera for the shot and putted the Vue rendered out backplate image as viewport background to start matching up and to find out how to place/direct the agents. Next to that some dummy geometry was placed on the terrain where the trees supposed to be to get them avoided nicely by the agents vision.

In the end the brain made the agents do the following: follow the terrain topology including banking and tilting them to the tangent; avoid each other based upon sound emission and flock them back together when wandering to far off; follow a flow field for the general running direction; getting random actions applied and blended onto the agents; randomize the playback rates of the actions for variety. Of course, lots of tweaking and testing was involved to get the crowd behaviour to work well but I was very happy in the end to use Massive for this which definitely provided some very good features to combine all these behaviours and keep a lot of control over what you are doing without the need to go into heavy programming. Figure 5 gives an overview of all forces working on the agents and as you see it can get quite tricky as you get opposite forces pushing them away from each other and pulling them back together which makes it necessary to go in and tweak the behaviour on a per shot basis.

Forces
Figure5. Agent Forces.

One thing I did miss in Massive though is the ability to go in and direct a single agent or a ‘make agent independent and tweak brain’ function. As all your agents, or at least groups of them, share the same brain it is sometimes hard to deal with that annoying one that is not doing what it should do. I ended up deleting those agents out or changing their start position to attempt to adjust their behaviour which is a pretty tedious process and finally some were even masked out in compositing.

As I mentioned before, integrating Massive into our project pipeline was one of the biggest challenges on the project. It is a great tool for what it does with artificial intelligence and crowd simulation but it requires some investigation and testing to get it to work properly. I hope reading this might give you some directions on how to integrate it on your project. I do not assume this is the ultimate way but it was sure close to the best way with the out of the box solutions. Contacting some industry companies and speaking to other Massive users it appeared that lots of them develop in house tools around it to fit it into their pipelines. Any comments or suggestions on this are very welcome.

Jo

Advertisements

Written by Jo Plaete

June 25, 2008 at 12:10 am

41 Responses

Subscribe to comments with RSS.

  1. […] Link […]

  2. Hi. I’ve a question. How did you render?. I’m trying render in mental ray but i don’t know how. I’ve the render_scrit.sh file but i can not execute it. Do you know why?. permission 755

    Dios

    October 7, 2008 at 10:46 pm

    • I have solution for you. I create a simple program that convert render_script.sh to render_script.bat, email me to get it for free. MASSIVE crowd to Maya, solution for rendering your crowd outside massive… Look at this project that i’ve worked with Brasilian company for World Cup 2014 2″Teaser | Colorados Temples” on my website

      Regards,

      Domingos

      February 19, 2011 at 2:31 pm

  3. @Dios:
    We were using prman for this project.
    We exported the ribs for each pass at simming time and post processed these with some python scripts for getting it to work with our setup(changing some paths, etc). Make sure you have all your files in place as massive will point back to the massive.dll or massive.so and your files(ribs,sims,..) at rendertime. I’ve never tried with mental ray but I suppose it’s the same idea using .mi files. The .sh script is usually some batch process generated by massive to automate your renderprocess. Give it a try looking inside if it’s actually pointing at the right things.. 755 should be fine for execution.

    joplaete

    October 7, 2008 at 11:43 pm

  4. “Make sure you have all your files in place as massive will point back to the massive.dll or massive.so and your files(ribs,sims,..) at rendertime”

    My mi, sims, and rib files must be in the massive instalation directory??. Because i’ve created a new directory for them y other partition.

    What i don’t understand yet is about render engine. You must install renderman in your computer, don’t you?, and this is the render engine that massive used. Do you know if i need to do the same with mental ray?. I’ve install xsi in my computer, but it’s this enough??

    Thank you so much

    Dios

    October 8, 2008 at 7:41 pm

  5. Hi Jo,

    I’ve got problems rendering width massive prime on windows. I have massive prime and pixars renderman server and studio running on my win xp. I tried hard, but the prman shaders won’t appear in massive (even when i enter the shaderpaths in the massive dialog). Nothing appears when i do a preview render in massive. Maybe you can help me with this problem.

    Thank you!

    joergsla

    November 13, 2008 at 11:33 pm

  6. hi joergsla

    set prman install path to eni.variable path.

    it will work.

    bharathi

    January 29, 2009 at 1:38 pm

  7. hi joplaete.

    where i can find level of detail option in massive?

    bharathi

    January 29, 2009 at 1:43 pm

  8. Hi Jo,

    How did you get AO rendered in renderman??? I have renderman and it is working, can use all the shaders but cannot find this renderman AO shader???

    Dragon

    February 15, 2009 at 3:04 am

  9. I have same problem in Prman with AO. I’m even try connect custom compiled prman AO shader, but it still cannot work.

    Mike

    February 17, 2009 at 10:11 pm

  10. hi jo, i am from India working as visual effects supervisor. i am very much keen in learning this software ( massive prime )hope u will guide me.

    rgds

    jitendraverma

    February 21, 2009 at 4:56 pm

  11. hi jo, i am from India working as visual effects supervisor. i am very much keen in learning this software ( massive prime )hope u will guide me.
    rgds

    jitendra

    February 21, 2009 at 4:57 pm

  12. @Dragon,Mike –
    Sent you my AO shaders and they seemed to work.
    If anybody else want them, drop me a mail, will try to do a small post sharing them soon.

    joplaete

    February 21, 2009 at 6:28 pm

  13. …[It was important though to keep the models fairly low poly for Massive but also to have the possibility to switch to a higher level of detail as agents came closer to the camera. This could be accomplished using the level of detail functionality inside Massive.]

    hi jo, can u explain the above. (technical)

    bharathi

    February 23, 2009 at 11:33 am

    • export it from motionbuilder to maya and from there you should be able to export a .ma file or fbx into massive, skeletons need to be compatible of course.. In the post i describe how we took our massive skeleton into maya -> motionbuilder and back, worked fine for us ! cheers

      Jo Plaete

      August 17, 2009 at 1:11 pm

  14. Joe, can you explain in total beginners terms, how do you scale an agent to fit your scene

    Mike Wharton

    May 8, 2009 at 8:04 pm

  15. Hi,
    I’m also fighting with AO in PRMan. Could U share this shader with me also ?

    Thanks a lot !

    Michal

    July 13, 2009 at 3:23 pm

  16. Hello, i wanted to know i have some motion capture data in the form of .c3d files and i can view them in motionbuilder, how do i get that motion imported into massive so that an agent exactly moves in the way the original data has been captured?

    Hassan Taimur

    July 29, 2009 at 3:36 pm

  17. Hello. I am from Korean and Vfx TD.
    I have a question.
    How can you rendered vue scenes on linux?

    Yeonsoo Kim

    September 3, 2009 at 11:16 am

  18. Hi Jo!

    Thanks for posting this breakdown! Could you also send me your ambient occlusion shader, as I’m trying to get ao and massive working with 3delight right now, without success.

    Josh

    September 22, 2009 at 4:29 pm

  19. Hi Jo…

    Thanks for the detailed pipeline..

    I was indeed cracking my brains to get an AOshader and the rib editing script to get the basic passes as output..

    Can you just help me out…and I bet that would be a BOMBASTIC HELP!!

    sreenu

    October 27, 2009 at 1:15 pm

  20. Hi Jo
    I’m a learner. And my english is limited.
    I’m dying to know how can i render the massive’s scene in maya?
    Can you help me、

    taotao

    January 6, 2010 at 9:45 am

  21. Hi,

    I have developed a pipeline that import massive scene into maya, but i’m not finished. Go to my website and see the result.

    drc

    January 22, 2010 at 3:49 pm

  22. Sorry, this is my url: My website

    drc

    January 22, 2010 at 3:53 pm

  23. This is amazing man, Thanks lots for sharing the information.

    i am going to start learning on massive but i was wondering do i have to render just with renderman ?!! can i use the maya’s mental ray ?!!

    Thanks again.

    Ahmed Mohsen

    June 2, 2010 at 5:26 pm

  24. Hi,

    If you want to rendering your crowd in maya
    mass2maya v0.5 plugin for Maya 2008/2009/2010 32 bit get it on my blog [http://visual3dtools.blogspot.com], it’s work with on win 32 bit and x64 with maya 32 bit

    i’m not using the old blogs.

    thanks

    Domingos (drc)

    June 2, 2010 at 7:18 pm

  25. hello,jo
    how to render Occlusions in massive?
    I can’t open raytrace in massive.

    leo yu

    June 21, 2010 at 9:04 am

    • I solved this problem.

      leo yu

      July 2, 2010 at 7:56 am

  26. how to import backgroung image for massive viewport.I want the details

    mathesh

    September 18, 2010 at 6:36 am

  27. […] Motionbuilder, Endorphin, Maya, 3dsMax and Massive After reading the Blog of Jo Plaete and his notes on Massive integration, I’ve started to do some “software […]

    Software Pipeline « Tom Sidji

    September 28, 2010 at 3:21 pm

  28. Cant get my massive to render agents in renderman, when i preview i only get the terrain render

    mansh

    February 19, 2011 at 10:31 am

  29. Hi any one help me….in massive two grops of peoples comes opposite dirction when they come near all are change the action and then fighting each other….pls how to do that…is it possible in motion tree or else do in brain….pls…i’m waiting for ur reply…

    Sathish

    May 18, 2011 at 2:36 pm

  30. thanks for the info, could you brief how to scale the agent in massive? thank you !

    wind

    June 15, 2012 at 11:57 am

  31. Hi
    I ‘ve been working on massive for a while and am rendering on Velocity and 3d Elight .
    my Q is how to render depth of field pass on these tow engines ??? . .
    please anyone send me an email explaining step by step the process and I will be really greatfull

    Basel

    June 20, 2012 at 2:58 pm

  32. […] reading the Blog of Jo Plaete and his notes on Massive integration, I’ve started to do some “software […]

  33. Hi Jo,
    could you explain me your Mocap pipeline in more detail. I’m having a problem with getting my Mocap data into Massive. I’ve tried exporting bvh or amc from motionbuilder (keyframed in 3ds Max) but that causes problems. Are there things to pay extra attention to?

    Ralf Bonten

    December 13, 2013 at 6:14 pm

  34. How do you use a ascii file as a action?

    Ralf Bonten

    December 13, 2013 at 6:49 pm

  35. Hello all, I was wondering if anyone had any ideas on how to use keyframe animation properly from maya to massive. I keep hitting a wall where the motion I set up in maya won’t transfer. Perhaps it is because the bones themselves are not moving but rather the IK handles? Any ideas?

    Ryan Hawk

    May 23, 2014 at 7:53 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: