MuJoCo running slowly due to a large number of objects in a simulation

Discussion in 'Bug Reports' started by David Kluger, Jul 10, 2015.

  1. We are trying to create a Box and Block Test (BBT) simulation that has 75 freely jointed bodies in the simulation. The program runs noticeably slower, presumably due to the program having to solve so many contact interactions with the 75 freely jointed blocks touching and knocking into each other. Is there a way to reduce the computational load in order to improve the VRE's performance for a simulation with so many freely jointed objects?

    See for a demo on the test we are trying to recreate in the VRE.
  2. Emo Todorov

    Emo Todorov Administrator Staff Member

    The default PGS solver scales as O(N^3) where N is 3*number_of_contacts. There is another solver (called Sparse) which scales as O(N) but it is not fine-tuned yet. You can try it and see what you get (from Settings / Physics / Solver). With your current simulation, what is the CPU time-per-step you are getting in the Info box. Is the system not able to simulate in real-time anymore? If so, have you tried making the simulation timestep slightly larger than the CPU time?
  3. Vikash Kumar

    Vikash Kumar Guest

    Another way to reduce the computational load is to reduce the dimentionality of the contacts, specified using "condim" in the xml. "condim" encodes the nature of the forces a contact generates :-
    condim=1; => Normal forces
    condim=3; => Normal forces, Tangential frictional forces
    condim=4; => Normal forces, Tangential frictional forces, Torsional frictional forces
    condim=6; => Normal forces, Tangential frictional forces, Torsional frictional forces, Rolling frictional forces.

    By default condim=3. If you think you can work without frictional forces for some objects (i.e. the objects will not penetrate but they will be smooth), make condim="1" for all such objects. This will give you some speedup. I'll definitely make the box's comdim=1;
  4. Note: the simulation is using 40 blocks rights now. It is running off my workstation computer, not the HAPTIX-supplied computer. Workstation specs: Intel Core2 @2.13 GHz with 8GB RAM (clocking out at a stellar 3.3 Windows Experience Score...). While real time factors and CPU timing are faster on the HAPTIX PC, they are still <1. OptiTrack is not connected, so MoCap is not running. Here are the results I am getting when changing the variables you suggested:

    1) Timestep=0.002; solver=PGS; condim_box=3; condim_block=3 (default options). Time=0.20x; CPU~8.5ms
    2) Timestep=0.005; solver=PGS; condim_box=3; condim_block=3. Time=0.50x; CPU~10ms
    3) Timestep>0.005; solver=PGS; condim_box=3; condim_block=3. Simulation is unstable, blocks fly around the simulation.
    4) Timestep=0.002; solver=Sparse; condim_box=3; condim_block=3. Time=0.31x; CPU~6ms; Simulation is quasi-stable, blocks jump and jitter, and MPL bobs up and down over the Mocap object.
    5) Timestep=0.005; solver=Sparse; condim_box=3; condim_block=3. Simulation is unstable, blocks fly around the simulation.
    6) Timestep=0.002; solver=PGS; condim_box=1; condim_block=1. Time=0.44x; CPU~4ms. Simulation is stable, but blocks are difficult to grab because they are very slippery.
    7) Timestep=0.005; solver=PGS; condim_box=1; condim_block=1. Time=0.9x; CPU~5ms. Simulation is stable, but blocks are difficult to grab because they are very slippery.
    8) Timestep=0.005; solver=PGS; condim_box=1; condim_block=3. Time=0.22x; CPU~9.5ms
    9) Timestep=0.005; solver=PGS; condim_box=3; condim_block=1 Time=0.52x; CPU~10ms. Simulation is stable, but blocks are difficult to grab because they are very slippery.

    In a nutshell, by changing the options that you suggested, I can only get the simulation to run fast enough when the blocks are too slippery to interact with (option-set #7). I have attached the model I built with the block and box test for your reference. The file is using default settings. You will also need the blue wood texture I added (also attached).

    Attached Files:

  5. Emo Todorov

    Emo Todorov Administrator Staff Member

    This kind of simulation needs a faster CPU, as well as some fine-tuning of the sparse solver which we haven't done yet, and a dedicated box-box collision function which we have not yet included (presently MuJoCo uses a general convex collision function for box-box, resulting in a single contact point that runs around trying to avoid penetrations everywhere, which in turn reduces stability).

    In the meantime, would it be possible to use capsules instead of boxes? They are better behaved under collision. Ellipsoids may also be an option.

    Re condim, the dimensionality of the contact space is defined as the max of the condim values of the two colliding geoms. So if all boxes and the ground have condim=1 while the hand geoms have condim=3 or 4 as in the latest model, only box-box and box-ground interactions should be slippery, while hand-box interactions should be normal... anyway, one shouldn't have to resort to slippery contacts.
  6. Re capsules or ellipsoids instead of boxes, this is a workaround we are a bit skeptical to use. We would like to create tests in the VREs using dimensions and geometries that have been widely accepted and validated in the scientific literature. Replacing the blocks for capsules or ellipsoids may be too big of a change for us to credibly relate our results back to those that are already published. However, a "Box and Capsule Test" may still provide valuable results and we will consider using it for testing our own software/hardware. I'll see if this makes a difference, consult the group, and get back to you.
  7. Emo Todorov

    Emo Todorov Administrator Staff Member

    I was able to get decent real-time simulation with your boxes on a good mobile processor (i7-4720HQ): CPU time around 2.7 msec, simulation timestep 3 msec. On the HAPTIX Xeon system it should be even better.

    The key is to increase the number of solver iterations for the sparse solver. This greatly improves stability for such complicated models, without making it much slower - because most of the CPU time is spent setting things up before the solver runs. Another issue was that the mocap weld constraint was under-damped (we need to fix this in the official model as well), but the main reason for the MPL bobbing was the insufficient number of solver iterations.

    Try the following two changes to your XML:

    <option timestep="0.003" apirate="50" iterations="100" solver="sparse"/>
    <weld body1="mocap" body2="forearm" solprm="1000 1000 200 10000"/>

    Some of the boxes are still jittering a bit, because of the single-contact-point issue discussed earlier. However even after we add a dedicated box-box collider, it will not help automatically for this model because it will gain stability at the price of increased number of contact points, thus slowing down the solver.

    So apart from adjusting parameters, I see three avenues for future algorithmic improvements here: (1) fine-tune the sparse solver, in particular trying to exploit nearly-decoupled systems via some non-trivial extension of the projected conjugate-gradient method; (2) generate lots of contact points to improve stability, and then fuse them in a smart way before running the solver; (3) take advantage of the idle CPU cores - these simulations are not easily amenable to parallel processing but there are some tricks one can try.

    Another thing I noticed while playing with this model is that MuJoCo HAPTIX becomes unresponsive when the CPU time is too long; good thing I have a simpler version (Sim) to test things. This needs to be addressed though...