This week saw the Mono2 update roll out to the main grid. This update has cycled through the testing process for weeks. Many were looking forward to its roll. Things did not go as well as expected. The problem, new fixes and coming fixes are discussed below.
The Mono2 roll out revealed a structural problem in the testing cycle. The percent of homestead regions in the release channels was too small to catch a problem in this upgrade. Hours after the Tuesday roll out the Lindens knew they had a problem. By noon Tuesday they knew the problem was big. None of the homestead regions were working well. By Wednesday morning they realized the scope of the problem.
Mean while some standard regions were getting excellent performance. Others were running into problems.
It did not take long to run down the problem and make a fix. That fix was rolled to the main grid Thursday morning. Performance improved on the homesteads.
Regions were suddenly seeing script performance degrade. The SIM FPS was spiking up and down and nearly empty regions were seeing Time Dilation and SIM FPS degrade. I was in some nearly deserted regions Wednesday. Performance was so bad I was changing viewers and checking my connection trying to figure out what was wrong.
The cause of these problems was a time allocation mistake. Quoting tehKellz, “The bug introduced on Tuesday used the wrong of two closely named variables and let homesteads run like normal regions. Which doesn’t work well when you put 4 per cpu on a host.”
Think of the server as a resource and each homestead gets 25% of that resource. Each region plans its use of the server according to what it thinks will be available. We can say it fits 250 milliseconds (ms) of stuff into its time slot. The Lindens optimize the allocation processes to give the best results for SL. Suddenly four regions thought they had 1,000 ms of 1,000 ms to fit things in. The server’s operating system, only having 1,000 ms per second to divide between them, had to try and fit four times the load, 4,000 ms of calcs, into 1 second. That doesn’t work well. The operating system’s time slicing is oriented toward operating system needs, which doesn’t match SL priorities. It functions, but is inefficient and performance degrades.
Thursday the problem was corrected. But, Kelly thinks there is still some problem and regions are still thinking they have a larger time share than they actually have. You may see that in SIM FPS jitter.
As I understand, the script time we see in the stats panels (Ctrl-Shift-1) has become somewhat removed from reality. Over the years those values have drifted away from reality as changes were made to the SL system. The values previously shown were different than the actual times. The Mono2 update pulls these values back toward reality. Script times change. So, people are seeing a change in the estate manager’s top scripts lists and script timings. Changes are expected to be drastic. So, it is not that Mono2 upgrade degrades a script’s performance, it is Mono2 improving the accuracy of the reporting.
The experience one has in a region should reveal whether the scripts are running better or not.
Kelly expects many benchmarking scripts to fail now. Soon we will see new benchmarking scripts that use the new LSL performance functions added to aid script writers.
Juicing Fixed… Sort of…
I had heard of scripts that JUICE. I did not understand what that meant. Think of steroids and juicing and you get the meaning. But, how does one juice a script? The mess up with Mono2 brought to light much of what is going on in the SL system and suggests what some have done to get an unfair advantage for their script.
Since the simulator software optimizes for what the region needs to do, it is possible to mislead it. Scripts can be written to make the simulator think a script really needs more time than it does. Mono2 takes steps to more quickly correct the time allocation for juiced scripts. A juiced script will still see better performance, but much less than they saw prior to the Mono2 upgrade.
The juicing issue will never be totally fixed as some legitimate scripts honestly need to do things that just mislead the simulator optimization.
Upgrades in the Release Channels
Le Tigre and Magnum are running the Mesh Prep upgrade. Since next Monday is a national holiday in the USA there will be no Tuesday roll outs. This will give Mesh Prep two more weeks in the RC. This updated is being labeled Mesh Prep 2.
The Mesh Prep 2 is a priority for the Lindens. The sooner it passes QA testing and moves out of the RC the sooner they can begin enabling mesh on the release channels.
We haven’t heard much about the coming of larger prims. But, they are still in the plan.
The hold up here is encroachment. When one allows their prim to hangover into another’s parcel we have an encroachment. The system has been changed to allow people to RETURN objects that encroach on their parcels. Apparently there are still problems with the process.
Those that want to help test the Mesh Prep can get the latest Mesh Project Viewer and use it in the release channels. Mesh and llCastRay() are not enabled. But, using the project viewer in those regions will test both the viewer and the sim software. Give it a try.
Driving vehicles in those regions with the project viewer is a good test of mesh and the new Havok physics version.
Simon Linden is working on sound and Private Parcel settings. This is a project that will probably make it into the RC in the next week or two.
Private Parcel settings would keep avatars in the parcel invisible to those not in the parcel. There already is a 512m limit on avatar visibility. I’m not sure if that means avatar impostures too. Nor am I sure what them means for those cam’ing around.
Some of the delay in getting it into the RC is the User Interface design. It is till changing. The feature will first appear in another project viewer. All viewers will be affected by setting changes made with the project viewer. Those changes are made to the parcel and that changes what the SIM sends to all viewers.
The fix for immortal prims is in the RC Blue Steel. So, with luck we will see it on the main grid in two weeks.
The fix and improve pace in SL is still very fast. The Mono2 upgrade is far more complex and far reaching than I typically get into. Work has been done to make it easier to work with the system and scripting languages in the future. The systems that support and depend on scripting have been revised and improved for better overall region performance. I’ll describe it as building foundation. The stage is set for significant future upgrades and new features.
I am anticipating some interesting news at the SLCC in August.