Some transcripts got posted and there are some bits of news.
Balancing
You may remember that the Lindens started balancing the regions by running adjacent regions in the same physical server, or at least physically nearby servers. They also made changes to the system so that popular regions would not load into the same server. The change improved performance and up time.
Since there was no new software roll to the main channel this week, they took the opportunity to do more balancing work. More busy regions were moved in to non-busy hosts. Oskar Linden says this makes a huge difference. They were going to get us some neat graphs and do a blog post, but that has never happened.
Over time as regions restart they drift… meaning the balance is lost. I suppose as busy regions crash more often they get moved to next available server more often and would tend to collect together. While rebalancing is a somewhat automated process Oskar says it takes some handholding to get it done well.
Region Conductor
You may remember part of the balancing act was improving the region conductor or concierge service. This is the service that finds a crashed region and brings it up in a new host. The idea is to have it avoid putting two busy regions in the same server. Coyot Linden explains, “This is kind of hard [to do] because usually by the time the conductor sees a region should be put on a host, the region is down and wants to come up. If it is down, it is hard to see how busy it is…”
So, I suppose that is why busy regions can drift into the same server.
Region Idling
Oskar talked some about region idling. The announcement is here and there are now FAQ’s.
Maestro Linden said, “It [an idling region] would be doing its usual work for 22ms, then sleep 100ms after each frame. The full frame time is roughly 100ms. But it gives a bit extra back to scripts… So a frame would start every 122ms, so the FPS goes to ~8.2.”
“Child agents wake up a sim. So if any avatar sees into the sim, it’s non-idle.”
Simon Linden added, “Some of that additional 100ms is given to scripts — about 25, I believe. … up to 25, I mean. Physical things will slow down, and time dilation will be high … just like a slow region.”
“It’s more like a loop lasting 100ms: sleep a few ms, check for network activity, repeat.”
Simon has said the region can break out of Idle while the 100ms sleep loop is running. This means spin up is almost instant.
Homeless asked, “If the avatar has his back to the sim, will it remain idle?”
Maestro Linden answered, “Only if you drop the network connection to the sim, Homeless. Usually that doesn’t depend on your camera angle, though.”
Simon Linden added, “If that network activity is a ‘get out of idling’ event, (incoming http, for example) it will break out at that point.”
“This is less about saving energy than trying to free the CPU from doing un-needed work and having it available for other regions or back-end processes on the same server.”
Maestro Linden elaborates, “If you have a clock with hands that move with key framed motion, it might run behind when idle.” Exactly what Maestro means probably depends on the script the clock is running.
Simon added, “The time in key framed motion is stretched by lag … it’s not real-time.”
“The sim has some threading, and should get more in the future as we go after lag-causing problems. There are also other non-simulator processes running on the server that communicate with the back-end. Code that launches regions, talks to databases, handles capabilities, etc. Those will benefit [from Region Idling].”
Breed-able Pets and Other Fun Monsters
There is some concern by pet and animal owners that Region Idling will be a problem for them.
So, far the Lindens have not seen a problem in testing. Simon points out, “I can’t promise that [meaning promise breed-able pets won’t be affected]. If people have poorly scripted breed-ables, they will be affected. For example, if someone programs a breed-able to eat every 162000 frames (1 hour at 45 fps) then they would eat much less on an idling region.”
Before you decide if you breed-able is fated for disaster, you should do some testing. I suspect breed-able designer figured out that a laggy region would cut into their feed sales. It would be in their interest to assure that feed consumption was not tied to region performance. However, if they did not take that into consideration, the pet owner would be the winner.
I hear a number of pets handle food consumption via calls to external servers. The breed-ables’ designers run the external servers and those are not affected be the Lab’s Region Idle. They continue to run at 100% available CPU cycles. Outgoing and incoming HTTP calls spin a sim region up to full speed. So, the feed-hunger cycle should be unchanged.
Presumably the breed-able is asking the external server if they are hungry. If so and the pet needed to get to food, then the time they need to get to food would be longer. So, how long does one have to feed a pet before it starves? How long does it take your pet to reach food? If it takes 5 minutes with a region at 45 FPS then at 8 FPS it will take about 28 minutes… maybe. That depends on how their movement is scripted. I don’t know of any pets that starve in an hour, so they seem safe to me, which some find unfortunate. But, that is another issue.
Idling Feature Request
Some want to have a feature in the Estate Managers Console to disable Region Idle. The thought is those running science simulations and eco-systems would need it. I doubt that.
The llGetEnv(), llGetRegionFPS(), and llGetRegionTimeDilation () functions in the Linden Scripting Language (LSL) provide features that should allow scripter’s to sense region performance and adapt. If they are not using those functions now, it is likely the scripts are written in ways that will not be affected by Idling. If they do use those features, they would already be compensating for the slow region frame rates.
Some assume that most people would use a console feature to disable Region Idling when there is no reason to do so. This would have a negative effect on the entire grid. So, I doubt we will see such a feature unless someone can post clear user cases for it and justify a benefit that out weights the negative impact.
Bots & Idling
If you have bots in your region, it is never going to go into Region Idle mode. These are the bots that have an AI program running on your computer controlling an avatar like bot in-world. The SL system sees them as it would any avatar. So, regions with those bots will stay spun up.
Summary
There is concern about how Region Idling will affect scripted operations. The Region Idle code is currently running in Blue Steel. If you have a concern, go test. Get any problems you find into a JIRA. If the Lindens are not hearing of or seeing any problems, this feature may roll to the main grid on Tuesday.
The Region Crossing Refactor Phase I code is testing on the other two release channels. If it holds up and there are no problems, this will be the code they roll out. In which case, you’ll have another week to test Region Idling. Don’t count on that extra week. The Crossing code is complex and has proven tough to sort out. It has been on and off the release channels a number of times. I would not be surprised if it revealed new problems and remained in testing.
Pingback: i have a feeling this is gona end badly - Page 4 - SLUniverse Forums
Thanks, Nalates. I’m a bit puzzled, though, how people can reliably test stuff with Region Idling. I mean, all I can think of is dropping something in a Blue Steel sandbox and hoping, at some point before it gets returned, that the sandbox starts idling for a bit. And that’s a bit hit-and-miss, to say the least.
I’m greatly reassured by your account of the meetings, though, and by some of the replies in the official forum. What would, though, really reassure me (and, I think, a lot of other people) is if one of the Lindens were to say, in terms, that if I set up something to IM me every 30 minutes on a timer and leave it on a sim, I’ll reliably receive IMs from it every half hour, to within a second or so, whether the region has been idling or not in the meantime. I think that’s what’s going to happen, but I’d feel happier if someone assured me it would.
I understand the concern. I doubt you will get the statement you want from the Lindens. They are in a no win situation with such statements.
Whether the script will reliably send every 30 minutes depends on how you write the script, not the region performance. If one uses llSleep or a Timer event, then it will be reliable. As bizarre as I find it, people use other ways to time script actions. Some count on the various throttling limits in some LSL functions. I’m told some use frame counting to coordinate animations. Some use FOR-LOOPS to force a delay and/or attempt to get more script time via the old flywheel idea (now effectively blocked). Anything that is not based on the wall clock will be affected by Idling.
To test one does have to hope they can find a region that will enter the Idle State. Islands are much more likely to idle. To know whether the script has experienced Region Idling, it has to be changed to record that info. I would use llGetEnv() and if Idling record the start time. Then check and record the time it comes out of the Idle state. Initially I think I will use a rather tight loop. Once I have a sense of how a region behaves a Timer event would be more efficient.
In general I think Region Idling is one of those things like: ‘I think this will work. We’ll try it and see if anyone notices a problem.’ That is an over simplification. I would bet the Lindens have QA’d this on ADITI with their own stuff to see what happened. The Lindens do miss thinking of scenarios residents know are common. But, no matter what they do they will miss some. Blue Steel is a good number of regions. It should be a good test. I just which they emailed or IM’d those in the regions to let them know what to watch for. But, too many of the residents are ignorant of the computer tech needed to understand. So, that would likely result in a flood of Chicken Little responses.
Pingback: #SL News Week 21