Server Updates
This last week we did get the roll to the main grid and a couple of small release candidates made their way to the RC channels.
The main channel got the package from Blue Steal as previously reported. That was a server crash fix package.
Blue Steel and Le Tigre got the same package, one that has some fixes for group SQL queries. These are not the Large Group Editing fixes. Those are still stuck in QA… I understand it is more the package they are in that is stuck rather than the actual group queries that are stuck. Simon Linden is NOT expecting the package with Large Group Editing to make it to an RC next week (42). Bummer.
Magnum continues to run last week’s (40) package, the backend infrastructure changes. The package that rolled to the main grid was added to this package. The combined packages will test this week.
The comments in the Deploys thread are few. So, it is probably safe to assume things are working well for everyone.
Future
Oskar Linden tells us there are packages in the queue that did NOT make it through QA and were held back from the RC channels. I presume this is the reason a small package is running on both Blue Steel and Le Tigre.
Network Test
Thursday’s meeting included a test Oskar was running on server network response. It is unclear if this is in response to Toysoldier’s flagging what I’ve labeled Sudden Massive Lag. Whatever, the ADITI grid went down and meeting attendees were dropped from the servers. It was a certificate problem. The server certificates expired during the meeting. They did get it fixed and resumed the meeting and made the test.
The meeting transcripts have yet to be posted, so I’m not sure how many made it back after certificate crash. I was showering and heading off to the VP debate. Inara Pey has information on the test in her article: Piling it on: the network optimization tests. She has a different idea of ‘huge’ than I do. Whatever, her take is the network performed better as she did not see the lag she associates with a high number of avatars coming into a region.
Some time ago the Lab made changes to reduce the region lag from incoming avatars. Since then the worst lag has been generated by avatars leaving a region. There is still room for improving the incoming avatar lag. So, Inara may be seeing a real improvement on the incoming side. We can hope.
Lately I’ve been arriving at events early and watching lag as others come in. They create way less lag than they did some months ago. Also, avatars leaving don’t seem to create the high lag spikes I once saw. But, I do find regions horribly lagged for no apparent reason. There are too few avatars to cause a problem.
Whether I was seeing the Sudden Massive Lag Toysoldier and event managers are complaining about or what is sometimes referred to a ‘stale’ servers, I can’t know from the user side of things.
I haven’t heard the Lindens talking much about the region crossing code. A Phase I change was made and the Phase II work was shelved while other infrastructure changes were being made. I have not heard any news lately on the ‘Multi-Threaded Region Crossing Code’. So, I’ve no idea where that project is.
We’ll probably hear more about the network tests next week (42).
Wind
I wrote in an earlier article about Average and local wind. See: SL News 2 Week 41. Andrew Linden has been working on SL Wind. As of Friday the changes have some instability Andrew is working to fix.
Interest List
This is the main project Andrew Linden is working on. Wind is sort of an associated project as they use the same communication channel. He has made some performance improvement but says it is not working as well as he wants. You can check it out on region Ahern, ADITI.
While it is pretty much all-around faster there are some things that should rez that don’t.
There is also a camera bug that is causing problems with the list. If you teleport in to a spot underground… your camera may be “broken” (cannot be moved or updated). This can happen if you come from a low altitude in the departing region and arrive in a region with terrain at a higher altitude. Normally the server detects the conflict and moves the avatar along with the camera to the surface. For some reason the camera is not moving with the avatar.
Andrew was running performance tests on the new interest list code. In one test in a mostly empty region with about 30 avatars running around the interest list was a slight bit worse in performance, about 5% worse. In another test where the crowd of avatars was NOT looking at a pile of dynamic objects behind them the new code showed about 40% improvement (less time spent running the interest list).
Possible Change
I found it interesting to hear Andrew explaining:
… in this case the “camera” I’m talking about is the server’s notion of it… which is used to figure out what objects to send to the viewer or not. The glitches in camera movement you’re (meeting attendee) talking about are viewer bugs.
The viewer tells the server where the camera is and the server obeys… up to some distance. The server will not accept camera positions that are too far from the avatar. It will clamp the distance.
Not in the old (currently on agni) code… the viewer can specify an arbitrarily far camera however… the viewer won’t actually use that location… it will slave to the agent’s position instead.
…
…if you move laterally then the object detail will degrade. In the new code… the object detail will follow the camera (not the avatar), as long as you stick within about 128 meters of the avatar. Beyond that the object detail will degrade.
…that’s the distance I’m using right now. Whatever it is… I’d like to keep it small enough so that your camera can’t wander into regions that are more than one neighbor region away.
It seems the server will restrict horizontal distance more than vertical. So, one can cam up or down and see things render. Cam’ing horizontally is more restricted. So, while you can cam vertically 1,500 meters and see things render, the server is not going to provide information for renders 700 meters away horizontally.
I am not sure how much the new interest list code is change affect our behavior. I know that lots of stuff does not render as I cam out horizontally now. And what does takes some time. It sounds like the interest list change will change and be more restrictive lateral-distance-wise, but provide faster nearby rendering.
In a practical sense, getting a 40% performance increase and faster rendering for nearby cam’ing at the expense of a more restrictive lateral cam’ing distance is a good thing in my mind. I usually use what I think of as medium distance caming when I am shopping, inside a store. I think this change is going to be improvement. But… we will see.
Ghost Objects
One of the things that happens in regions is things get deleted. However, the region server does not always get the message. So, some deleted items remain as ghost objects. One of the things Andrew says about that:
I did a minor adjustment there… in the first iteration the server will be sending KillObject messages for ALL objects that are deleted, not just those that are in view, which will help, but only a little bit.
Teleport Bug
Baker’s work on the Large Group Edit is pretty much done. Its release is hung up in QA because of other items in the package its bundled with. He has been assigned a teleport problem to fix while he waits for the Group code to make it to an RC. He puts it this way:
My news is that I started working on figuring out a bug with teleports. I’ve been investigating it for a week and a half now, and I may have found a way to repro it consistently. I’m not sure if it’s something I can fix, but I’ll at least figure out why it’s happening.
New Hardware
Some weeks ago the Lindens announced they were doing hardware upgrades. Simon and Andrew said a little about the Second Life hardware this last Friday.
There are more CPU cores, and the ones there are faster. The servers are not running Virtual Machines. They are basically Debian boxes with some server processes running.
Just an FYI
Oskar made it clear ahead of the tests that they had nothing to do with the main grid lag issues.
WRT teleports, while much has been done to do away with the large-scale viewer freezes which used to occur, ny experience has still be that when people enter / leave a crowded location, the viewer can “shutter” or suffer a “mini-freeze” – animation cycles are missed so that an avatar’s walk or motion “jumps”, rather than continuing smoothly. For me, this was absent from the Aditi tests, hence the comment.
Thanks for the information.