Scripting & Server News Week 42

For the release candidate on Le Tigre we have some new information that comes from the wiki release notes just recently updated. Some of those are:

Havok Update

The Havok Physics Engine updates to version 2012.1 in the release candidate. In week 35 I reported that this version was coming. At the time we had just updated to 2010.x, I think. This update enables Havok’s terrain optimized that simplifies the physics shape of the terrain for improved performance. This feature can be disabled by region owners and estate managers with the Region Debug Console command “set optimize_terrain false“.

Havok Physics

Some are reporting vehicle crossing are slower.

llGetSimStats

llGetSimStats() has yet to show up in the Linden Scripting Language (LSL)  pages of the wiki. Fortunately the release notes say:

Currently, this function accepts only [[SIM_STAT_PCT_CHARS_STEPPED]] as its parameter, which returns the % of Pathfinding characters skipped each frame, averaged over the last minute.  The returned value corresponds to the “Characters Updated” stat in the viewer’s Statistics Bar.

Continue reading

#SL News 1 Week 42

This Tuesday we got a roll to the main channel. This is the package that was on Blue Steel. This is the code to improve performance for Large Groups. It is not the Large Group Editing changes that Baker has made. Those changes are in a package still stuck in QA.

It has been quite since the roll out. That usually means things are working as expected.

Sever-Scripting Meeting September 2012

Magnum

Is going to continue to run the same package it has had for the last couple of weeks. This is a package that adapts the simulator to the new hardware. It contains various configuration changes for the Denebian OS running in the region servers. I have no information whether this is identical to last week or some configuration settings are changing. Whatever the case, it is expected users won’t see any difference.

Continue reading

#SL News 3 Week 41

Server Updates

This last week we did get the roll to the main grid and a couple of small release candidates made their way to the RC channels.

The main channel got the package from Blue Steal as previously reported. That was a server crash fix package.

Server & Scripting UG Oct 2012

Blue Steel and Le Tigre got the same package, one that has some fixes for group SQL queries. These are not the Large Group Editing fixes. Those are still stuck in QA… I understand it is more the package they are in that is stuck rather than the actual group queries that are stuck. Simon Linden is NOT expecting the package with Large Group Editing to make it to an RC next week (42). Bummer.

Magnum continues to run last week’s (40) package, the backend infrastructure changes. The package that rolled to the main grid was added to this package. The combined packages will test this week.

The comments in the Deploys thread are few. So, it is probably safe to assume things are working well for everyone.
Continue reading

Sudden Massive Lag Update

This continues to be a problem for event managers. Toysoldier is getting more and more Lindens to look at the forum thread for BUG-355Increased Instant Sim Lag & Avi Crashes During Major Events – sim network issue. (Limited access)

Andrew did not realize people were putting in the tech info needed to actually find the problem in the logs. I hope he has realized and taken a look at the thread: Increase in Instant SIM LAG & Crashes During Larger Events – Network Source?

Simon has responded in the thread: M7808.

Thanks for the reports, Toy … there’s some good information there that seems to shows the simulator getting into trouble when there are network problems.

If you are seeing the problem in your travels through the grid, read the thread so you know what to report and add your information. Please do not add noise. Just saying, “I saw that. Me too.” doesn’t help and actually hurts the process.

Toysoldier asks:

Next time one of your sims encounters one of these major lag events…. take a snapshot of the Advanced Performance screen and post it here.  The important stats to look at are related to the NETWORK stats. 

What has been initially noticed (and we would like more evidence of it from stale sims) is that some event triggers the the sim or the server kernel’s network driver or something related to the network to not process network packets at a normal flow.  Even at a low avi count and with other metrics looking normal, the network stats show unusually high pending downloads and very high un-acked packets.

But, we need more examples (snapshots of the performance stats) of a sim when it has fallen into this state.  Also make sure you mention date/time and the name of the sim. 

The thread: Increase in Instant SIM LAG & Crashes During Larger Events – Network Source?

#SL News 2 Week 41

Viewer 3.4.x Problem

Oz Linden has given us some news on the crashing problem they have had with the Beta and Development viewer. The Lindens believe the current Beta viewer (3-4-1-265642) has the memory leak, or whatever, fixed. The crash rate is back to low rates.

For the techies, the problem had to do with how the cURL wrapper was threaded. The cURL thing has to do with computer network communication over networks. It is a programming library that makes it easier for programmers to implement communication protocols like HTTP and HTTPS. It helps with encrypting communications.

Wind Vectors Displayed – Red Lines

One more round of testing in the Beta viewer is in progress. Once completed the basic fix and changes will move to the main release viewer, likely next week (42).

It takes a couple of days of testing to collect enough data to make a determination on whether a fix is working or not. The current Beta Viewer was compiled on Saturday and released on Monday. If you have Auto-Update on, you got it before it was on the web site.

Continue reading

#SL Testers Needed Oct 11

Oskar Linden posted in the SL Forum requesting help testing some network changes. I suspect this is related to the Sudden Lag I’ve been writing about. See: Networking Optimization Pile On Test – October 11th

Linden Lab has made some changes to the way regions handle network traffic. We need your help to test. This will help us insure that regions are communicating appropriately over the network. Mainly we are concerned with agents entering via direct login, teleport, and region crossing. As well as other functions such as IMs, Voice, and Group chat.

Tomorrow, October 11th, at 4PM Pacific time (right after the server beta user group) we will conduct these tests with as many people as we have. Testing will take place on ADITI and require out of world communication we will be coordinating via IRC. This will require an IRC client connected to EFNET in the channel #sltest. If you don’t know how to do that you have until tomorrow afternoon to figure it out. 🙂

The details on the tests are here:

 – https://wiki.secondlife.com/wiki/Networking_Optimization_Pile_On_Tests

I hope you can come and help us test these new changes.

__Oskar

The test procedure is described in the SL Wiki. See: Networking Optimization Pile On Tests.

These tests will be on the ADITI grid.

#SL News 1 Week 41

There was a roll to the main channel this week. It seems to have gone well. At least all the regions get a restart after 3 weeks or so. The package was the one running on the Blue Steel RC Channel. It is a minor update and should have no visible effect on the simulator behavior. This is a release needed to enable a coming release.

Server & Scripting UG Oct 2012

Magnum

Will get a new package. It will be the main channel code plus the changes to allow better use of the new hardware. Not knowing exactly what hardware the Lab is upgrading to… its hard to speculate on what the change may be. Presumably, if they are running Intel, they may be adding more threading features for the iCore series chips. If they run AMD, I have no clue. Continue reading

Sudden Lag on the Radar

Toysoldier Thor has been relentlessly tracking down the causes of his in-world event crashes and sudden massive lag. Region owners and event managers have been collecting and providing data on the problem in the forum. See my previous article: New Lag Problem. See the forum thread here: Major Increase in Instant SIM LAG & Crash during large events.

Simon Linden acknowledged the effort and is starting to get people on the trail of the problem.  From Toysoldier:

From Simon Linden…
” Thanks for the reports, Toy … there’s some good information there that seems to shows the simulator getting into trouble when there are network problems. ”

I think now that we have a possible smoking gun… the LL staff can focus their investigation on the NETWORK.

Having more sim /region owners post the same performance statistics during one of these extreme lag events would help us further confirm the root cause.

If you are seeing this problem, read the thread and post the requested information from your occurrence of the event. Filing a JIRA works too.