You probably remember that on the first of the month I posted Second Life Chat Load Test. That test was carried out last week. It was hoped we would see some of the numbers this week. That didn’t happen but we did get some of the story from Oskar Linden in the Beta Server Group meeting. I also posted #SL Server Follow Up Week 23, which has a section on chat. That article explains the inefficiencies in group chat.
The Test
Three regions were involved. One running the main grid server-trunk version (Oatmeal ) and the other two running the a test version with improved chat software. Also, the Utility Servers received new software weeks prior to the test. Nobody told me. But, Oskar just officially announced it here: UTIL hosts updated with Group Chat fixes. The test was pretty well covered in blogs, forum, and group notices. The turnout was good and the Lindens collected lots of good stuff… data. Oskar Linden said, “It was a huge success from a QA POV.”
The main things being tested were backward compatibility – old grid software to new grid software, multi-region chats, and large in region group chats.
What was Learned?
The changes worked pretty much as expected. Quoting Oskar:
Overall the findings were about what we had estimated. Group chat will work better in some situations, but it won’t work better in all situations.
The chat lag is a matter of the number of people in a group that are chatting or at least listening to the chat. No surprise. The number participating in the group chat in a given region is another big factor. So, 1,000 chatters in a region are unlikely to cause lag. A 1,000 chatters in 1,000 regions will cause chat lag.
The new upgrades to Utility Servers and simulators are referred to as Fast Chat. The surprising problem is that viewers are not keeping up with fast chat, which is why the changes to the Utility Servers and the simulators are not going to give the level of improvement we want. So, those testing the fast chat were crashing.
What Happens Now?
Step 1 was to update the Utility Servers on AGNI grid with the updated code. The Utility Servers got the newer updates 6/9. The next step is to roll out simulator changes to a Release Channel next Wednesday.
Then throttles and caps will be added to the software. This will prevent griefers overloading a target and crashing their viewer. So, no super-spamming.
The viewer then has to be enhanced to carry the faster chat. That request is already in cue.
Will We Notice a Change?
May be. The change will be there. The problems will be reduced. But, having only half the chat posts dropped as before is not an overwhelming improvement, from the user’s perspective. Technologically, it is a big improvement. Having a better understanding of the process and bottlenecks will allow them to create effective improvements and we should see those changes coming along.
There is more to come. The Lab is committed to fixing chat and getting it working well.
One of the things we can do in the mean time is to close off group chats, if we aren’t listening. The new SLV2 feature of having groups down in the lower right corner probably allows more of us to let them remain open. I am guessing more people are just having them sitting open but not really paying attention. That could create a large load on the chat server for no purpose. Is there really anything else that has changed in chat since SLV2 came out?
FYI
The test was in more than 3 regions. There were about 15 oatmeal regions, and 30 or so group chat test regions.
Thx. I appreciate the correction.
Yes, my viewer was locking for 7 minutes between frame refreshes during the heavy load testing, and took quite a bit of time after to catch up. There’s already noticeable improvement from what I’ve seen since 6/9, can’t wait for the other improvements, the stress testing was impressive after so long is dead… dog… slow groups.
Phoenix/Firestorm Viewer Support group chat (very big group, very active chat) has observed a rather dramatic improvement in posting lag/dropped calls since ~Friday.
It seems that not everyone is seeing this improvement, but I do not see a pattern so far.
LL has done something without telling us, and for once it is something good.
When I was helping with the testing and got “stuck” (i.e., felt the affect on my avatar in the form of lagging) I found that closing the group chat window and re-opening it snapped me right back to full speed.
And yes, sometimes people forget that everything contributes to lag (even just opening profiles which requires fetching of the info, the picture, the links, etc.), so the less you have going on that you aren’t actually using the better. Closing chat windows that are not in use is a good suggestion. UNFORTUNATELY for all those using Viewer 2 it’s not so apparent they are open as they minimize to unobtrusive little gray boxes in the lower right corner.
So any Linden listening, make them not so hidey. A brighter color might be a good plan, and even a slow blink. 😉