condition between the omgr thread and the conmgr thread. Now when the omgr
thread processes an event that is going out to the clients, it flattents
the message itself for each client that is to receive the message and the
flattened data is posted to the conmgr outgoing queue.
This means that once an event is finished processing, no further
modifications to any of the data associated with the event can effect the
data queued up to be sent to the client. This is a good thing, it will
eliminate or illuminate a very baffling class of bugs that we've sort of
been ignoring because we knew this could be the cause.
We used to take an event and flatten it directly into the direct buffer
from which we would do our socket write. Now we flatten it into a
temporary byte array. This means a metric shitload more garbage generation
and collection. We used to do the flattening on the conmgr thread, now we
do it on the omgr thread. This means a big redistribution of CPU demand.
Either of those things could result in a significant negative impact on
our performance, but we'll just have to deploy this stuff and find out.
Whee! If it turns out to be a serious problem, there are potential
optimizations that could be done by keeping a pool of direct buffers
around and flattening messages into them, relying on the fact that the
outgoing conmgr queue generally doesn't grow too large and we could
allocate tens to a hundred megabytes of memory for the outgoing queue if
we really needed to.
I'd also like to test the overflow handling stuff more. It didn't really
change in that everything just deals with arrays of bytes now instead of
unflattened messages, but I'll be more comfortable once I've seen all this
in action on ice where there may be few users, but they are just as likely
to experience lag and receive an overflow queue as users on the higher
traffic servers. There is code to log when overflow queues are created and
finally flushed and how much use they got while they were around, so that
should give us an indication of whether things are operating properly.
git-svn-id: svn+ssh://src.earth.threerings.net/narya/trunk@3419 542714f4-19e9-0310-aa3c-eee0fc999fb1
Michael Bayne
a285695ca8