Xbox LIVE Indie Games
Sort Discussions: Previous Discussion Next Discussion
Page 1 of 1 (24 posts)

PacketWriter boxes values?

Last post 7/25/2009 11:10 AM by Manly Games. 23 replies.
  • 6/19/2008 2:43 PM

    PacketWriter boxes values?

    Hi,

     I just spent the entire day profiling my application trying to figure out why I'm getting 13000 - 20000 boxed values per second on the xbox in multiplayer. After a lot of work on this it seems the culprit is PacketWriter. This is what the code looks like:

    public override void Serialise(PacketWriter writer) 
            { 
                header.Serialise(writer); 
                if (Application.GetGamePadState(PlayerIndex.One).IsButtonUp(Microsoft.Xna.Framework.Input.Buttons.DPadDown)) 
                { 
                    VectorWriter.Write3(writer, ref position); 
                    QuaternionWriter.Write(writer, ref rotation); 
                    VectorWriter.Write3(writer, ref direction); 
                    writer.Write(velocity); 
                    writer.Write(health); 
                    writer.Write(rage); 
                    writer.Write(stamina);    
                } 
            } 

    As you can see a lot of code serialization doesn't happen then DPadDown is pressed. When that happens the boxing goes down to a level of about 1000 - 5000 boxings per second. Any ideas as to why this could be happening? I thought it might be VectorWriter/QuaternionWriter because they do a 'ref' on the values but when I changed it, no difference was seen in the remote perf mon. Any ideas?

    Regards,
    Pontus

  • 6/19/2008 3:08 PM In reply to

    Re: PacketWriter boxes values?

    This code is rather old from back when we were using System.Net networking capabilities on windows (before XNA 2.0) so we had to write our own Vector and Quaternion serialization. I just updated the code to use the build in ones from the XNA framework, it now looks like this:
    public override void Serialise(PacketWriter writer) 
            { 
                header.Serialise(writer); 
                if (Application.GetGamePadState(PlayerIndex.One).IsButtonUp(Microsoft.Xna.Framework.Input.Buttons.DPadDown)) 
                { 
                    writer.Write(position); 
                    writer.Write(rotation); 
                    writer.Write(direction); 
                    writer.Write(velocity); 
                    writer.Write(health); 
                    writer.Write(rage); 
                    writer.Write(stamina);    
                } 
            }           

     

    The boxed values are all the same after the fix so this is still an issue. I've been scratching my own head on this forever now and I could really use a second opinion here.

    Regards,
    Pontus

  • 6/19/2008 3:39 PM In reply to

    Re: PacketWriter boxes values?

    Do you have an up-to-date Windows build of the game?  If so, I would use CLR Profiler to help you determine exactly where the boxing is occurring.  Using the Xbox RPM tool, you need to do some creative debugging to see where your boxing is occurring and this is not always very robust.  I am not seeing anything that should be causing boxing in the code you posted, which makes me believe the culprit may actually be elsewhere.  Chances are the packet writers/readers on Xbox need to flip bytes to handle endian issues, but I don't see why that would cause boxing.
  • 6/19/2008 3:53 PM In reply to

    Re: PacketWriter boxes values?

    I'm pretty confident that our PacketWriter implementation does not create any garbage.

    It's definitely boxed values you are seeing, and not just allocations? If you're seeing a lot of allocations from sending network packets, the usual cause is that you might not be reading packets as fast as they are coming in, so the incoming packet queue is just growing forever (that can be caused by forgetting to loop for as long as IsDataAvailable is true, or if you forget to read packets for one or more of the local network gamers).

    The only other thing I can suggest is to use binary-search to narrow things down further. Take out each line one by one, and watch how the performance counters change in response.

    The tricky thing here is to make sure that no other part of the program could be affecting your measurements. For instance, does pressing this gamepad button also change behaviors somewhere in your update code? By sending or not sending this extra data, could that be altering the behavior of your packet reader on the other end, or some update logic further down the line, which might actually be creating the garbage?

    Also, what happens in your header.Serialize call?

  • 6/19/2008 5:19 PM In reply to

    Re: PacketWriter boxes values?

    ShawMishrak:
    Do you have an up-to-date Windows build of the game?  If so, I would use CLR Profiler to help you determine exactly where the boxing is occurring.
    Sorry I forgot to mention that I have done this quite extensively but at the moment I am at a point where on pc the garbage is almost never collected during runtime but on the xbox the garbage gets collected around once every three seconds. So I'm thinking that something differs between xbox and pc which led me to try the rpm which is saying that I get around 13 000 - 20 000 boxed values each second. This might be one of the big issues.

     

    Shawn Hargreaves:
    It's definitely boxed values you are seeing, and not just allocations?

    Well unless the RPM fools me, I'm looking at the boxed values delta and I see it change significantly when I press DPadDown. Anyway allocations shouldn't be occurring here either right? Atleast not managed allocations?

    Shawn Hargreaves:
    If you're seeing a lot of allocations from sending network packets, the usual cause is that you might not be reading packets as fast as they are coming in, so the incoming packet queue is just growing forever (that can be caused by forgetting to loop for as long as IsDataAvailable is true, or if you forget to read packets for one or more of the local network gamers).
    Unless I'm missing something, that I exactly what I'm doing, the code is posted below if you wan't to look at it:

            public void Recieve() 
            { 
                NetworkSession session = NetworkEndPoint.Instance.NetworkSession; 
                if (session == null
                    return
                     
                foreach (LocalNetworkGamer gamer in session.LocalGamers) 
                { 
                    // Keep reading while packets are available. 
                    while (gamer.IsDataAvailable) 
                    { 
    #if !XBOX 
                        //Shortfuse.Engine.Script.Console.Log("Data available for " + gamer.Gamertag, Microsoft.Xna.Framework.Graphics.Color.White); 
    #endif 
                        NetworkGamer sender; 
     
                        // Read a single packet. 
                        gamer.ReceiveData(packetReader, out sender); 
    #if !XBOX 
                        //Shortfuse.Engine.Script.Console.Log("Data came from " + sender.Gamertag, Microsoft.Xna.Framework.Graphics.Color.White); 
    #endif 
                        if (sender != null && !sender.IsLocal) 
                        { 
    #if !XBOX 
                            //Shortfuse.Engine.Script.Console.Log("Sender is not local", Microsoft.Xna.Framework.Graphics.Color.White); 
    #endif 
                            byte packageType = packetReader.ReadByte(); 
                            if(packageType == (byte)PackageType.EVENT) 
                            { 
                                Event e = EventFactory.Construct(packetReader); 
                                EventManager.Instance.PostEvent(e, false); 
                                EventHeader header = e.Header; 
                                header.Sender = sender.Machine; 
                                e.Header = header; 

    #if !XBOX 
                                //Shortfuse.Engine.Script.Console.Log("Data is an Event of type " + e.GetType().Name, Microsoft.Xna.Framework.Graphics.Color.White); 
    #endif 
                            } 
     
                            else 
                            { 
                                List<byte> data = dataDictionary[sender]; 
                                data.Add(packageType); 
                                data.AddRange(packetReader.ReadBytes(packetReader.Length - packetReader.Position)); 
                            } 
                             
                        } 
                    } 
                } 
            } 

     Also note that this function gets executed once every frame and the frame rate right now is at ~25 in the current setup I'm using.

     

    Shawn Hargreaves:
    The only other thing I can suggest is to use binary-search to narrow things down further. Take out each line one by one, and watch how the performance counters change in response.
    Yes that's exactly what I've been doing all day and tracing it down led me to Serialize on the particular event type you see in the OP.

    Shawn Hargreaves:
    For instance, does pressing this gamepad button also change behaviors somewhere in your update code?

    Well that's the thing, I chose this button because it isn't used anywhere else, but just to be safe I did a "Find all references" on it and it was used in two places, the first is the one you see above. The second is one level down in the call stack where I say "if I didn't write all data for this event, then don't send it". The code looks like this:

    public void Dispatch(Event eventToDispatch) 
            { 
                NetworkSession session = NetworkEndPoint.Instance.NetworkSession; 
                if (session != null && session.LocalGamers.Count > 0) 
                { 
                    LocalNetworkGamer sender = session.LocalGamers[0]; 
                    for(int i = 0; i < machines.Count; ++i) 
                    { 
                        NetworkMachine machine = machines[i]; 
     
                        if (machine.Gamers.Count > 0) 
                        { 
                            packetWriter.Write((byte)PackageType.EVENT); 
                             
                            //----ATTENTION---- Possible culprite here!!!! 
                            eventToDispatch.Serialise(packetWriter); 
                            //----ATTENTION---- Possible culprite here!!!! 
     
                            NetworkGamer recipient = FindFirstGamer(machine); 
     
                            SendDataOptions sendOptions = SendDataOptions.None; 
                            sendOptions = eventToDispatch.Header.Reliability; 
     
                            if (Application.GetGamePadState(Microsoft.Xna.Framework.PlayerIndex.One).IsButtonUp(Microsoft.Xna.Framework.Input.Buttons.DPadDown)) 
                                if (recipient != null
                                    sender.SendData(packetWriter, sendOptions, recipient); 
                             
                        } 
                    } 
                } 
            } 
    So what this tells me is that if I DON'T press down DPadDown everyting gets executed as usual and I get around 16 000 boxed values/second. When I DO press DPadDown I only serialize the header (seen in the OP as header.Serialize()) and nothing else (well a byte flag but then really nothing else). I also don't send the data when the button is pressed (because this is multiplayer and I don't want the client to crash during the profile) and I get around 4000 boxed values when the button is pressed.

     To make sure that it really is the serialization and not the sending that's doing the boxing I tried doing all serialization and not send but I got 16000 boxed values again so it can't be the sending.

  • 8/15/2008 4:04 PM In reply to

    Re: PacketWriter boxes values?

    Hi again, so I've come back to this problem and gone a bit further and found the actual issue. It seems that BinaryWriter.Write(float) needs to box values (I guess in order to convert into a serialisable format) I'm not sure who to contact about this since it's an implementation of the Compact .Net framework of the Xbox 360.

    I made a test project which lets you write floating point numbers when holding down A and integers when holding down B to a PacketWriter. It's clear (by viewing from the Remote performance monitor) that no boxed values are generated when writing integers but when floating point numbers gets written on the other hand, lots of values gets boxed. Please deploy the program to your xbox and try the program through the Remote performance monitor to see for yourself.

    This is an issue that affects everyone that are making a network game and are worried about the garbage collector so I'm hoping for a solution to come from this. Maybe this could be fixed without boxing values, but I'm not sure how really. Who do I contact to suggest a fix? Is there a way to convert a float into byte format in C# that does not produce boxing?

    PacketWriterBoxingTest.zip

    Regards

  • 8/15/2008 4:56 PM In reply to

    Re: PacketWriter boxes values?

    You can work around that by writing an unsafe function that turns a float into an int, and write that.

      public static unsafe FloatToInt(float f, ref int i) {
        i = *(int *)&f;
      }

      public static unsafe IntToFloat(int i, ref float f) {
        f = *(float *)&i;
      }

     


  • 8/15/2008 5:02 PM In reply to

    Re: PacketWriter boxes values?

    Thanks for the quick reply!

    I think that's what's happening inside BinaryWriter.Write(float) (which I guess is the source of the boxing). Also, is it possible to run code compiled with /usafe turned on the xbox?

  • 8/18/2008 11:24 AM In reply to

    Re: PacketWriter boxes values?

    Fancy that, it worked! In a platonic sense I love you! I just pasted in the code, made som refactoring, allowed unsafe code and whopedidoo 5000 down to 60 boxed value types to per second. Practically this meant that the GC got invoked once per 66 seconds instead of once per 9 seconds....quite the difference!

    The only thing that worries me is the unsafe part. Is this really platform independent code? Isn't this endian dependent? Does it matter? Will Microsoft allow unsafe code to be run on the xbox in the end product?

    Anyway thanks alot for the help.


  • 8/18/2008 5:15 PM In reply to

    Re: PacketWriter boxes values?

    There's no issue with unsafe code on Xbox or otherwise, even on XBLCG. Not sure about endian issues, but I suspect it's all pretty well handled behind the scenes.
  • 8/18/2008 5:26 PM In reply to

    Re: PacketWriter boxes values?

    Alright cool, then this issue is pretty much taken care of. Thanks alot!
  • 8/18/2008 6:04 PM In reply to

    Re: PacketWriter boxes values?

    HerrUppoHoppa:
    Hi again, so I've come back to this problem and gone a bit further and found the actual issue. It seems that BinaryWriter.Write(float) needs to box values (I guess in order to convert into a serialisable format) I'm not sure who to contact about this since it's an implementation of the Compact .Net framework of the Xbox 360.

    I made a test project which lets you write floating point numbers when holding down A and integers when holding down B to a PacketWriter. It's clear (by viewing from the Remote performance monitor) that no boxed values are generated when writing integers but when floating point numbers gets written on the other hand, lots of values gets boxed. Please deploy the program to your xbox and try the program through the Remote performance monitor to see for yourself.

    This is an issue that affects everyone that are making a network game and are worried about the garbage collector so I'm hoping for a solution to come from this. Maybe this could be fixed without boxing values, but I'm not sure how really. Who do I contact to suggest a fix? Is there a way to convert a float into byte format in C# that does not produce boxing?

    PacketWriterBoxingTest.zip

    Regards

    For any issues using XNA Game Studio, regardless of whether it's potentially a .NET CF issue, you can file bugs on our connect site. Could you attach that zip file to the bug as well, and reference this thread?

    Thanks!

  • 8/18/2008 6:37 PM In reply to

    Re: PacketWriter boxes values?

    I am at a point where on pc the garbage is almost never collected during runtime but on the xbox the garbage gets collected around once every three seconds.

     

    Note that the desktop CLR collects much less often than the Xbox CLR, because it's a much better collector. The Xbox CLR does a stop-and-collect for every megabyte of memory that's allocated, so if you have any kind of heap usage, it will stop and collect quite often.

    You should design your game so that there are ZERO dynamically allocated objects during normal play. That's the only way to ensure a consistent gameplay experience. Usually, that's quite possible, by pre-allocating buffers, using fixed-size arrays with a "used elements" counter, etc.

  • 8/19/2008 8:51 AM In reply to

    Re: PacketWriter boxes values?

    Eli Tayrien:
    For any issues using XNA Game Studio, regardless of whether it's potentially a .NET CF issue, you can file bugs on our connect site. Could you attach that zip file to the bug as well, and reference this thread?
    Done! Here's the link.

     

  • 8/19/2008 10:49 AM In reply to

    Re: PacketWriter boxes values?

    jwatte:
    You should design your game so that there are ZERO dynamically allocated objects during normal play. That's the only way to ensure a consistent gameplay experience. Usually, that's quite possible, by pre-allocating buffers, using fixed-size arrays with a "used elements" counter, etc.
    Yes I've done some work pooling events and such and gotten down the garbage to almost nothing. What garbage remains comes from using reflection at run time to detect event type which is something I'll have to solve using less pretty type detecting mechanisms when time comes.
  • 6/29/2009 5:55 AM In reply to

    Re: PacketWriter boxes values?

    HerrUppoHoppa:
    jwatte:
    You should design your game so that there are ZERO dynamically allocated objects during normal play. That's the only way to ensure a consistent gameplay experience. Usually, that's quite possible, by pre-allocating buffers, using fixed-size arrays with a "used elements" counter, etc.
    Yes I've done some work pooling events and such and gotten down the garbage to almost nothing. What garbage remains comes from using reflection at run time to detect event type which is something I'll have to solve using less pretty type detecting mechanisms when time comes.


    Can you tell me what exactly the problem was? I have got the same problem. Without networking my garbage is zero but when turned on and enough local players force the console to write a lot of packets the GC-party begins :)
  • 6/29/2009 8:36 PM In reply to

    Re: PacketWriter boxes values?

    The float overload of BinaryWriter.Write() does some boxing on the 360 which leads to collections. jwatte was kind enough to write out an alternative way of writing to a stream. There isn't much more to say about it but if you have any questions, feel free to ask.

    Regards
  • 6/30/2009 3:20 PM In reply to

    Re: PacketWriter boxes values?

    This was an issue in old Game Studio versions, but was fixed some time ago.
  • 6/30/2009 4:56 PM In reply to

    Re: PacketWriter boxes values?

    Good to know, I never got a message saying it was resolved so I figured it wasn't. Ignore this problem then, unless you're using an old game studio version.

    Regards
  • 7/23/2009 5:05 PM In reply to

    Re: PacketWriter boxes values?

    Shawn Hargreaves:
    This was an issue in old Game Studio versions, but was fixed some time ago.


    Actually, it seems to be unfixed. I just recently noticed a ton of boxed values which I tracked down to binarywriters outputting floats. The boxing only occurs on the Xbox, nothing shows up in the CLR profiler. Here's some reproduction code:

    MemoryStream stream = new MemoryStream(4); 
    BinaryWriter writer = new BinaryWriter(stream); 
     
    float v = 0; 
    while (true
        writer.Write(v); // Boxes values like crazy; change v to be an int and it will not box values anymore. 
        writer.Seek(0, SeekOrigin.Begin); 

  • 7/24/2009 3:15 PM In reply to

    Re: PacketWriter boxes values?

    Are you running XNA 3.1?
  • 7/24/2009 3:33 PM In reply to

    Re: PacketWriter boxes values?

    Answer
    Reply Quote
    BinaryWriter != PacketWriter.

    PacketWriter does not box floats.
  • 7/24/2009 3:34 PM In reply to

    Re: PacketWriter boxes values?

    How are you determining it is boxing, out of curiosity?
  • 7/25/2009 11:10 AM In reply to

    Re: PacketWriter boxes values?

    dadoo Games:
    How are you determining it is boxing, out of curiosity?



    By hooking up the remote performance monitor to an Xbox. Very handy tool.
Page 1 of 1 (24 posts) Previous Discussion Next Discussion
var gDomain='m.webtrends.com'; var gDcsId='dcschd84w10000w4lw9hcqmsz_8n3x'; var gTrackEvents=1; var gFpc='WT_FPC'; /*<\/scr"+"ipt>");} /*]]>*/
DCSIMG