Xbox LIVE Indie Games
Sort Discussions: Previous Discussion Next Discussion
Page 1 of 2 (26 posts) 1 2 Next >

Xen 1.5 released: New GPU Particle System

Last post 3/19/2009 4:37 AM by guysherman.com. 25 replies.
  • 3/9/2009 12:06 PM

    Xen 1.5 released: New GPU Particle System

    Hi everyone. A few days ago I released a new version of xen, my long term project to make a simple, robust graphics API for XNA
    I've been working on xen for well over a year now, and this is the biggest release I've made.

    I don't normally advertise directly like this, but it's at the stage where I'm really happy with the api - and I'd really appreciate if some of you decide to take the time and have a look. I greatly appreciate any and all feedback :-)

    You can download it from here (2mb with source):
    www.codeplex.com/xen
    It's fully documented and includes 25 tutorials.

    -------------------
    A bit of background:

    There are two parts to Xen, the core Xen library, and Xen.Ex (which is the 'extensions and experiments' library).

    If you have ever struggled with Vertex/Index Buffers, Vertex Declarations, Render Target management, Render State problems, integrating shaders or higher level things like camera control - then Xen may save a lot of effort.

    For example, Xen generates vertex declarations itself (internally using reflection - details) and shader bindings are generated by an Visual Studio plugin that automatically creates strongly typed classes for each shader (Shader Parameters are generated as class properties of their native type - details)


    Xen.Ex is where I've had a lot of fun, and where the biggest changes occured for 1.5.

    Three main features of Ex are the Model importer, the 'material shader' and the new particle system.
    - The model system is a high performance model drawer that supports animations (thread aware).
    - The MaterialShader is similar to a BasicEffect, supporting upto 10 lights, animation, instancing, etc :-)

    - And finally, the new Particle System.
    Here I feel I've struck a good balance of flexibility and performance. The system is programmable - using XML. It supports hotloading on the PC, and if you really push it, can managed 1,000,000 particles on the xbox at well over 30fps (The xbox is a particle monster :)

    ---------------

    I have a couple of videos on youtube:
    http://www.youtube.com/watch?v=7wWsYTYGX78 (The particle system, HD)
    http://www.youtube.com/watch?v=rII5iRLN_lw (This video is a lot older - it shows 500 animated actors running on the xbox at 20fps)

    The particle system in action on the xbox (screenshots):


    ----------------

    If you do give it ago, I hope you enjoy it
    Thankyou! :-)
  • 3/9/2009 7:17 PM In reply to

    Re: Xen 1.5 :-)

    The videos were pretty impressive. Definitely worth a bump.
  • 3/9/2009 8:52 PM In reply to

    Re: Xen 1.5 :-)

    video are really impressive. I4m going to test it right now but do you have any roadmap or plan for the future ? I don't know if I'll be able to integrated it to my engine but I was wondering if your library have any limitations of use.

    cheers
  • 3/10/2009 7:45 AM In reply to

    Re: Xen 1.5 :-)

    Akai kaze:
    video are really impressive. I4m going to test it right now but do you have any roadmap or plan for the future ? I don't know if I'll be able to integrated it to my engine but I was wondering if your library have any limitations of use.

    cheers


    Thanks for the positive comments :-)

    I'd like to know how you get on. As you may have already discovered, xen isn't something that can be slotted alongside an existing codebase of typical XNA code. This design was intentional, as it allows much greater control of many aspects of rendering, which (I feel) results in a cleaner API where fewer things can go wrong. This isn't going to work for everyone, and I understand that.
    Very early versions of xen (over a year ago) were built to mix with general XNA code, but I found maintaining a peaceful coexistence was very demanding and imposed a lot of limitations.
    This is not to say you cannot use existing XNA code within xen - you can - but you need to be aware of the transitions between the two. However most existing XNA code should port over easily, and should end up simpler as a result.

    The roadmap right now is very speculative. Xen itself is practically unchanged from the first release - it's the new additions to Ex that have been the most significant. So in that respect, I don't see too many changes. I'll continue to look at ways to improve performance behind the scenes, but beyond that there is nothing set in stone.

    Thanks :-)

  • 3/11/2009 12:39 PM In reply to

    Re: Xen 1.5 :-)

    That stress test video is really impressive.
    Great stuff!
  • 3/11/2009 7:40 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Great job, looks like you put a lot of nice polishing work on this.  Hopefully I'll have some time to learn something from the tutorials.

    That's a nifty little performance gadget you got there, is that something part of Xen?
  • 3/12/2009 12:06 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Sad.  I got a host of errors.

    Looks quite impressive too.
  • 3/12/2009 12:33 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    I just take a  look and try to implement your library which is fact is not really a library. Doing such a complex particles system is indeed really hard with all the optimization but it make things really hard to integrate. I don't know for othe ruser but I'm really interested in Xen but the fact it has his own way of working makes it almost a game framework. If you are don't have any game framework it may be pretty cool but for people which already use gamescreen and such it could be really tricky. As you said it was meant to be this way so sad I think I wont be able to use it in my engine.

    nice job anyway.
  • 3/12/2009 12:43 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    SRH:
    Sad.  I got a host of errors.


    Did you run the "xen prebuild.bat" file?  It says to do that in the directions; after I ran the file it works fine.

    Some of the effects look simply amazing.  I really hope I can work this into my game.
  • 3/12/2009 12:49 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    It was running that file that I got the errors.  I tried running the tutorials anyway, but was unable to.
  • 3/12/2009 12:56 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    I agree that it looks very nice.  I was especially interested at looking into the particles stuff... which is something I don't know how to effectively implement.
  • 3/12/2009 11:01 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Thanks for the feedback guys :-) The stess test particle system was really fun to make - although I was worried the xbox was getting a bit too warm!

    SRH:
    Sad.  I got a host of errors.

    Looks quite impressive too.


    Sorry it's not working for you. Can you please send me the output from the prebuild, this will help me track things down.
    To do this, edit the prebuild.bat file and remove the two 'pause' (one near the start, one near the end) then run:

    "xen prebuild.bat" > log.txt

    In the command prompt (or another batch file). This will write everything out to a log file. If you could send me this log file, I'd greatly appreciate it!

    Akai kaze:
    I don't know for othe ruser but I'm really interested in Xen but the fact it has his own way of working makes it almost a game framework. If you are don't have any game framework it may be pretty cool but for people which already use gamescreen and such it could be really tricky. As you said it was meant to be this way so sad I think I wont be able to use it in my engine.


    Yes. I totally understand this won't work out for everyone, and if you have an existing project then chances are it's simply not an option. It was a concious decision on my part - and is why I call it an API alternative to XNA.
    There are tradeoffs either way (building on top of XNA or building around it) but I feel that I personally couldn't have done what I've managed without the approach I've taken. This decision wasn't taken lightly, and was made from direct experience with the WhaleWatch project (in the screen shots section), which was a commercial project I worked on, where we used an early version of xen (with typical XNA code and xen code was mixed and matched).


    charris:
    That's a nifty little performance gadget you got there, is that something part of Xen?

    Yes, it is (in Debug builds). This is one of the nice up sides to the code being lower level, it means I can keep track of a rather ludicrous number of statistics (from vertex bytes copied through to the average complexity of preshaders :). Of course, most people don't care for these :-) So I provide the essentials by default. It's amazing how useful the little bugger can be :-)

    The CPU usage is only on the xbox. When the stats are on screen, I spawn ultra low priority threads and thrash them, timing how much processing time they get :-) It's a nasty hack but it's a good indicator. The garbage collection graph simply keeps a WeakReference to an object(). It's helped me track down a couple of nasty garbage bugs in the particle system, which was good.


    Once again, thanks everyone :) I really appreciate it!
  • 3/12/2009 10:23 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Maybe I'm blind, but I haven't figured out how to PM for this forum...

    I've generated a log file.  What is your e-mail?
  • 3/13/2009 7:25 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    It seems this forum doesn't support PM. :-/

    If you have a codeplex account, you can send me a message through this page. Otherwise, you can easily use pastebin. I make an effort to never publically expose my email address for security, it's not I don't trust you - try putting your own email address into google (in quotes), and see what pops up :-)

    Thanks again!
  • 3/15/2009 2:20 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Great stuff SU.  I was really impressed with the performance you got with so many animated characters.

    While I have skeletal animation working with my collada files, I'm finding that I can only have about 4 characters at one time before my frame rates go to about 30. 

    How much of this was done outside of C# and XNA?  Meaning, are you really just relying on XNA objects (like Effect & VertexBuffers) and using them super-smartly or are you writing much lower level code?


    The thing that makes me hesitatant to use your dll's is simplay that I don't know the kind of plans or support you'll have going forward.  I know that Microsoft will support the core XNA objects or when people share raw code you can learn how it works and support it yourself.  With binaries, it's tougher because the user has such little visibility to the inner workings.  This is just a general comment.


    Anyway, very nice work.

  • 3/16/2009 2:41 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    tryz:

    The thing that makes me hesitatant to use your dll's is simplay that I don't know the kind of plans or support you'll have going forward.  I know that Microsoft will support the core XNA objects or when people share raw code you can learn how it works and support it yourself.  With binaries, it's tougher because the user has such little visibility to the inner workings.  This is just a general comment.


    I'm confused... the full source seems to be included in the zip when I downloaded the release.  I haven't tried to build it myself... but at first glance the solution and source seem to be all there.

    tryz:

    How much of this was done outside of C# and XNA?  Meaning, are you really just relying on XNA objects (like Effect & VertexBuffers) and using them super-smartly or are you writing much lower level code?


    To me, it looks like everything's done in C# & XNA.  I don't think there is any other way.  I was under the impression that attempting to use libs written in C++ / MC++ with XNA C# libs would fail miserably.  Am I wrong here?


    SU... Xen looks pretty neat.  I plan on checking it out a bit more as soon as I'm less busy.  Nice Work.

    -Rick
  • 3/16/2009 4:56 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    I apologize for the confusion, the dll source is in the folders.  I opened the solution in the root and the references pointed to DLLs vs. projects.  However, you're right, the source is in the... errr... 'src' folder.  Guess what I'll be reading this week :)

    I'm pretty sure there isn't any limit in C# (and therefore XNA) that prevents people from using DLLs that are built in other languages (including C++).  I'd wager that most of the system includes are in something other than C#.

    SU, again, very nice!!
  • 3/16/2009 9:14 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Im focusing on 2D games right now. But I still took a look at the videos just to see and I am very impressed. Looks awesome. Great job and nicely done!
  • 3/16/2009 1:12 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Sorry for the late reply.

    tryz, Xen is 100% safe C#. No C++ trickery :-)
    The animation stress test has the advantage that the Tiny model is rendered in a single draw call. On the xbox especially this is a big advantage.

    The biggest part of xen, is I've written a Visual Studio custom tool plugin to compile shaders (which for various reasons I cannot publish the source to yet). This plugin pre-processes shaders, and strips them down to their bare essentials. This bypasses the effect system entirely.
    I do as much as I possibly can to avoid unneeded API calls, and avoid copying data when possible. This goes for render states, textures, samplers, right through the vertex declarations and such.

    The result of this, is the animation stress test example, even though it's drawing 500 instances of the Tiny model, only makes around ~1250 API calls. The main part is simply 500 pairs of 'SetVertexShaderConstants()' and 'DrawIndexedPrimitives' (although the xbox requires the vertex shader is reset). Everything else is avoided.
    The animations are spun off as thread tasks. On the xbox this sample is *very slightly* GPU limited as the poly count is huge, but it doesn't take much at all to push to CPU limited.

    When things calm down a bit (my work life is a bit hectic right now) I'll be looking at what will happen for 1.6. I'll probably look at experimenting with putting API calls on their own buffered thread, to help the xbox out a bit more - but as you might imagine this would be pretty tricky to pull off. But overall I'm very happy with where things stand right now.

    Thanks! :-)

  • 3/16/2009 11:59 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Well, the results speak for themselves.  Certainly very impressive performance.

    One of my pains is that I'm rendering multiple meshes for my skinned object.  This allows me to swap out meshes.  For example, I can swap feet out for boots.  Unfortunately, the performance cost is pretty high.

    I'll certainly poke around a bit.  You've got a lot of really nice work.

    Thanks :)
  • 3/18/2009 3:36 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    I found the 2D particle system fun to mess around with. I edited fireworks.particles.xml, and managed to make your fireworks demo uglier. See image

    My edited fireworks demo

    I really don't think your totally off base creating an API. That's one of the main points of XNA, is to make game programming easier, and to speed up the process for indie developers. Your graphics API obviously makes creating graphics easier, and that's a good thing. I may not mind trading off a little bit of flexibility, if I can create a nicer looking game a lot faster.
    Right now I'm still at the learning tutorials stage with XNA. However I might be interested in working through some tutorials in both straight XNA as well as your framework. That way I could give you some feedback as far as how much easier it is, etc.

    I'm not good with understanding EULAs. So in non-lawyer's terms; does it cost me anything to create and distribute a game built on top of your framework?
  • 3/18/2009 3:53 AM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    RetroGamer79:
    I'm not good with understanding EULAs. So in non-lawyer's terms; does it cost me anything to create and distribute a game built on top of your framework?

    It's MS-Pl, which basically means you can do whatever you want. Obviously I'd appreciate mention, and if you make millions then perhaps a donation :P but legally you don't have to do anything.

  • 3/18/2009 9:57 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    StatusUnknown:
    RetroGamer79:
    I'm not good with understanding EULAs. So in non-lawyer's terms; does it cost me anything to create and distribute a game built on top of your framework?

    It's MS-Pl, which basically means you can do whatever you want. Obviously I'd appreciate mention, and if you make millions then perhaps a donation :P but legally you don't have to do anything.




    Thanks for the answer. Mention = Absolutely. The millions are rolling in as we speak ;).

    These are probably some newb questions, but if I were to create my own full-on game engine with sound, map editors, etc (nothing cutting edge by any means) that incorporated your graphics API:

    1. All graphics would need to use your API and not go down to the base XNA level, right?
    2. I noticed that the Overriding Update and Draw functions that you provide, do not receive the GameTime variable like they do in the default XNA GameStudio projects. I assume it's wrapped up inside. How do I access it, and do other items such as sound fit right in to the update framework?
  • 3/18/2009 10:44 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    Hi SU,

    I really like Xen, and am using it in a new project I am working on. One query though - is it by design that the XenFX custom tool interprets all shader parameters which don't have a semantic as a float?

    Very good work, and I look forward to what your work in the future.

    Guy Sherman.
  • 3/18/2009 11:50 PM In reply to

    Re: Xen 1.5 released: New GPU Particle System

    DynamicSamurai:
    is it by design that the XenFX custom tool interprets all shader parameters which don't have a semantic as a float?

    Hi, the properties generated match what the DirectX shader compiler exports. Which means no matter what you do, the value will be a float. So even if you define an integer in your shader, it will be compiled as float. Only DirectX 10 hardware properly supports ints, which XNA doesn't expose.

    RetroGamer79:
    1. All graphics would need to use your API and not go down to the base XNA level, right?

    Yes and no, but mostly Yes. :-)
    You can directly access the GraphicsDevice, using Begin/End GetGraphicsDevice() however you need specify what you are doing. Things like XNA vertex buffers can still be used (although you draw them differently, using state.DrawVertexBuffer()). The disadvantage is you lose a lot of the state tracking and statistics that get done behind the scenes.

    RetroGamer79:
    2. I noticed that the Overriding Update and Draw functions that you provide, do not receive the GameTime variable like they do in the default XNA GameStudio projects. I assume it's wrapped up inside. How do I access it, and do other items such as sound fit right in to the update framework?

    The DrawState and UpdateState objects both expose similar properties as GameTime (eg, state.TotalTimeTicks, state.TotalRealTimeTicks, state.DeltaTimeSeconds, and a bunch more). Xen provides no sound API so you just use standard XNA sounds.
    XNA GameComponents are still exposed through state.Application.XnaComponents, which can be used for non-drawing objects.

Page 1 of 2 (26 posts) 1 2 Next > Previous Discussion Next Discussion