Performance issues on v0.3 real-time Samples

Sep 20, 2010 at 4:03 PM
Edited Sep 20, 2010 at 4:08 PM

 

After trying out almost every "good" real-time commercial WPF chart out there - Syncfusion, Visifire, Modulus StockChartX etc. - I started to think the high performance charts in WPF are a myth.   Our requirements are simple, 4 realtime charts in one window for our traders.  The only thing the charts have to do is display real-time price ticks from live market data.  This is the most basic requirement for a "realtime" chart - display live data as it becomes available and re-render itself.  The other thing we wanted to do is keep the charts open during trading hours for 4-5 hours at a time, which is standard practice in trading.

Most of the commercial charts failed with just one chart in a matter of minutes, with respect to CPU performance.  This was without any hookup to the backend market data servers.  For testing we randomly generated the ticks inside the WPF codebehind, in order to eliminate any server latency issues.  Simulating even slow moving markets, say only 1 tick per second the performance was not good.  We intensively profiled all these commercial packages (with SciTech, .DotTrace, WPF toolkit and several other profilers both for memory and cpu performance) and most of them seemed to point to fundamental flaws with how Microsoft System.Windows.Shapes was written the deeper we looked.  When I came across the great reviews D3 had for its performance, I was very excited and wanted to give it a try.  Unfortunatelty Im sad to say I have to keep looking -- or at this point I will probably write one myself.

In downloaded the latest DynamicDataDisplay_v0.3_Binaries_&_Samples.zip and ran Samples/v0.3/Simulation.  On an Intel Xeon 4 core 2.33Ghz, 3 GB RAM machine, running only the single chart sample racked up overall CPU usage to 50%.  On two other separate Intel Xeon 2.2.7 Ghz 8 core, 3.5 GB machines the CPU performance was between 10-20% for SimulationSample.exe.  With 4 charts the CPU utilization was close to 100%.

I just wanted to share my findings out here, for any one else grappling with streaming charts in WPF.

 

 

 

 

Sep 20, 2010 at 8:12 PM

We too are running into performance problems with trying to do our charting.  We need to have a line graph that maps up to 200 points per second and keeps the history of past points.   This needs to run for up to an hour.  We usually start seeing performance hiccups after only a few minutes.  We are now pursuing writing directly to DirectX but are finding this to be not very intuitive and slow going. 

I'd love to hear about findings anyone has for doing data intensive real-time charting in WPF.

Sep 21, 2010 at 1:43 AM

Pardon me for saying so, but with all due respect, I find the data and the conclusions on both the above posts a bit puzzling.

@ gorter : What you are describing, right here, is a de facto memory leak. In fact it is probably the most accurate definition of a memory leak. Without having seen any of the performance hiccups you talk about, and since you don't describe them in detail, I could imagine that after a few minutes of your data flowing, there would be quite a few objects lying around and never freed. What happens next is that garbage collections happen more frequently, and take more time (sometimes right up to a full second for generation 2 collections). This would probably not be helped by any directX as it is not caused by the grid itself. This is a normal behaviour of the .net framework and is to be expected. You should probably look for improvements around your data structures, advanced DataBinding, and a very different design alltogether. Another grid will not solve those problems. No .Net Grid can.

 

@asifrah : I am surprised that you are surprised. Also, may I humbly question your benchmarking process. Everything GUI related will happen on a GUI thread. That means at maximum on one processor. On an 8 core machine, that means you will get maximum 100/8 = 12.5% total CPU usage from this grid. Not sure how you get 100% CPU usage, apart from maybe launching 4 separate processes with a grid each. That is not a good benchmark. What you want to benchmark is what happens when you have 4 grid in the same process. How does it affect the rate of processing, does the performance degrade gracefully, does it degrade by increasing latency (and by how much) or does it degrade by freezing regularly. A good run in a profiler would also help somewhat in highlighting some performance bottlenecks that could be in your code.

As a side note, I am not sure why people usually think 100% CPU usage, or high memory consumption is that bad. In fact, I would want my software, if written to be high perf, to use up all available resources. If it doesn't, then surely, that means it is doing something else, like waiting for I/O, or twiddling its thumbs. The only case when I like to see low CPU usage is if that software was designed to be scalable (usually, we are talking server-side here, not GUI) and that it meant it could process many things at a time. This usually would mean decreased latency (a software using only 2% of the CPU is NOT processing things as fast as possible). What you want to be thinking about when designing software is not CPU usage. It is scalability, latency. That is a very different approach.

One big pitfall people fall into also, is the one where you think you app actually NEEDS to update every single bit of data coming in. Now I don't know about your traders, but ours, they don't have bionic eyes just yet. They will probably get some grafted as soon as the technology allows, but we're still not quite there yet. Updating a number (or a graph) 200 times a second is just plain silly. Now, processing 200 updates a second is reasonable goal, but on another thread. What you want to do is your GUI to "reflect" the result of those 200 updates, but only by a "snapshot" that will aggregate them, and display them to the trader at a much more sensible (and readable) rate. I have found that 2 times per second is already quite enough for most people to endure. Your high level testosterone fueled trader can have 3 times /sec and he will do just as good a job. More, and you are designing something that only a machine could process. You are also wasting valuable resources in the GUI thread, that could be defered to one of the other 7 cores.

Now pardon my tone here, but when I read those posts, it felt just like what the juniors (junior in the field of realtime front office dev, they could be seasoned geeks in another field) guys we see around would say in their first few months. The bad case of "premature optimization", with a whiff of not understanding the real implications of data intensive work.

If I am completely wrong, feel free to open up the can of bird names ;-)

Sep 21, 2010 at 3:47 AM

Michel,  

Many people consider high CPU utilisation by an application to be bad because they would like their PC's to be responsive to other applications.  If one app is using continually using 100% of the CPU other applications are likely to lose their responsiveness.

BTW, I'm not sure if you realise that asifrah & gorter are referring to the performance of the dynamic data display. This is a graphing control and (as far as I know) it has no grid features.

Grant.

Sep 21, 2010 at 4:45 AM
Edited Sep 21, 2010 at 4:52 AM

@alexis_michel:   Can of bird names it is ;).   

I do think you took this personally for one reason or the other.  Before you dive into this sort discussion, please test the samples for yourself.  Execute SimulationSample.exe 4 times so that you have 4 instances running.  Each running sample has only one chart.  If you believe that is an unfair test, take the sample code and paste it 4 times into a WrapPanel so that all the charts fit nicely in one window, and is inside one process.  Now run the process.  If you do better than 50% CPU on an average machine, let us know.   

Also Im not sure what sort of software development school is teaching developers to max out the CPU or memory on a machine.  Did you happen to notice that the windows operating system is close to unresponsive with an application running at those levels?  

Fundamentally what we are taking about here is a problem with WPF in and of itself as far as charting goes.   The problem lies in the System.Windows.Shapes namespace.  Take the simple case of the Line class in System.Windows.Shapes.  In order to draw 1 line four coordinates need to be set - X1, Y1, X2, and Y2 - each of which is exposed as a .NET property of type double.  Now draw a line once every second, NOT 200 times a second.  It is well known that even the individuals with the best eyesight strain to distinguish beyond 50 frames per second.   Now what happens when we set one of these coordinates?  

[ This is actual .NET 3.5 framework code ] 

public void set_X1(double value)
{
   base.SetValue(X1Property, value);
}

..and what happens inside setValue?  


public void SetValue(DependencyProperty dp, object value)
{
   base.VerifyAccess();
   PropertyMetadata metadata = this.SetupPropertyChange(dp);
   this.SetValueCommon(dp, value, metadata, false, OperationType.Unknown, false);
}

So what does this all mean?  What exactly are we talking about here?  First there is a boxing operation when a double value is implicitly converted into System.Object.   Then SetValue causes a DependencyPropertyChanged.  Now do this for all 4 coordinates.   Now say a chart is made up of 50 lines, so do this 50 times for each line - remember all the coordinates on the chart will be redrawn every time there is an update.  Now do all this once per second, i.e. 60 times a minute.  Now keep doing this for one hour, assuming you keep the chart running for an hour.  So what do we finally have?

After one hour we have  4 * 50 * 60 * 60 = 720,000 Dependency Property Changes and 720,000 boxing operations.  And that is for only one chart, updating once a second.

Now imagine having 4 charts, running for 4 hours under the same scenario?  11,520,000.  That is over 11 million boxing operations and DP changes!

So while WPF is great framework -- System.Windows.Shapes which all WPF charts seem to be based on, doesn't seem to be that agreeable.  It is as if it was added onto .NET as an afterthought without much thought how it will actually work with live graphics.

I believe it is because of limitations like these other contributors of D3 have started writing a DirectX version of D3.  See the following thread: http://dynamicdatadisplay.codeplex.com/Thread/View.aspx?ThreadId=74901.  A barebones version of this code is available in the nightly build in the downloads section of this project.

So the real question is, how does a developer with a limited amount of time work around this?  One is to implement their own charts with classes in the System.Windows.Media.Drawing namespace. The other and more simpler way, is to have a queueing mechanism for their observable collection so as to display limited number of live datapoints at any one time, as new points are queued and displayed, older points are dequeued and fall of the chart.  I have actually tried this, and performance-wise it works out ok.  

Of couse one could also consider embedding charting packages that use ActiveX, DirectX or other technologies, but they will not have inherent support for themes.

@alexis_michel please try to restrain yourself if you do not actually have something useful or productive to add ;).  Unless of course you work for microsoft and have taken an unspoken offense.

 

 

 

 

 

 

 

 

Sep 29, 2010 at 3:54 PM

I have changed from using the D3 charting tools to running a Windows Forms Chart within a 'WindowsFormHost' control in my WPF application.  Performance is no longer a problem.

Sep 29, 2010 at 4:25 PM

Hi Gorter,

I have the same problem (in exactly the same type of application - Forex) - what Windows Forms chart are you using?

 

Sep 29, 2010 at 4:36 PM

I am using a FastLine chart.  Basically I took the 'RealTimeTimer' sample and have adapted for our use.  Still a work in progress, but seems to be working much better than previous attempts with other charts.

Sep 29, 2010 at 4:41 PM

Thank you for the info - let us know how things progress.

Sep 30, 2010 at 2:26 PM

Hey Guys,

any plans when exactly (at least month) the new version with DirectX-Rendering will appear?

It would be great, so we could adapt out application development plans to your schedule.

 

greetz,

Mav