Updated DirectX library

Oct 26, 2010 at 2:16 AM
Edited Oct 26, 2010 at 10:39 PM

Hi everyone,

I had a need to improve the performance of the LineGraph in DynamicDataDisplay, so I took the example DirectX library from the latest nightly build and improved upon it myself. Please keep in mind that I developed it for my project's needs, so you may have to modify some things to fit yours, but I will try to explain the things you an change to make it easier.

First, I want to go over the changes:

  • I upgraded to the latest SlimDX library from June of this year, so you will need to download that SDK and/or binaries in order to build the project.
  • I scrapped the DirectXHost class after working on it quite a bit in favor of the example SlimDXControl included in the SlimDX SDK.
  • I moved the TransparencyEffect into its own effect library to make compiling easier to handle. It uses the ShaderBuildTask from the WPFToolkit. This may not be necessary, but I left it just in case.
  • I replicated the class hierarchy from the main D3 library in order to host a DirectXLineGraph just like you would a normal WPF LineGraph in the ChartPlotter class.
  • I implemented a timer to control the rendering loop instead of the CompositionTarget.Rendering event. It seemed net some more performance since you are not relying on the WPF rendering loop to trigger.
  • I created some extension methods that duplicate the AddLineGraph() methods ,which you can look at and duplicate if you need to.

Now, to build it, you will need the following:

To use the new DirectXLineGraph, there has to be a SlimDXControl in the ChartPlotter.Children collection. You can add it in XAML or code. Then, you can add a new line graph. Here is an example in code:

 

_chartPlotter.Children.Add(new SlimDXControl());
_chartPlotter.AddDXLineGraph(pointCollection, Colors.Green, "description");

 

I have the line graph background color set to black, but you can change that in the SlimDXControl. Look for this line and change the color:

 

private Color4 _backBufferColor = new Color4(1, 0, 0, 0);

 

If I think of any more tips, I will edit this thread. Also, feel free to post any questions, comments or suggestions you have regarding my changes. I hope you guys can make use of it.

Edit: I mistakenly put ViewportElement2D instead of PlotterElement in the last line of the build instructions.

Oct 26, 2010 at 8:35 AM
Edited Oct 26, 2010 at 8:43 AM
 
Awesome!
 
Thank you Jason for sharing your hard work.
 
David Roh

Oct 26, 2010 at 10:38 PM

I will be posting an update tomorrow that addresses some pretty major performance issues. Stay tuned.

Oct 27, 2010 at 2:46 PM
Edited Oct 27, 2010 at 2:49 PM

I am getting some exceptions with the Line Description in the D3 library once I start using DXLineGraph

Below Image shows the code for adding the DirectXLineGraph

 

Below Code show the exception that happens inside the D3 library since it cannot find a LineGraph to add the description

 

Are you seeing this error.

Thanks

Kishore Vanapalli

Oct 27, 2010 at 4:58 PM

Yes. This is change I forgot to document. Since it was expecting the concrete class LineGraph, I had to change it to an interface reference.

Change that line(29) to:

IOneDimensionalChart g = graph as IOneDimensionalChart;
Oct 27, 2010 at 5:09 PM

I also ran into an issue yesterday while I was profiling the memory usage of my application. It was generating a lot of garbage, causing way too many Gen0 collections. I tracked the issue down to the GetPoints() method of IPointSource. This method returns an IEnumerable of Points that was being converted into an array of Points using the .ToArray() method.

The current work around for me is to add seperate float arrays for my X and Y graph values and completely bypassing the standard D3 point datasource system. I realize this may not work for some people, as I developed for my case, but does anyone have the need to chart anything other than double or float values in a line graph?

Oct 27, 2010 at 5:13 PM

DateTime on the independent axis.

Oct 27, 2010 at 7:42 PM

I am getting a blank screen with no plot in the graph region

  private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            DirectXLineGraph lineGraph = null;
            double[] yPts = new double[] { 11, 44, 22, 55, 33, 88, 66, 77, 99, 110 };
            double[] xPts = new double[] { 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 };

            IPointDataSource dSource = xPts.AsXDataSource().Join(yPts.AsYDataSource());
            lineGraph = plotter1.AddDXLineGraph(dSource, Colors.Blue, 1, "Phase");

            //StartToLoadData();
        }

Below is the image. 38 is the Frames that is shown in the debug mode.

Oct 27, 2010 at 8:04 PM
Edited Oct 27, 2010 at 8:05 PM

It looks like the values you are graphing fall outside the range of your chartplotter. Can you zoom the viewport out, or change your values?

Also, I noticed the TransparencyEffect is used to set the backbuffer color to transparent, if you want to use the graph background color instead. I have this re-enabled on the newer version of the library. If you want to enable it yourself, just add this line in the constructor of SlimDXControl. The backbuffer color does need to be set to black:

Effect = new Transparency();
Oct 27, 2010 at 9:02 PM

Hi Foxman13, and thanks for the work you've put into this DirectX implementation.

Like kvanapal, I'm having a similar issue - I don't see the graph Line that I should (I don't see any line at all), and like kvanapal I can see the legend and frame-rate (as well as the axis etc).

There is definitely a SlimDXControl added to the plotter, and I have followed the example and changes that you have suggested in your previous posts.

Any ideas what could be going wrong here?

 

Thanks for your time,

pete

Oct 27, 2010 at 9:10 PM

Are you seeing any first chance exceptions in the output window during debug?

Oct 27, 2010 at 9:31 PM

There is just one exception thrown, but I tracked this down to the creation of the Driect3DEx object (see code snippet below). I'm running Windows XP, and I understand that the Direct3DEx class is only available with Windows Vista onwards?

Anyway, yes there is an exception but nothing that I thought would cause the issue which I am experiencing.

private void InitializeDirect3D()
{
DirectXStatus = DirectXStatus.Unavailable_Unknown;
ReleaseDevice();

...other code

try
            {
                _direct3DEx = new Direct3DEx(); <-- Exception thrown here
                UseDeviceEx = true;
            }
            catch
            {
                try
                {
                    _direct3D = new Direct3D();
                    UseDeviceEx = false;
                }
                catch (Direct3DX9NotFoundException)
                {
                    DirectXStatus = DirectXStatus.Unavailable_MissingDirectX;
                    return;
                }
                catch
                {
                    DirectXStatus = DirectXStatus.Unavailable_Unknown;
                    return;
                }
            }

Hope this can help you,

Pete

Oct 27, 2010 at 9:41 PM

I found the issue. To work around the problem for now, go to DirectXChart.cs and find the OnMainLoop() method. In it, change the following:

            // only process our rendering if the data is updated
            //if (_renderUpdate)
            //{
            //    OnDirectXRender();
            //}

            OnDirectXRender();
Oct 27, 2010 at 10:09 PM
Edited Oct 28, 2010 at 2:22 PM

Ok everything seems to work. I did not see anything because my values were way above the range of the Axis. The default range is 0-1 on both the axes

We need to fix the method plotter.FitToView(). 

otherwise we have to set the viewport all the time after we render

OR

is the FitToView() already fixed??

Oct 27, 2010 at 10:15 PM

Let me take a look at FitToView() and see if I can get it going. I will also look at implementing a HighPerformanceDataSource type class to fix some of the garbage collection issues. I will keep everyone updated.

Oct 28, 2010 at 12:27 AM

How do you make the DirectX work with terminal services on a computer ???

Oct 28, 2010 at 1:40 AM

If you are talking about a remote desktop situation, I am pretty sure you can't share a DirectX device over that protocol.

Oct 28, 2010 at 8:56 AM

Hi Jason,

 

thanks a lot for your effort. I will check this out tonight @ home. There are some question i have beforhand:

- I will not be able to see the plot over a remote desktop connection?
- Am i able to host more than one of these controls in one application the same time?

 

Regards

joe

Oct 28, 2010 at 1:57 PM

Hi Joe,

Hardware accelerated graphics such as DirectX will not display over a remote desktop connection. This is dues to the surface actually being drawn on the GPU and the RDP knows nothing about it. Everything rendered over RDP is software-only.

Here is a little snippet that explains some of the problems with DirectX and RDP: http://social.msdn.microsoft.com/Forums/en-US/gametechnologiesdirectx101/thread/8b037258-431b-4286-b4f5-7070ca0afd98/

Also, you can host multiple plotters in the same application. However, I wouldn't recommend displaying more than 2-3 at a time, as it will bog down your video card.

Oct 28, 2010 at 2:31 PM

If I use the transparency effect in the constructor it is throwing an exception 

 

Oct 28, 2010 at 3:14 PM

Try placing it above InitializeComponent();

Oct 28, 2010 at 5:22 PM
Edited Oct 28, 2010 at 5:24 PM

Although you are using FrequencyFilter with your DirectXLineGraph it seems to me that that Filter is not being called and it is trying to plot all the thousands of points in the data source. At 20K point data source the graph just dies.

I have uploaded a working sample of the application that switches between 5K, 10K, 20K, 30K and 100K points

here is the link D3.DirectX.zip

You can clearly see that the application does not respond at20K points. It is clearly not using the FrequencyFilter that is added to the graph.

Oct 28, 2010 at 6:50 PM

You are correct. I should have mentioned that this wasn't a full blown replication of the LineGraph, as I developed it for my project's use cases. For example, the most points I would ever need to plot at once is 3000 and that represents a rolling time window on a data set.

Howver, if that is something you guys need, let me take a look at it and see what I can do. Or, feel free to add the additional functionality yourself.

Oct 28, 2010 at 6:53 PM

Yes I would be more than happy to help you develop this further. Just let me know where how I can help you add this filtering in. I can take this from there. I am still trying to get my head around what you have done. I am totally new to DirectX programming so still learning. But I am pretty well versed with how D3 works. Let me know where can I help and lets get it working.

kishore

 

Oct 28, 2010 at 6:58 PM
Edited Oct 28, 2010 at 7:04 PM

I suggest we setup a development project then based on the community's contributions. I will create a Codeplex project and update you shortly. Then, we can have source control and a release system for changes.

Oct 28, 2010 at 10:49 PM
Edited Nov 2, 2010 at 6:06 PM

There is something wrong with the Transparency. The graph area is only transparent on the areas where we paint it. The rest of the graph area is white. See picture below

Nov 2, 2010 at 3:40 PM
Foxman13 wrote:

I suggest we setup a development project then based on the community's contributions. I will create a Codeplex project and update you shortly. Then, we can have source control and a release system for changes.

 Did someone create a fork of this project that will actually have work being done on it?  If so, where is it?

Nov 2, 2010 at 6:19 PM

we got the project set up here http://d3commdev.codeplex.com. Trying to get the DirectX to work more appropriately using filters.

 

Kishore

Nov 2, 2010 at 6:21 PM

You have to be a developer or coordinator to have access to it right now.  I am neither.

Nov 2, 2010 at 6:25 PM

There is not any code posted to that site yet. I apologize for the delay, but i have been sick the past few days.

On Nov 2, 2010 1:21 PM, "iguanamind" <notifications@codeplex.com> wrote:
> From: iguanamind
>
> You have to be a developer or coordinator to have access to it right now. I am neither.
>
>
Nov 2, 2010 at 6:28 PM

Hi Jason, Kishore,

I am also very interested in this project - please keep us updated on your progress on this thread and let us know when you have a discussion opened on the new project site.

Thank you for all the hard work that you are doing - it is appreciated.

David

Nov 2, 2010 at 6:31 PM

NP!  Been battling something myself.  I am interested in a fork on this since nothing is being done (no releases or patches).  I probably don't have the same interest that generated this particular thread, but I am interested in helping.  I'm sure when you open up the new project, you will be more clear on purposes.

Nov 2, 2010 at 6:33 PM
kvanapal wrote:

There is something wrong with the Transparency. The graph area is only transparent on the areas where we paint it. The rest of the graph area is white. See picture below

 Interesting. I haven't seen this issue, but that is probably because I hide the grid lines in my charts. I am also using a black background.

Nov 3, 2010 at 3:53 PM

If u hide the axis Grid then it is not a problem. But I would like to see the Axis grid.

I am trying to overlay the curves on top of the Axis Grid so I am seeing the problem. Let me know if you find a fix.

Nov 3, 2010 at 4:03 PM

I am working on a much complex problem set for the graphing library.

I need to be able to plot upwards of 50K points and may be upto 200K. I need Zoom & Pan, need Axis on the right, need Logarithmic Axis on X axis. Thats is basic set of features that I need. Although D3 provides all the above Its rendering is very slow, so I am trying to see if DirectX can scale and it does. But the thing is I need to add all these bunch of features again back into the DirectX. I am new to DirectX and battling with the API.

Nov 3, 2010 at 4:28 PM
We'll work on these features. Zoom and Pan work, along with the Axis positioning. I have it working in my project.
We will need to focus on the DataSource side of things though, because that seems to be the biggest performance issue(aside from drawing) when you are graphing large amounts of data and/or data in real-time.

On Wed, Nov 3, 2010 at 11:03 AM, kvanapal <notifications@codeplex.com> wrote:

From: kvanapal

I am working on a much complex problem set for the graphing library.

I need to be able to plot upwards of 50K points and may be upto 200K. I need Zoom & Pan, need Axis on the right, need Logarithmic Axis on X axis. Thats is basic set of features that I need. Although D3 provides all the above Its rendering is very slow, so I am trying to see if DirectX can scale and it does. But the thing is I need to add all these bunch of features again back into the DirectX. I am new to DirectX and battling with the API.

Read the full discussion online.

To add a post to this discussion, reply to this email (dynamicdatadisplay@discussions.codeplex.com)

To start a new discussion for this project, email dynamicdatadisplay@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com




--
Jason Fox
Nov 3, 2010 at 5:21 PM

Oh thts good. I think I have the filtering enabled and working. I also realized that the Viewport is not set to the right bounds to start with.

Foxman13 wrote:
We'll work on these features. Zoom and Pan work, along with the Axis positioning. I have it working in my project.
We will need to focus on the DataSource side of things though, because that seems to be the biggest performance issue(aside from drawing) when you are graphing large amounts of data and/or data in real-time.

On Wed, Nov 3, 2010 at 11:03 AM, kvanapal <notifications@codeplex.com> wrote:

From: kvanapal

I am working on a much complex problem set for the graphing library.

I need to be able to plot upwards of 50K points and may be upto 200K. I need Zoom & Pan, need Axis on the right, need Logarithmic Axis on X axis. Thats is basic set of features that I need. Although D3 provides all the above Its rendering is very slow, so I am trying to see if DirectX can scale and it does. But the thing is I need to add all these bunch of features again back into the DirectX. I am new to DirectX and battling with the API.

Read the full discussion online.

To add a post to this discussion, reply to this email (dynamicdatadisplay@discussions.codeplex.com)

To start a new discussion for this project, email dynamicdatadisplay@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com




--
Jason Fox

Kishore

Nov 3, 2010 at 6:09 PM
Are you graphing real-time data, many points of data, or a combination of both?

On Wed, Nov 3, 2010 at 12:21 PM, kvanapal <notifications@codeplex.com> wrote:

From: kvanapal

Oh thts good. I think I have the filtering enabled and working. I also realized that the Viewport is not set to the right bounds to start with.

Foxman13 wrote:
We'll work on these features. Zoom and Pan work, along with the Axis positioning. I have it working in my project.
We will need to focus on the DataSource side of things though, because that seems to be the biggest performance issue(aside from drawing) when you are graphing large amounts of data and/or data in real-time.

On Wed, Nov 3, 2010 at 11:03 AM, kvanapal <notifications@codeplex.com> wrote:

From: kvanapal

I am working on a much complex problem set for the graphing library.

I need to be able to plot upwards of 50K points and may be upto 200K. I need Zoom & Pan, need Axis on the right, need Logarithmic Axis on X axis. Thats is basic set of features that I need. Although D3 provides all the above Its rendering is very slow, so I am trying to see if DirectX can scale and it does. But the thing is I need to add all these bunch of features again back into the DirectX. I am new to DirectX and battling with the API.

Read the full discussion online.

To add a post to this discussion, reply to this email (dynamicdatadisplay@discussions.codeplex.com)

To start a new discussion for this project, email dynamicdatadisplay@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com




--
Jason Fox

Kishore

Read the full discussion online.

To add a post to this discussion, reply to this email (dynamicdatadisplay@discussions.codeplex.com)

To start a new discussion for this project, email dynamicdatadisplay@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com




--
Jason Fox
Nov 4, 2010 at 2:43 PM

I have the filtering working on the ChartPlotter. I am not plotting real time data, I have to plot a data file which are really huge.

kishore

Nov 4, 2010 at 3:18 PM

Awesome! Can you send me your code? I will merge it with the changes I made for drawing and upload it as the initial verison on the community codeplex project.

Nov 4, 2010 at 5:24 PM
Edited Nov 4, 2010 at 6:54 PM
Foxman13 wrote:

Awesome! Can you send me your code? I will merge it with the changes I made for drawing and upload it as the initial verison on the community codeplex project.

Here is the code please get it and sync it with our changes. D3.DirectX.zip

DirectXPointsGraphBase has update method that currently does nothing. I have updated it to call UpdateCore which is a virtual method and is implemented in DirectXLineGraph. So i added UpdateCore to DirectXLineGraph and a whole slew of things that it needs to support the method. 

What is also important is a change in the AddDXLineGraph in the Plotter2DExtensions.cs You have to add the graph to the plotter before you set the datasource to the graph so the sequence of things is important so that the graph knows its plotter and the viewport and then knows how to deal with the data source when it is set. So this what I did 

public static DirectXLineGraph AddDXLineGraph(this Plotter2D plotter, IPointDataSource pointSource, Pen linePen, Description description)
        {
            if (pointSource == null)
                throw new ArgumentNullException("pointSource");
            if (linePen == null)
                throw new ArgumentNullException("linePen");
            
            DirectXLineGraph graph = new DirectXLineGraph();            
            graph.LinePen = linePen;            
            if (description != null)
            {
                NewLegend.SetDescription(graph, description.Brief);
                graph.Description = description;
            }
            
            graph.AddToPlotter(plotter);
            graph.DataSource = pointSource;            
            graph.Filters.Add(new FrequencyFilter());
            return graph;
        }

The DirectXChart inherits from ViewportElement2D which provides the Viewport and the Plotter2D properties. But we need to override these two properties in the DirectXChart so I have the two new properties in DirectXChart.cs file and these two functions are important for the filtering to work

new protected Plotter2D Plotter2D
{
    get { return (Plotter2D)this.plotter; }
}

        
        
new protected Viewport2D Viewport
{
   get
   {
      return Plotter2D.Viewport;
   }
}

In your SlimDXControl.cs in the onRendering method you are setting the _needsRendering to always true and there by 
redrawing it several times and we dont want to do that. I am not a DirectX expert so I will leave that for you to change. 
I made some changes that work ok for me but you want to further polish that.

In my test application I am trying to add LineGraph to the plotter in the Window_ContentRendered method but you can do that in Window_Loaded as well
without any problem

 

Nov 5, 2010 at 10:01 PM

When are you guys going to open-up the http://d3commdev.codeplex.com/ for the public? I wanna try the latest community version but it's locked.

Nov 5, 2010 at 10:26 PM

We are working on getting the initial code check-in done. Once we do, we will make it public for you guys to do bug testing and provide feedback.

Nov 5, 2010 at 11:52 PM

I'm evaluating some charting components and I'm intending to evaluate any working sample of D3 running over SlimDX. I'm afraid I don't have much time to make some decisions about using D3 or trying something else. If you could send me anything to evaluate ASAP would be of great help. I know it's not even ready for bug testing and feedback, but I would like to see it running so I could measure if it can fit my purpose.

I'm intending to contribute with code as well over the next months -- if I choose D3 as my charting engine. Feel free to drop me and e-mail at diego.frata [at] gmail . com

Editor
Nov 7, 2010 at 8:12 AM

Hi!

My name is Mikhail Brinchuk, and I am a developer of DynamicDataDisplay.

I'm searching now a remote job, so if you or your company can offer it to me, I will be very grateful.

If you want to customize DynamicDataDisplay somehow, I can help you to do it.

My CV is here - https://sites.google.com/site/thecentury/

Best regards,

Mikhail.

 

Nov 8, 2010 at 3:23 PM
Edited Nov 8, 2010 at 3:25 PM

Hi everyone. We have the community version of DynamicDataDisplay up over at http://d3commdev.codeplex.com/. This is just focusing on the DirectX version of the LineGraph at this point in time, so if you have need of plotting many thousands of points, then please go there and grab the latest source. If time permits, we will look at fixing some of the other issues plaguing D3 and possibly implementing missing features.

If you run across any issues, please use the issue tracker to submit issues and we will take a look at them.

The community version is released under the same license as D3 to comply with the Ms-RL license. However, I am working to contact somebody at Microsoft to change the license from Ms-RL to Ms-PL, so a lot of us would feel better about using it in our projects.

Thank!

Dec 13, 2011 at 10:28 AM
Edited Dec 13, 2011 at 10:31 AM

>>The community version is released under the same license as D3 to comply with the Ms-RL license. However, I am working to contact somebody at Microsoft to change the license from Ms-RL to Ms-PL, so a lot of us would feel better about using it in our projects.

Hi, I'm wondering if there are still plans to change the license from MS-RL to MS-PL?
That would definitely make me feel better!

Thanks in advance.