Plotting real-time data over a long period of time

Feb 16, 2013 at 8:28 AM

Currently I'm using D3 to plot real time simulation data. A typical simulation would run for several days and generate new values every second. In my application I would be having multiple ChartPlotters and each plotter will have multiple line graphs. I want each chart to keep track of the latest values (Currently I use FollowWidthRestriction but I'm not sure what is the correct value for width to use) and let the user pan/zoom to view the whole data since the simulation start.

I realize that there is a performance issue when plotting large number of points. So I'm wondering what would be a good solution for this? Are filters going to help with the performance issues? Do I need to store the data in some other structure than the ObservableDataSource?

I have noticed some similar discussions but there wasn't a definitive answer. I hope that it is possible to achieve these requirements with D3.

Best Regards,
Mar 7, 2013 at 2:26 AM

View can not handle more data points (Max arround 10000 data points).

I have one suggestion, Use some fixed span like 1 hour + 10% buferfer.
For example if you started simulation at 10:00 AM, keep your chart span from 9:00 Am to 10:06 AM.
Data starting position is at 10:00 Am and data plots up to 10:06, thenat 10:06 you shift your chart span to 9:06 to 10:12, Shift chart span again at 10:12
..... like this

When the time reachs 11:06 your chart span will be 10:06 to 11:12, remove data from 10:00 to 10:06 from your chart line graph, other wise again you will get performance issues after some time if line graph grows.

there were few discussion earlier Check link, the version in this link may be very old.

Jun 18, 2013 at 7:39 AM
Edited Jun 18, 2013 at 7:46 AM
Performance is not primarily related to the total number of points, but with the number of pixels that are redrawn. If you look at the class MarkerPointsGraph you'll see that D3 is trying each time to redraw all the points in DataSourse. To avoid this, you can limit the amount of redrawing changing class MarkerPointsGraph. The following is a derived class. This is not my code, but I have not found the original post. My charts are drawn all day with no problems. Sorry for the Visual Basic.
mports Microsoft.Research.DynamicDataDisplay.DataSources
Imports Microsoft.Research.DynamicDataDisplay.PointMarkers
Imports Microsoft.Research.DynamicDataDisplay.Common
Imports Microsoft.Research.DynamicDataDisplay.Charts
Imports Microsoft.Research.DynamicDataDisplay
Imports System.Diagnostics

Public Class FilteredMarkerPointsGraph
    Inherits MarkerPointsGraph

    Sub New()
    End Sub

    Sub New(dataSource As IPointDataSource)
    End Sub

    Protected Overrides Sub OnRenderCore(dc As DrawingContext, state As RenderState)
        '' base.OnRenderCore
        If Me.DataSource Is Nothing Then Return
        If Me.Marker Is Nothing Then Return

        Dim left = Viewport.Visible.Location.X
        Dim right = Viewport.Visible.Location.X + Viewport.Visible.Size.Width
        Dim top = Viewport.Visible.Location.Y
        Dim bottom = Viewport.Visible.Location.Y + Viewport.Visible.Size.Height
        Dim transform = Plotter2D.Viewport.Transform
        Dim bounds As DataRect = DataRect.Empty
        Using enumerator As IPointEnumerator = Me.DataSource.GetEnumerator(GetContext())
            Dim point As Point = New Point()
            Do While enumerator.MoveNext()
                If point.X >= left AndAlso point.X <= right AndAlso point.Y >= top AndAlso point.Y <= bottom Then
                    Dim screenPoint As Point = point.DataToScreen(transform)
                    bounds = DataRect.Union(bounds, point)
                    Dim ta = DirectCast(Marker, ITransformAware)
                    If ta IsNot Nothing Then
                        ta.Transform = transform
                        Marker.Render(dc, screenPoint)
                    End If
                End If
        End Using
        Viewport2D.SetContentBounds(Me, bounds)
    End Sub
End Class