Statistics of runs

Datamin is a full-cycle data streaming software, therefore not only does it help manage and execute pipelines but also collects detailed statistics of runs for each pipeline and task.

What information is collected:

  • Starting time

  • Ending time

  • Duration in milliseconds

  • Task ID and type

  • Task output

  • Success or failure

Having this information for each of the pipelines Datamin provides users with multiple different ways of working with the statistics:

  • A main dashboard that contains accumulated statistics about all pipelines and metrics

  • Log of runs for every pipeline and tasks

  • Aggregated statistics for every pipeline and task within the selected date range

  • Powerful API to pull statistics in different forms that can be integrated with your BI software if you want to use these historical data for your reports and dashboards

  • Slow task profiler that allows you to have constant control over your bottlenecks

Last updated