Technology Used: Ignition 8.0, MS SQL Server
How to get more information out of the vast amounts of data collected from the SCADA system in a meaningful way?
The client uses the Ignition platform as a SCADA solution, and for collecting vast amounts of data from numerous sources. The raw data is received with a resolution of a few seconds from a vast number of tags and stored in Ignition’s Tag Historian.
The mantra these days is: “as much data as possible,” which often means you sample or log data with a very high resolution, which is also the case here. The general idea is that storing data is not costly, and therefore “the more, the better.” But that does not necessarily make us any smarter!
One thing is storing data in a database; retrieving data can be a completely different thing. Pure data is all noise and challenging for human beings to understand without some supporting narrative.
A trend tool consisting of data aggregation and user-friendly charting.
We started with a close dialogue with the people in charge of daily operations and mapped what kind of information is crucial to support their business operations. Based on these talks, the decision was to create a data aggregation solution next to the Tag Historian.
The solution aggregates raw data into meaningful time intervals. The aggregation process turns the enormous amount of raw data into more accessible aggregations: 1 minute, 5 minutes, 1 hour and one day.
With the data aggregation mapping in place, we created a trend tool consisting of a front-end in Ignition based on the Easy Chart component plus some search functions, and a back-end in MS SQL Server consisting of the data aggregation pyramid and a search facility.
There are two clear benefits of the aggregation solution: Speed and information.
Speed is related to how long time does a query take to return the information you want? An essential issue for urgent situations, for example, handling a break-down. Without data aggregation, your queries will be directly on the raw data. That is not a problem if you ask for “all data for these 10 tags for the past 2 hours.” But if you ask for “all data for these 10 tags for the past month,” your query takes considerably longer.
Information is related to creating more meaning in the data. Aggregation makes it easier to identify patterns and trends in data, which are not immediately visible. The aggregation process reduces random noise in the data. Random noise can be many things, such as logging mistakes or clear outliers. When looking directly at the raw data, such random noise will make it difficult to spot the more important underlying trends. Aggregated trends help to study the ecosystem of the data, variation and deviation.