{"id":60681,"date":"2025-09-26T11:08:52","date_gmt":"2025-09-26T11:08:52","guid":{"rendered":"https:\/\/becolve.com\/blog\/investigating-the-mysteries-of-big-data\/"},"modified":"2025-09-26T11:08:52","modified_gmt":"2025-09-26T11:08:52","slug":"investigating-the-mysteries-of-big-data","status":"publish","type":"blog","link":"https:\/\/becolve.com\/en\/blog\/investigating-the-mysteries-of-big-data\/","title":{"rendered":"Investigating the Mysteries of Big Data"},"content":{"rendered":"

\"Data<\/p>\n

A few days ago, the magazine AutomationWorld<\/strong> published an interesting article<\/a> about the future jobs that will be generated due to the digital transformation processes in which we are all involved. One of them, which gives the article its name, is that of Data Detective<\/strong>, a position in charge of \u2018investigating the mysteries of Big Data, uncovering secrets within the content by examining sensors, devices, fog computing and neural networks.\u2019<\/em> <\/p>\n

From my point of view, I agree that this position, with this name or another, will end up existing, although it is true that I would add a nuance. A good Data Detective should investigate why the results extracted from the data do not apply or coincide with reality.<\/strong> <\/p>\n

Currently, all process and infrastructure industries are working on or evaluating data analysis tools, whether they are analytical tools, Business Intelligence<\/strong>, preventive maintenance, resource optimization, etc. But the approach of these integrations, for the most part, is usually done from ‘above’. This type of tool feeds on data stored in databases<\/strong>, and it is from these that they draw their conclusions from large processes and complex algorithms. <\/p>\n

\"Data<\/p>\n

For the industrial and infrastructure world, there is a fundamental problem in this. This data stored in databases<\/strong> has not been stored there for the purpose of being exploited by this type of tool, but has been generated and contextualized for another type of application \u2013 a SCADA<\/strong> typically -. This differential fact implies that these tools are working on a sampling field that is not properly designed or prepared for it. In the industrial world today there is a maxim: the technological bottleneck for the use of these tools is a correct acquisition of field data. <\/p>\n

To give an example, in an automotive parts manufacturing industry it is known that 6% of the parts are defective and are discarded at the time. That information is what reaches the repositories using their SCADA and ERP<\/strong>. However, the most interesting thing would be for these tools to know more about the discarded parts, their manufacturing route and the different elements that were involved, material, supplier, stamping machinery, transport belts, active personnel, etc. But in order to make these correlations, the system needs to meet certain characteristics that it will not have. Faced with this scenario, these tools will either not know how to draw a conclusion or, even worse, they will draw the wrong one. <\/p>\n

That is why I believe that we will reach the figure of the Data Detective<\/strong>. I do not venture into whether it will be an internal position of the companies or external consultancies, but I am clear about its mission. Investigate why the contracted exploitation tools are not having the desired ROI<\/strong>, that is, once the tool and its algorithms have been reviewed and it has been verified that everything is correct at that level, where are the contextualization failures of the data sources? In its correct acquisition, in its manipulation, in the sampling frequency, in its temporal stamping, in the volume with which it is worked, in that I need to acquire information from some key elements…? <\/p>\n

\"Data<\/p>\n

This Data Detective will most likely have to investigate, coordinate and understand the multiple actors involved in the process<\/strong> and that, most likely, is that in case these tools do not have the expected results, no one will feel responsible for it. And, much to our regret, they will surely all be right. If the data sources have not been properly prepared for this, a preventive maintenance application will be able to work with what it can, but it is evident that it will not be able to extract the most accurate conclusions, for example. <\/p>\n

Therefore, to avoid having to need a Data Detective we can propose several initiatives, not mutually exclusive:<\/strong><\/p>\n