Weak Signals: Hidden Meanings & Relationships in Data
In addition to the challenges of silos, some of the information captured – like a comment in a field notebook, a question in an email message, or a temperature reading in a sensor log – provides only weak-signal intelligence about quality issues. The signal is “weak” because its meaning can only be understood once it is connected with other data, and a pattern revealed.
For example, how could a supervisor know that a single defect – a specific circuit that overheats – is behind multiple, scattered reports of “bridges” or “capacitors” or “switches” “overheating” or “smoking” or “burning” or “sparking”?
The supervisor could not know unless some type of semantic classification were applied to the textual data, and the notes cross-referenced with other data like location, customer, or product or part ID numbers. Sometimes, however, signals are buried in data sets so large that the conventional tools normally used to detect such connections or patterns simply cannot be used.
This is a challenge that must be overcome in order to detect and address issues as early as possible, ideally through design changes, preventive maintenance, or revised usage guidelines, rather than through warranty claims, recalls, and lawsuits.
Fortunately, it is now easier to access and use dedicated tools designed for large, heterogeneous data environments, and aimed at performing advanced analytics (possibly using machine learning), so that silos of information can be bridged and weak-signal data transformed into clear and timely intelligence.
Cognitive Insight Engines
In this context, analytics systems known as “insight engines” are using big data processing tools, search engine indexing and advanced analytics to enable users to collect, organize, enhance, explore and analyze quality-related data across large and diverse data collections.
Such insight engines are designed to augment the cognitive processes that human beings follow when exploring or analyzing information.
“Insight engines augment search technology with artificial intelligence to deliver insights – in context and using various modalities – derived from the full range of enterprise content and data.”
Gartner, Inc., Summary, Magic Quadrant for Insight Engines
Some insight engines have functions specifically tailored for issue or event detection and investigation, while others are more geared to semantic search and discovery. What they have in common, though, is that all of them can provide unified access to diverse internal and external data collections, with those that can aggregate unstructured content (like customer complaints in online forums) and structured or semi-structured data (like database records or sensor log files) of greatest value to quality and reliability analytics.
To better understand how insight engines work, and what kinds of analytic processes can be used to address quality issues, in the next blog installment we’ll look at the EXALEAD insight engine and its use of advanced analytics to deliver quality and reliability intelligence. Read Part 4: EXALEAD Asset Quality Intelligence solution.
Part 3: Weak-signal intelligence and Cognitive Insight Engines