What do you need to know to set up machine data acquisition correctly?
One of the first steps towards digitalization in industry is machine data acquisition. Data is collected, stored and evaluated via networked machines. The available data creates a completely new level of transparency, allowing better production control and uncovering potential for process optimization. Here we will highlight the most important basics of machine data acquisition in manufacturing and production. Finally, you will find an example of the simple machine data acquisition in practice.
Inhaltsverzeichnis
Inhaltsverzeichnis
- Cloud
- OPC UA
- MQTT
- Networking
- Data storage
- Types of Machine Data
- Industry 4.0 and IoT
- Edge-Computing / Fog-Computing
- Costs and Benefits
- Security
- Communication Protocols
- Communication Standards
- Connection of legacy systems
- Target systems
- OEE
- Practical Examples
- Simple machine data acquisition in practice
1. Cloud
Machine data acquisition has been around much longer than cloud technology. But the cloud is definitely one of the most interesting targets for machine data, as cloud computing offers many new possibilities and resources.
The applications in the cloud are manifold and expanding daily. Data storage in cloud can be carried out in relational databases, NoSLQ databases and so-called DataLakes. Basic applications for using the data are dashboards and monitoring, regardless of location. Another field of application is BigData and machine learning. Here, knowledge is gained from the large amount of recorded data using algorithms and self-learning systems, which can be used to optimise processes and to detect emerging error situations (predictive maintenance).
Data delivery to the cloud must be adapted to the technical requirements of the respective cloud platform operator. Many platforms can be supplied with data via MQTT, but some OPC UA servers can also be connected as data sources. Last but not least, the databases can also be described directly.
We documented connections to the major platforms here:
2. OPC UA
The communication standard OPC UA defines the mechanisms for data access (see What is OPC UA?). Every machine, every sensor and every other system that provides OPC UA data must be connected in the same way. The mechanisms for describing data structures and data points defined in OPC UA allow machine data acquisition to be set up completely independently of the underlying machine types and proprietary protocols. OPC UA also specifies how data point hierarchies can be set up and searched. A data provision of manufacturing and production data, which is basically based on OPC UA, is the key to a good machine data acquisition infrastructure, because it is:
- Expandable at any time
- Independent of manufacturer
- Platform independent
- Interoperable
- Cross-industry standard
3. MQTT
The MQTT Protocol was designed as a particularly lightweight protocol for low bandwidth and low-power devices (see What is MQTT?) Based on TCP/IP, data is sent according to the publisher/subscriber principle. Communication always runs via a broker, which as a central participant receives all messages and distributes them to recipients who are interested in selected message types. The sender defines the message type for each message (topic). The advantage of being lightweight is a disadvantage with regard to the definition of the structure. For the data of a message, the so-called payload, there are no specifications regarding format and structure. Very often data is packaged in a JSON or XML structure, but also here the sender and receiver must follow common definitions which are made outside of MQTT. In order to realise data acquisition from MQTT messages, the respective JSON/XML structures must be processed and evaluated (see Data acquisition with the MQTT plug-in).
4. Networking
IP networks
In order to operate automated machine data acquisition, the basic prerequisite is end-to-end networking. In industry, IP-based networking has become established for this purpose. Machines and sensors must therefore be equipped with network components that allow them to be connected to the network. The medium used for IP communication can be quite different (Ethernet cable, W-LAN, 5G, etc.). The ability of machines to connect to the network varies greatly. Modern machines have control systems that are equipped with a network connection by default. It only has to be activated and configured. For new acquisitions should this be one of the standard requirements. For older machines it has to be clarified if, how and by whom an interface can be upgraded. For machines where upgrading is too costly or not possible at all, there are alternative solutions (see Connection of old systems).
Bus systems
Networks within machines, plants or buildings, which are built with special (non-IP based) bus systems, are not suitable for machine data acquisition. Bus systems such as Profibus, Profinet, RS458, RS232, KNX, LON, BACNet, MBus, CAN should be connected to a higher-level control system with network connection. Alternatively, a hardware gateway solution can be used, which converts the bus-specific physics and communication to IP-based communication (for example by Wachendorff).
5. Data storage
Machine data is normally stored to generate data series that can be analysed. Where and how data is stored depends on its structure and application. If data is collected specifically for a target system, the target system will also handle the data storage itself.
The most sensible way to store machine data is to store it in a central database. Classical relational databases are often used for this purpose. In these databases (e.g. Microsoft SQL Server, Oracle, MySQL) data is stored in fixed table structures and can be made available from there to various target systems.
In addition to relational databases, more and more databases with new concepts are gaining acceptance. In the so-called NoSQL databases, data is not stored in tables but in loose structures, which allow a completely new type of access, larger amounts of data and a high degree of flexibility with regard to the structure of the data. Special database types are available for different types of data (document oriented, graph databases, time series, key-value, multi-value).
Which database should be used in particular must be checked against the target systems, the type of data and the types of use.
6. Types of Machine Data
The term “machine data” summarises all data that can be retrieved from a machine, typically at run time. However, what the content of data is and how it is to be interpreted can be very different. In principle, a distinction can be made between simple data points and complex data sets.
Simple data points that are recorded continuously or when changes occur are process data. This data is recorded and stored with the current value and a time stamp as reference. This allows the status of the machines to be analysed using the historical data series.
More complex data from the machine are recorded as coherent data sets. In addition to the optional time stamp, these data contain above all unique references to data objects for which they provide associated data. These can be production or manufacturing orders or also material numbers, machine numbers, recipe numbers, batch numbers, etc. The data supplied in the data record for this purpose is then dependent on the application. For example, it can be test results, consumption data, finished goods information, traceability data and similar. The aim of the industry-specific communication standards is to describe many of these applications in general terms and thus to define data records that are as similar as possible, independent of the machine manufacturer.
7. Industry 4.0 and IoT
Machine data acquisition is an important foundation for the two topics Industry 4.0 and IoT (Internet of Things).
Industry 4.0
Industry 4.0 describes the complete networking and communication of systems in production among each other. Industry 4.0 networking makes it possible to develop production towards self-optimisation and self-control. Much of the data exchanged between the systems is machine data. A well developed data acquisition system is therefore an ideal starting point for Industry 4.0.
IoT
With the Internet of Things (IoT), an important use case is the transfer of status information to the cloud. Typically a digital twin is mapped there, which virtually has the same state as the real object. Using an existing machine data acquisition system, the machine can be quickly integrated into an IoT platform and the concept of the digital twin can be realised for the machine.
8. Edge-Computing / Fog-Computing
With cloud computing, applications are moved to the cloud and are therefore no longer directly available in the production network. However, there remain tasks that have to be run close to production, i.e. at the “edge”. Data must be pre-processed, algorithms must be available to production without latency through data transfers and data must be compressed for transfer to the cloud. This is exactly the area of Edge Computing. Especially for machine data acquisition, this is an important concept if data acquisition for the cloud is to take place. Small integrated devices with corresponding software running on them are regarded as a platform for edge computing.
Fog computing, in contrast, is somewhat broader and also describes locally operating server systems that manage sensitive data or run applications with a demand for high availability. Fog computing also connects the edge and the cloud.
9. Costs and Benefits
Of course, there are also costs involved in setting up machine data acquisition. These have to be invested in physical networking, the expansion of control systems, software for recording and storage and, last but not least, the work involved in planning and implementation. But is the effort worth it?
The benefits of the recorded machine data are manifold and the investment is almost always worthwhile. Especially the use of the data in different systems for ever new applications constantly increases the value of machine data acquisition. Some of the many profitable applications are:
- Automation of manual capture
- Optimisation of production processes based on the knowledge gained from machine data
- Improvement of reaction times within production through fast information
- More precise planning options for personnel and raw materials through more accurate time recording and material consumption reports
- Avoidance / shortening of machine failures by immediate fault message distribution
- Improved customer service through more accurate information on delivery times
- Optimised machine maintenance by more precise maintenance cycle compliance and predictive maintenance
- … and many more
10. Security
If a machine is networked for machine data acquisition, IT security must also be considered from the outset. Because with networking, a machine is always more vulnerable than without networking. That is a fact. But there are many ways to reduce the resulting risks to a minimum by using generally accepted security mechanisms.
Network disconnection
The most basic protection against unauthorised access is strict network separation. A clear distinction must be made between the different network areas, how they are connected and which communication between the networks is permitted. This is implemented with a firewall. For production networks, solutions are also available which segment the production network as such again and seal off individual production islands separately. Only the paths required for machine data acquisition are opened for defined clients.
Certificates
The standardised communication protocols also contain integrated mechanisms to guarantee the security of data collection. Thus, security has been implemented as a standard requirement in the OPC UA concept. In addition to encrypted transmission, OPC UA also provides for certificate exchange between client and server. Unfortunately, many other protocols originate from a time when the focus was not yet on security, so that integrated mechanisms are missing and always have to be configured additionally. OPC UA is therefore also a good choice from a “security” perspective.
Access control
Last but not least, user authentication is also required for machine data acquisition in order to restrict access to the data by role. For modern systems for data storage (databases / cloud) these functionalities are standard and only need to be actively used.
11. Communication Protocols
Proprietary
In manufacturing and production, there is usually a heterogeneous machine park and, in addition, a multitude of intelligent sensors and devices. The communication protocols spoken for data retrieval via IP-based communication can be proprietary or standardised.
In case of a proprietary protocol, the manufacturer has defined his own protocol. This allows access to all the special possibilities of the data source and, ideally, the communication speed is highly optimised. However, the special protocol must also be implemented by the client for machine data acquisition in order to connect the data. To access such controllers and devices, software components are particularly suitable which already implement the proprietary protocols and translate them into a standardised protocol. A good example of this is the Kepware OPC Server, which uses over 160 drivers to speak many of these protocols and then converts them into OPC UA, among others.
Standardised
The standardised protocols have been defined by associations and consortia in order to avoid the disadvantages of proprietary protocols and to allow easy connection of data sources and data sinks. Besides quite hardware-related standards like ModbusOnTCP or BACNet/IP, large and widespread standards like MQTT, MTConnect and OPC UA have been established. MQTT is primarily found in the IoT environment. MTConnect has been developed especially for machine tools and is mainly used in North America. The OPC standard for machine data acquisition has become established for the industry over the last 20 years. More and more controllers and devices provide OPC UA directly as communication protocol. For others there is the conversion of proprietary protocols to OPC UA with corresponding OPC servers. If OPC UA is defined as the lowest common denominator for data acquisition, all data sources and clients can be aligned to it, thus creating expandable and standardised machine data acquisition.
12. Communication Standards
In addition to the pure standardisation of communication protocols, there are also efforts in many industries to standardise the data structures for machine data acquisition. Various working groups have jointly defined which data is exchanged with machines and in what form. Examples of this among many others are EUROMAP 63 (injection moulding), Weihenstephan Standard (beverages), PackML (packaging), IEC 61850 (electrical switchgear). If a machine implements the relevant industry standard, it is ensured that certain data is provided by the machine. The detailed coordination and definition of interfaces at data point level is no longer necessary and the data acquisition setup is greatly accelerated.
A combination of industry-specific standards and the standardised communication protocol OPC UA form the recently advanced Companion Specifications of the OPC Foundation. The advantages are combined in the specifications, so that a machine can be expected to have predefined OPC UA data point structures when implementing a Companion Specification. Examples are EUROMAP 77 (injection moulding), OPC UA PackML (packaging), AutoID (Ident), Umati (CNC), IEC 61850 (electrical switchgear). Many more are currently being worked on.
13. Connection of legacy systems
Not all machines can be easily incorporated into a modern network. Especially older machines are difficult to integrate. The reasons are frequent:
- no control available, purely mechanical machine or only simple electronics
- Controller cannot be extended by a network connection
- Costs, effort and risk are too high for network upgrade
- Lack of know-how for upgrading
However, if the machine data of such a plant are nevertheless important information, a lot of data can be collected by small controllers. These are installed on or in the machines. Small controllers, such as those from WAGO (PFC200), Beckhoff (Embedded PC) or Siemens (S7-1200), can record and count mechanical states and processes via sensors, but electrical signals can also be tapped into the old system via wiring. As an interface for machine data acquisition, the small controllers are all equipped with OPC UA and thus integrate perfectly into the standardised infrastructure.
14. Target systems
Machine data acquisition serves the transparency of the production process. The data acquisition as such can always be viewed and set up independently. However, only the transfer of data to target systems results in high-grade knowledge and application possibilities. In production and manufacturing there are different systems which are used to control, monitor and plan production. Which systems are used depends on the industry and size of production. These systems can often be found:
- ERP (Enterprise Ressource Planning)
- MES (Manufacturing Execution System)
- Quality assurance
- EnMS (energy data)
- Service / Maintenance
- Reporting
Each of these systems is dependent on information from production and can work more efficiently if data from machine data acquisition is automatically fed into the system instead of being entered by manual processes.
15. OEE – Overall Equipment Efficency
The OEE code number is a special application of machine data acquisition. For the OEE, fixed formulas and data points are specified which are used for calculation. OEE is a product of three individual factors:
OEE = utilisation factor (UF) x efficiency factor (EF) x quality rate (QR)
Degree of utilisation = Real available time / planned occupancy time
The degree of utilisation describes the ratio between the planned operating time of the machine and the actual operating time, excluding the times during which the machines were out of order due to malfunctions and similar. In the best case, the machine provides a signal for the machine data acquisition, which signals that the machine is ready for operation and which can then be recorded, or faults are recorded, the duration of which is then billed as downtime.
Performance ratio = actual production quantity / target production quantity (cycle rate x machine running time)
To determine the degree of efficiency, the actual output of the machines is divided by the target quantity based on the output quantity. This yields the percentage of the expected production output.
Quality rate = yield / total quantity
The quality rate relates the number of good quality products to the total production quantity including rejects.
16. Practical Examples
Practical examples demonstrate the motivation and benefits of implemented machine data acquisition. Here you can read a few reports from our customers:
- Automotive supplier Auria Solutions
- Chainsaw manufacturer Stihl
- Colour measurement and control systems XRrite
- Dairy factory DMK Altentreptow
- Label manufacturer HERMA
Simple machine data acquisition in practice
In practice, machine data acquisition can optimize a company’s production. With the transparency gained, data is collected, stored and evaluated, and better production control is achieved.
The captured and stored data can be used by different departments. The production department monitors the current production speed in real time and intervenes in production at an early stage if necessary. Thus, production utilization can be improved or even production stops can be prevented. ERP systems such as SAP or Web Services can additionally record and evaluate the machine data. Thus, other internal departments such as controlling or logistics benefit from this and are automatically informed about completed orders or provided with current data. Real-time machine data acquisition, realizable with software such as OPC Router, provides company-wide transparency, enables real-time production control and optimizes every production.
Further information
Further information
Learn more about Industry 4.0! The fourth industrial revolution through digitalization: people, machines and products are directly networked with each other. Read more in our article!
OPC UA enables standardized access to machines, devices and other systems in Industrie 4.0, thus ensuring manufacturer-independent data exchange. In our Knowledge Base you will find an overview of the functionality and terms of the most important communication protocol for Industrie 4.0 and the Industrial Internet of Things (IIoT).
OEE (Overall Equipment Effectiveness) measures effectiveness in the areas of availability, performance and quality. The OEE figure provides an overview of the overall equipment effectiveness and shows optimization opportunities. Learn in our OEE article all the important basics and how they calculate the OEE to improve your production.
Further interesting articles on the topics of Industry 4.0, cloud, technology, alerting and practical application examples as well as case studies can be found in our Knowledge Base.