SphereGen

SphereGen Logo

Using Extended Reality (XR) to advance Industry 4.0  

iot industry 4.0 concept,industrial engineer(blurred) using hololens

As manufacturers increasingly embrace Industry 4.0 principles, it is important to understand the breadth of technologies that encompass Digital Transformation. These technologies include IoT, Extended Reality (AR/VR/MR), robotics, additive manufacturing, cloud computing, AI, and of course cyber security. This blog is about using Extended Reality in Manufacturing to advance the implementation of Industry 4.0

Extended Reality is the overarching term for Augmented (AR), Virtual (VR) and Mixed Reality (MR). XR applications represent a new age of process efficiencies, connectivity, and mobility. Manufacturers who adopt the use of this technology are offered new opportunities to compete not only among their peers in the US, but also globally.

As Extended Reality is a core tenet of I4.0, let’s explore what AR/VR/MR means and how manufacturing companies have benefited.

Augmented Reality (AR)

In AR, a digital image is overlayed on a real-world environment. An everyday example would be the Virtual Yellow 1st & Ten® line and line of scrimmage seen in an NFL broadcast. Brands use AR extensively in the consumer products industry to assist their retail buyers in making informed decisions with virtual, in situ “try-ons.” AR visualizations are typically deployed to a smartphone or tablet, while their 3D-only counterparts are available on webpages to increase ecommerce conversion rates. These models can also be embedded in a learning management system (LMS) to improve technical training and comprehension.

Virtual Reality (VR)

VR replaces the real world with a virtual world.  The most common experience is in gaming where players are depicted as avatars, existing and interacting within a virtual environment through a VR headset. Many thought leaders have envisioned a future where everyone will wear headsets and interact as avatars – living, working, and playing in these virtual worlds, sometimes referred to as Metaverse. As spatial computing improves, particularly data storage, competition among hardware vendors will increase. Headset brands such as HTC Vive, Meta Quest, HP Reverb and Apple Vision Pro will vie for domination in the market with some winners, some losers, and some consolidation.

Mixed Reality (MR)

In short Mixed Reality is the projection of holographic images onto a real-world setting.  Many people believe that Mixed Reality is simply an extension of Augmented Reality as viewed through a headset. Mixed Reality often includes other elements such as spatial awareness, perceptive distance and relationship between holograms and the ability to interact and manipulate virtual 3D objects. Leading hardware brands for industrial applications include HoloLens, Magic Leap, Lenovo, and Varjo with Apple entering in 2024.

Together, these three components are termed Extended Reality, encompassing all three technologies. In the past 5 years we have seen an explosion of XR applications worldwide. According to studies, the top use-cases for XR are defense and manufacturing followed by education and medical technology.

The Benefits of using Extended Reality in Manufacturing

Training 

One of the major goals of Industry 4.0 is to improve efficiency within the production process. Rethinking how employees are trained can improve the process and result in more effective outcomes. XR applications not only increase employee engagement within the training process, but also improve comprehension and retention of subject material.

Taking a look at traditional training methods used by most companies gives us insight as to why XR applications can be so much more impactful. Consider a typical training session where employees receive instruction by sitting in a classroom taking notes or watching videos. They will then try to apply what they have learned by working on physical machinery. Their reference materials are either paper-based manuals or videos served on tablets, laptops, or phones. Their instruction requires one or more trainers to be present providing active instruction at multiple points during the training.

Now picture this – a trainee dons a headset and begins an immersive training session. An XR application dynamically leads the trainee through a guided training course. As each part of a piece of equipment is explained, MR headsets can place depictive holograms over the physical equipment with arrows and signals, guiding them exactly how and where to operate or assemble the piece in front of them. Trainees can walk around the equipment, with the virtual guides following them, to better understand the operational perspective of controls and their positioning. They can reset the guided instructions to restart an area of training so they can practice until they feel they have mastered the step. Guided steps can require specific actions to be taken or ensure quiz completion before moving onto the next step. With instructions projected in the field of view, trainees are free to use both hands to work on equipment.

Although VR training is different, as the trainee is immersed in a completely virtual environment, the effectiveness of simulated training is equally as important. Trainees can practice in simulated space which is completely risk-free.

Why is this type of training impactful?

Visualization and interactive learning are more engaging. This aspect of learning in a 3D medium is what improves knowledge retention and comprehension, compared to passive settings such as classroom instruction, individual mentoring, or instructional videos. Trainees are able to use self-guided learning which allows them to move at their own pace at a reassuring comfort level. Virtual interactive training leads to safer training environments eliminating accidents and reducing stress. Trainers’ time is freed up allowing them to concentrate on evaluating the process and the end result of competency, instead of spending time instructing and travelling to classroom sites.

Extended Reality provides a dynamic alternative to traditional training in immersive risk-free environments ensuring businesses have a safe, confident, and competitive labor force. In addition, XR applications can be developed to be multilingual, ensuring that the same self-guided training program can be used in every location worldwide.

On the Job Technical instruction

In a factory setup, frontline workers, especially the newer ones, often require on-duty guidance from trained instructors or professionals. MR training guides provide impeccable three-dimensional step-by-step guidance so frontline workers can perform their job without relying on trainers and their availability. These virtual guides give great confidence to shop floor workers when approaching a new task or handling hazardous material/procedures.

The instructional guides direct workers to look where arrows, pointers, or hand gestures appear, highlighting where the next action is required. Workers’ hands are free to do the work before them as all instructions are in their field of view. Direction of gaze is all that is required to display the next step of instruction.

The chance for making errors is reduced and skillsets are improved, which enhances safety and security in the workplace.

Remote Field Service

All companies have a need at some point which requires the presence of a Subject Matter Expert (SME) on-site to fix or oversee downtime issues. The availability of SME’s is often limited and deploying them onsite is expensive and time-consuming.

MR applications facilitate remote assistance which connects frontline workers, either in the field or on the shop floor, to SME personnel through shared visualizations in real time. Technicians are empowered to collaborate by working together from separate locations yet seeing the same on-site view. They can examine equipment, processes, and service history to determine the best course of action for repair, thereby improving first time fix rates and reducing errors. In an industrial or lab setting, a technician can render digital manuals and videos to diagnose issues with complex devices, speeding up the time to repair and return the machinery to production status.

The pandemic highlighted the need to help manufacturers deal with issues when a machine or process goes down and they lack an on-site subject matter expert. For example, during the pandemic when travel to other countries was limited, one of our Manufacturing customers faced the challenge of needing support for equipment breakdowns in remote locations. Support relied on phone calls between the shop floor and a specialist, resulting in a slow turnaround on technical support calls. Available solutions were inefficient, involving cumbersome remote viewing methods such as video calls using a phone, iPad, and Laptops or if possible, extensive coordination for travel. Our client needed a solution to increase turnaround time on service calls to reduce production downtime and travel expense.

The solution resulted in implementing an MR solution which supports remote field assistance. The manufacturer purchased (1-2) MR headsets per factory so that when a piece of equipment requires service, a specialist or SME can evaluate the equipment remotely. Using annotations and visual cues, the specialist can walk the technician through the solution instead of having to travel to that location. Our client was able to successfully have SMEs remotely assist their technicians to resolve issues in an expedited timeframe, reducing downtime and cutting all travel costs.

Another of our customers, Ulbrich Steel, uses their remote assist application to expedite SME assistance during third shift hours. The MR application allows SMEs to respond from home, expertly guiding on-site technicians through questions and issues. This use case also results in expedited resolution, reducing overall downtime.

In addition to faster resolution times, the MR applications give the advantage of working hands free and heads up. There is no reason to consult paper manuals or look at other screens. SMEs can position helpful pdfs or guides within the technician’s field of view for referral. With the SME witnessing the same view of the technician, the SME can pinpoint where the technician is looking to ensure that their gaze is focusing on the right area of the work at hand.

Remote inspections and audit

The remote assist applications can act as “remote eyes” for the auditors or inspectors conducting technical inspections on the factory floor. This can serve internal inspections or virtual Gemba walks. A staff member wearing an MR headset can walk the shop floor, showing auditors exactly how operations are running. Auditors can perform quality and compliance checks or identify areas of concern, with no travelling required.

Enhanced Product Design, Development and Sales

As mentioned earlier, visualization is one of the key components to the benefits of XR. The references sited thus far have applied to the areas of training and instruction related to active production lines. However, XR is also extremely beneficial to the product lifecycle, beginning with design and carrying all the way through to the Sales process.

Using XR technologies, three-dimensional models can be used in product design and development. 3D tools allow collaboration for sharing design iterations across distributed project development teams. Digital renderings foster rapid prototyping which ensures that the design process is not hindered by 2D or 3D printing limitations. Design and quality issues are resolved before a product is launched.

The 3D models that are used in the ideation and design phase can then be used as a base to build sales and marketing product demonstrations. The models become a focus point of virtual catalogues and product demonstrations. Imagine changing tradeshow experiences to allow potential customers to virtually interact with your line of products. There are no size and weight limitations in showing virtual products, therefore no blockers exist to present a complete product line. Customers see products in detail and at scale as virtual representations can be manipulated to produce both results.

Marketers can build a product story which customers have continued access to long after the tradeshow ends. In addition, the tradeshow setup need only include a booth and headsets – no more shipping and rigging costs. XR sales presentations elicit emotional responses from buyers, allowing them to interact with your product line in as much detail as needed, resulting in better informed decision-making and higher close rates. Text and voice content can be presented as multi-lingual, saving the need to produce printed materials in multiple versions.

As a result, the overall product lifecycle is improved, from design to customer experience. Cost savings are realized in many steps of the lifecycle, enabling a better-quality product with less material and product waste along the way.

Enhanced Process Optimization Using Digital Twins

A digital twin is an exact virtual copy of an object – which could be a product, piece of equipment, person, process, supply chain, or even a complete business ecosystem. A comprehensive digital twin may be used to create virtual simulations to improve manufacturing and process efficiency. How is that done? Smart devices are interconnected and visualized in 3D through XR. IoT (Internet of Things) Sensors are outfitted to the vital areas of the object. These sensors produce data that is fed back to the counterpart digital twin through the cloud, to monitor performance such as temperature, energy consumption or atmosphere. This data can be viewed and manipulated on the digital twin creating virtual simulations. Virtual Twins can be used to monitor a product’s performance while improving design, operations, predictive maintenance, and end-of-life. Adjustments can be made to the digital twin to see how the system would change in real life before making any changes to the original system.

This cloud-connected data allows for structural views as well as operational. Even a Digital Twin of a building can be created to optimize the performance and efficiency of a physical plant’s energy and infrastructure.

It is estimated that the global digital twin market size was valued at USD 11.12 billion in 2022 and is projected to exhibit a compound annual growth rate (CAGR) of 37.5% from 2023 to 2030.

As more companies embrace IoT and big data, digital twin technology is becoming more popular. Below are seven ways that digital twins are used to improve manufacturing operations.

1. Testing Prototypes

Digital twins can be virtual prototypes during the design phase and be adjusted to test different simulations or designs before investing in a solid prototype. This saves time and money by reducing the number of iterations required to get the product into production.

2. Process Optimization

Sensors on a manufacturing line can be used to create a digital twin of the process and analyze important performance indicators. Adjustments to the digital twin can identify new ways to optimize production, reduce variances, and help with root-cause analysis.

3. Quality Management

Monitoring and responding to data from IoT sensors during production is essential for maintaining top quality and eliminating rework. The digital twin can model every part of the production process to identify where variances occur, or if better materials or processes can be used.

4. Supply Chain Management

Supply chains and logistics/distribution firms rely on digital twins to track and analyze key performance indicators, such as packaging performance, fleet management, and route efficiency. They are especially useful for optimizing just-in-time or just-in-sequence production and analyzing distribution routes.

5. Predictive Maintenance

Digital twins for individual equipment or manufacturing processes can identify variances that indicate the need for preventative repairs or maintenance before serious problems occur. They can also help optimize load levels, tool calibration, and cycle times.

6. Cross-Discipline Collaboration

The ready availability of operational data from digital twins makes it easy to share across disciplines, enabling collaboration, improved communication, and faster decision-making. Engineering, production, sales, and marketing can all work together, using the same data, to make more informed decisions.

7. Analyze the Customer Experience

Digital twins are often used to collect data over time that deliver insights into product performance, distribution, and end-user experience. This data can be used to help engineers and designers improve the customer response to the product, especially through customization and ease of use.

Summary

There is a shortage of workers skilled in advanced manufacturing technology. Industry 4.0 and digital transformation is an opportunity to enhance the capabilities of our workforce, not by replacing them but by upskilling in fields like automation and Augmented Reality to help maximize the efforts of people so they can do more with the resources they have.

For small and medium sized businesses (SMB), which make up a large part of the Manufacturing industry, companies may feel they lack the resources to adopt these recent technologies. The perception is that this type of technology is out of reach except for the largest of manufacturers. Many states have identified challenges to SMB adoption, such as which technology platform to choose, initial investment assessment and sourcing, a lack of digital training or digital culture, and cybersecurity concerns to protect intellectual property.

It is important for those companies to rely on organizations within their state which support initiatives for Industry 4.0 adoption. Affiliate entities may support digital transformation with technology education, demonstrations, grants, and adoption assistance. Education, research, and training play a significant role in State initiatives. Colleges and universities are active participants helping to shape the future of advanced manufacturing technology. Partnerships between state government, I4.0 service providers, consultancies and academia will mark a clear path forward for manufacturers of any size.

The benefits of using Extended Reality in Manufacturing are well beyond just supporting innovative initiatives. It is clear that XR delivers time and cost savings in use cases across multiple departments within a company.

This type of Industry 4.0 technology has clearly moved beyond “nice to have” status. This technology should be implemented in companies to optimize production processes and help companies remain competitive.

About SphereGen

SphereGen logo on white background

SphereGen is a unique solutions provider that specializes in cloud-based applications, Intelligent Automation, and Extended Reality (AR/VR/MR). We offer full-stack custom application development to help customers employ innovative technology to solve business problems.

Learn more about what we do in XR: https://www.spheregen.com/extended-reality

microsoft partner badge
uipath silver partner badge