Developing a virtual reality and AI-based framework for advanced digital manufacturing and nearshoring opportunities in Mexico

Ponce, Pedro; Anthony, Brian; Bradley, Russel; Maldonado-Romo, Javier; Méndez, Juana Isabel; Montesinos, Luis; Molina, Arturo

doi:10.1038/s41598-024-61514-4

Download PDF

Article
Open access
Published: 16 May 2024

Developing a virtual reality and AI-based framework for advanced digital manufacturing and nearshoring opportunities in Mexico

Pedro Ponce¹,
Brian Anthony²,
Russel Bradley²,
Javier Maldonado-Romo¹,
Juana Isabel Méndez¹,
Luis Montesinos¹ &
…
Arturo Molina¹

Scientific Reports volume 14, Article number: 11214 (2024) Cite this article

747 Accesses
Metrics details

Subjects

Abstract

The growing expansion of the manufacturing sector, particularly in Mexico, has revealed a spectrum of nearshoring opportunities yet is paralleled by a discernible void in educational tools for various stakeholders, such as engineers, students, and decision-makers. This paper introduces a state-of-the-art framework, incorporating virtual reality (VR) and artificial intelligence (AI) to metamorphose the pedagogy of advanced manufacturing systems. Through a case study focused on the design, production, and evaluation of a robotic platform, the framework endeavors to offer an exhaustive educational experience via an interactive VR environment, encapsulating (1) Robotic platform system design and modeling, enabling users to immerse themselves in the design and simulation of robotic platforms under varied conditions; (2) Virtual manufacturing company, presenting a detailed virtual manufacturing setup to enhance users’ comprehension of manufacturing processes and systems, and problem-solving in realistic settings; and (3) Product evaluation, wherein users employ VR to meticulously assess the robotic platform, ensuring optimal functionality and customer satisfaction. This innovative framework melds theoretical acumen with practical application in advanced manufacturing, preparing entities to navigate Mexico’s manufacturing sector’s vibrant and competitive nearshoring landscape. It creates an immersive environment for understanding modern manufacturing challenges, fostering Mexico’s manufacturing sector growth, and maximizing nearshoring opportunities for stakeholders.

Knowledge-integrated machine learning for materials: lessons from gameplaying and robotics

Article 24 January 2023

Sustainable manufacturing through application of reconfigurable and intelligent systems in production processes: a system perspective

Article Open access 16 December 2023

Extended reality for biomedicine

Article 02 March 2023

Introduction

Nearshoring involves a transnational organization establishing manufacturing, assembly, or sourcing processes in the country or continent where they intend to sell their products, reducing operational interruptions and ensuring the availability of supplies and efficient product delivery to the target market. Mexico has become an attractive option for companies looking to relocate their operations, particularly due to its proximity to the United States and the ease of exporting to this market under the United States–Mexico–Canada Agreement (USMCA). The Inter-American Development Bank (IADB) estimated that nearshoring could add $78 billion, $64 billion in exported goods¹. Table 1 lists potential opportunities in nearshoring and shows the top 5 Latin American and Caribbean countries (LAC), in which Mexico is at the top position. Quick wins refers to the objective of securing 15% of the U.S. imports for the top 50 products that each LAC country was already exporting to the U.S. Within the context of intra-LAC quick wins, the goal is to attain a 15% market share of LAC imports for the top 50 products that each LAC country was exporting. In the case of medium-term opportunities, the aim is to achieve a 15% market share of U.S. imports for the top 50 products that LAC countries were exporting to Europe.

Table 1 Top five countries with potential opportunities of nearshoring for trade in goods (US$ millions)¹.

Full size table

The IADB estimates that Mexico could generate up to $35.3 billion annually from nearshoring exports. Nevertheless, these imply several challenges, such as:

Selecting the ideal business location (site selection), talent acquisition, and meeting trade agreement requirements.
Reducing greenhouse gas emissions and creating a regulatory framework to facilitate digital commerce.
Creating jobs, generating wealth, and offering hope for citizens seeking a better future.

Therefore, Mexico must develop a production ecosystem with a steady supply of raw materials, skilled labor, and improved logistics infrastructure and security². Moreover, one of the sectors that most benefit from nearshoring is manufacturing. Thus, the future of manufacturing, especially within the context of Industry 4.0, needs a workforce proficient in a new set of skills, given the technological advancements and sustainability needs that are reshaping the sector globally. Several recent papers have embarked on the journey to identify and analyze these future skills, providing a roadmap for what will come in the manufacturing industry:

Rodzalan et al. (2022) undertook a systematic literature review to explore the current and future skills that align with the demands of Industry 4.0, with a particular focus on Malaysia. This paper shows the impact of Industry 4.0 on the education and manufacturing industries, highlighting the imperative to bridge the gap between current and future skills of Industry 4.0³.
Jurczuk and Florea (2022) identified existing gaps and future needs for digital design skills to support and comprehend the automation of business processes. A survey conducted in manufacturing companies across six European countries formulated a future-oriented digital design competence framework that addresses the requirements of process design and automation in the Factory of the Future⁴.
Li (2022) emphasized that by 2025, 50% of all employees will need reskilling due to adopting new technology and that over two-thirds of skills considered necessary in today’s job requirements will change. The paper provides a blueprint as a reference for individuals to learn and acquire new skills and knowledge, suggesting that life-long learning should be integrated into an organization’s strategic goals⁵.
Akyaziet al. (2022) developed an automated skill database for the manufacturing industry, particularly focusing on transversal occupations of this sector related to industrial symbiosis (IS) and energy efficiency (EE). The database incorporates each profile’s current and future skill needs, providing a valuable perspective on the future skills requirements resulting from industrial changes and sustainability needs⁶.

Incorporating artificial intelligence (AI) and virtual reality (VR) into this context, these technologies are becoming pivotal in enhancing skills and providing innovative solutions in the manufacturing sector. For instance, the Umeed: VR Game Using NLP Models and Latent Semantic Analysis for Conversation Therapy for People with Speech Disorders paper discusses using VR and NLP for training and skill enhancement through interactive, conversational scenarios, which can be extrapolated to manufacturing, where communication and interaction with systems and teams are crucial⁷. Moreover, the paper Priority Experience Replay DQN for Training an Agent in Virtual Reality Game for Kids with Paraplegia describes how an AI agent was created and then taught using the deep reinforcement learning deep Q-learning network (DQN) algorithm in a virtual reality game, providing feedback specifically suited to the individual’s skills and development. This concept of using AI to provide real-time, personalized feedback in a VR environment can be adapted to train manufacturing professionals in various tasks and operations, ensuring safety and precision⁸. Other type of approaches that involve VR focus in manufacturing, such as the Automatic assembly simulation of product in a virtual environment based on interaction feature pair, describes the automatic assembly simulation that is achieved from two main aspects: the interaction sequence provides the assembly order of the parts and the motion planning with a collision-free path for each part in a virtual environment⁹. Testing the reliability of monocular obstacle detection methods in a simulated 3D factory environment presents two proposed strategies for obstacle identification in an industrial context for driverless cars. The first method is based on convolutional neural networks (CNNs) for semantic segmentation, while the second method utilizes CNNs for depth estimation. Furthermore, a 3D virtual environment is developed in order to generate RGB pictures, ground truth segmentation, and depth images¹⁰. A survey on HoloLens AR in support of human-centric intelligent manufacturing offers a comprehensive review of the existing body of research on the application of HoloLens in industrial operations during the past several years. It aims to provide a detailed overview of the technical aspects of HoloLens-driven intelligent manufacturing¹¹. Finally, Towards user empowerment in product design: a mixed reality tool for interactive virtual prototyping enables user participation throughout the first phases of the design process and facilitates the direct incorporation of their preferences, input, and specifications into the design models. The virtual environment and user interfaces provide the capacity for convenient customization to suit certain design environments¹².

While the papers above provide a comprehensive overview of the skills and technological advancements in the global manufacturing sector, there is a noticeable gap in the literature regarding developing a virtual reality and AI-based framework for Advanced Manufacturing Education and Nearshoring Opportunities in Mexico. This could be an area ripe for exploration and research, considering the growing importance of nearshoring in developed countries and the potential of AI and VR in enhancing manufacturing education and operations. Developing a virtual reality (VR) and artificial intelligence (AI)-based framework for advanced manufacturing education and nearshoring opportunities, particularly in Mexico, can offer many advantages and significantly empower nearshoring endeavors.

Enhanced training and skill development can be achieved through VR, providing immersive training experiences and enabling workers to learn and practice skills in a safe, controlled virtual environment. At the same time, AI can personalize training programs, adapting them to individual learning curves and providing targeted feedback to enhance skill acquisition. AI algorithms can also improve operational efficiency by optimizing manufacturing processes, reducing waste, and enhancing productivity. VR is used for virtual prototyping and testing, reducing the need for physical models and enabling more rapid product development. Safety and risk management can also be enhanced with VR simulations that train workers to handle hazardous situations without real-world risk. AI predicts and prevents operational failures, enhancing workplace safety. Global collaboration can be facilitated through VR, enabling virtual collaboration and allowing experts worldwide to interact remotely with the manufacturing environment. In contrast, AI can facilitate communication and coordination among global teams, streamlining international collaboration.

Figure 1 illustrates an example of a virtual reality application in providing immersive training within a production line context (as shown in Fig. 1a). This visual representation showcases the capacity of virtual reality technology to create an environment where workers can undergo training in an immersive and controlled virtual setting. Moreover, it demonstrates how critical system information can be seamlessly integrated and presented within this immersive environment (Fig. 1b), thereby facilitating an effective learning experience for the workforce.

Additionally, a significant challenge is the maintenance of the line of production. Manufacturing systems are designed to be useful for years and experience progressive degradation, causing signs of oldness like noise, temperature changes, and pollution. Real-time sensing techniques and monitoring software enable continuous-time stochastic models for condition-based preventive maintenance. However, the side effects of degradation and the costs due to aging can lead to better-quality items with increased usage¹³. Likewise, existing maintenance based on time where maintenance scheduling literature is categorized into four main groups: ’Scheduling with a rate-modifying activity (RMA),’ ’Scheduling in a flexible window,’ ’Scheduling before reaching machine age,’ and ’Scheduling with a preventive maintenance policy.’ The first category deals with RMA, the second category involves advance interval scheduling, the third category requires scheduling before reaching a certain age, and the last category focuses on machine availability, considering two main objectives: (I) Maximizing machine availability, the goal of which is to maximize machine availability by determining the interval between preventive maintenance activities based on the reliability model of a system and the failure and repair data. (II) Maintaining a minimum reliability threshold assumes that the failure rate of a system tends to increase with time, making it susceptible to failures caused by age or wear¹⁴.

Likewise, Todescato et al.¹⁵ highlights the benefits of implementing reconfigurable and intelligent systems in production processes for sustainable manufacturing using digital technologies. These systems offer flexibility, rapid adaptation, customization capabilities, and integration of hardware and software flexibility. They also enhance sustainability by incorporating sustainability metrics into scheduling and decision-making processes. These systems can improve product quality and sustainability by integrating economic, environmental, and social dimensions, ensuring company growth. The holistic approach to flexibility and sustainability combines automation, information technologies, and artificial intelligence.

By strategically devising and adopting a virtual reality (VR) and artificial intelligence (AI)-infused framework for advancing manufacturing training, Mexico has the potential to substantially elevate its manufacturing capabilities. This concerted effort can position Mexico as an enticing nearshoring hub for international enterprises, particularly those originating from developed nations seeking a blend of skilled labor, technological prowess, and geographical proximity. The successful execution of such a strategy could potentially bolster the manufacturing sector within Mexico and foster a mutually beneficial rapport with its neighboring nations. This synergy, in turn, holds the promise of fostering economic expansion and technological progress across the region, filling a gap detected in the state of the art for nearshoring.

Likewise, including a case study enhances the practical relevance. The authors provide a concrete example of how the proposed framework can be implemented and its potential impact on assembling a virtual robot based on a physical representation in a virtual factory to estimate the production line and elements to produce, using a novel selection of elements to obtain the suitable final product to release that satisfy the needs on a specific environment.

Proposed framework

Nearshoring entails outsourcing work to neighboring countries with lower wages to leverage geographic and temporal proximity, utilizing cultural, economic, and political linkages. Considering evaluation factors in distance, trade-offs, and supply costs is essential, integrating quantitative chain cost calculations and qualitative assessments¹⁶. Likewise, nearshoring is a business strategy involving outsourcing to neighboring countries and offers advantages over traditional offshoring. It emphasizes proximity, fostering better communication and cultural alignment. Cost savings are achieved without sacrificing quality, leading to higher customer satisfaction. Nearshore resources enhance project management, expertise, and risk management through local insights and explicit governance models¹⁷.

In response to the rapid expansion of the manufacturing sector, particularly in regions like Mexico, there is a pressing need to equip engineers, students, operators, and decision-makers with the knowledge and skills essential for success in this dynamic industry. To address these needs to face nearshoring, a comprehensive educational framework has been developed, which can be employed in areas such as immersive digital mockups for realistic design validation, realism at scale, VR multi-user conferencing, reduction of energy, large-scale world simulation, and virtual factories^18,19,20. In this way, digital tools allow the development of a new way of interacting with manufacturing scenarios in an avant-garde way to improve processes, products, and services. In addition, it allows the incorporation of scaling considerations to expand since data are collected that provide information for decision-making in real-time.

This framework leverages innovative technologies such as Virtual Reality (VR) and Artificial Intelligence (AI) to revolutionize the pedagogy of advanced manufacturing systems, offering an immersive and transformative learning experience. Figure 2 depicts the proposed framework that unfolds through a well-defined step-by-step process:

Step 1: Training gaps identification and manufacturing processes requirements: This initial step involves conducting a thorough assessment of the target audience to identify specific educational gaps and requirements related to manufacturing processes. The insights gained from this assessment form the foundation for the framework’s development. Thus, the framework is introduced with a clear objective: to bridge training gaps and empower learners to excel in manufacturing processes. The integration of virtual reality (VR) and artificial intelligence (AI) is highlighted, as these technologies are at the framework’s core and play a pivotal role in the educational journey.
Step 2: Virtual product design and modeling: This step empowers users with the skills and knowledge needed to comprehend, design, and simulate robotic platforms. The virtual design studio allows users to create, modify, and fine-tune robotic platform designs in an interactive VR environment. At the same time, the Simulation Lab provides insights into how different parameters and configurations impact performance and functionality.
Step 3: Virtual manufacturing company development: This step provides a comprehensive understanding of manufacturing processes, equipment utilization, machinery maintenance, and inventory management. Users engage in hands-on activities within the virtual factory, interacting with production lines and applying problem-solving skills in realistic manufacturing settings.
Step 4: Virtual product evaluation: Immersive environment testing and adjustment to evaluate the final product, a robotic platform, using VR technology. The users assess the robotic platform’s performance under different conditions and scenarios, and simulated customer interactions and feedback enable users to refine the product based on customer expectations. After this step, feedback and adjustments are performed based on the key takeaways and benefits. The framework’s role in preparing individuals and organizations for success in the manufacturing sector is emphasized. Continuous updates and enhancements ensure the framework remains agile and adaptable to evolving industry trends and technologies.
Step 5: Implementation and evaluation: This step outlines how educational institutions or organizations can implement the framework, including technical requirements and instructor training. A plan for evaluating the framework’s effectiveness, including assessing user learning outcomes and collecting valuable feedback for further improvements, is also provided.
Step 6: Dissemination and adoption: Strategies for disseminating the framework to the target audience, such as marketing campaigns and outreach efforts, are developed. Educational institutions and organizations are encouraged to adopt the framework, with its potential benefits in enhancing education in manufacturing processes emphasized.

Step 1: Gaps identification and manufacturing processes requirements

In the case of interacting with multiple robots, the social behavior and preferences of end-users are collected. This feature enables them to review and modify any unacceptable actions or behavior. Moreover, robots communicate with other social products to better understand end-user behavior, which in turn helps improve robot-user engagement. It is pertinent to note that traditional robot design approaches dismiss the social aspects of human–robot interaction. Positive social aspects boost the robot’s performance when operators are involved. Sometimes, it is necessary to collect end-user data to understand robot control behavior.

Likewise, Méndez et al. (2019); Molina et al. (2021) proposed using the S4 framework to design functional prototypes and provide user-friendly sensing, smart, sustainable, and social features^21,22. They defined each S as:

Sensing (S1): the ability of a system to recognize occurrences, gather information, and assess variations by utilizing sensors that enable the observation of physical or environmental circumstances.
Smart (S2): denotes the integration of physical components, intelligent elements, and connectivity enhancements, enabling a product to possess intelligence and the ability to connect with other devices.
Sustainable (S3): Considers the social, environmental, and economic factors to produce balanced and optimized performance. Here, the social aspect pertains to how a product positively impacts people’s quality of life.
Social (S4): refers to the ability to independently observe, record, analyze, and modify consumer behavior, as well as adapt its online/offline features to enhance its performance or market acceptance.

Step 2: Virtual product design and modeling–robotic platform system design and modeling

Several robotic platforms have been developed and installed in the industry. For instance, Kumar et al.²³ presented a meticulous design of a skid-steering mobile platform coupled with a Cartesian serial manipulator to execute various agricultural tasks, as detailed in their work. Their exploration into conceptual and component design using Solidworks, simulation on undulating terrains with ADAMS software, and the analysis of control mechanisms to scrutinize errors and performance provides a comprehensive view into the multifaceted nature of robotic design in agriculture. In a distinct approach, Kharzhevskyi et al.²⁴ focuse on the pivotal aspect of wheel design for robotic platforms, especially those intended for reconnaissance in challenging terrains. Their work underscores the imperative of calculating design strength, ensuring minimal deformation, and maintaining the highest safety factor for wheel shaft and L bracket thickness, safeguarding the robotic platform’s robustness and reliability in demanding conditions. Transitioning to the industrial manufacturing domain, Zhang et al.²⁵ illuminate the significance of high-fidelity simulation environments in evaluating industrial automation, presenting a simulation platform that meticulously reconstructs the behavioral patterns of real systems by incorporating the impact of robot dynamics into the control logic of industrial tasks. Their work accentuates the necessity of achieving high simulation fidelity, particularly in scenarios involving cooperative robots and modular manufacturing devices, to ensure accurate and reliable outcomes in simulated industrial manufacturing scenarios.

Moreover, the integration of digital twins and web services in robotic deburring within intelligent manufacturing is explored by Stan et al.²⁶. Their development of a digital twin for a robotic deburring work cell, alongside process planning and robot programming, not only enhances the operational efficiency of the work cell but also introduces a new web platform that enables remote monitoring, triggers alerts for unexpected events, and facilitates control to authorized personnel. Furthermore, their exploration into energy consumption strategies and the results of energy-efficient motion planning, alongside signal-based scheduling optimization of the robotic deburring cell, provide valuable insights into energy-aware robotic manufacturing. Expanding the scope with additional recent works, Molotla et al.²⁷ introduce a configurable hybrid integral manufacturing platform that utilizes an industrial robot arm for both subtractive and additive manufacturing processes, demonstrating the feasibility of integrating various manufacturing tools controlled by a single system. Kim and Park²⁸ explore the application of robotic platforms in the garment industry, focusing on the automatic alignment and placement of fabric patterns, although further details are not available. Park et al.²⁹ present an autonomous robotic bin-picking platform that combines human demonstration with a collaborative robot and YOLOv5 neural network model for object localization without requiring prior CAD models or training datasets. Lastly, , Formoso et al.³⁰ introduce the Mobile Meta-Material Interior Co-Integrator (MMIC-I), a robotic assembler designed for ease of assembly and a simple design with a low number of unique parts, providing a solution for in-space assembly of large-scale space structures.

The contributions above highlight the remarkable strides in robotic platform system design and modeling within various manufacturing contexts. However, the rapid evolution of manufacturing demands, technological advancements, and the increasing complexity of production processes underscore the need to continue developing and innovating robotic platforms. The future of manufacturing hinges on the ability to create robotic platforms that are adept at navigating the complexities and variances within manufacturing environments and capable of adapting to evolving technologies and methodologies. This necessitates continuous exploration and development in the field, where robotic platforms are designed with a foresight of future manufacturing challenges and technological shifts. Moreover, integrating emerging technologies such as artificial intelligence, machine learning, and the Internet of Things (IoT) into robotic platforms will enhance their capabilities, enabling them to operate with increased autonomy, precision, and efficiency. Furthermore, the development of robotic platforms should also consider aspects of sustainability, energy efficiency, and adaptability to various manufacturing scenarios, ensuring their applicability and efficacy in future manufacturing landscapes. The ongoing creation and enhancement of robotic platforms remains paramount to ensuring that the manufacturing sector can meet future demands, embrace technological advancements, and navigate the challenges of the ever-evolving industrial landscape. The collective efforts of researchers, engineers, and industry professionals will be crucial in steering the direction toward innovative, efficient, and sustainable robotic platform designs in manufacturing.

In Industry 4.0, various types of robots have found applications in manufacturing processes. Each type of robot serves specific purposes and offers unique advantages in modern manufacturing. Figure 3 exemplifies seven different types of robots in manufacturing for Industry 4.0:

(I) Delta robots These robots are widely used in pick-and-place operations, packaging, and high-speed assembly tasks. They are known for their exceptional speed and precision, making them suitable for tasks that require rapid and accurate movements, such as placing small components onto electronic circuit boards.
(II) Articulated robots They have a wide range of applications in manufacturing, including welding, painting, material handling, and assembly. They are known for their flexibility and dexterity, with multiple joints and the ability to mimic human arm movements.
(III) Mobile robots These are used for tasks that require autonomous navigation in dynamic environments. They are commonly employed in logistics, warehousing, and material transport within manufacturing facilities. They can also be utilized for inventory management and surveillance.
(IV) Delivery robots Employed in the logistics and supply chain sectors. They are designed to transport goods within a facility, optimizing the movement of materials, components, and products. They can work autonomously or in collaboration with human workers.
(V) Aerial robots (Drones) They have monitoring, inspection, and surveillance applications in manufacturing. They are used for assessing equipment, inspecting infrastructure, and capturing aerial imagery. Drones are particularly valuable in large-scale facilities or outdoor settings.
(VI) Collaborative robots (Cobot) They are designed to work alongside human operators, fostering human–robot collaboration. They are used in tasks that require both automation and human oversight, such as precision assembly and quality control, and tasks that demand adaptability and fine motor skills.
(VII) SCARA robots Selective Compliance Assembly Robot Arm (SCARA) robots are employed in assembly processes, material handling, and packaging applications. They are known for their speed, repeatability, and precision, making them suitable for tasks that demand high accuracy.

In Industry 4.0, these robots are often equipped with advanced sensors, vision systems, and connectivity features, allowing them to be part of the Internet of Things (IoT) ecosystem. They can communicate with other machinery and systems, enabling data-driven decision-making and enhanced automation. The choice of robot type depends on the specific manufacturing process, the need for precision, speed, flexibility, and the level of interaction with human workers. Manufacturers select the robot type that best suits their operational requirements.

Step 3: Virtual manufacturing company development

In the study of virtual manufacturing companies, many innovative methodologies and technologies have emerged, each contributing substantively to the design and implementation of these digital entities. The integration of computational facilities and contemporary technologies, such as blockchain and machine learning, has been instrumental in amplifying product development and manufacturing capabilities. Vijayakumar³¹, in On blockchain technology and machine learning algorithms in concurrent engineering, shows the significance of integrating technologies like remote monitoring, artificial intelligence, machine learning, and blockchain in concurrent engineering applications, thus enhancing product development and manufacturing capabilities across various sectors. In a parallel, Serradilla et al.³² delineate a methodology that amalgamates data-driven techniques with domain knowledge in Methodology for data-driven predictive maintenance models design, development and implementation on manufacturing guided by domain knowledge. While sequential and ordered, this methodology encapsulates the essence of flexibility, ensuring that predictive maintenance (PdM) systems are meticulously tailored according to specific business and process characteristics, thereby intertwining with the concept of predictive maintenance in industrial settings. In a more specific instance of implementing virtual design in manufacturing, Starikov et al.³³ introduce the concept of the Virtual Furniture Design Bureau (VFDB) in Virtual Furniture Design Bureau: Distributed Design in Multi-Agent Environment Using Cloud Technologies. The VFDB orchestrates a unified system environment for the distributed design of complex cabinet furniture products and manufacturing processes, employing object structural attribute models (OSAM) and a multi-agent approach, thereby facilitating distributed design and ensuring that the design process is managed and streamlined through a simulated multi-agent environment. Figure 4 shows an example of virtual sampling for anomaly detection using computer vision.

In a network-oriented perspective, Effendi and Efendi³⁴ navigate through the design of a Virtual Local Area Network (VLAN) to manage broadcast and traffic on a computer network in Design and implementation of virtual local area network at PT. eds manufacturing Indonesia. This exploration into VLAN technology provides a lens through which the configuration of a virtualized computer network can be viewed, offering insights into managing network traffic and broadcasts in a virtual manufacturing setup. Marconi et al.³⁵ propose an integrated shoe design and manufacturing approach in A Digitally-enabled Integrated Approach to Design and Manufacture Shoe Lasts. This approach, enabled by CAD/CAM technologies, integrates design and manufacturing phases and employs haptic technologies to interact with virtual models, simplifying subsequent planning and manufacturing operations. Zhou et al.³⁶, in Integration of Advanced Simulation and Visualization for Manufacturing Process Optimization, likely delve into the integration of advanced simulation and visualization techniques to optimize manufacturing processes, providing a framework through which manufacturing processes can be visualized and optimized through advanced simulation techniques. In a parallel exploration of the Metaverse and virtual reality in manufacturing, Mourtzis³⁷ in The Metaverse in Industry 5.0: A Human-Centric Approach towards Personalized Value Creation, delves into the alignment of the Metaverse concept with Industry 5.0, representing a digital ecosystem where collaboration fosters innovation and enhances productivity through the integration of technologies like AI, VR, and IoT. tegration of technologies like AI, VR, and IoT. Chu et al.³⁸, in An Exemplary Case of Industrial Metaverse: Engineering Product Demonstration Using Extended Reality Technologies, present a framework that incrementally integrates virtual and real environments, developing VR, AR, and MR tools to support technical marketing, enabling real-time interaction for potential customers to evaluate industrial coolers without geographic and temporal restrictions.

Muthmainnah et al.³⁹, in Impact of Metaverse Technology on Student Engagement and Academic Performance: The Mediating Role of Learning Motivation, explore the impact of Metaverse technology on student engagement and academic performance, revealing that student involvement can be positively impacted by the employment of Metaverse technology, which in turn improves academic performance. Wang et al.⁴⁰, in Nanomaterial-based flexible sensors for Metaverse and virtual reality applications, delve into the advancements in nanomaterial-based flexible sensors (NMFSs), which can be tightly attached to the human skin or integrated with clothing to monitor human physiological information, provide medical data, or explore Metaverse spaces.

Finally, Li et al.⁴¹ mention that ensuring seamless integration and interaction between multiple platforms, devices, and data formats is crucial for achieving a ubiquitous semantic Metaverse with AI and VR technologies. This approach requires the development of intelligent algorithms that can allocate resources such as computational power and bandwidth in real-time based on user demand and distribution.

Maintenance

Genetic algorithms (GAs) are optimization algorithms based on natural evolution, transforming a population of individuals through mutation and crossover. They consist of a data structure, a fitness function, genetic operators, and a selection method for selecting more fit individuals for the next generation⁴².

Likewise, production scheduling and preventive maintenance (PM) planning are significant issues in the manufacturing industry. Equipment failures can disrupt schedules, while recommended intervals are often delayed to expedite production. Despite trade-offs, these activities are typically planned and executed independently⁴³.

The model’s adaptability to various factory settings is a significant feature. It allows for the adjustment of parameters like the cost of preventive maintenance (CMP), the cost of failure (CF), equipment criticality (CE), and logistic costs (CL), along with the probability function. This flexibility makes the model applicable across different operational scenarios.

In a VR factory environment, this model provides a realistic simulation platform. It enables the analysis and visualization of different maintenance schedules and their implications, aiding in making informed decisions. The model identifies the most efficient maintenance strategy by balancing regular maintenance costs against equipment failure risks and potential expenses.

Comparing preventive and predictive maintenance mandates a comprehensive evaluation encompassing critical facets. A meticulous cost-benefit analysis is imperative to juxtapose implementation and operational costs against benefits such as downtime reduction, extended equipment life, and enhanced production efficiency. Despite upfront expenses, predictive maintenance offers a more favorable return on investment over time, driven by optimized resource utilization and reduced emergency repairs. Furthermore, its superior risk management capabilities bolster manufacturing stability^44,45. Table 2 summarizes the information for each type of maintenance.

Table 2 Differences between preventive maintenance and predictive maintenance.

Full size table

Integrating this detailed cost function within the maintenance optimization model using a GA enhances the model’s precision and relevance in VR factory settings. It facilitates optimal maintenance planning, which is crucial for maintaining operational efficiency and reducing unnecessary expenses in various factory environments.

Sustainable and circular production

Although the design of the robotic system considers sustainability, it also highlights the consideration of sustainability in the production of the robotic system. Furthermore, It is essential to mention that the line of production must consider a circular economy approach to manage the waste generated by the process to create another type of value for the process or another type of process according to⁴⁶, the waste is value for other business using strategies, such as durable design, maintenance, repair, re-use, re-manufacturing, refurbishing, and recycling⁴⁷. In addition, the 9R framework promotes a circular economy approach by examining how products and materials can be used and reused at their highest value while minimizing waste and environmental impact⁴⁸. Namely, the 9Rs are Rethink, Reduce, Reuse, Repair, Refurbish, Remanufacture, Repurpose, Recycle and Recover (from R1 to R9).

Step 4: Product evaluation

Researchers have been exploring various methodologies and technologies to enhance the precision and efficiency of the evaluation process and products. A notable approach, as proposed by Bortolini et al.⁴⁹. in their paper A two-step methodology for product platform design and assessment in high-variety manufacturing, involves a two-step methodology for product platform design and assessment in high-variety manufacturing, which utilizes a modified algorithm for solving the longest common subsequence (LCS) problem and k-medoids clustering. This methodology identifies the platform structure and assigns variants to the platforms, subsequently assessing them against various industrial and market metrics. Meanwhile, the emergence of virtual reality (VR) has opened new avenues in product evaluation, providing a unique, immersive environment where products can be evaluated in a detailed and interactive manner. Wang and Liu⁵⁰ have delved into creating a virtual evaluation system specifically for product designing using VR. However, the specifics of their methodology are not detailed in the available information. Their paper is titled A virtual evaluation system for product designing using virtual reality.

On a similar note, a study by Palacios-Ibáñez et al.⁵¹ titled Consumer Subjective Impressions in Virtual Reality Environments: The Role of the Visualization Technique in Product Evaluation explored how different visualization techniques in VR environments can influence consumer subjective impressions during product evaluation. Their research, which involved evaluating different designs of a watering can in various VR settings, highlighted the potential perceptual differences that might arise when a product is assessed using different visualization techniques. Moreover, while evaluating a blood pressure monitor, Hinricher et al.⁵² investigated the effects of VR and the test environment on user experience, usability, and mental workload. Their findings, documented in Effects of virtual reality and test environment on user experience, usability, and mental workload in the evaluation of a blood pressure monitor , underscored that while VR evaluations should focus on objective criteria like user errors, subjective criteria such as user experience can be significantly influenced by VR. Another insightful research by Felip et al.⁵³, titled Touch Matters: The Impact of Physical Contact on Haptic Product Perception in Virtual Reality, emphasized the impact of physical contact on haptic product perception in VR. Their work demonstrated that the means of allowing a product to be touched, which elicits a greater sense of presence, may positively impact evaluations of haptic features, especially for products with high haptic importance. In a different context, the VoicePrivacy 2020 Challenge, as documented by Tomashenko et al.⁵⁴ in The VoicePrivacy 2020 Challenge Evaluation Plan, aimed to promote the development of privacy preservation tools for speech technology. Although not directly related to product evaluation, the methodologies and metrics developed during this challenge could offer insights into the development of evaluation methodologies in other domains.

Step 5: Implementation and evaluation

Advanced data analytics and AI-driven feedback mechanisms are pivotal in this VR and AI-based framework. The system continuously collects and analyzes their interactions and performance data as users engage in the various modules. Artificial Intelligence algorithms, powered by machine learning, process this data to provide personalized feedback to each user. This feedback evaluates their progress and identifies areas where improvement is needed. For instance, AI algorithms can detect patterns in design choices or manufacturing process simulations and suggest alternative approaches based on best practices and historical data. This data-driven approach ensures learners receive tailored guidance, making their educational journey more efficient and effective. Furthermore, the system can offer insights into industry trends and emerging technologies by analyzing a vast dataset of manufacturing scenarios, equipping users with the latest knowledge to make informed nearshoring decisions. Data analytics and AI-driven feedback make this framework a dynamic and continuously evolving educational resource, aligning users with the ever-changing landscape of advanced manufacturing in Mexico and beyond.

Step 6: Dissemination and adoption

Dissemination strategies are designed to spread the word about the framework, emphasizing its benefits and functionality. To achieve this, various marketing campaigns, both online and offline, will be employed. Additionally, workshops and seminars can be hosted, providing a hands-on experience to showcase the framework’s capabilities. Collaboration with educational institutions, industry partners, and government bodies will leverage resources and networks to enhance outreach. Online platforms and resources can be established, serving as hubs for information, discussions, and support. Furthermore, the framework’s flexibility, which allows customization to meet specific needs, will be emphasized, making it a more attractive choice for potential adopters. Success stories and case studies will be shared, demonstrating the positive impact of early adoption on education and manufacturing processes. Additionally, the commitment to continuous improvement will be reiterated, assuring adopters that the framework will be updated and aligned with evolving industry trends.

Study case in design a virtual production based on a physical robot

This section describes the framework development applied in the virtual commissioning of a designed robot. The first stage consists of designing a virtual representation of a physical prototype, followed by a production line generated by the design features, and finally, the implementation stage in a particular environment. The COVID-19 pandemic prompted the development of new technology, including robots, to address public safety, clinical care, and work continuity. This robotic system is teleoperated through an intelligent glove, the robotic system is modular, this means that new attachments can be integrated according to the needs of each task to be solved. In addition, it has sensors to provide feedback to users to help them make decisions. Because it is a teleoperated system, the interface is a glove that indicates movements to interact with the environment in real time. The glove has a haptic feedback system to provide a more realistic experience of interacting with the environment, since in case of collisions, the user receives a response in vibration mode and also generates a stiffness to represent the forces of the environment on the robotic system. Furthermore, The S4 methodology, which has been used to create innovative products, is a low-cost robot⁵⁵.

Four basic models, the function, the solution, the virtual, and the physical models, are determinant factors. Mechanical and software simulations can create a quick prototype. Bidirectional and visual communication are built into the robot to improve end-user-teleoperator engagement. Displayed eyes in robots foster a sense of social interaction, allowing people to interact with them rather than just seeing a robot. These features give the robot social qualities and responses to facilitate operator interaction. Robots can send messages to adjust driving circumstances; extra information appears if the operator drives the robot at high speed for a long time and the user adjusts the robot’s control.

Step 1: Gaps identification and manufacturing processes requirements

S4 framework involves features that must be considered to develop sustainable products. Therefore, a virtual representation of the physical platform is developed to model the capabilities and describe the performance in time.

For each of the dimensions of the S4 framework, the following features are described: (1) Sensing, using a water sensor, proximity, infrared, camera, flow control, temperature, microphone, and RFID sensor. (2) Smart, including actuators such as ultraviolet light, BLCD wheels, embedded systems, and wireless connectivity. (3) Sustainable, low operating cost, low maintenance cost, intuitive interface, low energy consumption, safety, reduced environmental impact, and ergonomic assembly. (4) Social, the user acquires greater knowledge in their assigned tasks and the system is considered as a complement to the operator. These characteristics defined in the physical prototype have to be represented in the virtual model.

Figure 5 describes the modules based on the physical platform. For instance, Fig. 5a and b describe the interaction using a smart interface, such as a glove, which aims to control a robotic platform or a machine to transfer knowledge to training. Figure 5c shows a virtual representation of the physical robot shown the virtual components. Finally, Fig. 5d illustrates an exploited representation of each element.

Step 2: Virtual product design and modeling

The design of the robot is then scaled for production to meet customer demand. This consists of translating the design into production plans and assembly steps and then designing a production system capable of producing the robots at scale. The Bill of Materials and virtual representation of the physical design of the robot from the previous step are used to define the production and assembly steps. Figure 6 describes the interaction with the virtual system in real-time. Parts of the robot can be classified into commercial-off-the-shelf components, outsourced components, or manufactured parts. Each part’s manufacturing process is defined by considering the material, geometry, and mechanical properties. Machines are selected according to the specifications required to manufacture the parts.

Since the design requires the virtual representation of the physical prototype, the workstations where workers develop the system modules are designed virtually, too. For instance, Figure 7a depicts the virtual representation of the structure and drive module of the physical prototype for the design and construction of virtual process improvements. Figure 7b illustrates the workstation for the electronics module. Each workstation has the tools and materials available in the physical world to perform similar tasks. The virtual representation workstations are required because the product design considers collaborative work where multiple specialists coexist in the design of the systems. Thus, a connection between the two environments must be maintained to reduce workflows. In addition, virtual workstations increase worker safety, improve the training of workers, reduce development time and cost, execute work tasks in parallel, and increase design quality.

Step 3: Virtual manufacturing company development

Based on the virtualization of the production plan in the previous section, the production system can be described as a transfer line with the following stages: reception of raw materials and components, preparation of materials, assembly of the chassis and structure, installation of actuators and sensors, installation of the control system, partial tests, final assembly, and final tests.

A transfer line discrete event simulator can be run in the VR engine to simulate factory operations. The simulator takes in parameters from the production equipment manufacturers and the process plan developed in the previous stage. These include the mean time to fail (MTTF), mean time to repair (MTTR), cycle time distributions, and available buffer spaces between machines. The transfer line is modeled as a network of queues in the discrete event simulator. Through sampling cycle times on each station and failure and repair times from probability distributions, the simulation considers the variability of the processes. It can present production metrics such as overall equipment effectiveness, throughput rate, yield rate, lead time, etc. The simulation can be combined with additional data, such as demand forecasts and inventory policies, to increase the similarity to real-world production operations in a factory setting.

One of the advantages of using virtual reality implementations is the distribution of the assembly stages of the prototype to simulate and analyze the efficiency of the stages. This proposal represents the stages above in a virtual environment designed to describe the production line. However, according to the assembly’s inventory, elements, and scalability, the line of production must adjust each time the product is designed. For instance, for a primary assembly, the elements involved in the line of production are limited due to the prototype’s elemental features, as shown in Fig. 8. Figure 9 depicts the cycle of all stages of product assembly production considering the scalability and each product module. The raw material location is transported by payload vehicles to the assembly line. There are four autonomous arms. The first one cuts the aluminum sheets. The second arm assembles the chassis and the structure. The third arm installs components, sensors, and system control installation. The last robotic arm evaluates partial tests by simulating the system integration. Figure 10 illustrates a dashboard that describes the features of the line of production to give essential information to the operator to decide in real-time.

To improve the performance of the robotic platform, beginning from a prototype, the features are adjusted according to the environment based on a knowledge database. Figure 11 describes the workflow to modify the features of the robotic platform. The first step is to collect data from experts (I). After that, a knowledge database is generated (II) to develop an initial prototype. Prototype (III) involves the main features according to the expert knowledge, and an environment is defined to evaluate the performance, collecting data generated by the robotic platform to make decisions (IV). A convolutional neuronal network (CNN) (V) estimates the performance and evaluates the generated value with the expected value. The criteria generated by CNN take the best features to adapt the current model to reduce the error in the evaluation stage (VI). The robotic platform’s features are modified for each iteration until a final design (VII). For this proposal, an agricultural environment is to evaluate the virtual system.

The procedural content generation (PCG) approach is employed to expand generative algorithms beyond constructive approaches used in the game industry to produce a limited range of content, emphasizing the integration of constraints into search-based PCG like a virtual prototype⁵⁷. In this proposal, PCG operates based on four key constraints: (I) the weight of the material chassis, (II) the overall weight of the prototype, (III) torque in the attraction system, and (IV) the shape of assets to generate potential features of the virtual prototype.

Moreover, each constraint undergoes a normalization process and is subsequently multiplied by a constant determined through expert feedback. This systematic approach assigns weights to the most crucial features, aiming to derive the optimal combination during the search evaluation, prioritizing characteristics carrying the highest weighting.

Given the characteristics of a virtual prototype, a particular environment is created for proof-of-concept testing to evaluate the system. Each particular environment has specific features and conditions that are difficult to replicate in the real world for use in vision systems. Due to the requirements of the features of each sample, the virtual tools generate synthetic samples to train an object detection system to be implemented in the real world. However, they must have adequate detail to be detected in the physical world⁵⁸. Synthetic samples are an alternative to increasing the diversity of samples in particular cases, such as in agriculture. The samples from the physical world of desired conditions, such as crop diseases, are complicated because they have to physically generate the crops with the conditions to collect them to obtain the samples.

In this case, the test scenario is a greenhouse to detect anomalies in tomato crops in an environment that requires a rail to guide the robot’s path to move in a controlled scenario. The system has to move using a rail and detect anomalies in the crop to eliminate them and take tomatoes with better conditions. Then, an environment has been recreated that assimilates a tomato harvesting greenhouse with tomatoes with different characteristics to detect them in the system and remove them. In addition, there is a rail to help guide the system’s displacement. Since the robotic system is modular, the sprinkler is designed to irrigate water on its way in case it requires moisture in the soil. Figure 12 illustrates the robotic platform in the greenhouse environment.

Maintenance scenarios

Incorporating the detailed cost function into the Maintenance Optimization Model using a Genetic Algorithm in a VR factory setting provides a practical approach for scheduling maintenance. This model employs a genetic algorithm to determine the optimal timing for equipment maintenance by considering various influencing factors⁵⁹. The key elements of this model, including the cost function, enable it to simulate factory conditions within a VR environment, enhancing decision-making capabilities.

The cost function (t, CMP, CF, CE, CL) is pivotal to this model. It defines the total cost of maintenance based on the probability of maintenance necessity, expressed through the function Equation 1. This probability function indicates that the likelihood of requiring maintenance increases over time, reflecting the equipment’s natural wear and tear. The total cost calculation combines preventive maintenance costs, represented by the costs associated with potential equipment failure. This formulation effectively balances the direct and indirect maintenance costs against the severity and probability of equipment failure expressed in Equation 2.

$$\begin{aligned} f(t)=1 - e^{-0.01 t} \end{aligned}$$

(1)

The cost function for maintenance represents the probability that maintenance is necessary or effective at time t. In a simplified model, this probability could increase with time, reflecting the increasing likelihood of equipment failure. However, this probability could be based on a factory model’s historical data, predictive analysis, and other operational factors.

CMP - cost of preventive maintenance This is the direct cost of performing preventive maintenance. It would include labor, materials, tools, and any other expenses directly related to executing maintenance.
CL - logistic cost This cost encompasses indirect expenses associated with performing maintenance, such as work planning and scheduling, spare parts inventory management, and any other logistic expenses.
CE - equipment criticality This factor multiplies the cost of equipment failure based on the importance of the equipment. More critical equipment (i.e., those whose failure would have more severe consequences) would have a higher factor. This reflects the potential operational and financial impact of a failure.
CF - cost of failure This is the cost of equipment failure if maintenance is not performed. It would include repairs, downtime, production loss, product quality impact, and other equipment failure costs.

$$\begin{aligned} Cost \ function(t) = f(t) * (CMP + CL) + (1 - f(f)) * CE * CF \end{aligned}$$

(2)

The total cost function aims to balance the cost of regular maintenance with the risk and cost of potential equipment failure. The optimization of this function would seek the point at which the combined cost of preventive maintenance and the risk-adjusted for the criticality of failure is minimized.

For instance, the cost function for the values with CMP =50, CF = 500, CE =0.1, and CL = 15 gives an optimized value of 50 and time of 0, which requires immediate maintenance, as shown in Fig. 13. On the other hand, CMP =5, CF = 50, CE =1.5, and CL = 5 give a minimized value of 11.68 and a time of 365 that requires maintenance for more than a year, as is shown in Fig. 14.

The maintenance model that has been suggested determines the characteristics of the assembly process based on a virtual model. It is important to note that the model operates in parallel to enable real-time behavior and is modular. This approach makes it easy to add new features or variables that need to be considered to provide a more realistic behavior.