Industrial and Manufacturing Engineering, Engineering
6
Scopus Publications
Scopus Publications
Towards a Digital Twin-Assisted Risk Assessment Approach for Experimental Robotic Workcell Design Jinha Park, Simon Chris Vinkel, Ole Wennerberg Nielsen, Christian Schlette 2025 5th International Conference on Robotics Automation and Artificial Intelligence Raai 2025, 2025 Unlike industrial installations, where a single risk assessment is typically sufficient for a fixed configuration, research environments require adaptable strategies capable of addressing multiple experimental scenarios and frequent reconfigurations. This creates unique safety challenges arising from the dynamic nature of experimental setups and the spatial limitations of laboratory environments. To address these challenges, this paper presents a digital twin-assisted approach that enables systematic risk assessment and reduction throughout the design process of experimental robotic workcells. Early integration of digital twin technology during the design phase allows potential hazards to be identified and mitigated before physical implementation. A case study demonstrates the practical application of this approach, showing how digital twin technology can be used not only to design the workcell layout but also to enable iterative risk assessment and reduction throughout the development of a multi-purpose robotic workcell in an academic research laboratory. Ultimately, this study aims to establish a systematic approach that highlights the potential of digital twin-based risk management to enhance safety and design consistency in experimental robotic workcells.
Towards Safe Human-Robot Interaction: A Pilot Study on a Deep Learning-Assisted Workspace Monitoring System Jinha Park, Chen Li, Zhuangzhuang Dai, Christian Schlette Proceedings 2024 IEEE 24th International Conference on Software Quality Reliability and Security Companion Qrs C 2024, 2024 This pilot study aims to explore the potential of a deep learning-assisted workspace monitoring system in ensuring safety in both social and industrial human-robot interaction settings. For this purpose, two vision sensors are used to collect multi-view datasets from different perspectives, with a single participant involved in 12 defined movement scenarios. The Residual Network (ResNet 18), a deep learning model, is employed to detect upper body movements based on the collected datasets. The experimental results demonstrate the accurate prediction of upper body movements by the proposed approach. Furthermore, the results also indicate the potential integration of this approach, which utilizes multiple inputs from various sensors, with the existing system introduced in previous work to facilitate a more dynamic workspace monitoring system for safety purposes.
Digitalizing Manual Processes Using Digital Twins and Product Lifecycle Management for Safe Human-Robot Interaction Scenarios Jinha Park, Felix Casser, Christian Schlette Icac 2023 28th International Conference on Automation and Computing, 2023 Human-Robot Interaction (HRI) can potentially enhance productivity, quality, and safety in manufacturing industries. However, transitioning from manual processes to HRI settings poses several challenges for companies, such as integration issues, knowledge transfer, and ensuring the safety of operators and equipment. To tackle these challenges, this paper suggests an approach to digitalizing manual processes using Product Lifecycle Management (PLM) and Digital Twins (DTs) technology. The proposed approach involves creating a comprehensive Bill of Process (BOP) using PLM software to describe manual processes as digitalized data. The created BOP, containing all product-related information, can be exported and imported into the database of the DT platform to generate elaborated digital models within the DT platform. These models can be utilized for the implementation of safe HRI settings, in conjunction with other existing digital models on the DT platform. Furthermore, the rest of the data, which is hard to be described in PLM software, can be added manually and directly through the GUI of the DT platform. The resulting digital models and information are designed to enable cooperation with DTs of robots and sensors in complex scenarios.
Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye Tracking and Multimodal Sensing Study Zhuangzhuang Dai, Jinha Park, Aleksandra Kaszowska, Chen Li Icac 2023 28th International Conference on Automation and Computing, 2023 The advent of industrial robotics and autonomous systems endow human-robot collaboration in a massive scale. However, current industrial robots are restrained in co-working with human in close proximity due to inability of interpreting human agents' attention. Human attention study is non-trivial since it involves multiple aspects of the mind: perception, memory, problem solving, and consciousness. Human attention lapses are particularly problematic and potentially catastrophic in industrial workplace, from assembling electronics to oper-ating machines. Attention is indeed complex and cannot be easily measured with single-modality sensors. Eye state, head pose, posture, and manifold environment stimulus could all play a part in attention lapses. To this end, we propose a pipeline to annotate multimodal dataset of human attention tracking, including eye tracking, fixation detection, third-person surveil-lance camera, and sound. We produce a pilot dataset containing two fully annotated phone assembly sequences in a realistic manufacturing environment. We evaluate existing fatigue and drowsiness prediction methods for attention lapse detection. Experimental results show that human attention lapses in production scenarios are more subtle and imperceptible than well-studied fatigue and drowsiness.
A Digital Twin-based Workspace Monitoring System for Safe Human-Robot Collaboration Jinha Park, Lars Caroe Sorensen, Simon Faarvang Mathiesen, Christian Schlette 2022 10th International Conference on Control Mechatronics and Automation Iccma 2022, 2022 Human-Robot Collaboration (HRC) requires rigorous safety standards to ensure the safety of the operator. To this end, one promising approach is to install safety-rated sensors in the workcell while considering collaboration methods such as the Safety-rated Monitored Stop (SMS) and Speed and Separation Monitoring (SSM). This paper proposes a Digital Twin (DT)-based workspace monitoring system to implement the approach for safe HRC with both industrial robots (IRs) and collaborative robots (cobots). In the DT framework, the pose of the operator is provided by laser scanners and a body-tracking camera, and the sensor data are processed to calculate the distance between the operator and the hazardous area in order to control the speed of the robots. We carry out experiments to demonstrate the speed scaling function of robots, and the results show that robots can dynamically adapt their movements according to the distance to guarantee a safe environment.
How can i help you? An intelligent virtual assistant for industrial robots Chen Li, Jinha Park, Hahyeon Kim, Dimitrios Chrysostomou ACM IEEE International Conference on Human Robot Interaction, 2021 In the light of recent trends toward introducing Artificial Intelligence (AI) to enhance Human-Robot Interaction (HRI), intelligent virtual assistants (VA) driven by Natural Language Processing (NLP) receives ample attention in the manufacturing domain. However, most VAs either tightly bind with a specific robotic system or lack efficient human-robot communication. In this work, we implement a layer of interaction between the robotic system and the human operator. This interaction is achieved using a novel VA, called Max, as an intelligent and robust interface. We expand the research work in three directions. Firstly, we introduce a RESTful style Client-Server architecture for Max. Secondly, inspired by studies of human-human conversations, we embed conversation strategies into human-robot dialog policy generation to create a more natural and humanized conversation environment. Finally, we evaluate Max over multiple real-world scenarios from the exploration of an unknown environment to package delivery, with the means of an industrial robot.
