“Don’t automate, augment!”

This is the takeaway of the seminal book on the future of work by Davenport and Kirby.^*1 The emergence of cyber-physical systems makes radical new products and systems possible and challenges the role of humankind. Throughout the design, manufacturing, use, maintenance, and end-of-life stages, digital aspects (sensing, inferencing, connecting) influence the physical (digital fabrication, robotics) and vice versa. A key takeaway is that such innovations can augment human capabilities to extend our mental and physical skills with computational and robotic support – a notion called “augmented well-being.” Furthermore, agile development methods, complemented by mixed-reality systems and 3D-printing systems, enable us to create and adapt such systems on the fly, with almost instant turnaround times. Following this line of thought, our special issue is entitled “Augmented Prototyping and Fabrication for Advanced Product Design and Manufacturing.”

Heavily inspired by the framework of Prof. Jun Rekimoto’s Augmented Human framework,^*2 we can discern two orthogonal axes: cognitive versus physical and reflective versus active. As depicted in Fig. 1, this creates four different quadrants with important scientific domains that need to be juxtaposed. The contributions in this special issue are valuable steps towards this concept and are briefly discussed below.

AR/VR

To drive AR to the next level, robust tracking and tracing techniques are essential. The paper by Sumiyoshi et al. presents a new algorithm for object recognition and pose estimation in a strongly cluttered environment. As an example of how AR/VR can reshape human skills training, the development report of Komizunai et al. demonstrates an endotracheal suctioning simulator that establishes an optimized, spatial display with projector-based AR.

Robotics/Cyborg

Shor et al. present an augmentation display that uses haptics to go beyond the visual senses. The display has all the elements of a robotic system and is directly coupled to the human hand. In a completely different way, the article by Mitani et al. presents a development in soft robotics: a tongue simulator development (smart sensing and production of soft material), with a detailed account of the production and the technical performance. Finally, to consider novel human-robot interaction, human body tracking is essential. The system presented by Maruyama et al. introduces human motion capture based on IME, in this case the motion of cycling.

Co-making

Augmented well-being has to consider human-centered design and new collaborative environments where the stakeholders involved in whole product life-cycle work together to deliver better solutions. Inoue et al. propose a generalized decision-making scheme for universal design which considers anthropometric diversity. In the paper by Tanaka et al., paper inspection documents are electronically superimposed on 3D design models to enable design-inspection collaboration and more reliable maintenance activities for large-scale infrastructures.

Artificial Intelligence

Nakamura et al. propose an optimization-based search for interference-free paths and the poses of equipment in cluttered indoor environments, captured by interactive RGBD scans. AR-based guidance is provided to the user.

Finally, the editors would like to express their gratitude to the authors for their exceptional contributions and to the anonymous reviewers for their devoted work. We expect that this special issue will encourage a new departure for research on augmented prototyping for product design and manufacturing.

^*1 T. H. Davenport and J. Kirby, “Only Humans Need Apply: Winners and Losers in the Age of Smart Machines,” Harper Business, 2016.

^*2 https://member-cnki-net-s-1416.res.gxlib.org.cn:443/rwt/1416/https/NSRXELUTMWWXT5LQPSYT655TMH/about/ [Accessed June 21, 2019]

This design paper describes the development of custom built interface between a force-replicating virtual reality (VR) haptic interface glove, and a user. The ability to convey haptic information – both kinematic and tactile – is a critical barrier in creating comprehensive simulations. Haptic interface gloves can convey haptic information, but often the haptic “signal” is diluted by sensory “noise,” miscuing the user’s brain. Our goal is to convey compelling interactions – such as grasping, squeezing, and pressing – with virtual objects by improving one such haptic interface glove, the SenseGlove, through a redesign of the user-glove interface, soft glove. The redesign revolves around three critical design factors – comfort, realism, and performance – and three critical design areas – thimble/fingertip, palm, and haptic feedback. This paper introduces the redesign method and compares the two designs with a quantitative user study. The benefit of the improved soft glove can be shown by a significant improvement of the design factors, quantified through QUESI, NASA-TLX, and comfort questionnaires.

While several methods have been proposed for detecting three-dimensional (3D) objects in semi-real time by sparsely acquiring features from 3D point clouds, the detection of strongly occluded objects still poses difficulties. Herein, we propose a method of detecting strongly occluded objects by setting up virtual auxiliary point clouds in the vicinity of the target object. By generating auxiliary point clouds only in the occluded space estimated from a detected object at the front of the sensor-observed region, i.e., the occluder, the processing efficiency and accuracy are improved. Experiments are performed with various strongly occluded scenes based on real environmental data, and the results confirm that the proposed method is capable of achieving a mean processing time of 0.5 s for detecting strongly occluded objects.

While loading equipment into factories or facilities, pre-loading planning on computers to determine a loading object’s posture and its loading path will help reduce trials and errors at the site location. During such pre-loading planning, we need to detect the shape data of both the factory area and loading object and their interference states and then plan the loading object’s posture accordingly. In practice, however, there are a considerable number of issues while acquiring the latest factory area data and while grasping where the loading object in complex postures will result in interference, both simultaneously and for a short time period. In this study, we aimed at resolving the above-mentioned difficulties by developing a system that can detect where the loading object will interfere by means of polygonal shapes and can visualize the area by means of point clouds, both working in conjunction so that the developed system can automatically plan a loading object’s posture and can visualize its planned posture through augmented reality.

Virtual reality (VR), augmented reality (AR), and mixed reality technologies are utilized at various stages of product lifecycle. For products with long lifecycles such as bridges and dams, the maintenance and inspection stages are very important to keep the product safe and well-functioning. One of the advantages of VR/AR is the ability to add important information such as past inspection data. Past inspection information is summarized in a document consisting of the 2D sketches of bridge degradation drawings. However, this degradation sketch is in 2D, and it has no correspondence with the 3D world. In this study, we propose a method to associate important information of 2D sketches with a 3D industry foundation classes (IFC) model, which is a standardized computer aided design model. To display a VR image of a bridge during the inspection process, the proposed method is applied to the 3D IFC model of the bridge and 2D degradation sketch of the inspection report.

This paper describes a next-generation nursing education simulator, the endotracheal suctioning training environment simulator (ESTE-SIM), which is capable of interactively reproducing vital reactions. With the spread of home treatment, care providers who have received a certain level of nursing education should be increased, not limited to conventional health-care professionals. A great gap exists between simulations under restricted conditions that have been practiced in conventional nursing education and those in the actual clinical site, thus creating a burden on nurses and patients. If a simulator that approaches real clinical situations can be developed, it will not only contribute to lessening the burden on nurses but also improve the quality of nursing care. The ESTE-SIM, which simulates endotracheal suctioning, can measure the movements of the suction catheter inserted in the trachea. The measurement information is used to estimate the progress of the nursing maneuver, which is then used to reproduce vital reactions, including dynamic facial expression changes based on projection mapping and monitor-displayed vital signs. To design and control the vital reactions, a mathematical model to determine the behavior of the simulator is formulated based on the actual measurement data of the vital reactions of patients and the experiential knowledge of nurses. By integrating these element technologies, we developed a novel interactive nursing education simulator capable of recreating typical vital reactions that occur during the basic endotracheal suctioning maneuver.

In the recent years, Japan has been facing the problem of an aged society, and oral care is an important aspect in maintaining the oral health of elderly persons. Swallowing disorders, caused by various factors, are one of the most severe oral problems associated with advancing age. A mealtime assistant can help elderly persons in this situation, and thus, mealtime assistance is considered as an essential skill for students in nursing education. There are, however, some problems in mealtime assistant training in nursing schools. For example, there are few instructors compared to the number of students and few opportunities in clinical situations. Therefore, training through simulation has received significant attention as an effective educational process for students in medical and nursing schools. We have attempted to develop an oral care simulator and corresponding software to provide nursing students with an effective oral care training environment. Previously, we developed a prototype model of an oral care simulator for learning tooth brushing. In this study, we developed a tongue model for a mealtime assistant simulation model. Incorrect meal assistance may lead to mouth injury and aspiration pneumonia. To prevent such situations, the caregiver must serve the meal on an appropriate position on the tongue using a spoon, and must release it safely. This paper describes a summary of the developed mealtime assistant simulator and the procedures of tongue model development using three-dimensional computer-assisted design (3D-CAD) and 3D printing. We also developed a sensor system to be embeded in the tongue model to detect spoon motion. The sensor system for the tongue model was evaluated through spoon detection experiments.

The measurement of human motion is an important aspect of ergonomic mobility design, in which the mobility product is evaluated based on human factors obtained by digital human (DH) technologies. The optical motion-capture (MoCap) system has been widely used for measuring human motion in laboratories. However, it is generally difficult to measure human motion using mobility products in real-world scenarios, e.g., riding a bicycle on an outdoor slope, owing to unstable lighting conditions and camera arrangements. On the other hand, the inertial-measurement-unit (IMU)-based MoCap system does not require any optical devices, providing the potential for measuring riding motion even in outdoor environments. However, in general, the estimated motion is not necessarily accurate as there are many errors due to the nature of the IMU itself, such as drift and calibration errors. Thus, it is infeasible to apply the IMU-based system to riding motion estimation. In this study, we develop a new riding MoCap system using IMUs. The proposed system estimates product and human riding motions by combining the IMU orientation with contact constraints between the product and DH, e.g., DH hands in contact with handles. The proposed system is demonstrated with a bicycle ergometer, including the handles, seat, backrest, and foot pedals, as in general mobility products. The proposed system is further validated by comparing the estimated joint angles and positions with those of the optical MoCap for three different subjects. The experiment reveals both the effectiveness and limitations of the proposed system. It is confirmed that the proposed system improves the joint position estimation accuracy compared with a system using only IMUs. The angle estimation accuracy is also improved for near joints. However, it is observed that the angle accuracy decreases for a few joints. This is explained by the fact that the proposed system modifies the orientations of all body segments to satisfy the contact constraints, even if the orientations of a few joints are correct. This further confirms that the elapsed time using the proposed system is sufficient for real-time application.

To achieve a universal design that satisfies diverse user requirements associated with aging and internationalization, designers must make a decision based on diverse user requirements. Designers have generally incorporated average human physical characteristics in their designs. Thus, user limitations are critically important. Traditional design methods often regard engineering and product design as iterative processes based on point values. However, when user information is represented as a point value, the resulting product satisfies only that specific user group and does not necessarily satisfy diverse user groups. This study proposes a universal design method that obtains diversely ranged design solutions for user requirements. The proposed method defines diverse user requirements, design variables, and user characteristics as sets, which range in value. To represent user information accurately, users are classified into numerous groups using classification techniques. Design variables are divided into two types: control and noise. Control factors are designer-controllable variables that are based on design specifications. Noise factors are designer-uncontrollable variables representing user characteristics. To derive a ranged design solution set, designers clarify the relationship between performance and design variables. Ranged solutions satisfying required performance are derived for each group using all relational expressions and ranged variable values. The combinations of divided design variables that cannot satisfy the required performance are eliminated from the design proposal, and the narrowed range of design variables become ranged solutions. The ranged solutions are derived for each group, and the common range of design variables is the ranged solution for all users. This paper chooses the design problem of the strap height of a train as a case study of the proposed universal design method. In this case study, we consider diverse user requirements based on the variability of physical characteristics. This paper discusses the suitability of our proposed approach for obtaining ranged solutions that reflect the physical characteristics of diverse users.

We previously designed a compact computer numerical control (CNC) lathe that arranges its heat sources so as to reduce their thermal deformation. However, a compact lathe often undergoes large deformation owing to unexpected thermal conditions arising out of the work environment or from operation of the lathe itself. Hence, we propose a method to determine equations predicting thermal deformation in a CNC lathe from temperatures measured at a few specific points. These equations enable one to effectively compensate for lathe thermal deformation. However, they cannot be applied to cutting operations involving a coolant fluid because the coolant fluid flow may lead to a complicated thermal deformation scenario. In this study, we attempted to more accurately compensate for thermal deformation, for cutting operations involving a coolant fluid, by adding simple calibration coefficients to the prediction equations. We applied the modified equations to a numerically controlled controller and validated our approach for cutting operations using a coolant fluid under various conditions.

In the agricultural production and processing industry, technological processes in which electroactivated water and solutions based on it are effectively used are very widespread. The development of installations for the electroactivation of water and the optimization of operating modes are undoubtedly topical issues, the solution of which will make it possible to use them more effectively in agricultural practice. The results have been presented of studying the operational characteristics of a nonflowing device for the electroactivation of tap water both with an unchanged area of the separation membrane between the anode and cathode compartments and when this area changes its dimensions. Studies have made it possible to establish that, at a constant value of the working area of the separation membrane, the minimum consumption of electricity corresponds to the smallest interelectrode distance. Approaches for implementing effective operation of the electrotechnological installation are proposed.

The factors influencing rotate vector (RV) reducer dynamic transmission error were studied using virtual prototyping technology, which contained the elastic deformation, working load, part manufacturing error, and assembly clearance. According to the error transmission relationship of the RV reducer, 15 influencing factors were selected to design an orthogonal simulation test. The virtual prototype of the RV reducer was built using CREO and ANSYS, and imported into ADAMS for multi-body dynamics simulation. The simulation method reliability was verified via experiments. The results show that the circle center radius error of the pin gear, the amount of equidistant modification of the cycloid gear, the amount of radial-moving modification of the cycloid gear, the clearance between the support bushing and planet carrier, and the clearance between the crankshaft and the support bushing were positively correlated with the RV reducer dynamic transmission error. Among these, the circle center radius error of the pin gear has the greatest influence on the dynamic transmission error of the RV reducer followed by the amount of equidistant modification of the cycloid gear. The elastic deformation of the part and the load fluctuation show a certain gain effect on the transmission error, the elastic deformation of the cycloid gear has a great influence, and the elastic deformation of the pin gear has the least.