Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
npj Flexible Electronics volume 6, Article number: 59 (2022 ) Cite this article
The rapid rise of the Internet of things (IoT) have brought the progress of electronic skin (e-skin). E-skin is used to imitate or even surpass the functions of human skin. Thermoregulating is one of the crucial functions of human skin, it is significant to develop a universal way to realize e-skin thermoregulating. Here, inspired by the sweat gland structure in human skin, we report a simple method for achieving dynamic thermoregulating, attributing to the temperature of microencapsulated paraffin remains unchanged when phase change occurs. Combining with the principle of triboelectric nanogenerator, a deep learning model is employed to recognize the output signals of handwriting different letters on ME-skin, and the recognition accuracy reaches 98.13%. Finally, real-time recognition and display of handwritings are successfully implemented by the ME-skin, which provides a general solution for thermoregulating e-skin and application direction for e-skin in the field of IoT.
As a huge network combing various information sensing devices and the Internet, the Internet of things (IoT) tremendously promoted the development of the fourth industrial revolution for realizing real-time communication among humans, machines, and things1,2,3,4. With the rapid development of IoT and fifth-generation wireless networks, human-machine interface (HMI) with electronic skin (e-skin) as a sensor is widely used in smart homes, human health monitoring, intelligent control, and other fields5,6,7,8. E-skin, a device imitating human skin, has been studied to have a variety of skin-like functions, such as flexibility9, tactile sensing10,11, air permeability12, and thermoregulating ability13. Among them, temperature regulation is one of the important functions of human skin14,15. At present, thermoregulating of e-skin generally increases the heat dissipation of skin by using porous materials or reduces the external heat transfer by using thermal insulation materials13,16. These methods only consider from the perspective of supercooling or overheating, which are difficult to realize the dynamic adjustment of temperature. Too low or too high ambient temperature will affect human physiological activities and then affect people’s health17,18. Therefore, it is critical for e-skin to have a dynamic thermoregulating ability like human skin.
Thermoregulating of human skin is mainly realized through the sweat glands of subcutaneous tissue. When the ambient temperature is high, sweat glands relax and enter the active state to release the heat of the human body in time. When ambient is low, a part of sweat glands becomes inactive to reduce the heat dissipation of the human body19,20. Phase change materials (PCM) have similar properties, which absorb heat when ambient is high conversion to latent heat and release stored latent heat when ambient is low, the temperature of PCMs is almost constant during this process21,22,23. Our previous work has proved the feasibility of using liquid metal as PCM for dynamic thermoregulating24. Paraffin has been proved to be a PCM with high melting enthalpy and a green material for energy conservation and environmental protection21,22. The higher the melting enthalpy, the higher the heat storage capacity23. When paraffin is used as PCM in e-skin, it is inevitable to consider the leakage of paraffin. As a common method of material treatment, microencapsulation provide an effective way to avoid the leakage of PCM25,26,27. In the meantime, microcapsule technology has the minimum influence of PCM during phase transition on the original performance of e-skin.
Moreover, one of the important functions of e-skin is to be applied as a sensor for the HMI system28,29,30,31,32. According to different principles, there are capacitive sensors33,34, resistive sensors35, and triboelectric sensors36, etc. Triboelectric nanogenerator (TENG) has been extensively discussed because of its low cost, simple structure, and low-power consumption37,38,39,40. However, the signal of TENG will be greatly affected by the environment, how to accurately identify the output signal of TENG-based sensors is still challenging. Deep learning has made many achievements in search technology, data mining, machine translation, natural language, and other fields, and solved many complex pattern recognition problems41,42,43,44,45. It is of great research value to apply deep learning in accurate sensing and real-time signal processing of e-skin46,47. Previous studies have used the deep learning model to realize the accurate recognition of different people’s writing habits48. However, deep learning model for the accurate handwriting recognition and real-time display is still challenging. Traditional handwriting equipment needs complex circuit system and external power supply. Directly using this handwriting equipment on e-skin will bring poor user experience. It is attractive to write directly on the e-skin without relying on paper, pen or complex electronic devices. Therefore, it is necessary to develop an e-skin that can not only realize dynamic thermoregulating, but also realize accurate and portable handwriting recognition system.
Herein, we developed an e-skin based on microencapsulated paraffin (M-paraffin) with the dynamic thermoregulating ability and realized a handwriting screen system with real-time display based on deep learning. Inspired by the structure of sweat glands in the subcutaneous tissue to regulate body temperature, microcapsules are concentrated on the lower surface of silicone elastomer by a simple stripping method, which not only affects the performance of the material itself as little as possible but also makes the prepared microcapsules e-skin (ME-skin) have the function of dynamic thermoregulating. This method of embedding microcapsules with dynamic thermoregulating ability into the surface of e-skin is generally applicable. Combined with the principle of TENG, this ME-skin can detect human motion, such as finger bending, wrist bending, arm bending, and so on. When a finger writes different letters on ME-skin, ME-skin will also produce different signals. The signals generated by ME-skin are processed through deep learning (average correct rate: 98.13%), and a self-powered real-time handwriting screen is successfully demonstrated. ME-skin not only provides a simple and feasible idea for thermoregulating e-skin but also provides a model for the communication between human-machine and human-human. It has unlimited application possibilities in artificial intelligence and virtual reality/augmented reality (VR/AR).
Paraffin has been widely used as a phase change material because of its high energy storage density49,50. A microcapsule technology was used to wrap paraffin (See Supplementary Figs. 1, 2 and Experimental Section for the detailed preparation of M-paraffin). A conceptual diagram of a dynamic thermoregulating e-skin is demonstrated in Fig. 1a. When the external environment is hot, the M-paraffin absorbs heat from solid to liquid state, and when the external environment is cold, the M-paraffin releases heat from liquid to the solid state. A photograph of M-paraffin is given in Fig. 1b, paraffin was wrapped in microcapsule and appears white. The existence of microcapsule prevented the leakage of paraffin in the process of phase transformation. The scanning electron microscope (SEM) image taken by dipping a small amount of M-paraffin with a toothpick is shown in Fig. 1c. The average diameter of M-paraffin is about 30 μm and the paraffin is well encapsulated by microcapsules. Figure 1d shows the detailed preparation process of ME-skin. First, the M-paraffin was dispersed into the ethanol solution, stirring the mixed solution with a glass rod to disperse the M-paraffin evenly. After that, the ethanol solution with M-paraffin was dropped on a glass sheet. When the ethanol volatilizes completely, the prepared silicone solution was dropped on the M-paraffin, curing it in a drying oven. Then, a tweezer was used to slowly tear off the silicone elastomer with M-paraffin, and the ME-skin was finally prepared. The thickness of the ME-skin measured with a vernier caliper was about 480 μm, as shown in Supplementary Fig. 3. It is specified that the side without M-paraffin is the front of the ME-skin.
a Schematic of ME-skin with thermoregulating ability based on M-paraffin. b Photograph of M-paraffin. c The SEM image of M-paraffin. Scale bar: 20 μm. d Detailed preparation process of ME-skin. e–g The SEM image of ME-skin with front, back, and cross-section. Scale bar: 50 μm. h Stress-strain curves of ME-skin and silicone elastomer. i The mechanical deformation capacity of a 10 × 15 mm ME-skin. Images of the initial state, stretching, twisting, rolling, and release state.
As shown in Fig. 1e–g, the front, back, and sectional images of ME-skin were taken by SEM, which reveal that almost all M-paraffin wax was gathered on the back of ME-skin, and these M-paraffins presented were fully wrapped by silicone elastomer. The SEM image in Fig. 1g shows that the thickness of the M-paraffin layer was only about 60 μm, which is thin compared with the thickness of the whole ME-skin. A silicone elastomer without M-paraffin was prepared for a comparison object. The stress-strain curves of the two films were tested, as illustrated in Fig. 1h. The size of the two films was 10 × 15 mm. The silicone elastomer with M-paraffin exhibit almost the same mechanical performance as the silicone elastomer without M-paraffin. It is noticed that several slight fluctuations appear in the stress-strain curve of silicone elastomer with M-paraffin during stretching, which is attributed to tiny cracks generated inside silicone elastomer around M-paraffin when the film is stretched. It is worth mentioning the phase change materials are embedded into one side of the e-skin by using this simple preparation method, which will not only affect the performance of the materials themselves as little as possible but also endow an e-skin with a certain dynamic thermoregulating function. Figure 1i shows the prepared 16×24 mm ME-skin has excellent mechanical properties and could still be restored to its original state without fracture after stretching, twisting, and rolling. Then, some relevant experiments were carried out to prove the adhesion stability of M-paraffin on the surface of silicone elastomer. Supplementary Fig. 4a shows the optical photo of the surface of the ME-skin in the original state. After that, the ME-skin was stretched repeatedly 200 times (the stretching rate is 120%), the photos at the same position of the ME-skin were taken as shown in Supplementary Fig. 4b. It can be found that the surface of the ME-skin shows almost no change, which proves the good adhesion between M-paraffin and silicone elastomer.
Sweat gland perspiration, for mammals, is an important way to regulate body temperature19,20, as shown in Fig. 2a. When the ambient temperature is high or human activities generate a lot of heat, sweat glands enter an active state and take away the heat of the human body through perspiration. When the ambient temperature is low, most sweat glands enter an inactive state to reduce the loss of heat51,52. Inspired by the regulation of sweat glands on human body temperature, we put forward the idea of using M-paraffin PCM (MPCM) to achieve dynamic thermoregulating on e-skin. When the environment is relatively hot, MPCM absorbs heat and converts it into its latent heat. When the environment is cold, MPCM releases the previously stored heat to maintain a constant temperature. A differential scanning calorimetry (DSC) curve of M-paraffin was measured in Fig. 2b, which indicates M-paraffin’s phase transition temperature is about 31.1 °C and melting enthalpy is 161.92 J g−1. According to previous reports, the comfortable temperature of human skin is in the range of 30–34 °C53,54, and the phase transition point of M-paraffin is just in this temperature range, which indicates the regulation ability of ME-skin on the comfortable temperature of human skin. Supplementary Table 1 shows some PCMs used in recent papers. The melting point of M-paraffin used in our work is within the comfortable range of skin temperature. At the same time, compared with other PCMs, encapsulation with microcapsules can effectively prevent the leakage of PCMs. Moreover, higher melting enthalpy allows M-paraffin to absorb or release more heat during the phase transition, which is crucial for realizing dynamic thermoregulating of ME-skin. Supplementary Fig. 5 studied the thermal stability of M-paraffin, the melting enthalpy of M-paraffin hardly decreased after 5 cycles of heating-cooling experiments, which proves the excellent stability of M-paraffin in the process of phase transitions.
a Schematic diagram of ME-skin inspired by human skin sweat gland structure. b The DSC curve of ME-skin. c Two layers of 5 × 5 cm ME-skin and two layers of silicone elastomer without M-paraffin were placed on a film heater, and the temperature sensors were placed in the middle of the two films. d The measured temperature change with time when ME-skin and silicone elastomer were laid flat. The temperature of the film heater was set to 35 °C. e When the heating temperature of the film heater was 35 °C, the temperature changes of curly ME-skin and silicone elastomer. f The experimental group and the control group were placed on the glass to study the temperature changes under the environment of air conditioning operation. g When the air conditioning temperature was 16 °C, ME-skin and silicone elastomer without M-paraffin temperature versus time curve at an initial temperature of 36 °C. h The time-temperature curve of curly ME-skin and silicone elastomer. The air conditioning temperature was set to 16 °C.
To further demonstrate the thermoregulating function of ME-skin, the temperature changes of ME-skin under high temperature and low-temperature environments were discussed respectively. Figure 2c shows the dynamic thermoregulating ability of ME-skin at 35 °C ambient temperature. A patch temperature sensor was placed in the middle of the two ME-skin (5 × 5 cm), a control group with the same conditions but no M-paraffin was set. The experimental group and the control group were placed on a film heater and the heating temperature was set to 35 °C. The temperature change trend of flat and curly ME-skin was studied respectively, as shown in Fig. 2d, e (The model of ME-skin in the curly state is shown in Supplementary Fig. 6a). The average temperature of ME-skin was about 2 °C lower than that of ordinary silicone elastomer, and the temperature difference between the two reached 4 °C at most. Figure 2f shows a schematic diagram of the test in an air conditioning temperature of 16 °C. The experimental group and the control group were investigated when both were heated to the same temperature. At 16 °C of air conditioning, the temperature of flat and curled ME-skin dropped slower than that of the silicone elastomer, about 1 °C higher on average, as shown in Fig. 2g, h (The model of ME-skin in the curly state is shown in Supplementary Fig. 6b). The curly ME-skin is less affected by the external environment, so it can better prove the dynamic thermoregulating of ME-skin. Obviously, ME-skin can realize dynamic thermoregulating to human body temperature, which is essential for the practical application of e-skin on the human body.
Silicone elastomer is an excellent electron negative material, which has been used in TENG extensively55,56,57. As presented in Fig. 3a, silicone elastomer with M-paraffin was used as the friction layer and the copper foil was used as the conductive layer for ME-skin to realize the self-powered function. It is worth mentioning that because the human skin is conductive58, when TENG was attached to the human body, the contact part must be insulated, so the three-layer film structure as shown in Fig. 3a was designed. Figure 3b depicts the detailed working principle of ME-skin as a single-electrode mode TENG. When a contact happens between human skin and silicone elastomer with M-paraffin, based on the principle of triboelectric effect, human skin is easy to lose electrons and be positively charged, and silicone elastomer with M-paraffin is negative charged (step i). After that, with the separation of human skin an M-paraffin silicone elastomer, the original electrostatic balance on ME-skin is broken, the electrons on the electrode will flow to the ground to achieve a new electrostatic equilibrium (step ii). Their own electrostatic equilibrium is formed between M-paraffin silicone elastomer and electrode when the human skin is far away from the maximum value. At this time, there will be no outflow of electrons (step iii). When the human skin moves to ME-skin again, the electrons flow from the ground to the electrode to maintain electrostatic equilibrium, an electrical signal in the opposite direction of step ii is generated (step iv). The open-circuit voltage, short-circuit current, and transferred charge of ME-skin are presented in Fig. 3c–e, the load resistance used is 1 GΩ. The average peak open-circuit voltage of 228 V and average peak short-circuit current of 3.82 μA are obtained by tapping a 5 × 5 cm ME-skin with the palm of your hand. The corresponding transfer charge is 20.3 nC, which is acquired by integrating the current signal, as plotted in Fig. 3e. The two processes of contact and separation between hand and ME-skin were difficult to be completely symmetrical. Consequently, the positive and negative of open circuit voltage and short-circuit current signal were also not symmetrical.
a The schematics diagram of ME-skin. b The working mechanism for ME-skin in a contact-separation mode. c–e The open-circuit voltage, short-circuit current and transferred charge of tapping ME-skin with a hand. f–h Fixing ME-skin on a finger, the open-circuit voltage of bending the finger 45°, 90°, and bending slowly 45°. i, j The open-circuit voltage of bending the wrist 30° forward i and reverse when ME-skin was fixed on the wrist. j, k Fixing ME-skin at the arm, the open-circuit voltage with bending the arm 45° repeatedly.
The working environment of TENG is complex and changeable. Supplementary Fig. 7 studies the output voltage characteristics of ME-skin in different environments. With the increase of contact-separation frequency, the voltage of ME-skin also showed an increasing trend (Supplementary Fig. 7a). When the frequency of contact-separation of ME-skin was controlled at 6 Hz, the voltage output also showed an increasing trend with the increase of contact force (Supplementary Fig. 7b). Taking some temperature values around the phase transition temperature of M-paraffin to test the voltage signal of ME-skin, it can be found that the temperature of 25-50 °C hardly affected the output of ME-skin (Supplementary Fig. 7c). While, the positive and negative charges generated by ME-skin were the same during a period of contact separation. TENG has been proved to be a self-powered sensor to detect human motion and respond in real-time. A 24 × 50 mm ME-skin was fixed on a finger by tape to record its open-circuit voltage signal, as presented in the inset of Fig. 3f. The signals of a finger bending 45°, bending 90°, and slow bending 45° were revealed, which is presented in Fig. 3f-h. It is worth mentioning that since silicone elastomer does not fix in the finger well, in order to obtain stable signal output, the minimum bending angle that can be accurately detected is 30°. In the same way, the voltage of wrist bending was measured. When lifting the hand up and down, ME-skin produced signals of almost the same size but opposite direction, as illustrated in Fig. 3i, j. Furthermore, Supplementary Fig. 8 shows the signal of wrist bending 60°. The larger the bending angle, the higher the corresponding voltage signal is, which is consistent with the previous experimental results in Fig. 3f, g. After that, the bending signals of fixing ME-skin on the inner side of the elbow were discussed, the open-circuit voltage of bending 45° is shown in Fig. 3k, and 90° is shown in Supplementary Fig. 9. Further, the durability of ME-skin was studied, as shown in Supplementary Fig. 10, the voltage signals of the initial state of the ME-skin (Supplementary Fig. 10a), the voltage signals after 1000 times vertical pressing (Supplementary Fig. 10b) and 1000 times bending (Supplementary Fig. 10c) were exhibited, respectively. The results indicate that the output signal of ME-skin maintains a certain stability without obvious decline. Therefore, ME-skin not only has a significant effect in dynamic thermoregulating but also has potential application value in human motion monitoring. All experiments were carried out when the load resistance is 1 GΩ.
Combined with the principle of TENG, a self-powered handwriting film was designed. A schematic diagram of the handwriting film is given in Fig. 4a. A finger slides on ME-skin to write the letter ‘A, B, C, D, E’, and the corresponding electrical signals are collected. Different letters have different signal characteristics. Analyzing these characteristics is crucial for accurately recognizing letters written by a finger on ME-skin. The signal generation of a single letter is mainly divided into two parts. The first part is produced by contact-separation between the finger and ME-skin, which has been described in detail in Fig. 3b. The second part is generated by the process of a finger sliding on the surface of the ME-skin, as shown in Fig. 4b. In addition, the surface of the ME-skin is not completely flat, and the signal difference caused by the roughness of the material surface also exists. An experimenter wrote the letters of ‘A’, ‘B’, ‘C’, and ‘D’, the corresponding output signals are shown in Fig. 4c–f, respectively. There are no two identical leaves in the world. The same is true for a single letter signal. Although the signals of different letters seem similar, there must be different characteristics of these signals, which could be to judge the letters. For the letters ‘A’, ‘B’, ‘C’, and ‘D’, their characteristics were not particularly obvious in the time domain.
a ME-skin was used as a self-powered handwriting sensor. When a finger wrote letters on ME-skin, the generated voltage signals were recorded by the signal acquisition device. b The working mechanism for ME-skin in a contact-sliding mode. c–f When a finger wrote “A, B, C, D” on ME-skin, the open-circuit voltage signals of ME-skin. Each letter was repeated three times. g–j Time-frequency plots of the open-circuit voltage signals of the letter “A, B, C, D”. Among them, the time domain of STFT was 0–20 s, and the frequency domain intercepted the part of 10–30 Hz.
Hence, the fast Fourier transform (FFT) was adopted to analyze the characteristics of the four signals in the frequency domain in Supplementary Fig. 11. For the frequency domain characteristics of a single cycle, the category of letters can be clearly identified and judged. However, FFT can not reflect the characteristics of a time dimension. When a person writes different letters, the time used is also diverse due to the diverse number of strokes (For example, it spent 2.0 s to write a letter ‘B’ and 1.1 s to write a letter ‘C’.), time-domain analysis is also necessary for more accurate letter recognition. The short-time Fourier transform (STFT) can analyze the frequency and time of the signal at the same time. As shown in Fig. 4g–j, STFT was applied for processing the signal of the letters “A”, “B”, “C”, and “D”, the length of each signal was set to 20 s. STFT was employed to analyze the signal characteristics of 10–30 Hz for accurately identifying the characteristics of the signal. The theoretical application of this ME-skin is on the human skin. Considering the possible influence of the bending of ME-skin on its signal, the voltage signals of ME-skin on the table and on the arm were tested and presented in Supplementary Fig. 12. Because of the softness of human skin, when writing letters on the arm, the contact force between the finger and ME-skin will be reduced, the generated voltage signal was small. While there is no significant difference in the characteristic of the signal. In addition, the influence of different temperatures on the voltage signal was also tested, as shown in Supplementary Fig. 13. When the ambient temperature was set at 25 °C, 30 °C and 35 °C, handwritten the letter ‘A’ on ME-skin. It can be found that the temperature has little effect on the output of the signal. It is worth mentioning that this temperature variation range includes the phase transition process of paraffin microcapsules from solid to liquid, which proves that the functions of dynamic thermoregulating and self-powered handwriting will not affect each other. The abbreviation of Xi’an Jiaotong University is “XJTU”, the voltage signals of these four letters were measured, as shown in Supplementary Fig. 14. Supplementary Fig. 15 demonstrates three features of the four signals that were extracted and analyzed. Three indicators were extracted to disperse the data, which were the average value, skewness, and rectified mean value (Supplementary Note 1). Each indicator was divided by its maximum value. This preliminary proves that the signals of each letter have their unique characteristics, and they can be distinguished by using an appropriate deep learning algorithm.
Although different features can be distinguished for different signals, the signals of each letter are different due to the strength of writing, speed of fingers, and the influence of some man-made environment. As shown in Supplementary Fig. 16, the signal of the letter ‘A’ was repeatedly measured nine times. Therefore, it is impossible to realize 100% recognition only according to the amplitude and frequency of the signal. The use of e-skin should not be limited in imitating human skin, but also make it realize the function beyond human skin, so it is essential to apply deep learning model to train and recognize data59. In our work, a real-time intelligent handwriting system was designed based on the deep learning model. As shown in Fig. 5a, different letters were written on ME-skin and these voltage signals were collected with an oscilloscope. These signals were then divided into two groups, one is the training group and the other one is the test group. The characteristics of these voltage signals were identified by deep learning. When the finger wrote letters on ME-skin again, the AI algorithm can recognize letters and display them in real-time. Corresponding a simple Schematic diagram of the handwriting system was shown in Supplementary Fig. 17.
a Flow-process diagram of the real-time handwriting system. b The detailed process and parameters of a CNN model. c Clustering results after processing the voltage signal of “A, B, C, D, X, J, T, U” by CNN model. d Confusion map of recognizing 8 letters. e, f Demonstration of writing down the four letters “X-J-T-U” with a finger on ME-skin and displaying these letters on LabVIEW section in real-time.
A 1D-CNN model was constructed for feature extraction and automatic recognition of voltage signals with different letters. The specific process of the CNN model is plotted in Fig. 5b, each data contains 200 data points. The deep learning model includes three convolutional layers, three max-pooling layers, and one fully connected layer that outputs prediction results of eight letters. By analyzing the characteristics of the data, the visualization of the data is depicted in Fig. 5c. The result was attributed to the distribution of t-distributed Stochastic Neighbor Embedding (t-SNE) embedded in the dimensions of principal component 1 (PC1) and dimensions of principal component 2 (PC2). The voltage signals of the eight letters of “A”, “B”, “C”, “D”, “X”, “J”, “T”, and “U” were visualized respectively. This is a satisfactory clustering result, in which twenty training sets of each letter were well dispersed. 100 sets of voltage signals for each letter were collected. 80 groups were training set to train the CNN model and 20 groups were the test set to verify the accuracy of the model, as shown in Fig. 5d. Before the training process, each data was preprocessed, including 0-1 normalization and resampling. The length of each signal was scaled to interval [0,1] (0-1 normalization). Convenient for model input, the length of the signal was resampled to 1024 (resampling). By training the 1D-CNN model, the recognition rate of training reached a satisfactory 98.13%.
However, not all letters have obvious characteristics. The writing of some letters is similar, such as “I” and “L”, “U” and “O”, “S” and “Z”, “U” and “V”. The open-circuit voltage signals handwritten with these letters are shown in Supplementary Fig. 18. The open-circuit voltage signals of these letters have a certain similarity. These signals were used to train the deep learning model, and 20 groups were selected as the test set, as shown in Supplementary Fig. 19. The result of recognition rate was lower than before, but still reached 90.71%, which further proves the reliability of the handwriting system. Furthermore, an intelligent handwriting screen system with a real-time display was developed to realize smart e-skin, as shown in Fig. 5e, f. Connecting ME-skin, signal acquisition system, and trained neural network database, when an experimenter wrote a letter on ME-skin, the signal acquisition system collects and transmits the voltage signal to the intelligent identification model. The module judged the type of signal according to the trained deep learning model and outputted a corresponding letter on the display screen. As displayed in Fig. 5e, f, and Supplementary Video 1, the experimenter wrote down ‘X-J-T-U’ on ME-skin once, and the corresponding letter of ‘X-J-T-U’ appeared in the LabVIEW software.
When another different experimenter writes the letter “J” and “U” on the handwriting system, there is a series of diversity in writing habits, writing strength, and writing frequency for different people, the generated voltage signals are different, as shown in Supplementary Fig. 20. Based on the voltage signals collected by the new experimenter, the deep learning model was modified accordingly, see Supplementary Note 2 for specific methods. The new experimenter’s handwritten ‘J’ signals could reach an accuracy of 90% and “U” signals could reach an accuracy of 80%. If the number of samples is increased, the recognition accuracy will be further improved. Considering the influence of some writing strokes on the experimental results, some additional experiments were carried out. As shown in Supplementary Fig. 21, the signals of four letters of “X, J, T, U” were re-tested. 400 groups of data samples were collected for each letter, of which 200 groups were normal writing habits and the other 200 groups were abnormal writing styles. For example, for the letter “T”, the difference is whether to write a horizontal or a vertical first. Each letter takes 320 groups of data samples as the training set and 80 groups of data samples as the test set. Supplementary Fig. 21a shows the data before clustering, Supplementary Fig. 21b shows the data after clustering, and Supplementary Fig. 21c shows the confusion matrix of the data set. It can be seen that even for the writing of different strokes writing order, the recognition rate of the deep learning model still reaches 99.06%. The increase in recognition rate is mainly due to the increase in test volume of each group of signals. On the basis of this experiment, the handwritten letter signals of another experimenter were added to the deep learning model. The signal of four letters of ‘X, J, T, U’ was collected, and the signal of each letter was measured for 200 groups. Add these 800 groups of data into the deep learning model, and the recognition results are shown in Supplementary Fig. 22. With the further increase in the amount of data, the accuracy of recognition has reached 99.58%. This fully proves the reliability of deep learning in letter recognition and the feasibility of the application of ME-skin on the smart handwriting screen. It is worth mentioning that in the cluster diagram in Supplementary Fig. 22b, it is obvious that there are obvious differences in the distribution of handwritten letters for different experimenters, which shed light on the potential application of our system in smart handwriting recognition.
In summary, we developed an e-skin with dynamic thermoregulating ability using M-paraffin as the main material. The preparation of the e-skin used a simple stripping method, which could also be applied to other e-skins easily. The phase transition point of M-paraffin was 31.1 °C, which is within the comfortable temperature range of human skin. The enthalpy of M-paraffin during phase transition was 161.92 J g−1, proving its excellent heat storage performance. Through comparative experiments, it is found that ME-skin using M-paraffin had a good thermoregulating ability when the ambient temperature is higher or lower than the comfortable temperature of human skin. Attaching ME-skin to the human body could detect the signals of fingers bending, wrist bending, and arm bending. When a finger repeatedly writes ‘A, B, C, D’ on ME-skin, the signal corresponding to each letter had its characteristics. A CNN was used to extract the features of the eight signals ‘A, B, C, D, X, J, T, U’, and clustering is successfully realized. 100 signals were tested for deep learning, of which 80 are training sets and 20 are test sets. The final recognition rate is more than 98%. Further, a real-time display handwriting screen system was developed based on LabVIEW. When a finger wrote the four letters of ‘X-J-T-U’ on ME-skin, the corresponding letters were displayed in real-time on the LabVIEW interface. This ME-skin not only provides a simple and feasible idea for thermoregulating e-skin but also provides a model for the information exchange between humans and machines.
The first stage was to add urea, melamine, and formaldehyde in an alkaline environment (PH = 8). The addition reaction appeared between melamine and formaldehyde, urea, and formaldehyde. After that, condensation polymerization occurs between hydroxyl and carboxyl groups of molecules to remove water molecules. As the polymerization proceeds to a certain extent, a water-soluble prepolymer solution was formed. The second stage was the emulsification of paraffin. Through the stirring of the stirrer, a shear force was provided to form small oil droplets and disperse the paraffin in the continuous phase. Due to its hydrophilicity and lipophilicity, the emulsifier would adsorb on the surface of paraffin oil droplets to wrap paraffin oil droplets. Paraffin oil droplets were dispersed in prepolymer solution. In the third stage, the pH value of the solution was adjusted to 2.5-3 (I.e., under acidic conditions). Further polycondensation reaction would take place between the prepolymer molecules to remove water molecules. The molecules were connected by a methylene ether bond or amide bond. A three-dimensional network structure was formed with the increasing molecular weight. Then, under the condition of the heating, it was solidified to form M-paraffin.
0.2 g M-paraffin was added to a small beaker, ethanol solution was dropped into the beaker and fully stirred for 5 min to completely disperse the M-paraffin. Dropping the mixed solution on the glass and volatilizing ethanol soliton. Part A and Part B of silicone elastomer were mixed in a ratio of 1:1 and stirred for 15 min. After the silicone solution was dropped on the glass with M-paraffin, it was dried at 60 °C for 4.5 h. Using a tweezer to tear off ME-skin. The thickness of the ME-skin was about 500 μm.
A GeminiSEM 500 SEM was employed to observe M-paraffin and the front, back surfaces, and sections of ME-skin. A DISCOVER DSC250 DSC was used to analyze the melting point and energy storage density of M-paraffin. A multifunctional tensile machine (JITAI-5KN) was used to test the stress-strain curve of the films. The temperature of the ME-skin and control group was measured by a dual-channel temperature sensor (UT325). A film heater was applied to provide a heat source. The voltage of ME-skin was measured by an oscilloscope (Tektronix TDS 2012C) and the current was measured by a low-noise current preamplifier (MODEL SR570). An oscilloscope (Tektronix MDO34) was used to communicate with LabVIEW software in real-time. The collected voltage signal was transmitted in real-time to the LabVIEW program. The LabVIEW program was directly called the sample library trained by the Python program to realize the recognition and display of the collected signals.
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Biggs, J. et al. A natively flexible 32-bit Arm microprocessor. Nature 598, 532–536 (2021).
Wu, C. et al. A phased array based on large-area electronics that operates at gigahertz frequency. Nat. Electron. 4, 757–766 (2021).
Li, A. et al. Demonstration of valley anisotropy utilized to enhance the thermoelectric power factor. Nat. Commun. 12, 5408 (2021).
Papadopoulos, N. et al. Touchscreen tags based on thin-film electronics for the Internet of Everything. Nat. Electron. 2, 606–611 (2019).
Zhu, M. L. et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications. Sci. Adv. 6, eaaz8693 (2020).
Dong, B. W. et al. Wearable triboelectric-human-machine interface (thmi) using robust nanophotonic readout. ACS Nano 14, 8915–8930 (2020).
Zhu, C. et al. Ultraelastic yarns from curcumin-assisted eld toward wearable human-machine interface textiles. Adv. Sci. 7, 2002009 (2020).
Chew, J. Y., Kawamoto, M., Okuma, T., Yoshida, E. & Kato, N. Adaptive attention-based human machine interface system for teleoperation of industrial vehicle. Sci. Rep. 11, 17284 (2021).
Yun, Y. H. et al. Extremely flexible and mechanically durable planar supercapacitors: High energy density and low-cost power source for E-skin electronics. Nano Energy 78, 105356 (2020).
Guo, Y. J. et al. Anodized Aluminum Oxide-Assisted Low-Cost Flexible Capacitive Pressure Sensors Based on Double-Sided Nanopillars by a Facile Fabrication Method. ACS Appl. Mater. Inter. 11, 48594–48603 (2019).
Zhang, C. et al. A stretchable dual-mode sensor array for multifunctional robotic electronic skin. Nano Energy 62, 164–170 (2019).
Yao, G. et al. Bioinspired triboelectric nanogenerators as self-powered electronic skin for robotic tactile sensing. Adv. Funct. Mater. 30, 1907312 (2020).
Huang, J. et al. Stretchable and heat-resistant protein-based electronic skin for human thermoregulation. Adv. Funct. Mater. 30, 1910547 (2020).
Nadel, E. R., Bullard, R. W. & Stolwijk, J. A. Importance of skin temperature in the regulation of sweating. J. Appl. Physiol. 31, 80–87 (1971).
Kenney, W. L. & Munce, T. A. Invited Review: Aging and human temperature regulation. J. Appl. Physiol. 95, 2598–2603 (2003).
Li, M. F. et al. A highly stretchable, breathable and thermoregulatory electronic skin based on the polyolefin elastomer nanofiber membrane. Appl. Surf. Sci. 486, 249–256 (2019).
Strigo, A., Carli, F. & Bushnell, M. C. Effect of ambient temperature on human pain and temperature perception. Anesthesiology 92, 699–707 (2000).
Barcroft, H. & Edholm, O. G. The effect of temperature on blood flow and deep temperature in the human forearm. J. Physiol. 102, 5–20 (1943).
Sato, K., Kang, W. H., Saga, K. & Sato, K. T. Biology of sweat glands and their disorders. I. Normal sweat gland function. J. Am. Acad. Dermatol. 20, 537–563 (1989).
Ferguson, K. A. & Dowling, D. F. The function of cattle sweat glands. Aust. J. Agr. Res. 6, 640–644 (1955).
Sharma, A., Tyagi, V. V., Chen, C. R. & Buddhi, D. Review on thermal energy storage with phase change materials and applications. Renew. Sust. Energy Rev. 13, 318–345 (2009).
Nazir, H. et al. Recent developments in phase change materials for energy storage applications: A review. Int. J. Heat. Mass Tran. 129, 491–523 (2019).
Jilte, R., Afzal, A. & Panchal, S. A novel battery thermal management system using nano-enhanced phase change materials. Energy 219, 119564 (2021).
Xiang, S. X. et al. Liquid-metal-based dynamic thermoregulating and self-powered electronic skin. Adv. Funct. Mater. 31, 2100940 (2021).
Pupa, P. et al. The efficacy of three double-microencapsulation methods for preservation of probiotic bacteria. Sci. Rep. 11, 13753 (2021).
Zamal, H. H., Barba, D., Aïssa, B., Haddad, E. & Rosei, F. Recovery of electro-mechanical properties inside self-healing composites through microencapsulation of carbon nanotubes. Sci. Rep. 10, 2973 (2020).
Ren, Q., Guo, P. & Zhu, J. Thermal management of electronic devices using pin-fin based cascade microencapsulated PCM/expanded graphite composite. Int. J. Heat. Mass Tran. 149, 119199 (2020).
Gong, S., Zhang, B., Zhang, J., Wang, Z. L. & Ren, K. Biocompatible poly(lactic acid)-based hybrid piezoelectric and electret nanogenerator for electronic skin applications. Adv. Funct. Mater. 30, 1908724 (2020).
Zhao, J. et al. Skin-inspired high-performance active-matrix circuitry for multimodal user-interaction. Adv. Funct. Mater. 31, 2105480 (2021).
An, T. et al. Self-powered gold nanowire tattoo triboelectric sensors for soft wearable human-machine interface. Nano Energy 77, 105295 (2020).
Chen, S. et al. Flexible and transparent sensors with hierarchically micro-nano texture for touchless sensing and controlling. Nano Energy 82, 105719 (2021).
Niu, H. S. et al. Highly Morphology-Controllable and Highly Sensitive Capacitive Tactile Sensor Based on Epidermis-Dermis-Inspired Interlocked Asymmetric-Nanocone Arrays for Detection of Tiny Pressure. Small 16, 1904774 (2020).
Pruvost, M., Smit, W. J., Monteux, C., Poulin, P. & Colin, A. Polymeric foams for flexible and highly sensitive low-pressure capacitive sensors. npj Flex. Electron. 3, 7 (2019).
Chen, S. et al. Multi-sized planar capacitive pressure sensor with ultra-high sensitivity. Nano Energy 87, 106178 (2021).
Tian, H. et al. A Graphene-based resistive pressure sensor with record-high sensitivity in a wide pressure range. Sci. Rep. 5, 8603 (2015).
Zheng, Y. et al. Highly sensitive electronic skin with a linear response based on the strategy of controlling the contact area. Nano Energy 85, 106013 (2021).
Fan, F. R., Tian, Z. Q. & Wang, Z. L. Flexible triboelectric generator. Nano Energy 1, 328–334 (2012).
Long, L. et al. High performance floating self-excited sliding triboelectric nanogenerator for micro mechanical energy harvesting. Nat. Commun. 12, 4689 (2021).
Wu, C., Wang, A. C., Ding, W., Guo, H. & Wang, Z. L. Triboelectric nanogenerator: a foundation of the energy for the new era. Adv. Energy Mater. 9, 1802906 (2019).
Jiang, C. M. et al. A multifunctional and highly flexible triboelectric nanogenerator based on MXene-enabled porous film integrated with laser-induced graphene electrode. Nano Energy 66, 104121 (2019).
Ji, X. L., Zhao, T. K., Zhao, X., Lu, X. F. & Li, T. H. Triboelectric Nanogenerator Based Smart Electronics via Machine Learning. Adv. Mater. Technol. 5, 1900921 (2020).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Wang, G., Ye, J. C. & De Man, B. Deep learning for tomographic image reconstruction. Nat. Mach. Intell. 2, 737–748 (2020).
Yang, L. et al. Particle-laden droplet-driven triboelectric nanogenerator for real-time sediment monitoring using a deep learning method. ACS Appl. Mater. Inter. 12, 38192–38201 (2020).
Shi, Q. et al. Deep learning enabled smart mats as a scalable floor monitoring system. Nat. Commun. 11, 4609 (2020).
Sohn, K. et al. An extremely simple macroscale electronic skin realized by deep machine learning. Sci. Rep. 7, 11061 (2017).
Lee, G. et al. Deep-learning-based deconvolution of mechanical stimuli with ti3c2tx mxene electromagnetic shield architecture via dual-mode wireless signal variation mechanism. ACS Nano 14, 11962–11972 (2020).
Zhang, W. Q. et al. Multilanguage-handwriting self-powered recognition based on triboelectric nanogenerator enabled machine learning. Nano Energy 77, 105174 (2020).
Li, M. A nano-graphite/paraffin phase change material with high thermal conductivity. Appl. Energy 106, 25–30 (2013).
Sanusi, O., Warzoha, R. & Fleischer, A. S. Energy storage and solidification of paraffin phase change material embedded with graphite nanofibers. Int. J. Heat. Mass Tran. 54, 4429–4436 (2011).
Best, A., Lieberman, D. E. & Kamilar, J. M. Diversity and evolution of human eccrine sweat gland density. J. Therm. Biol. 84, 331–338 (2019).
Randall, W. C. Sweat gland activity and changing patterns of sweat secretion on the skin surface. Am. J. Physiol. 147, 391–398 (1946).
Tan, C. et al. A high performance wearable strain sensor with advanced thermal management for motion monitoring. Nat. Commun. 11, 3530 (2020).
Griffit, W. & Veitch, R. Hot and crowded: Influence of population density and temperature on interpersonal affective behavior. J. Pers. Soc. Psychol. 17, 92–98 (1971).
Li, H. et al. 3D printed flexible triboelectric nanogenerator with viscoelastic inks for mechanical energy harvesting. Nano Energy 58, 447–454 (2019).
Yang, D. et al. Self-healing and elastic triboelectric nanogenerators for muscle motion monitoring and photothermal treatment. ACS Nano 15, 14653–14661 (2021).
Chen, S., Wang, Y., Yang, L., Karouta, F. & Sun, K. Electron-induced perpendicular graphene sheets embedded porous carbon film for flexible touch sensors. Nano-Micro Lett. 12, 136 (2020).
Dick, I. P. & Scott, R. C. Pig ear skin as an in-vitro model for human skin permeability. J. Pharm. Pharmacol. 44, 640–645 (2011).
Han, J. et al. Self-powered artificial mechanoreceptor based on triboelectrification for a neuromorphic tactile system. Adv. Sci. 9, 2105076 (2022).
This work was supported by the National Key Research and Development Program of China under Grant No. of 2018YFB1306100, the Fundamental Research Funds for the Central Universities, and SEM facility of Instrument Analysis Center of Xi’an Jiaotong University.
Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xi’an Jiaotong University, Xi’an, 710049, China
Shengxin Xiang, Jiafeng Tang, Lei Yang, Yanjie Guo & Zhibin Zhao
School of Aerospace Science and Technology, Xidian University, Xi’an, 710049, China
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
S.X., and L.Y. projected the idea and designed the experiments. J.T., Y.G., and Z.Z., wrote the control programs and algorithms for demonstration. S.X., J.T., Y.G., and L.Y. carried out the data analysis. S.X., and L.Y. wrote the manuscript. All authors discussed the results and commented on the manuscript.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Xiang, S., Tang, J., Yang, L. et al. Deep learning-enabled real-time personal handwriting electronic skin with dynamic thermoregulating ability. npj Flex Electron 6, 59 (2022). https://doi.org/10.1038/s41528-022-00195-3
DOI: https://doi.org/10.1038/s41528-022-00195-3
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
npj Flexible Electronics (npj Flex Electron) ISSN 2397-4621 (online)
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.