Introduction: In medical activities, patients have the right to protect their private parts, medical history, physical defects, special experiences, encounters, etc. from any form of external violations . It is no longer enough for medical institutions and practitioners to do their own confidentiality work well enough to meet the new challenges posed by the era of big data to protect these privacy from being leaked. Have you ever worried about privacy issues during your medical treatment? See how the Sun Yat-Sen University team protects patient privacy by putting digital masks on patients.
The degree of digitization of medical records is rising with the continuous involvement of information technology in the medical field, but the accompanying risk of data leakage makes some people hesitate in the process of promoting medical informatization . In particular, data such as facial images not only provide a large number of clues to identify diseases, but also provide basic information of patients such as gender, race, age and other biological characteristics, which are difficult to be completely anonymous. Conventional blurring and cropping methods not only have the possibility of losing important disease-related information, but also cannot avoid the recognition of patient identity by the face recognition system.
The COVID-19 pandemic has also seen a surge in demand for remote diagnostics. In the case of remote diagnosis of eye diseases, patients will inevitably have to provide a large amount of facial information. “Patients will want to know whether their sensitive information is safe and whether their privacy is protected,” said Lin Haotian, a professor at Sun Yat-sen University’s Zhongshan Ophthalmology Center.
To this end, Professor Lin’s team developed a new technology called “digital mask” based on 3D reconstruction and deep learning algorithms. It is capable of erasing identifiable features while preserving disease-related features required for diagnosis. On September 15, 2022, the research result “A digital mask to safeguard patient privacy” (Figure 1) was published in Nature Medicine[1]. The research results show that, under the premise of ensuring diagnosis, digital masks can prevent doctors and even face recognition AI from identifying patients, which not only protects patients’ privacy, but also further enhances people’s willingness to share health data, which will be beneficial to Medical data is better used for public medical research.
Figure 1 Research results (Source: [1])
01
Deep learning to extract facial features, 3D reconstruction to generate digital masks
Unlike previous methods of facial reconstruction, this study enhanced the reconstruction of the eye in response to the actual need for ophthalmic diagnosis. In the reconstruction process, the features of different parts of the face extracted by deep learning are used to digitize the shape and motion information of the 3D face, eyelids and eyeballs. The face, eyelid, and eyeball each have separate pre-defined parametric models. Among them, the face model is a bilinear model, and the face mesh Mf is represented by a shape vector wfs and a motion vector wfm. The eyelid model is similar to the face model, and the detailed eyelid information Me is represented by the eye shape vector wes and the eye motion vector wem. The reconstruction of the eyeball uses the simplified geometry and appearance eyeball model (SGAEM) previously studied by the team, which approximates the eyeball as a sphere, and uses three parameters such as eyeball radius, iris radius, and relative facial position to represent static characteristics. The rotation angle in polar coordinates represents the dynamic characteristics. These parameters are estimated by the 2D facial landmark Lface, the 2D eyelid landmark Leyelid, and the iris landmark Liris extracted from RGB space (Figure 2).
Figure 2 Development of digital mask system (Source: [1])
After the development of the digital mask, the research team conducted an assessment of the model’s suitability by analyzing data from 405 participants between May 2020 and September 2021. Participants included: (1) 100 strabismus outpatients; (2) 92 pediatric ophthalmology outpatients; (3) 102 thyroid-associated orbitopathy (TAO) outpatients; (4) 111 outpatients Ophthalmology clinic patient. The mean normalized pixel errors were 0.85%, 0.81%, 0.82%, 1.00%, and 1.52%, 1.24%, 1.52%, and 1.61% in eye reconstruction and eyelid reconstruction, respectively (Fig. 3), all at low levels and remains stable, indicating that the reconstruction of the digital mask is accurate.
Figure 3 Quantitative evaluation of digital masks (Source: [1])
02
Digital masks make the right trade-off between patient privacy features and disease diagnosis features
The research team then invited a total of 12 ophthalmologists, three from each of the four departments, to diagnose patients in their respective departments based on the original video and the video covered by the digital mask. if the doctorThe diagnosis of the original video and the video covered by the digital mask is consistent, which can prove the potential of the digital mask in clinical application. The Cohen Kappa coefficients showed a high degree of agreement: the k-values for strabismus, ptosis, and nystagmus were between 0.845 and 0.924 for both eyes, while the k-value for TAO for the right eye was 0.801. The diagnostic accuracy of the original video and the digital mask overlay video was comparable in all paired comparisons.
In a study comparing the identity-hiding abilities of digital masks and cropping treatments on patient facial images, the research team asked respondents to reconstruct or crop images from the digital masks, from five original images and an “Other” option to find the corresponding original image (Figure 4). The results showed that 27.3% of respondents made judgments based on reconstructed images of digital masks, while 91.3% of respondents made judgments based on cropped images. This shows that digital masks can effectively hide patient identity information compared to cropping. Of course, compared to the actual situation, the situational setting of the test will lead to a higher accuracy rate, because the respondents generally need to identify the patient within a range of far more than 5 people.
Figure 4 Clinical validation of digital masks (Source: [1])
03
Generous when privacy is guaranteed
To assess patients’ willingness to share facial images during digital mask use, the research team randomly selected 317 outpatients to participate in an empirical survey. The research team asked participants to watch the uploaded original video and the digital mask-processed reconstruction video, and complete a questionnaire on intention to use. The questionnaire includes five aspects: health support, privacy concerns, trust in doctors and medical platforms, use of digital masks and willingness to share information, and it is hypothesized that health support, privacy concerns, and the use of digital masks will affect patients’ trust in doctors and further Influence the willingness to share information (Figure 5). The results showed that 80% of the participants had privacy concerns, especially those with facial symptoms. And digital masks have a positive effect on patients’ trust in doctors (β=0.348, P
Figure 5 An empirical survey of patients’ willingness to share personal health information (Source: [1])
04
Skilled, digital masks have the ability to evade AI recognition
Researchers selected three well-known deep learning systems, including FaceNet, CosFace, and ArcFace, to conduct face recognition attacks on digital masks. In tests, the facial recognition system was asked to match a query image to a database of images from 405 patients. Query images may be raw images of patient videos, cropped images, or digital mask reconstructions. The results show that the face recognition system can easily match the correct identity when queried with the original image; cropped images reduce this ability to a limited extent; and digital mask reconstruction makes the face recognition system almost unable to identify the correct identity (Figure 6). This proves that digital mask technology has huge advantages in terms of privacy protection.
Figure 6 Using AI re-identification algorithm to verify the reliability of digital masks (Source: [1])
Compared to the current simple and crude but widely used cropping or strip overlay processing, digital masks are a more complex method and are not vulnerable to model inversion and reconstruction attacks. The obtained quantitative parameters (such as the degree of eye rotation, eyelid shape parameters, blink rate, and rotation frequency) are expected to provide clues for intelligent disease diagnosis or research on the relationship between diseases and facial features in the future. This technology can better promote the progress of health care services. Of course, there are still some limitations in this study, such as insufficient model capacity to reconstruct conjunctival hyperemia, eyelid edema, and abnormal tissue growth; secondly, when patient videos are exposed, the anti-recognition function of digital masks may fail ; Third, there is still a potential risk of being attacked.
Writing|Feng Lixiao
Typesetting|Feng Lixiao
References:
This article is an original creation of biological exploration. Personal forwarding and sharing are welcome. If any other media or website needs to be reprinted, the source Biological Discovery must be indicated before the text.