Researchers say they’ve developed a new AI foundation model trained on more than 40 million facial photographs that can help predict mortality risk in patients with cancer.
The technology is based on work from scientists at Mass General Brigham in Boston, which had earlier created a system called FaceAge that is designed to estimate a person’s biological age — indicative of physiological condition rather than chronological age — from a facial photograph.
The new system is called FAHR-FaceSurvival and is detailed in a study that was published Monday in the Lancet Digital Health.
“This is the first time that we’ve shown that we can actually predict directly your mortality risk — not just your biological age, but your mortality risk — from a photograph alone,” said Raymond Mak, a radiation oncologist at Mass General Brigham Cancer Institute and an author of the study.
Findings showed that among cancer patients with FAHR-FaceSurvival scores in the highest 10 percent, about half died within 90 days, versus about 2 percent of those in the lowest 10 percent.
Researchers used the biological-age estimates to help predict survival outcomes of people with cancer and found that patients whose biological age exceeded their chronological age tended to have poorer outcomes.
“The original FaceAge study that was reported on a year ago was a proof of principle that showed that we could train an algorithm to predict a biological age from a face photograph,” Mak said.
FAHR-FaceAge was “more accurate than our original FaceAge” both at estimating age and as a prognostic marker in cancer patients, Mak said. The other system, FAHR-FaceSurvival, directly predicts mortality risk in patients with cancer.
One of the most important findings, he said, was that researchers could use one face photograph and a foundation model to develop multiple algorithms to detect different health signals just from that simple image.
Using the new FAHR-Face foundation model, Mak and his team were able to draw on existing datasets containing photos of more than 30,000 cancer patients, along with information about their medical outcomes — an approach Mak described as novel.
“We could take those photographs and the known medical outcome — so their survival time, whether they died after their cancer diagnosis and when they died, or if they survived,” Mak said.
The team used that information to train FAHR-FaceSurvival.
What does AI see?
In the study, attention maps were generated from the trained models to understand which regions of the face the models focused on in making patient predictions.
FAHR-FaceSurvival homed in on the area below the eyes, including the nasal bridge. FAHR-FaceAge predominantly highlighted the nasolabial folds, forehead and temporalis area, underlining the importance of age-related changes in these areas.
What’s not understood is exactly what these algorithms detected in those areas or how they formed conclusions.
“One of the fascinating unanswered questions is what exactly the algorithm is seeing,” Suchi Saria, a professor and director of the John Hopkins University Machine Learning and Healthcare Lab who was not involved in the study, said in an email. “But we don’t yet know what physiological features in those regions are carrying the signal. Understanding that would help us determine whether the model is discovering a genuinely new biomarker or quantifying visible manifestations of illness that clinicians already recognize.”
When asked about possible theories as to what AI is picking up, Mak recalled Hippocrates’ writings about what he called “the face of death” in the Book of Prognostics, written in about 400 B.C.: “a sharp nose, hollow eyes, collapsed temples.”
As an oncologist, Mak said, he has seen patients nearing death develop a characteristic facial appearance, including hollowing around the temples, sunken eyes and loss of soft tissue around the nose. As a clinician, he suspects AI might be “looking at that loss of soft tissue, loss of muscle mass, but that’s speculation.”
He clarified that the algorithms are “complete computer vision only” and are not reaching out across the internet for clues. They have only seen a face photograph and either the age of that person or the date of death or survival time. In other words, AI hasn’t had access to the observations of Hippocrates or anyone else.
Adding value to what clinicians can do already
Emma Pierson, an assistant professor of computer science and core faculty in the Computational Precision Health program at the University of California at Berkeley who was not involved in the study, described the findings as “scientifically interesting.” But she questioned whether doctors are already implicitly assessing parts of a patient’s face — as Mak does as a clinician — even if not in a particularly systematic way.
Before the model is used in clinical practice, Pierson would want to see clear evidence that it improves on predictions in a way that otherwise could not be captured.
“Showing that a photograph predicts survival is not the same as showing that it tells us something a clinician doesn’t already know,” Saria said, “and that in turn is not the same as showing that using the prediction improves patient care. … The next experiment I’d want to see is: Take the best prediction you can make using the information already available to the clinician, then add the facial model and ask how much better the prediction gets.”
Pierson also noted that the study accounted for factors such as age, sex, race or ethnicity, cancer site and aspects of treatment. Based on her preliminary review, however, she did not see comprehensive measures of frailty — a patient’s level of physical vulnerability and ability to endure illness and treatment.
Although formal frailty assessments are not included in the new study’s survival analyses, Mak described a separate clinical trial of older lung cancer patients receiving radiation treatment; researchers are comparing FaceAge estimates with assessments of patients’ strength and physical reserve.
Pierson raised concerns about AI potentially generating racial bias.
“Any time you train something on someone’s face, you are implicitly including patient race in the model’s predictions,” she said, and the use of race in medical predictions is “enormously controversial.” For instance, if patients from a particular racial group have historically received poorer cancer treatment, AI trained on facial photographs could be “encoding racial bias into your model,” she said.
Researchers noted study limitations that include cancer-only cohorts from mainly high-income countries and called for broader validation and more representative patient populations. Prospective trials are needed to determine whether the algorithms provide benefits when used alongside existing prognostic models.
In a separate study published earlier this year, Mak and colleagues analyzed images taken over a period of years to assess how biological age estimates changed over time and how those changes were associated with survival outcomes.
Beyond cancer care, researchers also see broader possibilities using relatively limited clinical data.
Saria found the study to be “clever and quite well done as a demonstration that there is real prognostic information encoded in facial appearance. … To me, that’s the important scientific contribution: It suggests there is measurable health information in the face that we haven’t previously been able to quantify very well.”
“We have the potential to develop multiple health biomarkers from a single face photograph with multiple different AI algorithms,” Mak said, noting that no other imaging modality for obtaining a biomarker is as low-cost as a camera or webcam.
“We really feel like this is the beginning of a new, totally new field of medicine.”
The post How a photo of a patient’s face could predict whether they’ll survive cancer appeared first on Washington Post.
