Ukraine expands use of its facial recognition systems to identify dead Russian soldiers

29 March 2022 (Krakow, Poland) – As we had noted in a previous post, Ukraine’s defense ministry has used Clearview AI’s facial recognition technology for well over two years, a gift from Clearview.

NOTE: Ukraine received free access to the Clearview AI search engine for faces, first using it to vet people of interest at checkpoints, airports, plus other uses, via Lee Wolosky, an adviser to Clearview and former diplomat under U.S. presidents Barack Obama and Joe Biden. Despite repeated requests, Clearview did not offer the technology to Russia. 

Clearview has more than 3 billion images it scrapped from the Russian social media service VKontakte, plus access to military intelligence photo data bases and other Russia military social media sites. The VKontakte images make Clearview’s dataset more comprehensive than that of PimEyes, a publicly available image search engine that people have used to identify individuals in war photos. 

Many Western businesses have pledged help to Ukraine, providing internet hardware, cybersecurity tools and other support. For instance, multiple Western based companies that build and operate satellites and satellite software have provided Ukraine with near real-time satellite images to track Russian troop movements even at night or through cloud cover. Amazon has quietly set-up specialised cloud services to store “critical data”, as noted by a Ukraine government official, without elaborating. Amazon has declined to comment.

Clearview’s technology is also being used used to reunite refugees separated from their families, identify Russian operatives and help the government debunk false social media posts related to the war.

ABOVE: bodies of Russian soldiers lie outside a school destroyed by Russian shelling, not far from the center of Ukrainian city of Kharkiv, 24 March 2022. 

Now Ukraine has announced it is using facial recognition software to identify the bodies of Russian soldiers killed in combat and to trace their families to inform them of their deaths. It it part of a complex system that merges work of an OSINT team that finds the social media accounts of dead Russian soldiers, hospital records and other accessible government sources. From there, authorities are messaging relatives via email and private messaging to make arrangements to collect the body, as well as blasts via Russian information conduits:

“As a courtesy to the mothers of those soldiers, we are disseminating this information over social media to at least let families know that they they’ve lost their sons and to then enable them to come to collect their bodies”. 

No one but the Ukraine government knows the number of bodies identified through facial recognition but OSINT sources indicate there has been some success. The Kremlin’s official spokesman, who normally ignores these types of stories, in response to a request for comment on Ukraine’s use of Clearview, was almost immediate: “We have no knowledge of this. There are too many fakes coming out of Ukraine”. But when asked about the recovery of soldiers’ bodies from Ukraine, he said questions of casualties from the military operation was the competence of the Ministry of Defence. But when asked the defence ministry did not respond to a request for comment.

NOTE: Ukraine’s military and U.S. military analysts estimate some 15,000 Russian soldiers have been killed since Russia invaded on 24 February. Our Ukraine team will publish a post later this week to detail how those casualty figures have been computed via OSINT and what is known about Russian military operations, plus some sophisticated data analytics being used by our boss, Gregory Bufithis.

Ukraine also has the “Look For Your Own project”, a Telegram channel where it posts images of unidentified captured or killed Russian soldiers and invites claims from relatives. It has generated a stream of emails from Russian families, although everything received is being vetted to determine authenticity. The Ukrainian government even has an online form where Russian relatives can submit a claim to collect a body. 

The head of the forensic medicine department at a hospital in London told us fingerprints, dental records and DNA remain the most common ways of confirming someone’s identity. Obtaining pre-death samples of such data from enemy fighters is challenging, though, opening the door to innovative techniques such as facial recognition. But as we noted at the opening of this post, clouded eyes and injured and expressionless faces potentially make facial recognition unreliable on the dead. However, using the Clearview database can help Ukraine identify the dead more easily than trying to match fingerprints and works even if there is facial damage. And while decomposition reduces the technology’s effectiveness, a paper from a 2021 conference showed the technology is getting better.

So the technology is out here. We spoke to a doctor at the Institute of Legal Medicine and Forensic Sciences, a leading medical school in Berlin, who has extensive experience in war crimes investigations and medical forensics on battlefields and he noted identifying the deceased is probably the least dangerous way to deploy the technology in war, but once you introduce these systems and the associated databases to a war zone, you have no control over how it will be used and misused.