US Representative Jimmy Gomez was not particularly shocked to learn that he was among 28 Congress members facing criminal mugs in an experiment conducted by the ACLU in Northern California. In fact, the California Democrat had a pretty good idea of what many of the people on this list had in common.
"I told my co-workers that I would not be surprised if they were mostly colored or minorities," he told CNNMoney.
He was right.
On Thursday, the ACLU released the results of their tests on Amazon's controversial recognition software, which uses artificial intelligence to identify people, among other things.
After building a database of 25,000 publicly available arrest photos and comparing them with all 435 members of the congress, the ACLU determined that if Rekognition has maintained its defaults, legislators such as Luis Guitierrez of Illinois, John Lewis of Georgia, and Norma Torres of California, as a criminal.
Of the 28 legislatures, recognition was falsely identified, and about 40% were colored, a result that led the ACLU to call for a government moratorium on the use of facial recognition software ̵
"That's a big deal," Gomez told CNN. "The bias is digitized and used against people who already have many obstacles and struggles."
Although inaccuracy in facial recognition is a well-known problem for women and color people, experts point out that it is necessary to have broader discussions about the ethics of this technology and the responsibility of companies to ensure that they are used fairly.
To this end, Gomez and Lewis Amazon CEO Jeff Bezos sent a letter requesting "an immediate meeting to discuss how to address the defects of this technology to prevent inaccurate results."
In a separate letter, Senators Edward Markey, Mark DeSaulnier, and Luis Gutierrez Bezos have asked to provide details on internal accuracy or bias assessments that Amazon has performed on Rekognition. They also want to know which law enforcement or intelligence agencies are using recognition and whether Amazon is reviewing their use.
Amazon disagreed with the findings of the ACLU by arguing that the tool was not being used properly.
Related: Amazon Asked to Stop Selling Face Detection Technology
"We believe the results could potentially be improved by following the best practices for establishing confidence thresholds (this is the percentage Probability that found recognition has a match) that was used in the test, "said an Amazon spokesman in a statement. "While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals or other social media use cases, it would not be appropriate to identify people with a reasonable level of security Customers set a threshold of at least 95% or higher. "
Suresh Venkatasubramanian, a computer scientist at the University of Utah, said that this dispute would let Amazon off the hook easily. The company has an obligation to ensure that people know how to use their tools properly, he told CNNMoney.
"You had the choice not to kick Tech out if you thought it was sensitive," he said. "We do not let people prescribe drugs by accident."
Related: Microsoft wants regulation of facial recognition tech
Tech companies come to this view. In a blog post released earlier this month Microsoft ( ) President Brad Smith suggested that facial recognition technology be regulated for its "broad societal impact and potential for abuse" ,
Bezos has publicly approached the issue despite recent calls from shareholders and civil rights groups not to sell the technology to the government. In the CNNMoney statement, Amazon called Recognition "a good thing in the world".
Maybe or not, said Woodrow Hartzog, who teaches law and computer science at Northeastern University. "The idea that this is a neutral technology that can be used for good or evil and Amazon should not be responsible, I think is completely wrong," he said.
"It's not unreasonable to say that if you can create a product that can be harmful, then you should be responsible for the design decisions," he said, "and if you bring it out into the world, you do so in a safe and sustainable way. "
CNNMoney (New York) First published on July 26, 2018: 19:02 ET