Dangerous: OpenAI develops voice cloning tool in super election year
OpenAI, the world-famous creator of ChatGPT, has developed a whole range of other AI models in addition to GPT-4 that play into all areas of content. One of them is Voice Engine, which can be used to create artificial voices. Voice Engine uses text input and a single 15-second audio sample to produce natural-sounding speech that closely resembles the original speaker, according to OpenAI. It is only a small model that can generate “emotional and realistic voices” without a lot of data.
Of course, OpenAI is not alone with such tools. A whole range of startups such as Synthesia, Murf AI, Play.ht or ElevenLabs also offer the generation of synthetic voices in one form or another. The AI company around Sam Altman, which as the market leader is of course under special observation, has now refrained from a major release of the voice engine, which was already developed in 2022. Only a handful of partners have been allowed to use the clone voices under certain conditions.
“We recognize that producing language that resembles people’s voices poses serious risks that are particularly salient in an election year. “We work with U.S. and international partners in government, media, entertainment, education, civil society and beyond to ensure we incorporate their feedback in development,” OpenAI said. “We believe that any widespread use of synthetic speech technology should be accompanied by voice authentication, ensuring that the original speaker consciously adds their voice to the service, as well as a no-go voice list that prevents voice creation Identifies and prevents people who are too similar to prominent people.”
Only selected partners are allowed to use Voice Engine
The partners currently testing Voice Engine have agreed to usage policies that prohibit impersonating another person or entity without consent or legal permission. They must also obtain “the express and informed consent of the original speaker” whose voice is being synthesized. In addition, listeners must be made aware that the voices they are hearing were generated by AI. In addition, machine-recognizable audio watermarks are intended to ensure that audio data generated by Voice Engine can be traced.
There are of course numerous possibilities for fraud with fake voices, such as calls or interviews with well-known personalities. Before AI voices are used more widely, voice-based authentication should be eliminated as a security measure for access to bank accounts and other sensitive information, OpenAI says.
In Pakistan, AI videos have already been used in election campaigns. They may have helped Jailed Ex-Prime Minister Imran Khan’s opposition party PTI win the election. This made it possible for Khan to campaign behind bars without the opportunity to appear. It is also known that a married couple in the USA received fake calls from relatives who were said to have been held hostage. This shows that AI technologies for cloning voices are already in use worldwide – OpenAI would be another, but very well-known, company that provides such tools.
