ElevenLabs, a pioneering startup in the field of AI-powered voice cloning, has secured an impressive $80 million in a recent Series B funding round. The round, co-led by renowned investors such as Andreessen Horowitz, former GitHub CEO Nat Friedman, and entrepreneur Daniel Gross, has catapulted the company to unicorn status, valuing it at over $1 billion. With this influx of capital, ElevenLabs plans to further enhance its products, expand its infrastructure and team, engage in AI research, and prioritize safety measures to ensure the responsible and ethical development of AI technology.
AI-powered voice cloning
Founded in 2022 by Piotr Dabkowski, a former Google machine learning engineer, and Mati Staniszewski, a former Palantir deployment strategist, ElevenLabs set out to create AI-powered tools capable of producing and editing synthetic voices. Their inspiration stemmed from poorly dubbed American films, with the belief that AI could significantly improve this process. The startup launched its browser-based speech-generation app approximately a year ago, allowing users to create lifelike voices with adjustable parameters for intonation, emotion, cadence, and other vocal characteristics.
ElevenLabs offers both free and paid options for users. Free users can input text and receive a recording read aloud by default voices while paying customers can upload voice samples to craft unique styles using ElevenLabs’ voice cloning technology. Beyond this, the company is expanding its offerings to cater to audiobooks, dubbing for films and TV shows, and generating character voices for games and marketing activations.
Innovations and customers
One of ElevenLabs’ notable innovations is its “speech to speech” tool, designed to preserve a speaker’s voice, prosody, and intonation while eliminating background noise. It also translates and synchronizes speech with the source material, particularly valuable for movies and TV shows. Upcoming developments include a dubbing studio workflow with transcript and translation tools, as well as a subscription-based mobile app that narrates webpages and text using ElevenLabs’ voices.
The company’s innovations have attracted clients such as Paradox Interactive, a prominent game developer known for titles like Cities: Skylines 2 and Stellaris, and The Washington Post, among other media and entertainment companies. ElevenLabs boasts that users have generated audio content equivalent to more than 100 years of audio, with employees at 41% of Fortune 500 companies utilizing their platform.
Controversies and ethical concerns
Despite its success, ElevenLabs has not been without controversy. The platform has been used for malicious purposes, allowing individuals to clone voices and disseminate harmful content, including threats of violence and hate speech. In response, ElevenLabs has taken steps to root out abusive users and implemented tools to detect speech created using their platform. Furthermore, they plan to collaborate with distribution partners to make this tool available on third-party platforms.
The startup has also faced criticism from voice actors who claim that ElevenLabs uses samples of their voices without consent, potentially leading to the promotion of content they do not endorse. Some have experienced harassment campaigns involving cloned voices. There is also growing concern within the voice acting industry about the potential displacement of human actors by AI-generated voices, especially in lower-paying, entry-level roles.
ElevenLabs’ solution and competition
To address these concerns, ElevenLabs has introduced a marketplace for voices, allowing users to create, verify, and share voices. Creators receive compensation when others use their voices, giving them control over availability and compensation terms. However, instead of monetary compensation, creators currently receive credit toward ElevenLabs’ premium services.
The startup faces competition from other players in the synthetic voice space, including Papercup, Deepdub, Acapela, Respeecher, and Voice.ai, as well as established tech giants like Amazon, Microsoft, and Google. ElevenLabs plans to continue its growth, expanding its team from 40 to 100 employees by the end of the year, with a determination to make a significant impact in the rapidly growing synthetic voice market.
ElevenLabs, with an impressive $80 million in funding and a valuation of over $1 billion, is emerging as a dominant force in the AI-powered voice cloning industry. While its innovations have garnered praise and attracted top-tier clients, the startup has also faced criticism and ethical concerns related to the misuse of its technology and the potential threat it poses to the voice acting profession. Nonetheless, ElevenLabs is actively addressing these issues through improved safeguards and a marketplace for voice creators. As it faces competition from other emerging startups and tech giants, ElevenLabs remains committed to advancing the synthetic voice market while navigating the complexities of ethics and industry disruption.
