Block's AI Learns Language From Sound Alone

For developers running high-performance Mac hardware, a new way to process voice data is now available. Block, the payments company led by Jack Dorsey, released a system called Dessa that handles audio inputs and outputs without converting them to text. The model operates locally on a MacBook Pro M4 Max or a machine with equivalent power. This local execution means the data does not need to travel to a cloud server for processing. The system represents a shift in how voice interfaces might function on personal devices.

The development team describes Dessa as a true speech-to-speech model. They state that everything the system knows came from human speech. Ragavan Thurairatnam, who leads the project, shared a sample interaction where the model converses with a user. In that clip, the AI initially mistakes the speaker for a person named Ronaldo. Despite that error, the rest of the exchange flows with a natural rhythm. The team published several other examples on the model's website to demonstrate its capabilities.

One recorded exchange covers Christopher Nolan films. During that conversation, the model drifts into a discussion about Bryan Singer's Superman movies. In another sample, a user tells the model about a surprise trip to Paris. The website notes that the model responds with what sounds like teary gratitude. A further clip pairs Dessa with OpenAI's GPT-Live-1 voice model. In that specific test, the audio quality and response timing appear smoother than in other standalone examples.

The developers list three main reasons for creating the system: curiosity, better human-AI understanding, and safety. They compare the learning process to a baby acquiring language. Babies learn to talk without ever seeing a word written down. The team wanted to test if a model could do the same. They report that it can, though they note the task is very hard. The project skips the traditional step of training on written text.

The team argues that current voice assistants limit the potential of speech. They believe speech is the most natural interface for intelligence. However, they say today's systems flatten what people say into text. This process prevents the AI from keeping up with human timing. The team also points to a safety gap in current technology. They state that AI systems are gaining power while remaining nearly deaf to the emotion in voices. They describe a scenario where a person in serious pain speaks to a voice AI that carries on regardless. They position Dessa as a step toward fixing that disconnect.

Block plans to give early access to a small group of AI safety researchers. The announcement did not include a timeline for a wider public release. The company has published the model's details and sample recordings on the dedicated Dessa website.