How humans struggled for 66 years to make machines speak

Voice generation with the help of AI (artificial intelligence) has become commonplace. Today, machines can produce speech that sounds remarkably close to a human voice, often within seconds.

However, decades before AI became part of everyday life, humans had already achieved something that once seemed extraordinary: getting a machine to sing.

In 1961, a computer sang a love song for the first time. The voice was thin, mechanical, and slow. Yet the performance marked a turning point.

The song was ‘Daisy Bell’, and the machine was an IBM mainframe at Bell Telephone Laboratories.

What sounded then like little more than a technological curiosity would eventually become one of the most famous scenes in Hollywood history.

However, before machines could produce natural-sounding speech, researchers first had to solve a much more basic problem: how to make a machine produce a human-like voice at all.

A person does not simply pronounce letters. Sound is continuously shaped by the mouth and vocal tract. Voice changes with pitch, rhythm, emphasis, pauses, and emotion.

For machines, every one of those elements had to be translated into instructions they could reproduce.

Bell Telephone Laboratories, the research arm of the American telephone network, had been working on artificial speech for years. By the early 1960s, researchers John L Kelly Jr. and Carol Lochbaum were working on computer speech synthesis, while Max Mathews was developing ways for computers to create music.

Then came a simple but imaginative idea: why not make the computer sing instead of demonstrating the technology with a dry scientific phrase?

They chose ‘Daisy Bell’, written by Harry Dacre in 1892. Its familiar tune and simple melody made it suitable for the experiment.

Kelly and Lochbaum programmed the computer’s vocal part, while Mathews created the musical accompaniment.

The machine was not improvising. Researchers carefully instructed it to produce specific sounds in a particular order. The process involved punched cards, painstaking programming, and considerable computing time.

The Bell Labs demonstration reportedly took around 20 minutes to produce one minute of music.

By today’s standards, that sounds almost absurd. In 1961, it was astonishing.

The result became the earliest known recording of a computer-synthesised voice singing a song. The performance was later selected for preservation in the US National Recording Registry.

‘Daisy Bell’ had another destination.

Science-fiction writer Arthur C Clarke was among those who witnessed the demonstration. Years later, while working with Stanley Kubrick on 2001: A Space Odyssey, Clarke remembered the computer singing.

Near the end of the film, the HAL 9000 computer is being shut down. As its functions begin to fail, HAL starts singing: ‘Daisy, Daisy…’

A real computer had once struggled to make the song audible. A fictional computer now sang it as it lost its mind.

The connection gave ‘Daisy Bell’ a strange second life – transforming an early experiment in computer speech into a cultural symbol of artificial intelligence.

However, making a computer speak was only the beginning. Making it sound natural was a much harder problem.

For decades, researchers developed increasingly sophisticated ways of constructing artificial voices. Some systems assembled speech from fragments of recorded human voices. Others relied on mathematical models to generate sounds.

The machines could speak, but they often sounded unmistakably artificial. Plus, every improvement demanded more work.

If a system needed to produce a different sound, emphasis, or speaker, engineers often had to modify the underlying system or collect new recordings.

Stephen Hawking offered one of the most remarkable examples of this era.

After losing his natural ability to speak, Hawking used a speech synthesiser based on work by MIT researcher Dennis Klatt. The voice was electronic and distinctly American. Yet Hawking kept it for decades.

It was not a reproduction of Hawking’s biological voice. It became Hawking’s voice because it gave him something more important: the ability to speak.

Then machine learning changed the equation.

Instead of engineers having to describe every element of speech manually, neural networks could learn patterns from human voices.

In 2016, Google DeepMind introduced WaveNet, a neural network capable of generating raw audio one sample at a time. It produced speech that sounded more natural than many earlier text-to-speech systems.

As it learned from recordings of real human speakers, WaveNet could model different voices and reproduce subtle characteristics of speech, including incidental sounds such as breathing and mouth movements.

The technology was initially too slow for practical products. But progress came quickly.

In October 2017, DeepMind's Parallel WaveNet began generating Google Assistant voices in US English and Japanese. The technology was becoming faster while retaining the advantages of neural speech synthesis.

Then came voice cloning.

Research systems such as Google’s SV2TTS demonstrated in 2018 that a speaker’s voice could be reproduced from relatively short recordings.

In 2023, Microsoft’s VALL-E pushed the idea further, reducing the required reference recording to about three seconds while attempting to preserve aspects of the speaker’s emotion and acoustic environment.

The initial systems were far too slow for ordinary use. However, improved versions became dramatically faster, with some capable of producing one second of speech in roughly 50 milliseconds.

The direction of travel had changed. The machine was no longer simply being told every detail of how a voice should sound. It was learning what human voices sounded like. That distinction opened the door to the voice-generation systems people use today.

Now, a person can enter text and receive speech almost immediately. Systems can produce different voices, accents, rhythms, and styles. Some can adapt to a particular speaker from a relatively small amount of recorded speech.

The technology behind that process is enormously complicated.

For the user, however, it can be almost ridiculously simple: type. generate. listen.

That is the distance between 1961 and today.

The IBM mainframe needed researchers, punched cards, carefully designed speech models, and roughly 20 minutes to create one minute of music.

Modern AI can turn written words into a convincing human-like voice in seconds.

However, the most remarkable part is not simply how quickly the technology advanced. It is how convincingly artificial voices can now reproduce something deeply human.

Early systems essentially selected and assembled sounds according to instructions.

Modern systems can generate speech with variations in rhythm, emphasis, and delivery. Increasingly, they can also respond to a user's voice in real time, producing a generated response that can sound remarkably like a person speaking from the other side of a phone or laptop.

In 1961, researchers were asking whether a machine could make a computer speak or sing. Decades later, that question has largely been answered.

‘Daisy Bell’ was written in 1892. A computer sang it in 1961. HAL sang it in 2001: A Space Odyssey.

Today, machines can sing, speak, narrate, dub, and imitate voices almost on demand.

The little IBM recording was never meant to predict any of that. It simply proved that a machine could find its voice.