Stream your voice clone 🎼

We'll build a Django app that:

streams audio to the user's browser (easily with HttpStreamingResponse)
uses your cloned voice

Let's turn your voice into an infinitely-scalable instrument 🎉

Our final product will look like this (audio needed to hear the voice)

Let's go 🏎

Install and create your Django app

pip install --upgrade django elevenlabs python-dotenv

django-admin startproject core .
python manage.py startapp sim

Add our app sim to the INSTALLED_APPS in settings.py:

# settings.py
INSTALLED_APPS = [
   'sim',
   ...
]

Add your environment variables

Create a .env file in your project root and add the following:

ELEVENLABS_API_KEY=your-elevenlabs-api-key
VOICE1_ID=your-voice1-id
VOICE2_ID=your-voice2-id
VOICE2_ID=your-voice3-id

To get your ElevenLabs API key, sign up at ElevenLabs.
Add at least 3 voice ids you want to use from ElevenLabs to the .env file.

Add your urls

Add the following to your core/urls.py:

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
   path('admin/', admin.site.urls),
   path('', include('sim.urls')),
]

Create a urls.py file in your sim app and add the following:

from django.contrib import admin
from django.urls import path

from . import views


urlpatterns = [
   path('', views.index, name='index'),
   path('stream/', views.stream_audio, name='stream-audio'),
]

Add your templates

Create a templates directory in your sim app and add an index.html file:

<!DOCTYPE html>
<html lang="en">
<head>
   <meta charset="UTF-8">
   <meta name="viewport" content="width=device-width, initial-scale=1.0">
   <title>Stream your voice</title>

</head>
<body>
<div class="audio-player">
   <h1>Hear your voice</h1>
   <form action="{% url 'stream-audio' %}" method="get" target="audio_frame">
      <label for="text">Enter text:</label>
      <textarea name="text" id="text" required>The Western world is a beacon of happiness</textarea>
      <label for="voice">Choose a voice:</label>
      <select id="voice" name="voice">
         {% for voice in voices %}
         <option value="{{ voice.id }}">{{ voice.name }}</option>
         {% endfor %}
      </select>
      <button type="submit">Create audio</button>
   </form>
   <audio controls>
      <source src="" type="audio/mp3" id="audioSource">
      Your browser does not support the audio element.
   </audio>
</div>

<script>
   const form = document.querySelector('form');
   const audioSource = document.getElementById('audioSource');
   const audioPlayer = document.querySelector('audio');

   form.onsubmit = function(event) {
      event.preventDefault();
      const formData = new FormData(form);
      const url = `${form.action}?text=${encodeURIComponent(formData.get('text'))}&voice=${encodeURIComponent(formData.get('voice'))}`;
      audioSource.src = url;
      audioPlayer.load();
      audioPlayer.play();
   };
</script>
</body>
</html>

Create your views

Add the following to your sim/views.py:

import os

from django.shortcuts import render
from django.http import StreamingHttpResponse, HttpResponseBadRequest
from elevenlabs.client import ElevenLabs


def index(request):
   voices = [
      {"name": "Bruce", "id": os.environ['VOICE1_ID']},
      {"name": "Mike", "id": os.environ['VOICE2_ID']},
      {"name": "Shaneel", "id": os.environ['VOICE3_ID']},
   ]
   return render(request, 'index.html', context={"voices": voices})


def stream_audio(request):
   # Retrieve text and voice from query parameters
   text = request.GET.get('text')
   voice = request.GET.get('voice')

   if not text or not voice:
      return HttpResponseBadRequest("Text and voice parameters are required.")

   client = ElevenLabs(api_key=os.environ['ELEVENLABS_API_KEY'])
   try:
      audio_stream = client.generate(
         text=text,
         voice=voice,
         stream=True
      )
   except Exception as e:
      return HttpResponseBadRequest(f"Error generating audio: {str(e)}")

   response = StreamingHttpResponse(audio_stream, content_type='audio/mp3')
   return response

Run your app

python manage.py runserver

Visit the url mentioned in the terminal to see your app in action.

Add styling using Photon Designer

Currently, the app looks pretty basic:
Unstyled app

So, We'll use Photon Designer (my product) to style the app in three simple steps:

Visit Photon Designer and sign up (free currently)
paste our template into the editor
add a prompt "Make an elegant audio player for streaming audio, in the style of Apple Music, with a prominent play button" and press generate
Paste generated code and replace the content of the index.html file with it.

Here's a video of me doing this in real-time:

And here's how our final app looks after a couple of iterations with Photon Designer: Styled app

Add your voice clone

Finally, let's add your specific voice to the app, so that you can stream your voice clone to the user. This is very easy to do. We simply need to clone your voice and copy your voice clone id to the .env file. To do this:

Create a voice clone on ElevenLabs. See my short video below on how to do this.
Paste your voice clone id into the .env file

Here's a video from me showing how to clone your voice:

Congrats! ✅

You've successfully built a Django app that:

streams audio to the user's browser with Django and HttpStreamingResponse
turned your voice into an instrument that you can use to create audiobooks with your voice, or anything else 🎉

Here's the full repo for the app

P.S Photon Designer

As mentioned, I'm building: Photon Designer to style your HTML much faster. Currently free to use, so give it a try and let me know what you think 🚀