MemoMuse

Hum it. Produce it. Sell it. Voice memo → AI-generated full track with vocals & instruments in under a minute. List as vinyl on Shopify — with live support every step of the way.

Comment

MemoMuse — Full Product Overview

Voice memo → Produced music demo → Vinyl on Shopify.

Record a rough idea, pick a genre, tweak with studio controls, get a mixed track with vocals and instrumentation, sell it as a vinyl record, and manage customer support — all in one app.

What is MemoMuse?

MemoMuse is an end-to-end music creation and commerce platform with three core features:

  1. Music Studio — Turn voice memos into fully produced songs.
  2. Vinyl Storefront — Publish and sell generated tracks as vinyl records on Shopify.
  3. Voicemail Copilot (Customer Agent) — AI-powered customer support inbox, purpose-built for Shopify merchants.

Feature 1: Music Studio

The core pipeline transforms a rough voice idea into a fully mixed, produced track in six steps.

How It Works

Step Service What happens
1. Transcribe OpenAI Whisper (local, base) Raw audio → text transcript
2. Analyze Google Gemini 2.5 Flash Transcript → song title, lyrics (Verse/Chorus, 16–24 lines), style prompt, mood, BPM, key, and contains_lyrics detection (real words vs humming)
3. Store session Backboard.io REST API Persists transcript, lyrics, genre, mood for memory across runs. Optional — pipeline continues if unavailable
4. Refine lyrics Featherless AI (Qwen2.5-7B) LLM polish pass on lyrics. Optional — original lyrics used if unavailable
5. Generate audio Google Lyria Realtime + ElevenLabs Instrumental: Lyria generates 60s at 48 kHz from the style prompt. Vocals: ElevenLabs TTS for lyrics, STS for humming (preserves melody). Both run in parallel. Falls back to instrumental-only if vocals fail
6. Mix + Post-process pydub + FFmpeg Both tracks normalized to −20 dBFS. Bass/treble EQ, pitch shift, vocal balance applied. Exported as final MP3

Studio Controls

Control Range What it does
Voice 20+ ElevenLabs voices Selects vocal character (male, female, various accents) with preview
Bass -10 to +10 Low-frequency EQ — boosts/cuts under 250 Hz
Treble -10 to +10 High-frequency EQ — boosts/cuts above 4 kHz
Pitch -6 to +6 semitones Shifts the final mix up/down via sample rate manipulation
Vocal Mix -6 to +6 dB Adjusts vocal/instrumental balance

Frontend Features

  • Audio recording — MediaRecorder API with live waveform visualizer (40 animated bars).
  • Genre picker — 8 chips: Pop, Lo-Fi, Hip Hop, Cinematic, R&B, Indie Folk, Electronic, Jazz.
  • Pipeline progress — 4-step animated indicator (Transcribe → Analyze → Generate → Mix).
  • Live karaoke display — Lyrics highlighted line-by-line in sync with playback, auto-scrolling.
  • Vinyl record visual — CSS-animated spinning disc with song title and genre label.
  • Download MP3 — One-click download named after the song title.
  • Copy Lyrics — Copies generated lyrics to clipboard.

Feature 2: Vinyl Storefront (Shopify Integration)

Every generated track can be published and sold as a vinyl record product directly on your Shopify store — closing the loop from creative idea to real commerce.

  • One-click publish — The "Buy as Vinyl" button calls POST /api/publish, which uses the Shopify Admin API (write_products scope) to create a product instantly.
  • Auto-populated listing — Product includes song title, $24.99 price, full generated lyrics in description, and genre/mood/BPM/key metadata tagged under the "MemoMuse" vendor.
  • CDN-aware UX — Frontend waits 8 seconds after product creation before opening the Shopify store URL, ensuring the product is live and purchasable.
  • Storefront config — GET /api/config returns the Shopify storefront domain and token so the frontend can link directly to the product page.

Result: Artists go from humming an idea to having a purchasable product on their Shopify store in under two minutes.

Feature 3: Voicemail Copilot (Customer Agent)

Voicemail Copilot is MemoMuse's AI-powered customer service feature — a support inbox that lets Shopify merchants receive, analyze, and respond to customer voicemails, then push resolved tickets back to Shopify without leaving the app.

User Flow

  1. Open Voicemail Copilot — Visit /voicemail or click it in the nav.
  2. Add a voicemail — Upload an audio file or use a built-in demo (e.g., "Angry Customer — Late Order").
  3. Analyze — Click Analyze Voicemail; Gemini returns structured JSON and a new ticket appears in the inbox.
  4. Review ticket — See summary, classification chips (intent, sentiment, urgency), original audio playback, full transcript, and a suggested reply.
  5. Generate voice reply & push to Shopify — Edit the reply, click Generate Voice Reply (ElevenLabs TTS), then push the resolved ticket back to the Shopify store's customer service record.

Analysis Output

Field Type Description
transcript string Full text of the voicemail
intent enum ORDER_STATUS, RETURN, GENERAL_QUESTION, COMPLAINT, OTHER
sentiment enum POSITIVE, NEUTRAL, NEGATIVE
urgency enum LOW, MEDIUM, HIGH
summary string Short summary of the message
suggestedReply string AI-suggested reply text for the support agent

Full API Reference

Method Path Description
GET / Single-page frontend (Music Studio)
POST /generate Accepts audio + genre + studio JSON, runs pipeline, returns track data
GET /audio/{filename} Serves generated MP3 files from temp/
GET /api/voices Returns available ElevenLabs voices (id, name, gender, accent, preview URL)
POST /api/publish Creates a vinyl product on Shopify; returns product_url
GET /api/config Returns Shopify storefront domain + token
GET /voicemail Serves Voicemail Copilot SPA
POST /api/voicemail/analyze Analyzes voicemail audio via Gemini
POST /api/voicemail/tts Generates ElevenLabs voice reply
POST /api/voicemail/shopify-push Pushes resolved ticket to Shopify

Sponsor Integrations

Google — Gemini + Lyria

  • Gemini 2.5 Flash: Powers both Music Studio analysis (lyrics, mood, BPM, key) and Voicemail Copilot classification (intent, sentiment, urgency, suggested reply).
  • Lyria Realtime: Generates 60-second instrumentals via async WebSocket from a style prompt + BPM.

ElevenLabs — Voices + TTS + STS

  • TTS (eleven_multilingual_v2): Synthesizes lyrics into vocals and generates voice replies for Copilot.
  • STS (eleven_multilingual_sts_v2): When the user hums, preserves melody while applying the selected voice.
  • Voice Library: 20+ voices with metadata and preview URLs, cached after first call.

Shopify — Admin API

  • Music Studio: Publishes generated tracks as $24.99 vinyl record products.
  • Voicemail Copilot: Pushes resolved replies back to the store's customer service record.
  • Scope: Uses OAuth-acquired Admin API token with write_products scope.

Tech Stack

Component Technology
Runtime Python 3, FastAPI, uvicorn
Transcription OpenAI Whisper (local, base model)
LLM Analysis Google Gemini 2.5 Flash
Instrumental Google Lyria Realtime (experimental)
Vocals ElevenLabs TTS / STS + Voice Library
Lyric Refinement Featherless AI (Qwen2.5-7B-Instruct)
Session Memory Backboard.io REST API
E-Commerce Shopify Admin API
Audio Processing pydub + FFmpeg
Frontend Vanilla HTML/CSS/JS, MediaRecorder API

Setup & Environment

Environment Variables

GEMINI_API_KEY                       # Google AI Studio — Gemini + Lyria
ELEVENLABS_API_KEY                   # ElevenLabs TTS/STS + voice library
ELEVENLABS_VOICE_ID                  # Optional — default voice ID
FEATHERLESS_API_KEY                  # Optional — lyric refinement
BACKBOARD_API_KEY                    # Optional — session memory
SHOPIFY_ADMIN_TOKEN                  # Shopify Admin API (write_products scope)
NEXT_PUBLIC_SHOPIFY_STORE_DOMAIN     # e.g. yourstore.myshopify.com
SHOPIFY_STOREFRONT_TOKEN             # Shopify Storefront API token

Running the App

# Install dependencies
pip install -r requirements.txt

# Ensure ffmpeg is installed
brew install ffmpeg        # macOS
apt install ffmpeg         # Ubuntu/Debian

# Start the server
python main.py             # serves on http://localhost:8000

Built With

Share this project:

Updates