Gemini Multimodal Playground ✨

A Python application for having voice and video conversations with Google's new Gemini 2.0 model. Features real-time voice and video input and audio responses. Available in two versions: a full-stack web application and a standalone Python script.

Full-Stack Version

gemini.playground.full.stack.mp4

Getting Your Gemini API Key

1. Go to Google AI Studio
2. Sign in with your Google account
3. Click "Create API Key"
4. Copy the generated API key and paste it into the appropriate .env file

Prerequisites

1. Python 3.12 or higher
2. Node.js 18 or higher
3. A Google Cloud account
4. A Gemini API key

Backend Setup

1. Clone this repository
2. Create a virtual environment and activate it:

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install the required packages:

pip install -r requirements.txt

4. Create a .env file in the root directory with your API key:

GEMINI_API_KEY=your_api_key_here

5. Start the backend server:

python backend/main.py

Frontend Setup

1. Navigate to the frontend directory:

cd frontend

2. Install dependencies:

npm install

3. Start the development server:

npm run dev

4. Open http://localhost:3000 in your browser

Standalone Version

playground.demo.mp4

Prerequisites

Same as above, but only Python-related requirements are needed and Tkinter:

• On Ubuntu/Debian: sudo apt-get install python3-tk
• On Fedora: sudo dnf install python3-tkinter
• On macOS & Windows: Already included with Python

Installation

1. Clone this repository or download the standalone folder

2. Create a virtual environment and activate it:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install the required packages:

pip install -r requirements.txt

4. Create a .env file in the standalone directory with your API key:

GEMINI_API_KEY=your_api_key_here

Running the Standalone Application

1. Make sure your virtual environment is activated
2. Run the script:

python standalone.py

Configuration Options

Both versions provide several configuration options:

• System Prompt: The initial instructions given to Gemini about its role and behavior
• Voice: Choose from different voice options for Gemini's responses:
  • Puck
  • Charon
  • Kore
  • Fenrir
  • Aoede
• Enable Google Search: Allows Gemini to search the internet for current information
• Allow Interruptions: Enables interrupting Gemini while it's speaking

Troubleshooting

• Audio feedback loop issue - Gemini may interrupt itself when it detects its own voice output through your microphone. This occurs because the application processes all incoming audio, including Gemini's responses. To prevent this feedback loop, either:
  1. Disable the "Allow Interruptions" option in settings
  2. Use headphones/earphones to prevent your microphone from picking up Gemini's audio output