Convai microphone

A Convai project expects a microphone attached to the machine it runs on. In an app streamed by Eagle 3D Streaming there isn't one — the microphone is at the viewer's end. Here is how to connect the two.

Live demo  ·  demo project  ·  video walkthrough

Why it needs configuring at all

Convai listens to Unreal's audio input, and on a streaming machine that input is empty. The browser has the viewer's voice; Pixel Streaming can carry it; the work is pointing Convai at that stream instead of at the local device that does not exist.

Part 1. The character Blueprint

  1. Open your character Blueprint and add a PixelStreamingAudio component, wiring up the audio-streaming nodes.

    The audio nodes added to the character Blueprint

  2. Select the ConvaiPlayer component and, in its Pixel Streaming section, untick "Use Pixel Streaming Mic Input".

    Disabling Use Pixel Streaming Mic Input

  3. Select the PixelStreamingAudio component and, under Effects, set Base Submix to AudioInput.

    Setting the PixelStreamingAudio component's Base Submix to AudioInput

  4. Under Edit → Project Settings → Plugins → Convai → Convai API → API Key, enter the API key from your Convai dashboard.

    Entering the Convai API key in project settings

Part 2. On the platform

  1. Package and upload the project.

  2. Create a configuration for it, and on the configuration's Microphone tab turn on Use Browser side Microphone.

Both halves are required

The Blueprint work routes the audio inside Unreal; the configuration setting is what makes the browser capture it in the first place. With one but not the other, the character simply never hears anything.

When the character does not respond

Work through these in order — most reports are the first one.

Last updated