marcus welz

Android: Ducking Audio with TextToSpeech

The Android app I'm working on makes use of TextToSpeech. It's also very likely that the user is playing music in the background. In order for the music to not drown out the TTS output, we'll make use of audio ducking, a feature that's "new" as of API level 8 (Android 2.2).

The ingredients needed for this is an instance of the TextToSpeech class and the AudioManager.

In order to get spoken text, we use TextToSpeech.speak(String text, int mode, HashMap<String,String> params);

Now, in order to properly get any background music to duck when we start speaking, and raise the volume when we're done speaking, we'll need to know when the TTS is done blabbering. And for this, we can register a callback via TextToSpeech.setOnUtteranceCompletedListener(). There are, however, a few pitfalls that we'll need to watch out for:

1. TextToSpeech.setOnUtteranceCompletedListener() MUST be invoked after onInit() was called. Otherwise it won't necessarily register properly, and onUtteranceCompleted() may never be called.
2. When invoking TextToSpeech.speak(), we must provide an "Utterance ID" via the params HashMap using TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID. If there's no ID, onUtteranceCompleted() will not be invoked.
3. Because it is possible to queue up more utterances via TextToSpeech.speak() before the previous one has completed, we'll need to track how many we've queued up, so we know to only release audio focus after all of them have been completed. Otherwise we've got a race condition where we queued up two things to speak, but lose focus after the first one has completed. The result is a mess where music will cut in and out while text is being spoken.

With this in mind, we'll create a wrapper class which will encapsulate this functionality.

public class DuckingTTS {

    // Log tag
    private final String TAG = DuckingTTS.class.getName();

    // Debugging is on (set to false to cut down on spam)
    private static final boolean D = false;

    /**
     * Which audio stream to use. We use the music one here,
     * so that it'll be "in line" with the music volume from whatever is playing.
     */
    private final int STREAM_TYPE = AudioManager.STREAM_MUSIC;

    /**
     * Parameters we're feeding with each invocation of speak()
     */
    private HashMap ttsParams;

    /**
     * How many utterances are playing at a particular moment.
     */
    private int mUtterancesPlaying = 0;

    /**
     * The text-to-speech engine
     */
    private TextToSpeech mTts;

    /**
     * Whether TTS is initialized (onInit() called yet?)
     */
    private boolean mIsInitialized = false;

    /**
     * Queue up chatter when TTS is not initialized yet
     */
    private List queue = new ArrayList();

    /**
     * AudioManager injected via RoboGuice
     */
    @Inject
    private AudioManager mAm;

    /**
     * Context will be injected via RoboGuice
     * @param context
     */
    @Inject
    public DuckingTTS(Context context) {

         ttsParams = new HashMap();
         ttsParams.put(TextToSpeech.Engine.KEY_PARAM_STREAM, String.valueOf(STREAM_TYPE));
         ttsParams.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, "ID");

         mTts = new TextToSpeech(context, ttsOnInitListener);
         mIsInitialized = false;
    }

    public boolean speak(String text) {

        if (!mIsInitialized) {
            if (D) Log.d(TAG, "speak(\"" + text + "\") - queued");
            queue.add(text);
            return false;
        }

        // Tell it to speak
        if (D) Log.d(TAG, "speak(\"" + text + "\") - sending to TTS");
        mTts.speak(text, TextToSpeech.QUEUE_ADD, ttsParams);

        if (mAm == null) return true;

        // if this is the first utterance (e.g. we're not already talking) then request audio focus w/ducking.
        if (mUtterancesPlaying < 1) {
            int status = mAm.requestAudioFocus(audioFocus, STREAM_TYPE, AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK);

            if (status == AudioManager.AUDIOFOCUS_REQUEST_FAILED) {
                Log.e(TAG, "speak() audio focus request failed.");
            }
        }
        mUtterancesPlaying++;

        return true;
    }

    public void shutdown() {
        mTts.shutdown();
    }

    private TextToSpeech.OnInitListener ttsOnInitListener =
            new TextToSpeech.OnInitListener() {

        /**
         * Callback for when the TextToSpeech engine was initialized.
         * Result will tell us whether this was successful or not.
         *
         * @param status
         */
        public void onInit(int status) {

            if (status != TextToSpeech.SUCCESS) {
                return; // we just abort on failure, it's never fully initialized
                // this can be bad, by the way, because every speak() call will now add something to the queue.
            }

            mIsInitialized = true;
            mTts.setOnUtteranceCompletedListener(ttsOnUtteranceCompletedListener);

            // Also speak anything that was queued up so far.
            for (String text : queue) {
                speak(text);
            }
        }

    };

    private TextToSpeech.OnUtteranceCompletedListener ttsOnUtteranceCompletedListener =
            new TextToSpeech.OnUtteranceCompletedListener() {

        /**
         * Callback when TTS has completed an utterance.
         */
        public void onUtteranceCompleted(String utteranceId) {
            if (D) Log.d(TAG, "onUtteranceCompleted(\"" + utteranceId + "\")");
            mUtterancesPlaying--;

            if (mAm == null) return;

            // once we're done speaking, lose audio focus.
            if (mUtterancesPlaying < 1) {
                mUtterancesPlaying = 0;
                mAm.abandonAudioFocus(audioFocus);
            }
        }

    };

    private AudioManager.OnAudioFocusChangeListener audioFocus =
            new AudioManager.OnAudioFocusChangeListener() {

        public void onAudioFocusChange(int focusChange) {

            // I don't think we actually care.
            if (D) Log.d(TAG, "onAudioFocusChange(" + focusChange + ")");

        }
    };

}

That's it. I use it in a service, and I wire it in with a simple annotation using RoboGuice:

public class GpsService extends RoboService {

    @Inject
    private DuckingTTS mDuckingTTS;

    // ... rest of the class implementation ...
}

As the final disclaimer: The class works for me and my purposes at the moment, but it doesn't handle every error scenario. Also, don't forget to call shutdown() to release the TTS resources.