StudioSoundAudioProcessor

@RequiresApi(value = 27)
@ExperimentalApi
class StudioSoundAudioProcessor : BaseAudioProcessor

kotlin.Any
   ↳ androidx.media3.common.audio.BaseAudioProcessor
     ↳ androidx.media3.effect.playservices.StudioSoundAudioProcessor

AudioProcessor implementation for audio source separation using the StudioSound Google Play services module.

This AudioProcessor outputs 8 or 16 interleaved channels containing the separated sources from the received input stream, for mono or stereo input respectively. getResults provides information about each resulting source, including what label it was classified as and what channel indices in the output stream that source corresponds to.

The StudioSound module only supports up to 2 minutes of input. queueInput will throw if the caller attempts to process an input stream beyond 2 minutes.

Each instance only supports processing one input stream. Any attempt to reuse the processor after processing a stream will throw.

This AudioProcessor holds an internal ExecutorService for handling asynchronous events. If using this class as a standalone effect, call reset to release this ExecutorService and other resources.

Summary

Nested types

Type of separated source.

A record containing metadata about a separated source.

Constants
const Int

See CROWD.
const Int

See MUSIC.
const Int

See NATURE.
const Int

See NOISE.
const Int

See SPEECH.
const Int

See UNSPECIFIED.
const Int

See WIND.

Public functions
java-static ListenableFuture<StudioSoundAudioProcessor!>!
create(context: Context!)

Returns a future that provides a new StudioSoundAudioProcessor instance.
ByteBuffer!

Returns a buffer containing processed output data between its position and limit.
ImmutableList<StudioSoundAudioProcessor.SeparationResult!>?

Returns a list of SeparationResult that holds information about the channels and labels corresponding to each separated source, or null if processing has not completed.
Boolean

Returns whether this processor will return no more output from getOutput until flush has been called and more input has been queued.
Unit
queueInput(inputBuffer: ByteBuffer!)

Queues audio data between the position and limit of the inputBuffer for processing.

Protected functions
AudioProcessor.AudioFormat!

Called when the processor is configured for a new input format.
Unit

Called when the processor is flushed.
Unit

Called when the end-of-stream is queued to the processor.
Unit

Called when the processor is reset.

Inherited Constants
From androidx.media3.common.audio.AudioProcessor
const ByteBuffer!

An empty, direct ByteBuffer.

Inherited functions
From androidx.media3.common.audio.AudioProcessor
Long

Returns the expected duration of the output stream when the processor is applied given a input durationUs.
From androidx.media3.common.audio.BaseAudioProcessor
AudioProcessor.AudioFormat!

Configures the processor to process input audio with the specified format.
Unit

This function is deprecated.

Use flush instead.
Unit

Prepares the audio processor to receive a new stream of input in the last configured format and clears any buffered data and pending output.
Boolean

Returns whether the current output buffer has any data remaining.
Boolean

Returns whether the processor is configured and will process input buffers.
Unit

This function is deprecated.

Use onFlush instead.
Unit

Queues an end of stream signal.
ByteBuffer!

Replaces the current output buffer with a buffer of at least size bytes and returns it.
Unit

Resets the processor to its unconfigured state, releasing any resources.

Inherited properties
From androidx.media3.common.audio.BaseAudioProcessor
AudioProcessor.AudioFormat!

The current input audio format.
AudioProcessor.AudioFormat!

The current output audio format.

Constants

AUDIO_TYPE_CROWD

const val AUDIO_TYPE_CROWD = 2: Int

See CROWD.

AUDIO_TYPE_MUSIC

const val AUDIO_TYPE_MUSIC = 3: Int

See MUSIC.

AUDIO_TYPE_NATURE

const val AUDIO_TYPE_NATURE = 4: Int

See NATURE.

AUDIO_TYPE_NOISE

const val AUDIO_TYPE_NOISE = 6: Int

See NOISE.

AUDIO_TYPE_SPEECH

const val AUDIO_TYPE_SPEECH = 1: Int

See SPEECH.

AUDIO_TYPE_UNSPECIFIED

const val AUDIO_TYPE_UNSPECIFIED = 0: Int

See UNSPECIFIED.

AUDIO_TYPE_WIND

const val AUDIO_TYPE_WIND = 5: Int

See WIND.

Public functions

create

java-static fun create(context: Context!): ListenableFuture<StudioSoundAudioProcessor!>!

Returns a future that provides a new StudioSoundAudioProcessor instance.

This method verify that the current device supports the Google Play services StudioSound module, attempt to download and install it if needed, and then return a new instance of StudioSoundAudioProcessor.

Parameters
context: Context!

The Context.

getOutput

fun getOutput(): ByteBuffer!

Returns a buffer containing processed output data between its position and limit. The buffer will always be a direct byte buffer with native byte order. Calling this method invalidates any previously returned buffer. The buffer will be empty if no output is available.

Returns
ByteBuffer!

A buffer containing processed output data between its position and limit.

getResults

fun getResults(): ImmutableList<StudioSoundAudioProcessor.SeparationResult!>?

Returns a list of SeparationResult that holds information about the channels and labels corresponding to each separated source, or null if processing has not completed.

Any channels that do not map to a SeparationResult are empty and should be ignored.

isEnded

fun isEnded(): Boolean

Returns whether this processor will return no more output from getOutput until flush has been called and more input has been queued.

queueInput

fun queueInput(inputBuffer: ByteBuffer!): Unit

Queues audio data between the position and limit of the inputBuffer for processing. After calling this method, processed output may be available via getOutput. Calling queueInput(ByteBuffer) again invalidates any pending output.

Parameters
inputBuffer: ByteBuffer!

The input buffer to process. It must be a direct byte buffer with native byte order. Its contents are treated as read-only. Its position will be advanced by the number of bytes consumed (which may be zero). The caller retains ownership of the provided buffer.

Protected functions

onConfigure

protected fun onConfigure(inputAudioFormat: AudioProcessor.AudioFormat!): AudioProcessor.AudioFormat!

Called when the processor is configured for a new input format.

Throws
androidx.media3.common.audio.AudioProcessor.UnhandledAudioFormatException

onFlush

protected fun onFlush(streamMetadata: AudioProcessor.StreamMetadata!): Unit

Called when the processor is flushed.

onQueueEndOfStream

protected fun onQueueEndOfStream(): Unit

Called when the end-of-stream is queued to the processor.

onReset

protected fun onReset(): Unit

Called when the processor is reset.