The multistream API allows individual Opus streams to be combined into a single packet, enabling support for up to 255 channels. Unlike an elementary Opus stream, the encoder and decoder must negotiate the channel configuration before the decoder can successfully interpret the data in the packets produced by the encoder. Some basic information, such as packet duration, can be computed without any special negotiation.
The format for multistream Opus packets is defined in RFC 7845 and is based on the self-delimited
Opus framing described in Appendix B of RFC 6716. Normal Opus packets are just a degenerate case of
multistream Opus packets, and can be encoded or decoded with the multistream API by setting streams to 1 when initializing the encoder
or decoder.
Multistream Opus streams can contain up to 255 elementary Opus streams. These may be either "uncoupled" or "coupled", indicating that the decoder is configured to decode them to either 1 or 2 channels, respectively. The streams are ordered so that all coupled streams appear at the beginning.
A mapping table defines which decoded channel i should be used for each input/output (I/O) channel j. This table
is typically provided as an unsigned char array. Let i = mapping[j] be the index for I/O channel j. If i < 2*coupled_streams,
then I/O channel j is encoded as the left channel of stream (i/2) if i is even, or as the right channel of stream (i/2)
if i is odd. Otherwise, I/O channel j is encoded as mono in stream (i - coupled_streams), unless it has the special value 255,
in which case it is omitted from the encoding entirely (the decoder will reproduce it as silence). Each value i must either be the special
value 255 or be less than streams + coupled_streams.
The output channels specified by the encoder should use the Vorbis channel ordering. A decoder may wish to apply an additional permutation to the mapping the encoder used to achieve a different output channel order (e.g. for outputing in WAV order).
Each multistream packet contains an Opus packet for each stream, and all of the Opus packets in a single multistream packet must have the same duration. Therefore the duration of a multistream packet can be extracted from the TOC sequence of the first stream, which is located at the beginning of the packet, just like an elementary Opus stream:
int nb_samples, nb_frames;
nb_frames = opus_packet_get_nb_frames(data, len);
if (nb_frames < 1)
return nb_frames;
nb_samples = opus_packet_get_samples_per_frame(data, 48000) * nb_frames;
The general encoding and decoding process proceeds exactly the same as in the normal encoder and decoder APIs. See their documentation for an overview of how to use the corresponding multistream functions.
| Modifier and Type | Class and Description |
|---|---|
| public static class | OpusMultistream.
Contains the function pointers loaded from |
| Modifier and Type | Field and Description |
|---|---|
| public static final int | OPUS_MULTISTREAM_GET_DECODER_STATE_REQUEST
Multistream CTLs Enum values: |
| public static final int | OPUS_MULTISTREAM_GET_ENCODER_STATE_REQUEST
Multistream CTLs Enum values: |
| Access | Constructor and Description |
|---|---|
| protected |
| Modifier and Type | Method and Description | |||||
|---|---|---|---|---|---|---|
| public static int | nopus_multistream_decode(long st, long data, int
number of bytes in payload len, long pcm, int frame_size, int decode_fec)Unsafe version of: | |||||
| public static int | nopus_multistream_decode_float(long st, long data, int
number of bytes in payload len, long pcm, int frame_size, int decode_fec)Unsafe version of: | |||||
| public static long | nopus_multistream_decoder_create(int Fs, int
number of channels to decode. One of: channels, int streams, int coupled_streams, long mapping, long error)
Unsafe version of: | |||||
| public static int | nopus_multistream_decoder_init(long st, int Fs, int
number of channels to decode. One of: channels, int streams, int coupled_streams, long mapping)
Unsafe version of: | |||||
| public static int | nopus_multistream_encode(long st, long pcm, int frame_size, long data, int
size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as
the only bitrate control. Use max_data_bytes)SET_BITRATE_REQUEST to control the bitrate.Unsafe version of: | |||||
| public static int | nopus_multistream_encode_float(long st, long pcm, int frame_size, long data, int
size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as
the only bitrate control. Use max_data_bytes)SET_BITRATE_REQUEST to control the bitrate.Unsafe version of: | |||||
| public static long | nopus_multistream_encoder_create(int Fs, int
number of channels in the input signal. This must be at most 255. It may be greater than the number of coded channels ( channels, int streams, int coupled_streams, long mapping, int application, long error)streams +
coupled_streams).Unsafe version of: | |||||
| public static int | nopus_multistream_encoder_init(long st, int Fs, int
number of channels in the input signal. This must be at most 255. It may be greater than the number of coded channels ( channels, int streams, int coupled_streams, long mapping, int application)streams +
coupled_streams).Unsafe version of: | |||||
| public static long | nopus_multistream_surround_encoder_create(int Fs, int channels, int mapping_family, long streams, long coupled_streams, long mapping, int application, long error)
| |||||
| public static int | nopus_multistream_surround_encoder_init(long st, int Fs, int channels, long streams, long coupled_streams, long mapping, int application)
| |||||
| public static int | Returns: number of decoded samples on success or a negative error code on failuremultistream decoder st, ByteBuffer input payload (Use a data, ShortBuffer NULL pointer to indicate packet loss)output signal, with interleaved samples. This must contain room for pcm, int frame_size*channels samples.number of samples per channel of available space in frame_size, int pcm.
If this is less than the maximum packet duration (120ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case
of PLC ( flag (0 or 1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it
were lost. decode_fec)Decode a multistream Opus packet. | |||||
| public static int | Returns: number of decoded samples on success or a negative error code on failuremultistream decoder st, ByteBuffer input payload. Use a data, FloatBuffer NULL pointer to indicate packet lossoutput signal, with interleaved samples. (This must contain room for pcm, int frame_size*channels samples.)number of samples per channel of available space in pcm.
frame_size, int If this is less than the maximum packet duration (120ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case
of PLC ( flag (0 or 1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it
were lost. decode_fec)Decode a multistream Opus packet with floating point output. | |||||
| public static long | opus_multistream_decoder_create(int
sampling rate of input signal (Hz). One of: Fs, int
the total number of streams coded in the input. This must be no more than 255. streams, int number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of
coded channels ( coupled_streams, ByteBuffer streams + coupled_streams) must be no more than 255.mapping from coded channels to output channels mapping, IntBuffer returns error)OK on success, or a negative error code on failureAllocates and initializes a multistream decoder state. | |||||
| private static int | opus_multistream_decoder_ctl(long
decoder state st)Performs a CTL function on a multistream Opus decoder. | |||||
| public static int | opus_multistream_decoder_ctl(long
decoder state st, int CTL request request)Performs a CTL function on a multistream Opus decoder. | |||||
| public static int | opus_multistream_decoder_ctl(long
decoder state st, Opus.CTL request requestPerforms a CTL function on a multistream Opus decoder. | |||||
| public static void | opus_multistream_decoder_destroy(long
multistream decoder state to be freed st)Frees an | |||||
| public static int | Returns: the size in bytes on success, or a negative error code on failurethe total number of streams coded in the input. This must be no more than 255. streams, int number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of
coded channels ( coupled_streams)streams + coupled_streams) must be no more than 255.Gets the size of an | |||||
| public static int | Returns: OK on success, or a negative error code on failuremultistream decoder state to initialize st, int sampling rate of input signal (Hz). One of: Fs, int
the total number of streams coded in the input. This must be no more than 255. streams, int number streams to decode as coupled (2 channel) streams. This must be no larger than the total number of streams. Additionally, The total number of
coded channels ( coupled_streams, ByteBuffer streams + coupled_streams) must be no more than 255.mapping from coded channels to output channels mapping)Intialize a previously allocated decoder state object. | |||||
| public static int | Returns: the length of the encoded packet (in bytes) on success or a negative error code on failuremultistream encoder state st, ShortBuffer the input signal as interleaved samples. This must contain pcm, int frame_size*channels samples.number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the
permitted values are 120, 240, 480, 960, 1920, and 2880. Passing in a duration of less than 10 ms (480 samples at 48 kHz) will prevent the encoder
from using the LPC or hybrid modes. frame_size, ByteBuffer output payload. This must contain storage for at data)max_data_bytes.Encodes a multistream Opus frame. | |||||
| public static int | Returns: the length of the encoded packet (in bytes) on success or a negative error code on failuremultistream encoder state st, FloatBuffer the input signal as interleaved samples with a normal range of pcm, int +/-1.0. Samples with a range beyond +/-1.0 are supported but will be
clipped by decoders using the integer API and should only be used if it is known that the far end supports extended dynamic range. This must
contain frame_size*channels samples.number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the
permitted values are 120, 240, 480, 960, 1920, and 2880. Passing in a duration of less than 10 ms (480 samples at 48 kHz) will prevent the encoder
from using the LPC or hybrid modes. frame_size, ByteBuffer output payload. This must contain storage for at data)max_data_bytes.Encodes a multistream Opus frame from floating point input. | |||||
| public static long | opus_multistream_encoder_create(int
sampling rate of the input signal (in Hz). One of: Fs, int
the total number of streams to encode from the input. This must be no more than the number of channels. streams, int number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded
channels ( coupled_streams, ByteBuffer streams + coupled_streams) must be no more than the number of input channels.Mapping from encoded channels to input channels. As an extra constraint, the multistream encoder does not allow encoding coupled streams for which
one channel is unused since this is never a good idea. mapping, int the target encoder application application, IntBuffer returns error)OK on success, or a negative error code on failureAllocates and initializes a multistream encoder state. | |||||
| private static int | opus_multistream_encoder_ctl(long
encoder state st)Performs a CTL function on an multistream Opus encoder. | |||||
| public static int | opus_multistream_encoder_ctl(long
encoder state st, int CTL request request)Performs a CTL function on an multistream Opus encoder. | |||||
| public static int | opus_multistream_encoder_ctl(long
encoder state st, Opus.CTL request requestPerforms a CTL function on an multistream Opus encoder. | |||||
| public static void | opus_multistream_encoder_destroy(long
multistream encoder state to be freed st)Frees an | |||||
| public static int | Returns: the size in bytes on success, or a negative error code on errorthe total number of streams to encode from the input. This must be no more than 255. streams, int number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded
channels ( coupled_streams)streams + coupled_streams) must be no more than 255.Gets the size of an | |||||
| public static int | Returns: OK on success, or a negative error code on failuremultistream encoder state to initialize st, int sampling rate of input signal (Hz). One of: Fs, int
the total number of streams to encode from the input. This must be no more than the number of channels. streams, int number of coupled (2 channel) streams to encode. This must be no larger than the total number of streams. Additionally, The total number of encoded
channels ( coupled_streams, ByteBuffer streams + coupled_streams) must be no more than the number of input channels.mapping from encoded channels to input channels. As an extra constraint, the multistream encoder does not allow encoding coupled streams for which
one channel is unused since this is never a good idea. mapping, int the target encoder application application)Initialize a previously allocated multistream encoder state. | |||||
| public static long | opus_multistream_surround_encoder_create(int Fs, int mapping_family, IntBuffer streams, IntBuffer coupled_streams, ByteBuffer mapping, int application, IntBuffer error)
| |||||
| public static int | ||||||
| public static int | opus_multistream_surround_encoder_init(long st, int Fs, IntBuffer streams, IntBuffer coupled_streams, ByteBuffer mapping, int application)
|
| OPUS_MULTISTREAM_GET_DECODER_STATE_REQUEST | back to summary |
|---|---|
| public static final int OPUS_MULTISTREAM_GET_DECODER_STATE_REQUEST Multistream CTLs Enum values: | |
| OPUS_MULTISTREAM_GET_ENCODER_STATE_REQUEST | back to summary |
|---|---|
| public static final int OPUS_MULTISTREAM_GET_ENCODER_STATE_REQUEST Multistream CTLs Enum values: | |
| OpusMultistream | back to summary |
|---|---|
| protected OpusMultistream() | |
| nopus_multistream_decode | back to summary |
|---|---|
| public static int nopus_multistream_decode(long st, long data, int len, long pcm, int frame_size, int decode_fec) Unsafe version of:
| |
| nopus_multistream_decode_float | back to summary |
|---|---|
| public static int nopus_multistream_decode_float(long st, long data, int len, long pcm, int frame_size, int decode_fec) Unsafe version of:
| |
| nopus_multistream_decoder_create | back to summary | ||
|---|---|---|---|
| public static long nopus_multistream_decoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, long error) Unsafe version of:
| |||
| nopus_multistream_decoder_init | back to summary | ||
|---|---|---|---|
| public static int nopus_multistream_decoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping) Unsafe version of:
| |||
| nopus_multistream_encode | back to summary |
|---|---|
| public static int nopus_multistream_encode(long st, long pcm, int frame_size, long data, int max_data_bytes) Unsafe version of:
| |
| nopus_multistream_encode_float | back to summary |
|---|---|
| public static int nopus_multistream_encode_float(long st, long pcm, int frame_size, long data, int max_data_bytes) Unsafe version of:
| |
| nopus_multistream_encoder_create | back to summary |
|---|---|
| public static long nopus_multistream_encoder_create(int Fs, int channels, int streams, int coupled_streams, long mapping, int application, long error) Unsafe version of:
| |
| nopus_multistream_encoder_init | back to summary |
|---|---|
| public static int nopus_multistream_encoder_init(long st, int Fs, int channels, int streams, int coupled_streams, long mapping, int application) Unsafe version of:
| |
| nopus_multistream_surround_encoder_create | back to summary |
|---|---|
| public static long nopus_multistream_surround_encoder_create(int Fs, int channels, int mapping_family, long streams, long coupled_streams, long mapping, int application, long error) | |
| nopus_multistream_surround_encoder_init | back to summary |
|---|---|
| public static int nopus_multistream_surround_encoder_init(long st, int Fs, int channels, long streams, long coupled_streams, long mapping, int application) | |
| opus_multistream_decode | back to summary |
|---|---|
| public static int opus_multistream_decode(long st, ByteBuffer data, ShortBuffer pcm, int frame_size, int decode_fec) Decode a multistream Opus packet.
| |
| opus_multistream_decode_float | back to summary |
|---|---|
| public static int opus_multistream_decode_float(long st, ByteBuffer data, FloatBuffer pcm, int frame_size, int decode_fec) Decode a multistream Opus packet with floating point output.
| |
| opus_multistream_decoder_create | back to summary | |||||
|---|---|---|---|---|---|---|
| public static long opus_multistream_decoder_create(int Fs, int streams, int coupled_streams, ByteBuffer mapping, IntBuffer error) Allocates and initializes a multistream decoder state. Call
| ||||||
| opus_multistream_decoder_ctl | back to summary |
|---|---|
| private static int opus_multistream_decoder_ctl(long st) Performs a CTL function on a multistream Opus decoder.
| |
| opus_multistream_decoder_ctl | back to summary |
|---|---|
| public static int opus_multistream_decoder_ctl(long st, int request) Performs a CTL function on a multistream Opus decoder.
| |
| opus_multistream_decoder_ctl | back to summary |
|---|---|
| public static int opus_multistream_decoder_ctl(long st, Opus. Performs a CTL function on a multistream Opus decoder.
| |
| opus_multistream_decoder_destroy | back to summary |
|---|---|
| public static void opus_multistream_decoder_destroy(long st) Frees an
| |
| opus_multistream_decoder_get_size | back to summary |
|---|---|
| public static int opus_multistream_decoder_get_size(int streams, int coupled_streams) Gets the size of an
| |
| opus_multistream_decoder_init | back to summary | |||||
|---|---|---|---|---|---|---|
| public static int opus_multistream_decoder_init(long st, int Fs, int streams, int coupled_streams, ByteBuffer mapping) Intialize a previously allocated decoder state object. The memory pointed to by To reset a previously initialized state, use the
| ||||||
| opus_multistream_encode | back to summary |
|---|---|
| public static int opus_multistream_encode(long st, ShortBuffer pcm, int frame_size, ByteBuffer data) Encodes a multistream Opus frame.
| |
| opus_multistream_encode_float | back to summary |
|---|---|
| public static int opus_multistream_encode_float(long st, FloatBuffer pcm, int frame_size, ByteBuffer data) Encodes a multistream Opus frame from floating point input.
| |
| opus_multistream_encoder_create | back to summary | |||||
|---|---|---|---|---|---|---|
| public static long opus_multistream_encoder_create(int Fs, int streams, int coupled_streams, ByteBuffer mapping, int application, IntBuffer error) Allocates and initializes a multistream encoder state. Call
| ||||||
| opus_multistream_encoder_ctl | back to summary |
|---|---|
| private static int opus_multistream_encoder_ctl(long st) Performs a CTL function on an multistream Opus encoder.
| |
| opus_multistream_encoder_ctl | back to summary |
|---|---|
| public static int opus_multistream_encoder_ctl(long st, int request) Performs a CTL function on an multistream Opus encoder.
| |
| opus_multistream_encoder_ctl | back to summary |
|---|---|
| public static int opus_multistream_encoder_ctl(long st, Opus. Performs a CTL function on an multistream Opus encoder.
| |
| opus_multistream_encoder_destroy | back to summary |
|---|---|
| public static void opus_multistream_encoder_destroy(long st) Frees an
| |
| opus_multistream_encoder_get_size | back to summary |
|---|---|
| public static int opus_multistream_encoder_get_size(int streams, int coupled_streams) Gets the size of an
| |
| opus_multistream_encoder_init | back to summary | |||||
|---|---|---|---|---|---|---|
| public static int opus_multistream_encoder_init(long st, int Fs, int streams, int coupled_streams, ByteBuffer mapping, int application) Initialize a previously allocated multistream encoder state. The memory pointed to by To reset a previously initialized state, use the
| ||||||
| opus_multistream_surround_encoder_create | back to summary |
|---|---|
public static long opus_multistream_surround_encoder_create(int Fs, int mapping_family, IntBuffer streams, IntBuffer coupled_streams, ByteBuffer mapping, int application, IntBuffer error)
| |
| opus_multistream_surround_encoder_get_size | back to summary |
|---|---|
public static int opus_multistream_surround_encoder_get_size(int channels, int mapping_family)
| |
| opus_multistream_surround_encoder_init | back to summary |
|---|---|
| public static int opus_multistream_surround_encoder_init(long st, int Fs, IntBuffer streams, IntBuffer coupled_streams, ByteBuffer mapping, int application) | |
Opus.getLibrary().
| Modifier and Type | Field and Description |
|---|---|
| public static final long | multistream_decode
Function address. |
| public static final long | multistream_decode_float
Function address. |
| public static final long | multistream_decoder_create
Function address. |
| public static final long | multistream_decoder_ctl
Function address. |
| public static final long | multistream_decoder_destroy
Function address. |
| public static final long | multistream_decoder_get_size
Function address. |
| public static final long | multistream_decoder_init
Function address. |
| public static final long | multistream_encode
Function address. |
| public static final long | multistream_encode_float
Function address. |
| public static final long | multistream_encoder_create
Function address. |
| public static final long | multistream_encoder_ctl
Function address. |
| public static final long | multistream_encoder_destroy
Function address. |
| public static final long | multistream_encoder_get_size
Function address. |
| public static final long | multistream_encoder_init
Function address. |
| public static final long | multistream_surround_encoder_create
Function address. |
| public static final long | multistream_surround_encoder_get_size
Function address. |
| public static final long | multistream_surround_encoder_init
Function address. |
| Access | Constructor and Description |
|---|---|
| private |
| multistream_decode | back to summary |
|---|---|
| public static final long multistream_decode Function address. | |
| multistream_decode_float | back to summary |
|---|---|
| public static final long multistream_decode_float Function address. | |
| multistream_decoder_create | back to summary |
|---|---|
| public static final long multistream_decoder_create Function address. | |
| multistream_decoder_ctl | back to summary |
|---|---|
| public static final long multistream_decoder_ctl Function address. | |
| multistream_decoder_destroy | back to summary |
|---|---|
| public static final long multistream_decoder_destroy Function address. | |
| multistream_decoder_get_size | back to summary |
|---|---|
| public static final long multistream_decoder_get_size Function address. | |
| multistream_decoder_init | back to summary |
|---|---|
| public static final long multistream_decoder_init Function address. | |
| multistream_encode | back to summary |
|---|---|
| public static final long multistream_encode Function address. | |
| multistream_encode_float | back to summary |
|---|---|
| public static final long multistream_encode_float Function address. | |
| multistream_encoder_create | back to summary |
|---|---|
| public static final long multistream_encoder_create Function address. | |
| multistream_encoder_ctl | back to summary |
|---|---|
| public static final long multistream_encoder_ctl Function address. | |
| multistream_encoder_destroy | back to summary |
|---|---|
| public static final long multistream_encoder_destroy Function address. | |
| multistream_encoder_get_size | back to summary |
|---|---|
| public static final long multistream_encoder_get_size Function address. | |
| multistream_encoder_init | back to summary |
|---|---|
| public static final long multistream_encoder_init Function address. | |
| multistream_surround_encoder_create | back to summary |
|---|---|
| public static final long multistream_surround_encoder_create Function address. | |
| multistream_surround_encoder_get_size | back to summary |
|---|---|
| public static final long multistream_surround_encoder_get_size Function address. | |
| multistream_surround_encoder_init | back to summary |
|---|---|
| public static final long multistream_surround_encoder_init Function address. | |
| Functions | back to summary |
|---|---|
| private Functions() | |