Spaces:
Configuration error
Configuration error
File size: 7,241 Bytes
a2fcab8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
# Spatial Audio RFC (draft)
*This document describes an open metadata scheme by which MP4 multimedia containers may accommodate spatial and head-locked stereo audio. Comments are welcome on the [spatial-media-discuss](https://groups.google.com/forum/#!forum/spatial-media-discuss) mailing list or by [filing an issue](https://github.com/google/spatial-media/issues) on GitHub.*
------------------------------------------------------
## Metadata Format
### MP4
Spatial audio metadata is stored in a new box, `SA3D`, defined in this RFC.
#### Spatial Audio Box (SA3D)
##### Definition
Box Type: `SA3D`
Container: Sound Sample Description box (e.g., `mp4a`, `lpcm`, `sowt`, etc.)
Mandatory: No
Quantity: Zero or one
When present, provides additional information about the spatial audio content contained in this audio track.
##### Syntax
```
aligned(8) class SpatialAudioBox extends Box(‘SA3D’) {
unsigned int(8) version;
unsigned int(8) ambisonic_type;
unsigned int(32) ambisonic_order;
unsigned int(8) ambisonic_channel_ordering;
unsigned int(8) ambisonic_normalization;
unsigned int(32) num_channels;
for (i = 0; i < num_channels; i++) {
unsigned int(32) channel_map;
}
}
```
##### Semantics
- `version` is an 8-bit unsigned integer that specifies the version of this box. Must be set to `0`.
- `head_locked_stereo` is a 1-bit flag used to indicate that the stored audio track contains head-locked stereo audio in addition to ambisonics audio. The flag should be set if the track contains head-locked stereo and unset otherwise.
- `ambisonic_type` is a 7-bit unsigned integer that specifies the type of ambisonic audio represented; the following values are defined:
| `ambisonic_type` | Ambisonic Type Description |
|:-----------------|:---------------------------|
| `0` | **Periphonic**: Indicates that the audio stored is a periphonic ambisonic sound field (i.e., full 3D). |
- `ambisonic_order` is a 32-bit unsigned integer that specifies the order of the ambisonic sound field. If the `ambisonic_type` is `0` (*periphonic*), this is a non-negative integer representing the periphonic ambisonic order; in this case, it should take a value of `sqrt(n) - 1`, where `n` is the number of channels in the represented ambisonic audio data. For example, a *periphonic* ambisonic sound field with `ambisonic_order = 1` requires `(ambisonic_order + 1)^2 = 4` ambisonic components.
- `ambisonic_channel_ordering` is an 8-bit integer specifying the channel ordering (i.e., spherical harmonics component ordering) used in the represented ambisonic audio data; the following values are defined:
| `ambisonic_channel_ordering` | Channel Ordering Description |
|:-----------------------------|:-----------------------------|
| `0` | **ACN**: The channel ordering used is the *Ambisonic Channel Number* (ACN) system. In this, given a spherical harmonic of degree `l` and order `m`, the corresponding ordering index `n` is given by `n = l * (l + 1) + m`. |
- `ambisonic_normalization` is an 8-bit unsigned integer specifying the normalization (i.e., spherical harmonics normalization) used in the represented ambisonic audio data; the following values are defined:
| `ambisonic_normalization` | Normalization Description |
|:--------------------------|:--------------------------|
| `0` | **SN3D**: The normalization used is *Schmidt semi-normalization* (SN3D). In this, the spherical harmonic of degree `l` and order `m` is normalized according to `sqrt((2 - δ(m)) * ((l - m)! / (l + m)!))`, where `δ(m)` is the *Kronecker delta* function, such that `δ(0) = 1` and `δ(m) = 0` otherwise. |
- `num_channels` is a 32-bit unsigned integer specifying the number of audio channels contained in the given audio track.
- `channel_map` is a sequence of 32-bit unsigned integers that maps audio channels in a given audio track to ambisonic components, given the defined `ambisonic_channel_ordering`. The sequence of `channel_map` values should match the channel sequence within the given audio track.
For the example case of `ambisonic_type = 0` (Periphonic), consider a 4-channel audio track containing ambisonic components *W*, *X*, *Y*, *Z* at channel indexes `0`, `1`, `2`, `3`, respectively. For `ambisonic_channel_ordering = 0` (ACN), the ordering of components should be *W*, *Y*, *Z*, *X*, so the `channel_map` sequence should be `0`, `2`, `3`, `1`.
As a simpler example, for a 4-channel audio track containing ambisonic components *W*, *Y*, *Z*, *X* at channel indexes `0`, `1`, `2`, `3`, respectively, the `channel_map` sequence should be specified as `0`, `1`, `2`, `3` when `ambisonic_channel_ordering = 0` (ACN).
For the example case of `ambisonic_type = 0` (Periphonic) with `head_locked_stereo = 1`, the stored audio will consist of `4` ambisonic components *W*, *Y*, *Z*, *X* in addition to head-locked stereo components *L* and *R*. In this case, the SA3D atom will define `num_channels = 6` and a `channel_map` specified as `0`, `1`, `2`, `3`, `4`, `5` indicating that the channels are laid out in the file as *W*, *Y*, *Z*, *X*, *L*, *R*. This representation extends to different layouts of ambisonics and head-locked stereo components. For example, a channel layout of `4`, `5`, `0`, `1`, `2`, `3` indicates that the layout of the stored audio is *L*, *R*, *W*, *Y*, *Z*, *X*.
##### Example
Here is an example MP4 box hierarchy for a file containing the `SA3D` box:
- moov
- trak
- mdia
- minf
- stbl
- stsd
- mp4a
- esds
- SA3D
where the `SA3D` box has the following data:
| Field Name | Value |
|:-----------|:-----|
| `version` | `0` |
| `ambisonic_type` | `0` |
| `ambisonic_order` | `1` |
| `ambisonic_channel_ordering` | `0` |
| `ambisonic_normalization` | `0` |
| `num_channels` | `4` |
| `channel_map` | `0` |
| `channel_map` | `2` |
| `channel_map` | `3` |
| `channel_map` | `1` |
------------------------------------------------------
## Appendix 1 - Ambisonics
The traditional notion of ambisonics is used, where the sound field is represented by spherical harmonics coefficients using the *associated Legendre polynomials* (without *Condon-Shortley phase*) as the basis functions. Thus, the spherical harmonic of degree `l` and order `m` at elevation `E` and azimuth `A` is given by:
N(l, abs(m)) * P(l, abs(m), sin(E)) * T(m, A)
where:
- `N(l, m)` is the spherical harmonics normalization function used.
- `P(l, m, x)` is the (unnormalized) *associated Legendre polynomial*, without *Condon-Shortley phase*, of degree `l` and order `m` evaluated at `x`.
- `T(m, x)` is `sin(-m * x)` for `m < 0` and `cos(m * x)` otherwise.
### Conventions
#### Azimuth
- `A = 0`: The source is in front of the listener.
- `A` in `(0, pi/2)`: The source is in the forward-left quadrant.
- `A` in `(pi/2, pi)`: The source is in the back-left quadrant.
- `A` in `(-pi/2, 0)`: The source is in the forward-right quadrant.
- `A` in `(-pi, -pi/2)`: The source is in the back-right quadrant.
#### Elevation
- `E = 0`: The source is in the horizontal plane.
- `E` in `(0, pi/2]`: The source is above the listener.
- `E` in `[-pi/2, 0)`: The source is below the listener.
|