Stereoscopic 3D video

For stereoscopic video, a sometimes fairly good illusion of three-dimensional image, different image has to be served to each eye. (Assuming two eyes, side by side, human/mammal/bird/reptile class, no exotic arrangements.)

Ordinary displays tend to be flat, without depth, observed by both eyes from distance. Achieving per-eye delivery requires special measures and can be challenging, or compromises have to be taken.

If image from the same display is split to two channels, by filters or shutters, ghosting can become a problem; imperfect separation of left/right channels leave a weak "echo" image in the wrong channel. Can be significantly annoying.

Intro slides: https://www.iwssip.org/archive/2012/fileadmin/CONTENT/Logos/IWSSIP%202012%20Tutorial%20Coding%20of%20Stereoscopic%20and%203D%20Video.pdf

Signal description: https://etmriwi.home.xs4all.nl/forum/hdmi_spec1.4a_3dextraction.pdf

3d stereoscopy types

left/right in the same image

• Anaglyph 3D - uses color filter on eyes that split one image to two
• requires goggles with the optical filters - lightweight
• usually some ghosting
• does not require any special hardware
• works with any unmodified RGB display, color projector, printed pages
• filters screw up with color perception, intensely colored objects can be in one eye only; retinal rivalry issues
• cheap and easy
• usually red/cyan (most common), magenta-green, also yellow-blue or amber-blue
• hightech variants use triads of narrowband filters with nonoverlapping red-green-blue slivers for left and right eye

• polarization filters on eyes
• linear or circular
• linear very sensitive to head tilt, ghosting at even smaller angles becomes significant
• circular more forgiving
• difficult to make polarization on printed media
• easy for projectors - side-by-side pairs of projectors modified only by added polarizing filters can be used
• common in 3d cinemas
• passive system, glasses very lightweight, no batteries, no synchronization, cheap
• passive 3d LCD panels usually have opposite polarization on even-odd pixel lines
• loss of half vertical resolution in 3d mode
• can be considered a form of very close spatial separation
• active shutters can be synced via standard VESA Stereo connector, providing +5V, GND, and TTL-level signal (H for left eye, L for right eye).

• Autostereoscopy
• parallax barrier - some cellphone/LCD displays, Nintendo 3DS
• lenticular lens arrays
• usable even with printed media - lenticular printing

spatially separated images

• stereoscope - uses two lenses and pair of side by side images
• ancient (1830s)
• works with printed images
• display version used in many VR rigs (Google Cardboard class)
• KMQ viewer - uses optical prisms to deflect parallel axes of vision to crossing-in-19deg-angle top-bottom (over-under) axes
• top-bottom (TB) image, images for left/right eye located under each other
• works with printed pages

temporally separated images

• left-right eye image shown as separate whole intact images in an alternating sequence
• for passive systems, images differ in polarization
• especially suitable for DLP projectors (double the color filter on the rotating wheels, add polarization filters)
• for active systems, shutter goggles are needed
• LCD shutter panels on each eye, alternating open/closed for left/right eye
• heavier than passive, more expensive
• tends to ghost more than passive due to slow liquid crystals
• needs synchronization between display/projector and glasses - bluetooth or infrared, source of incompatibility
• less sensitive to coplanarity and angles, wider range of angles to watch the screen than with passives
• maintains resolution, but needs twice the framerate

Practical systems

Anaglyph based systems became rare. Due to their simplicity they still can be encountered in niche applications.

Most common systems encountered in the wild as of 2020 are:

• passive 3D TV/monitor
• circular polarization opposite on even/odd pixel lines
• results in two overlapping images with half the vertical resolution (each eye sees even or odd scanlines as black)
• more expensive to manufacture

• passive 3D projector
• linear or circular polarization
• in non-cinema settings rarely two side-by-side projectors
• usually polarization alternating for left/right image

• active 3D TV/monitor/projector
• no polarization on display, ordinary screen can be used - LCD, CRT, projector
• left/right eyes are shown as even/odd frames
• requires active shutter glasses synchronized with the display

• one-display-per-eye VR/AR glasses
• allows miniaturized OLED or LCOS displays
• needs two data streams, more complex driver
• eg. Epson Moverio BT-35

• one-display-for-both-eyes VR/AR glasses
• common in Google Cardboard stereoscope-class glasses
• uses SBS with 1:1 pixels

• auto-stereoscopic
• no need for eyewear

3D HDMI types

In context of HDMI displays (assume 1920x1080, other sizes are the same via simple proportions), several ways to deliver the 3d signal are used.

As 3d requires twice the information, one of compromises has to be taken:

• halve the resolution on one axis and encode two frames in one
• double the framerate, use even/odd frames
• double the frame resolution and split it to two (frame packed)

For video file compression, some formats are better than others; compression combines many pixels into macroblocks (8x8, 16x16, 32x32...) and sharp transitions inside may be corrupted (see the fringing around edges in JPEG images for an example). Having boundaries between the left and right images in the macroblock can introduce such fringing as crosstalk/ghosting.

Discrete images (top-bottom, side-by-side, frame sequential...) are much more friendly than more intimate mixing (alternate lines, alternate columns, checkerboard patterns...). With most resolutions, even the largest macroblocks can be vertically divided (1920 is 30 64-pixel macroblocks, 1280 is 20). Horizontally (for top-bottom) the division is less favorable, but with only one boundary through macroblocks the effects are sweeped to the topmost and bottommost edges of the image where they are way less conspicuous.

anaglyph

• old, rare to see in modern systems
• can be generated from other formats by a filter that combines two frames to one

individual independent streams

• each eye gets independent (synchronized-together) stream
• usually as editing pipe, unusual in consumer end except for multistream/multiview video file encoding, converted to something below by the player
• cameras must maintain sync (genlock)
• can be used with two-projectors 3d, two-display goggles
• support eg. on OpenGL and nVidia card supporting Quad Buffer Stereo

frame packed, 3DFP, FP

[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LLLLLL  LLLLLL  ]
[------------------]
[  RRRR    RRRR    ]
[  RR  RR  RR  RR  ]
[  RR  RR  RR  RR  ]
[  RRRR    RRRR    ]
[  RR RR   RR RR   ]
[  RR  RR  RR  RR  ]

• single frame for both eyes
• two 1920x1080 frames encoded in one 1920x2205 frame (1080(left)+45 blank lines (separator, equals vertical-blank)+1080(right))
• full 1080p/24 for each eye
• usual on BluRay 3D content
• double the bandwidth compared to 2d signal of same framerate
• 2 times clock frequency than 2d
• HDMI 1.4a 3D format for movies, mandatory support (default format), 1080i/p(?) @ 23.98/24Hz
• HDMI 1.4a 3D format for games, mandatory support (default format), 720p @ 59.94/60Hz
• interlaced formats encode as leftodd-space-rightodd-space-lefteven-space-righteven frames

frame sequential, FS, F/S, "page-flip", field-alternative

[  LL      LL      ][  RRRR    RRRR    ][  LL      LL      ][  RRRR    RRRR    ]
[  LL      LL      ][  RR  RR  RR  RR  ][  LL      LL      ][  RR  RR  RR  RR  ]
[  LL      LL      ][  RR  RR  RR  RR  ][  LL      LL      ][  RR  RR  RR  RR  ]
[  LL      LL      ][  RRRR    RRRR    ][  LL      LL      ][  RRRR    RRRR    ]
[  LL      LL      ][  RR RR   RR RR   ][  LL      LL      ][  RR RR   RR RR   ]
[  LLLLLL  LLLLLL  ][  RR  RR  RR  RR  ][  LLLLLL  LLLLLL  ][  RR  RR  RR  RR  ]
   ----> time

• left/right eye frame as even/odd
• each frame is full 1920x1080
• double the bandwidth is needed
• double the framerate is needed
• minimum 144 Hz refresh/framerate is needed for computer monitors, 120 Hz for projectors (720p@120 is common for cheaper "3D-ready" DLPs)
• native format for active shutter glasses - just sync the glasses and show unmodified video
• not provided by BluRay players; only on PC, eg. nVidia 3D Vision System
• when fed with non-3D signal shows good 2D signal
• HDMI 1.4a option

side-by-side full, SbS-Full

[  LL      LL        RRRR    RRRR    ]
[  LL      LL        RR  RR  RR  RR  ]
[  LL      LL        RR  RR  RR  RR  ]
[  LL      LL        RRRR    RRRR    ]
[  LL      LL        RR RR   RR RR   ]
[  LLLLLL  LLLLLL    RR  RR  RR  RR  ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 3840x1080, left on left, right on right, pixel aspect ratio 1:1
• double the bandwidth is needed
• 2 times clock frequency than 2d
• uncommon, horizontal variant on frame packing
• HDMI 1.4a option

top-bottom, top-and-bottom, 3DTB, TB, TaB, T/B, over/under, above/below, AB

[  LL      LL      ]
[  LL      LL      ]
[  LLLLLL  LLLLLL  ]
[  RRRRR   RRRRR   ]
[  RRRRR   RRRRR   ]
[  RR  RR  RR  RR  ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame as two 1920x540 frames, left on top, right on bottom, pixel aspect ratio from 1:1 to 2:1
• half of vertical resolution is lost
• suitable for passive LCDs - just split the frame line by line, reassemble top frame to even lines, bottom to odd lines (or vice versa) to match the filters
• unmodified standard video, easy to generate on ordinary hardware
• better for 720p as horizontal resolution defines depth perception
• HDMI 1.4a 3D format for broadcast TV, mandatory support, 1080p @ 23.98/24Hz, 720p @ 50 or 59.94/60Hz
• when fed with non-3D signal shows two halves stretched and overlapping

side-by-side, SBS (for TVs, interlaced)

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in interlaced pair of 1920x540 frames as two 960x540 frames, left on left, right on right, pixel aspect ratio from 1:1 to 1:2
• half of horizontal resolution is lost
• on passive LCDs, half of vertical resolution is also lost, effectively reached is 960x540 - TaB format is more friendly
• higher apparent framerate for the same bandwidth
• better than top-bottom for interlaced as every half-frame has full 540 lines
• when fed with non-3D signal shows two halves stretched and overlapping

side-by-side, 3DSBS, SBS, S/S, SBS half-horizontal, SBS-HH (for TVs, progressive)

[ L   L    RR  RR  ]
[ L   L    R R R R ]
[ L   L    R R R R ]
[ L   L    RR  R   ]
[ L   L    R R RR  ]
[ LLL LLL  R R R R ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame as two 960x1080 frames, left on left, right on right, pixel aspect ratio from 1:1 to 1:2
• half of horizontal resolution is lost
• on passive LCDs, half of vertical resolution is also lost, effectively reached is 960x540 - TaB format is more friendly
• very common video format, common on youtube
• unmodified standard video, easy to generate on ordinary hardware
• with VR, results in image half as wide (spheres get "thin")
• when fed with non-3D signal shows two halves stretched and overlapping
• HDMI 1.4a 3D format for broadcast TV, mandatory support, 1080i @ 50 or 59.94/60Hz

side-by-side, SBS (for VR)

[  LL        RRRR    ]
[  LL        RR  RR  ]
[  LL        RR  RR  ]
[  LL        RRRR    ]
[  LL        RR RR   ]
[  LLLLLL    RR  RR  ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame as two 960x1080 frames, left on left, right on right, pixel aspect ratio stays 1:1
• half of horizontal resolution is lost
• two square(ish) images, suitable for stereoscope-like optics
• google cardboard, many other VR systems
• very common video format, somewhat common on youtube
• unmodified standard video, easy to generate on ordinary hardware
• with tv, results in image twice as wide (spheres get "fat")
• when fed with non-3D signal shows good 2D signal

alternating scanlines, line-by-line, LbL, row-interlaced, horizontal-interlaced

[  LL      LL      ]
[  RRRRR   RRRRR   ]
[  LL      LL      ]
[  RRRRR   RRRRR   ]
[  LLLLLL  LLLLLL  ]
[  RR  RR  RR  RR  ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame, alternating scanlines, starting with left, then right, then again, pixel aspect ratio 1:1
• half of vertical resolution is lost
• compression may introduce artefacts (TaB and SBS have only one subframe boundary, alternating lines have boundary on each line)
• native for passive LCDs
• when fed with non-3D signal shows good 2D signal
• HDMI 1.4a option (half-vert-res or full, with doubling height??)
• uncommon

line-alternative (maybe LbL?)

[  LL      LL      ]
[  RRRR    RRRR    ]
[  LL      LL      ]
[  RR  RR  RR  RR  ]
[  LL      LL      ]
[  RR  RR  RR  RR  ]
[  LL      LL      ]
[  RRRR    RRRR    ]
[  LL      LL      ]
[  RR RR   RR RR   ]
[  LLLLLL  LLLLLL  ]
[  RR  RR  RR  RR  ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x2160 frame, alternating scanlines, starting with left, then right, then again, pixel aspect ratio 1:1
• half of vertical resolution is lost
• compression may introduce artefacts (TaB and SBS have only one subframe boundary, alternating lines have boundary on each line)
• uncommon

checkerboard, CB, C/B, CK, alternating pixels, SbS-OLER/SbS-ELOR/SbS-OLOR/SbS-ELER (quincunx subsamplings), quincunx, Qx

[ LR R    LR R     ]
[ RL  R   RL  R    ]
[ LR   R  LR   R   ]
[ RLR     RLR      ]
[ LR   R  LR   R   ]
[ RL LRL  RL LRL   ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame as two 960x1080 frames, left on odd pixels, right on even pixels (then swap on next line), pixel aspect ratio stays 1:1 (or 2:1, depending on how counted)
• half of horizontal resolution is lost
• two variants possible, with topmost leftmost as left or as right, can alternate frame by frame
• DLP alternative to LCDs' line-by-line, common with rear-projection large DLP TVs
• when fed with non-3D signal shows good 2D signal
• now uncommon, historical, used with DLP projectors
• superior apparent resolution than SbS and TAB
• two flavors
• native Qx, pixel-identical; lots of high frequencies, suffers under compression
• rebuilt Qx, effectively SbS, pixels reshuffled in output device
• HDMI 1.4a option, flavor 01xx in the SBS-HH group

alternating columns, column-interlaced, vertical-interlaced

[ LR R    LR R     ]
[ LR   R  LR   R   ]
[ LR   R  LR   R   ]
[ LR R    LR R     ]
[ LR   R  LR   R   ]
[ LRL LR  LRL LR   ]

• single frame for both eyes (3D mixed)
• two 1920x1080 frames encoded in one 1920x1080 frame as two 960x1080 frames, left on odd pixels, right on even pixels (then swap on next line), pixel aspect ratio stays 1:1 (or 2:1, depending on how counted)
• half of horizontal resolution is lost
• two variants possible, leftmost as left or as right
• when fed with non-3D signal shows good 2D signal

layered-depth video, LDV

described in paper: http://pixeltje.be/wp-content/uploads/Bartczaketal11.pdf

see also 2D-plus-depth

L-depth

[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LLLLLL  LLLLLL  ]
[------------------]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]

• single frame for both eyes
• only progressive (noninterlaced)
• 2 times clock frequency than 2d
• stereo has to be reconstructed from the data
• eye distances can be adjusted to a degree
• occluded parts of the images, shown to different eyes, have to be calculated from other frames
• natural for range imaging, eg. RGBD format (red/green/blue+depth)
• option in HDMI 1.4a

L-depth+GFX

[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LL      LL      ]
[  LLLLLL  LLLLLL  ]
[------------------]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]
[  depth           ]
[------------------]
[  graphics        ]
[  graphics        ]
[  graphics        ]
[  graphics        ]
[  graphics        ]
[  graphics        ]
[------------------]
[  graphics depth  ]
[  graphics depth  ]
[  graphics depth  ]
[  graphics depth  ]
[  graphics depth  ]
[  graphics depth  ]

• single frame for both eyes
• only progressive (noninterlaced)
• 4 times clock frequency than 2d
• stereo has to be reconstructed from the data
• eye distances can be adjusted to a degree
• like L-depth but textures for the occluded sides of the images (revealed by each eye angle) are added
• looks like a "halo" of the objects
• option in HDMI 1.4a

most common formats

streams on internet, video files

In contect of youtube streams, the most common 3d video is SBS-HH, TV flavor. The VR flavor is close second, if not first already. Top-bottom is fairly rare.

The Kodi player 3D uses filename conventions as a format hint.

• The presence of "3D" or "3d" (between separators, like '.', '-', '_'...) signals 3D content.
• The presence of "SBS", "sbs", "HSBS", or "hsbs" between separators signals 3D side-by-side content.
• The presence of "TAB", "tab", "HTAB", or "htab" between separators signals 3D top-bottom content.

Usually they come in the frame-compatible way, where two images are anamorphically squeezed to one of the same size.

Full-size images in double-length or double-width frames are comparatively rare.

3D players, BluRay...

The players usually support the frame-packed format as a default.

This one takes twice the bandwidth, but the movies usually are running in 24, 25, 29.97 or 30 fps, which gives plenty of margin.

format signalling

For HDMI displays, the signalling of 3d content presence can be done automatically. In the HDMI signal, the messages are transmitted in InfoFrames, in the Control Island.

3d displays usually have the possibility of manual selection of the 3d standard, for the case when the stream type is not in the HDMI stream.

The InfoFrame is a CEA-861 vendor-specific frame, has structure as follows (according to HDMI1.4a, 2010):

• header byte 0: 0x81, packet type
• header byte 1: 0x01, version
• header byte 2: 0b000vvvvv, 5 bits of length of payload (0x05, 0x06, or 0x07)
• payload byte 0: checksum
• payload byte 1: 0x03
• payload byte 2: 0x0C
• payload byte 3: 0x00, 0x000C03 LSbyte first, IEEE registration identifier of packet type
• payload byte 4: 0bfff00000, 3 bits of frame type (unlisted are reserved)
• 000 - plain HDMI, no additional data
• 001 - 1 byte of HDMI1.4a parameter values follows
• 010 - 3D format present, 3D_Structure and maybe 3D_Ext_Data follows
• payload byte 5: optional, 0bffffg000, 4 bits of 3D_Structure (unlisted are reserved), 1 bit for 3D_Metadata presence (g=1, metadata present)
• 0000 - frame packing, FP
• 0001 - (less common) field alternative
• 0010 - (less common) line alternative
• 0011 - (less common) side-by-side full
• 0100 - (less common) L + depth
• 0101 - (less common) L + depth + graphics + depth
• 0110 - top-bottom, TB
• 1000 - side-by-side, SbS-HH
• 1xxx - indicates presence of 3D_Ext_Data
• payload byte 6: optional, 0bffff0000, 4 bits of 3D_Ext_Data; present for 3D_Structure values of 1xxx
• 00xx - horizontal subsampling
• 0100 - SbS-OLOR - checkerboard/quincux
• 0101 - SbS-OLER - checkerboard/quincux
• 0110 - SbS-ELOR - checkerboard/quincux
• 0111 - SbS-ELER - checkerboard/quincux
• 1xxx - indicates presence of

Metadata for parallax_zero, parallax_scale, dref and wref can be present in further bytes.

HDMI 3D formats

primary

 720p@50,59.94/60       - FP, SbS-HH, TB
 720p@23.98/24,29.97/30 - FP
1080i@50,59.94/60       - FP, SbS-HH
1080p@23.98/24          - FP, SbS-HH, TB
1080p@29.97/30          - FP,         TB
1080p@50,59.94/60       -             TB

secondary

 720x240p@50            - FP, SbS-HH, TB
1440x240p@50            - FP, SbS-HH, TB
2880x240p@59.94/60      - FP, SbS-HH, TB
 720x288p@50            - FP, SbS-HH, TB
1440x288p@50            - FP, SbS-HH, TB
2880x288p@50            - FP, SbS-HH, TB
 640x480p@59.94/60      - FP, SbS-HH, TB
 720x480i@59.94/60      - FP, SbS-HH, TB
 720x480p@59.94/60      - FP, SbS-HH, TB
 720x480i@119.88/120    - FP, SbS-HH, TB
 720x480p@119.88/120    - FP, SbS-HH, TB
 720x480i@239.76/240    - FP, SbS-HH, TB
 720x480p@239.76/240    - FP, SbS-HH, TB
1440x480i@59.94/60      - FP, SbS-HH, TB
1440x480p@59.94/60      - FP, SbS-HH, TB
1440x480i@119.88/120    - FP, SbS-HH, TB
1440x480i@239.76/240    - FP, SbS-HH, TB
2880x480i@59.94/60      - FP, SbS-HH, TB
2880x480p@59.94/60      - FP, SbS-HH, TB
 720x576i@50            - FP, SbS-HH, TB
 720x576p@50            - FP, SbS-HH, TB
 720x576i@100           - FP, SbS-HH, TB
 720x576p@100           - FP, SbS-HH, TB
 720x576i@200           - FP, SbS-HH, TB
 720x576p@200           - FP, SbS-HH, TB
1440x576i@50            - FP, SbS-HH, TB
1440x576p@50            - FP, SbS-HH, TB
1440x576i@100           - FP, SbS-HH, TB
1440x576i@200           - FP, SbS-HH, TB
2880@576i@50            - FP, SbS-HH, TB
2880@576p@50            - FP, SbS-HH, TB
1280x720p@23.98/24      -     SbS-HH, TB
1280x720p@25            - FP, SbS-HH, TB
1280x720p@29.97/30      -     SbS-HH, TB
1280x720p@100           - FP, SbS-HH, TB
1280x720p@119.88/120    - FP, SbS-HH, TB
1920x1080p@25           - FP, SbS-HH, TB
1920x1080p@29.97/30     -     SbS-HH
1920x1080i@50           - FP, SbS-HH, TB   1250 lines total
1920x1080i@50,59.94/60  -             TB
1920x1080p@59.94/60     - FP, SbS-HH
1920x1080i@100          - FP, SbS-HH
1920x1080p@100          -     SbS-HH, TB
1920x1080i@119.88/120   - FP, SbS-HH, TB
1920x1080p@119.88/120   -     SbS-HH, TB

software

for 3D signalling to TV (from omxplayer):

• CONF_FLAGS_FORMAT_NONE - no 3D
• CONF_FLAGS_FORMAT_SBS - side-by-side
• CONF_FLAGS_FORMAT_TB - top-bottom
• CONF_FLAGS_FORMAT_FP - frame-packed

on raspi, in /opt-vc/include/interface/vmcs_host/vc_hdmi_property.h:

 * 3D structure: param1 - 3D structure (e.g. top/bottom side by side) (default value is none, i.e. 2D)
 *               param2 - n/a at the moment, may be used in the future
 *
 * 3D structure is auto reset to "2D" every time HDMI is power on. Only affect CEA formats.
 */
 * Matched to the 3d struct bit fields stored internally to represent 3D support in EDID
 */
typedef enum {
   HDMI_3D_FORMAT_NONE = 0,           /** plain and simple 2D! */
   HDMI_3D_FORMAT_SBS_HALF = (1<<7),  /** side by side half horizontal */
   HDMI_3D_FORMAT_TB_HALF  = (1<<6),  /** top and bottom half vertical */
   HDMI_3D_FORMAT_FRAME_PACKING = (1<<8),  /** frame packed */
   HDMI_3D_FORMAT_FRAME_SEQUENTIAL = (1<<9),  /** Output left on even frames and right on odd frames (typically 120Hz)*/
   /* More 3D structs, e.g. full frame packing, may be added here */
   HDMI_3D_FORMAT_INVALID = 0xFFFF
} HDMI_3D_FORMAT_T;

in /opt-vc/include/interface/vmcs_host/vc_hdmi.h:

 * All possible 3D structures
 * to be used in decoded 3D modes (e.g. HDMI_3D_SUPPORTED_MODE)
 */
typedef enum {
   HDMI_3D_STRUCT_NONE                                   = 0,
   HDMI_3D_STRUCT_FRAME_PACKING                          = (1<<0),
   HDMI_3D_STRUCT_FIELD_ALTERNATIVE                      = (1<<1),
   HDMI_3D_STRUCT_LINE_ALTERNATIVE                       = (1<<2),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_FULL                      = (1<<3),
   HDMI_3D_STRUCT_L_DEPTH                                = (1<<4),
   HDMI_3D_STRUCT_L_DEPTH_GRAPHICS_GRAPHICS_DEPTH        = (1<<5),
   HDMI_3D_STRUCT_TOP_AND_BOTTOM                         = (1<<6),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_HALF_HORIZONTAL           = (1<<7),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_HALF_ODD_LEFT_ODD_RIGHT   = (1<<8),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_HALF_ODD_LEFT_EVEN_RIGHT  = (1<<9),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_HALF_EVEN_LEFT_ODD_RIGHT  = (1<<10),
   HDMI_3D_STRUCT_SIDE_BY_SIDE_HALF_EVEN_LEFT_EVEN_RIGHT = (1<<11),
   HDMI_3D_STRUCT_FRAME_SEQUENTIAL                       = (1<<12),
} HDMI_3D_STRUCT_T;

On raspi, 3D mode is set by call to m_BcmHost.vc_tv_hdmi_set_property(&property), with HDMI_PROPERTY_PARAM_T struct as parameter

On raspi, in tvservice.c, 3D format names are: (see HDMI_3D_STRUCT_T bit fields):

• FP - frame packing - two full-size images in twice as high frame
• F-Alt - field-alternative - alternating frames, twice the framerate
• L-Alt - line-alternative - even line left, odd line right
• SbS-Full - two full-size images in twice as wide frame
• Ldep - L-depth
• Ldep+Gfx - L-depth graphics
• TopBot - top-and-bottom, TB - two horizontally squished images
• SbS-HH - side-by-side half-horizontal - two vertically squished images
• SbS-OLOR - checkerboard/quincux - side-by-side half odd left odd right
• SbS-OLER - checkerboard/quincux - side-by-side half odd left even right
• SbS-ELOR - checkerboard/quincux - side-by-side half even left odd right
• SbS-ELER - checkerboard/quincux - side-by-side half even left even right

format support considerations

resolution-halving approaches, frame-compatible formats (FCF)

The SbS-HH and TB (and less commonly line-alternating) approaches divide the standard-size screen vertically or horizontally and squeeze the left and right image in. This can work on any ordinary video hardware with a single framebuffer (not counting doublebuffering etc...). The loss of half the resolution is cost of simplicity.

Standard video-handling pipelines can be used for processing the data - compressing, decompressing, editing, sending around...; the only difference to ordinary 2D video is the two frames being anamorphically squeezed together (or not even anamorphically if it is stereoscope-class VR).

The emergence of 4K displays is lending itself to such 3D video/image; the resolution that is normally unnecessarily high still provides a very acceptable full-HD even after losing half in each direction.

The line-alternating approach used to be employed in SDTV 3D, where even and odd frames recorded the left and right eye.

Aspect ratio correction can result in letterboxing, shifting the images around; vertical bars destroy TB, horizontal bars screw SbS.

SbS can be shot by lens using prisms/mirrors and anamorphic optics to handle the stereo acquisition before the light hits the sensor.

resolution-preserving approaches

The other approach is stretching the frame sent to the output device to twice the height or width (FP, SbS-Full, L-Alt - FP is standard), or twice as fast (F-Alt). This requires less ordinary hardware, usually requires direct support in the signal sending device (FP is common for BluRay players). Some nVidia videocards support quadruple buffers (two doublebuffers, one for each eye).

The resulting video is significantly more complex to handle, may not be supported by all software and hardware. Things can get quirky pretty fast.

exotics

The mosaicing approaches (SbS-OLOR/OLER/ELOR/ELER) are legacy techniques used by some early 3d DLP projectors. Rare to see in the wild.

The Ldep and Ldep+Gfx encode image and per-pixel depth (and graphics and graphics-depth) into each frame.

quad-buffering

Some videocards (NVidias, cards supporting AMD HD3D offer quad buffering. It is a bog-standard double-buffering, with separate buffer pairs for left and right eye, handled synchronously.

javascript rendering frameworks (Babylon.js)

• a choice of stereoscopic cameras, derivatives of ordinary cameras
• stereo cams take two additional parameters - inter-eye distance, and a boolean for side-by-side or top-bottom
• cams automatically squish the vertical or horizontal image aspect ratio
• image behaves like ordinary screen at ordinary resolution
• add-on controls in DOM that are not 3d objects will either show on only one eye or have to be "manually" duplicated to proper other-eye position in the render loop
• cursor is shown on one-eye only; hiding the "real" cursor and replacing it with a pair of manually positioned objects may be an expedient solution

As the image behaves like ordinary image/video, its further handling requires no other considerations. For the cost of losing half of vertical or horizontal resolution, the stereoscopic rendering can be done on unaugmented hardware and software; any WebGL supporting browser should do the job.

The video 3d flag in HDMI signal will not be set without special system calls. This may be solved by some independent process signalled to via HTTP request (or websocket/MQTT or so...), or left to be handled manually.

hardware support tests

passive-3d LG TV, tested

• with babylon.js on laptop, over HDMI, manually set 3d format
• with omxplayer from raspberry pi, commandline-option set 3d format

Epson Moverio BT-35 goggles, untested, researched

• wearable 720p, one OLED display for each eye
• BT-35 model has HDMI input, USB interface (accel/gyro/magneto, camera)
• support for 3d/stereo side-by-side, reducing per-eye resolution to 640x720p
• probably has to be told to switch to 3d mode via USB
• command in SDK, SDK for Windows and Android, for linux/raspi will likely have to be handled by libusb

TODO

• research the issues with parallel or converging camera angles
• human vision limitations, optimal dimensioning