Vertical positioning of subtitles & SMPTE standard versions

[Antti N]

Hi guys,
while getting used to the recent changes in DCP-o-Matic regarding vertical subtitle positioning, I've realized that there's always been an important difference between SMPTE ST 428-7:2010 and ST 428-7:2014 in this matter.

For example, if Valign is set to "top", the 2010 spec defines Vposition as "[t]he distance between the top primary image border and the top subtitle instance border". This is what D-o-M seems to do currently.

In contrast, the 2014 version states that it's "[t]he vertical distance between the top primary picture edge and the baseline for the characters drawn". This is in accordance with Interop/CineCanvas.

Now, the difference between subtitle instance border and text baseline is usually negligible if Valign is set to "bottom" – but it's quite significant when Valign is "top" (the latter being what D-o-M uses when adding subtitles to the project as srt's).

Judging from recent experience, it would seem to me that most (all?) real-world D-Cinema equipment follows the 2014 positioning logic, even when the subtitle XML declares itself as ST 428-8:2010. The result being that SMPTE subtitles are drawn much higher than intended whenever Valign is set to "top". I can do some more precise testing in the field to verify this, but I've encountered such an issue at least on Barco ICMP, Dolby IMS3000 and Dolby DSS200 + ICP systems, all running recent firmware/software versions.

Meanwhile: are there perhaps plans to migrate D-o-M's SMPTE subtitle format to ST 428-7:2014 anytime soon?

[Carsten]

Puh... That calls for some extensive testing. Also important how the preview and player deals with this, because that is what most people will use to verify vertical positioning. Does RDD52 deal with this?

[Antti N]

Does RDD52 deal with this?

Couldn't find anything at a quick glance, except that ST 428-7:2014 is the only spec version listed among "normative references"…

[HaroldHallikainen]

This is a problem. I was chair of the subtitling committee trying to improve the rendering of Japanese subtitles. As pointed out, there are differences in the vertical position specification in the two versions. We ran extensive tests and found only one server used the 2014 version for vertical positioning (use of the horizontal baseline). Three other servers used the bounding box as specified in the previous version. There are also differences in horizontal positioning, but they are not as visible.

In general, I think subtitle rendering engines should use the information that is in the font file. For horizontal text, I think the postion should be based on the horizontal origin of the first character (for left align), which is to the left of the glyph and on the baseline. To test this, we rendered a pipe character. Some systems left a small space (left side bearing) between the left side of the glyph, while others put the glyph right at the specified position (used the bounding box instead of the horizontal origin). For right align, I'd use the horizontal origin of the last character plus the advance width, which is where the next character would be positioned. For center alignment, the mid-point of these two positions would be used. But, in general, the horizontal baseline should be used for vertical positioning of horizontal text (like the 2014 standard says).

However, for vertical text, I believe the vertical origin of the character should be used for positioning. The vertical origin is normally above the character and centered above the character (but not always - for example the vertical origin of a comma is to the right of the glyph so that when the vertical text is rendered, the comma is to the left of center). Vertical positioning of vertical text based on the bounding box is close to the use of the vertical origin (closer than use of the horizontal baseline), but still not correct.

All subtitle rendering systems properly space horizontal text based on the advance width (the distance from the horizontal origin of one character to the horizontal origin of the next).

However, for vertical text, instead of using the advance height (distance from the vertical origin of one character to the next), the subtitle rendering systems use an unspecified distance between bounding boxes. This results in Japanese 111 (three ichi characters) showing up as a 3 (three horizontal lines within one cell.

Further the font compressors commonly used for subtitles remove the vertical metrics (such as advance height) making it impossible for a system to render vertical text properly.

I was pushing for subtitle systems to accurately render the font files. But, that would require updates in all systems, which not many really wanted to do.

For now, for horizontal text, recognize that some systems use the horizontal baseline for vertical positioning, and some use the bounding box. You need to position subtitles such that this variation does not throw the subtitle off the screen.

Good luck!

Harold

[carl]

Goodness me, what a mess

Thanks for the note on the difference between the 2010 and 2014 standards - I hadn't noticed that at all.

Judging from recent experience, it would seem to me that most (all?) real-world D-Cinema equipment follows the 2014 positioning logic

We ran extensive tests and found only one server used the 2014 version for vertical positioning

I wonder which of these observations is currently more accurate.

Then again, I guess it doesn't really matter which version has the "majority" - so long as a nontrivial number of projection systems use one standard or the other lots of subtitles are going to be in the wrong place and the DCP authoring software is always going to be blamed (especially if it is open-source).

The pragmatic answer is probably to do whatever the popular commercial authoring programs do (whatever that is). I don't know what else we can reasonably do, other than get into some ridiculous situation where we show the user some "error bars"-style-rectangle of where their subtitles might end up.

I guess referencing all subtitles from the bottom of the frame might reduce the potential error slightly.

[HaroldHallikainen]

Yes, a mess, especially for vertical text. I think all that can be done right now is to define a safe area for subtitles to ensure they do not go off the screen. As Carl points out, bottom align will have a smaller error since the distance from the horizontal baseline to the bottom of the bounding box is considerably less than the distance from the horizontal baseline to the top of the bounding box.

My statement about one server using the 2014 version (vertical position using horizontal baseline) and the rest using bounding boxes is from extensive tests done on four servers (and projector rendering) in Japan. I have the photographs and wrote a fairly extensive report based on measurements of the subtitles in the photographs. This was about 2 years ago.

Good luck!

Harold

[Antti N]

Judging from recent experience, it would seem to me that most (all?) real-world D-Cinema equipment follows the 2014 positioning logic

We ran extensive tests and found only one server used the 2014 version for vertical positioning

I wonder which of these observations is currently more accurate.

To be honest, what I wrote there was mostly based on some quick observations made during a film festival, so I didn't really have the luxury of time to do any rigorous testing. So I guess I'd rather go with Harold there. But I'll try to prepare some proper test content next week, check it on as many setups as possible during December and share the results here. I can probably do at least Barco ICMP and ICMP-X, Dolby IMS3000 (possibly IMS2000 as well), Dolby DSS200 + ICP and Sony SRX-R51x/XCT-S10.

Harold: which hardware did you test in Japan? Just wondering, since I don't really have any idea which brands and models are popular over there.

I guess referencing all subtitles from the bottom of the frame might reduce the potential error slightly.

My thoughts exactly, and that would actually be a huge improvement. With Valign="bottom", a line of borderless 42pt Arial would only be shifted by ~12 pixels between 2010 and 2014 positioning – compared with something like 41 pixels when Valign is set to "top".

[HaroldHallikainen]

Thanks for the comments! The SMTPE group was working on improving the rendering of Japanese subtitles. The vertical position tests were done using vertical Japanese text. The test DCP was created by a postproduction house in Japan. They also sent me photographs of the projected output. The tests were run in December 2020. I then used GIMP to analyze the photographs, measuring distances in pixels, etc. Tests were run on rendering engines in three servers and projector rendered subtitles. I've attached a portion of the summary report showing how each of the four systems positioned vertical text vertically.

Other rendering attributes studied included:
* Vertical text intercharacter spacing
* Conversion to bold
* Conversion to italic
* Horizontal position

In June 2020, I retired. I continued my SMPTE work for another year, then let my membership lapse. Others are continuing the work, including guidance on how to author subtitles to minimize the differences on playback on various systems.

It will be interesting to see the results of your tests!

Harold

[Antti N]

All right, here goes my progress report…!

I used DCP-o-Matic 2.16.18 to create a couple of test clips, all with both Interop and SMPTE versions. I tried to keep it simple and narrow: all tests use the same font (the D-o-M default Liberation Sans, but at size 42), no bold or italics, no vertical or RTL text.

Each test clip has two lines of subtitles, positioned alternately like this:

Code: Select all

<Text VAlign="bottom" VPosition="14">Line 1</Text>
<Text VAlign="bottom" VPosition="8">Line 2</Text>

…and alternately like this:

Code: Select all

<Text VAlign="top" VPosition="86">Line 1</Text>
<Text VAlign="top" VPosition="92">Line 2</Text>

If I'm reading the Interop/Cinecanvas and SMPTE 2014 specs correctly, those should be rendered identically: 8% from picture bottom to font baseline equals 92% from picture top to font baseline. With an active area height of 1080 pixels, that should be 86 px from the bottom (86.4 rounded down).

Systems tested so far:

- Barco ICMP (s/w versions 1.4.4.1 and 1.4.4.6)
- Barco ICMP-X (s/w version 1.4.4.3)
- Sony XCT-S10 (s/w version 1.53.4)
- Dolby DSS200 + TI ICP (subtitles rendered by the latter, which has s/w version 4.5.454)

Results:

First, none of the tested systems made any difference between Interop and SMPTE (even though the SMPTE test files have an xmlns attribute pointing to http://www.smpte-ra.org/schemas/428-7/2010/DCST).
In other words, none of them seem to follow the outdated SMPTE ST 428-7:2010.

The Sony XCT-S10 renders all test clips identically:
- Font baseline of the second subtitle line is drawn at exactly 87 px from bottom.
- Adding a border or a shadow doesn't affect the position of the baseline: it remains at 87 px, and eg. a one-pixel border then extends below the baseline to 86 px.
- The presence of ascenders and/or descenders in the text makes no difference: baseline stays at 87 px.

The ICP and all Barco ICMP(-X) results were almost identical to the Sony – except for a 1 px difference when VAlign is set to "bottom":
- VAlign="bottom" VPosition="8": baseline drawn at 86 px
- VAlign="top" VPosition="92": baseline drawn at 87 px

Not sure where the 1 pixel difference is coming from – possibly a rounding error? Of course, it's insignificant for most purposes anyway.

I'm still going to test at least IMS3000 within a week or two, and possibly IMS2000 as well.

I've attached the base subtitle file that I used for creating my test clips. (The only attribute that I changed was Effect, since I wanted to try out all three options: none, border and shadow.)

[Antti N]

The vertical position tests were done using vertical Japanese text.

I wonder if this explains the crucial difference between Harold's test results and mine, or is there perhaps more to it…?

There actually was a time when I kept wondering why so many of the Asian DCP's that I get for film festivals have either burnt-in or PNG subtitles. Not wondering anymore.