View Bug Details

IDProjectCategoryView StatusLast Update
0002505DCP-o-maticCompatibilitypublic2023-11-18 20:30
Reporterjsalvo Assigned Tocarl  
PrioritynormalSeverityminorReproducibilityhave not tried
Status feedbackResolutionfixed 
PlatformMacOSOS XOS Version10.12
Product Version2.16.57 
Target Version2.16.x 
Summary0002505: Closed-Caption (timed text) track file exceeds recommend size.
Description

When creating a DCP with a closed caption track (imported from SRT), the resulting encoded MXF "sub" track is a minimum of ~423KB... regardless of the number of captions. Caption tracks are generally very sparse and from most other authors will result in files sizes measured in 10s of KBs. Many players will fail to load caption tracks larger than 256KB.

While the captions do seem to load correctly in DCP-o-Matic player, so the data is encoded in a readable way... assuming the player can load... the captions could fail to load on many commercial projection systems.

Steps To Reproduce

Create DCP with closed caption tracks.

Additional Information

ISDCF recommends as follow:
"Some closed captioning systems may be limited in the size of the timed text file they can handle. Most
of the existing system supports timed text XML file size of at least 256kB. Some system may support
higher size (e.g., up to 2MB). A timed text reel typically runs 100kB, so this should not be an issue. It is
possible to use a single timed text file for the entire composition and break it into reels using EntryPoint
and Duration, however, this may not work in all current implementations if the file size is larger than
256kB."
https://isdcf.com/papers/ISDCF-Doc9-Closed-Caption-Authoring-Best-Practices.pdf

Tagscaptions, correctness
Branch2525-ccap-fonts
Estimated weeks required
Estimated work requiredUndecided

Relationships

related to 0002506 acknowledgedcarl libdcp cleanup: use of SubtitleAsset for closed captions seems wrong 

Activities

jsalvo

2023-04-05 17:33

reporter  

jsalvo

2023-04-05 17:35

reporter   ~0005612

Should specify... this issue is with SMPTE DCPs, Interop DCPs produce sparse XML files, that seem to be of an appropriate size.

carl

2023-04-05 21:25

administrator   ~0005614

Thanks for the report. DoM is adding font data to the closed caption MXF (which is not needed). Fixed by 577602f032c0a129612efc8015f846f4bf6e64c9 which will be in 2.16.52.

overlookmotel

2023-05-16 11:18

developer   ~0005685

I know this has already been closed, but just checking something...

I had read "timed text file" in the ICDCF doc as referring to the XML file, not the MXF. i.e. in SMPTE DCPs, the XML is wrapped in an MXF, but I'd understood the 256KB limit as being on size of the XML itself (once unwrapped from the MXF), not the size of the MXF in total.

Am I wrong about that?

The document also says that any font that is included is ignored, so there's no benefit in including one. So this change is a good one regardless. But what I'm wondering is if we should recall and remake any DCPs with closed captions made previously with older versions of DCP-o-matic, as they may not play correctly?

carl

2023-05-16 22:18

administrator   ~0005691

Last edited: 2023-05-16 22:19

I think you are right. Bv2.1 7.2.1 says that the XML must be less than 256kB, and the cumulative file size (including XML, PNGs, and fonts) must be less than 115Mb, with a font smaller than 10Mb. So I'm not sure if the original complaint is correct. However, it seems to be pointless to include fonts with closed captions, so we might as well not do it.

carl

2023-05-16 22:20

administrator   ~0005692

83d63bb331190db4372de4d0aaf8413c51f1eee4 made these fonts come back.

carl

2023-05-16 22:21

administrator   ~0005693

Fixing this on the misleadingly-titled 2525-ccap-fonts

overlookmotel

2023-10-14 12:52

developer   ~0006007

Is the intent to remove font TTFs from CCAP caption tracks again?

Personally, I think it's fine as is in 2.16.65 - font TTFs are included.

In theory font TTFs are extraneous, but pretty much all CCAP DCPs we see from commercial sources (Deluxe etc) do include TTFs. So that seems to be the "vanilla" approach, and it's always possible there's some good reason for that - there may some projector model out there that will baulk at a missing TTF. So in my view the lowest risk approach is to follow the crowd on this, and the downside of a few extra 100KBs in a DCP is pretty insignificant.

If you agree, I believe this issue can be closed.

jsalvo

2023-10-14 19:49

reporter   ~0006010

"the downside of a few extra 100KBs in a DCP is pretty insignificant."
Failing QC is not "insignificant" to me or my clients. We need to be able to generate DCPs that conform to the spec, and particularly with festival (like Sundance) leaning in to accessibility and requiring closed captions... we need to be able to pass QC with those packages.

jsalvo

2023-10-14 19:53

reporter   ~0006011

To add, I've not heard of a single projector "baulking at a missing TTF"... but I have personally had DCP-O-MATIC created DCPs rejected for excessively large CC assets. So at least from my anecdotal datapoint, the harm actual/experienced harm of leaving the TTFs is far greater than the purely hypothetical harm of removing them.

Perhaps the Solomon's solution here is just to make it this a check box in preferences.

carl

2023-10-14 22:29

administrator   ~0006013

Last edited: 2023-10-14 22:29

@jsalvo do you know how large the CC assets where when they were rejected? Or do you have examples of MXFs from DCPs that were rejected?

jsalvo

2023-10-14 22:42

reporter   ~0006015

@carl yes, they're in the attached images. 423K. With the QC report citing the 256K limit of the specifications.

overlookmotel

2023-10-15 20:01

developer   ~0006018

@jsalvo To clarify, I was not saying that failing QC is insignificant. Of course it's not! My point was that the extra data of a few font TTFs is insignificant in the context of a DCP which overall will weigh up to 300GB for a feature.

A SMPTE captions MXF packages up the captions XML and font TTF in one file.

My understanding of the sections of the spec quoted above is that the size limit for the MXF is 115Mb, and the 256KB limit only applies to the XML itself, which is only a fraction of the MXF's overall size. So it surprises me that this DCP would fail QC.

Are you able to share a copy of the QC report? It'd be useful to know what QC software is producing the fail.

If it's sensitive to share the QC report publicly, perhaps you'd be willing to share it privately with Carl?

Bug History

Date Modified Username Field Change
2023-04-05 17:33 jsalvo New Bug
2023-04-05 17:33 jsalvo Tag Attached: captions
2023-04-05 17:33 jsalvo Tag Attached: correctness
2023-04-05 17:33 jsalvo File Added: Screenshot 2023-04-05 at 12.24.12 PM.png
2023-04-05 17:33 jsalvo File Added: Screenshot 2023-04-05 at 12.24.29 PM.png
2023-04-05 17:33 jsalvo File Added: Screenshot 2023-04-05 at 12.24.48 PM.png
2023-04-05 17:33 jsalvo File Added: Screenshot 2023-04-05 at 12.25.01 PM.png
2023-04-05 17:33 jsalvo File Added: Screenshot 2023-04-05 at 12.26.03 PM.png
2023-04-05 17:35 jsalvo Note Added: 0005612
2023-04-05 21:24 carl Assigned To => carl
2023-04-05 21:24 carl Status new => confirmed
2023-04-05 21:25 carl Note Added: 0005614
2023-04-05 21:26 carl Status confirmed => resolved
2023-04-05 21:26 carl Resolution open => fixed
2023-04-05 21:26 carl Relationship added related to 0002506
2023-05-16 11:18 overlookmotel Note Added: 0005685
2023-05-16 22:18 carl Note Added: 0005691
2023-05-16 22:19 carl Note Edited: 0005691
2023-05-16 22:20 carl Status resolved => confirmed
2023-05-16 22:20 carl Product Version 2.16.48 => 2.16.57
2023-05-16 22:20 carl Estimated work required => Undecided
2023-05-16 22:20 carl Note Added: 0005692
2023-05-16 22:21 carl Branch => 2525-ccap-fonts
2023-05-16 22:21 carl Note Added: 0005693
2023-08-13 12:23 carl Target Version => 2.16.x
2023-10-14 12:52 overlookmotel Note Added: 0006007
2023-10-14 19:49 jsalvo Note Added: 0006010
2023-10-14 19:53 jsalvo Note Added: 0006011
2023-10-14 22:29 carl Note Added: 0006013
2023-10-14 22:29 carl Note Edited: 0006013
2023-10-14 22:42 jsalvo Note Added: 0006015
2023-10-15 08:14 carl Target Version 2.16.x => 2.16.66
2023-10-15 08:25 carl Target Version 2.16.66 => 2.16.67
2023-10-15 20:01 overlookmotel Note Added: 0006018
2023-10-19 23:03 carl Status confirmed => feedback
2023-11-18 20:30 carl Target Version 2.16.67 => 2.16.x