problem creating DCP on Mac Mini (2018), Mojave

[Carsten]

Yes, I saw that APFS->HFs+ downgrade app, but it would only be a very last resort. So far, I can only suspect that APFS is causing this, but we can't be sure.

Let's see. I assume you used Paragon extFS driver for OS X to write to that ext2 disc?

I understand there have been quite a few issues reported with APFS generally (not here or specifically DCP-o-matic related). However, I would assume that many DCP-o-matic users now use Mojave and APFS. And so far we have zero reports about specific problems of this combination. Weird...

We can only try to track the issue down by trial and error...maybe Carl comes up with an idea.

Regarding 'Ingest' - Cinema servers usually do not play the movie from the transport drive. The DCP is copied onto the server's internal RAID. The process of copying a DCP, small or large, is called 'Ingest'. They do not call it 'copy', because typically, an ingest is more than a plain copy - usually, the DCP is also verified during copying. So, if there is an issue with a DCP, it SHOULD usually already occur during ingest of the server sees there is something wrong. Rarely, a DCP is ingested, but still doesn't play. The reasons are usually very specific. I know the Doremis usually give more detailed errors/warnings during ingest than an ICMP.


- Carsten

[pontorosso]

Many thanks for explaining "Ingest". As far as I know, the not running DCPs could - in these words - not be ingested. Still waiting for an answer by the cinema and hopefully Carl can explain, what "error - 101" is meaning.

At the moment I am creating a DCP right on a DX115 DC (with a Move Dock)/ CRU. When creating a DCP in its whole length of 19 Minutes, I cannot play it on the DCPomatic player. Trying with 10 minutes of the movie, suddenly I got the message: could not read frame. The same with the film in 18 minutes length.

I got various Messages:

could not read frame - this message came on different Positionspapier of the movie
130 Butler video Buffer reached 480 frames
marker is not compliant with its position

Quvis player can read the versions I tried.

From my side I have to give up - there is no certain pazttern, where the mistiges appear and why...

[Carsten]

I understand this is very confusing. You may actually have multiple different issues here.

The strategy of reading the video files may be different - it may be that a corrupted file is read over by one player, but not accepted by another player.

A cinema server though, checks files during ingest and may refuse to play them if any errors have been found.

I have also seen the occasional issue when writing immediately to USB connected drives. Also, the alien file system drivers like Paragon are not as solid as native file systems like HFS+.

Then there is: https://dcpomatic.com/mantis/view.php?id=1631

How large are these drives you are using in the CRU, btw?

This may be very complicated to solve from a distance. Apropos distance - where are you located, Gerhard?

Let's see what the cinema says...

[pontorosso]

My answers are inside the text:

I understand this is very confusing. You may actually have multiple different issues here.

The strategy of reading the video files may be different - it may be that a corrupted file is read over by one player, but not accepted by another player.
- YES, THAT´S WHAT I´M THINKING, TOO. PROBLEMS OF NOT TO BE ABLE TO READ A FRAME APPEAR ON DIFFERENT PLACES, BUT MOST OF THE TIME AT THE SAME.

A cinema server though, checks files during ingest and may refuse to play them if any errors have been found. - SO IT DID THE LAST TIME WHEN ALL THIS STARTED.

I have also seen the occasional issue when writing immediately to USB connected drives. Also, the alien file system drivers like Paragon are not as solid as native file systems like HFS+.

Then there is: https://dcpomatic.com/mantis/view.php?id=1631

How large are these drives you are using in the CRU, btw? - THE ONE IS 2TB, AND I ALSO HAVE A DC MINI WITH 1TB. I HAD THE TEST MOVIE WITH DIFFERENT TIMES ON BOTH DRIVES WITH THE SAME EFFECT.

This may be very complicated to solve from a distance. Apropos distance - where are you located, Gerhard? I AM LIVING IN WELS, UPPER AUSTRIA, YOU MAY KNOW ANDRÄ, WHO IS WORKING IN OUR LOCAL CINEMA AND TO WHOM I GAVE MY FIRST "ERROR DRIVE".

[Carsten]

Now - none of the following is addressing your specific issue with that MacMini, but...

I am not a huge fan of Paragon software, I understand though that many use it (there is also hardly an alternative solution for ext2/3 available).

I would try to avoid creating a DCP directly onto an ext2/ext3 drive, in OS X and Windows, as these drivers are not as stable as native filesystem drivers.
Instead, I would usually try to create the DCP on HFS+ or NTFS, then just copy to an ext2/ext3 drive. That may be safer.

The disc sizes you mention, you should be safe - discs larger than 2TB may cause strange issues when used over USB.
Can you make sure your DCP drives use an MBR partition table? OS X allows to choose GUID/GPT or MBR when initializing the drive with disc utility. Always use MBR.

Okay, may be not too heplful, but, for now, if you are after the encoding speed of the MacMini, you may use your old machine running DCP-o-matic master, and setup the MacMini as a networked encoding server. At some time, we will have sorted out the problem.

- Carsten

[pontorosso]

An interesting experience: I again created a DCP on an other external drive - also with extFS2, also 10 minutes long - and this time it could be played by the dcpomatic player. - Perhaps everything is indeed so unstable, that I was just lucky this time ??!! The drive is 256GB.

You say: "networked encoding server" - how does this work, please?

[carl]

I'm really confused about what tests have been done. It sounds like fairly fundamental file corruption is happening somewhere.

Can we start by establishing a few basics?

For now, I'd suggest you run only the following test:

1. Make a DCP somewhere using DCP-o-matic 2.14.11
2. Load that DCP into the DCP-o-matic 2.14.11 player and do "verify", and see if it is OK.

Can you tell us whether this works on

1. the internal drive? (and can you confirm this is APFS?)
2. an external drive formatted with HFS?
3. the same external drive formatted with EXT2?

Thanks!

For extra points, run the same tests a few times (deleting the DCP-o-matic project between tests).

[pontorosso]

Thanks, Carl.
Before running your suggestions I can say: I already made DCP on the internal drive with APFS, also on an external one with HFS+ and also on exFS2/3.
I created those in different lengths of the same movie.

2 minutes examples worked all the time properly. 8 minute and 18 minutes of this 19 minutes film crashed at two different positions. In spite of the crash I could load those 2 DCP files on the dcpomatic player, but then came the crash.

Anyway, I´ll do it again as you told me, with a different film. In my opinion the strange thing is, that I can load and play some DCPs, but then they crash.
And different lenghts seem to raise different results for Playback.

I´ll tell you what happens.

[carl]

Thanks for the information.

So, if I'm right: DCP-o-matic crashes when creating DCPs in your 8 and 18 minute tests, even on the internal APFS drive. Is that correct?

That's the thing I would like to focus on, I think. Which crash do you see on the DCP creation?

[pontorosso]

Carl, I did everything you told me to do (at 4:55). Here are the results:

I created 3 DCPs from 3 different films.

All 3 DCPs had the following Patterns:

1) created on an external drive with extFS: not playable by the DCPomatic player, and error 101

2) created on external drive with HFS+: playable by the DCPomatic player and verify ok

3) created on internal drive with APFS: playable by the DCPomatic player and verify ok

next: copying DCP from internal to external HFS+: playable by the DCPomatic player and verify ok

copying from an external drive with extFS to another external drive with exFS: it says "missing files" (and I could it see: not everything was copied,
something with ASSETMAP was wrong)

copying from internal drive to external extFS: not working, error 101.

OK, these are my results and it is strange again, that this time there were no crashes, but only either playable or not playable.
Anyway: the biggest problem seems to be to play from an external drive with extFS formatted.

BUT: perhaps this is not an assurance, that the on the dcpomatic player working files would also work on a cinema server, wouldn´t it???

So perhaps all this helps to enclose the problem.

The question is: are cinema servers able to read HFS+? I always used extFS, because I´ve heard, all servers could read this.

Thanks for your help!