Let's Talk About "The Process", Shall We?

[Liberated Images]

Hello again, friends!  I hope all is well for you all.  I am posting today to ask about workflow habits.  Do you have a reliable way of delivering DCPs?  One in which they ingest every time?  I'm not sure exactly what I'm doing wrong- I do have more success than failure, but I have had many a DCP fail to ingest.  I'd like to be able to expand my delivery of DCPs but I want to be at least mostly sure that the receiver will be able to use the DCP on the first try around.

Perhaps some background info will help, so I will briefly describe how I typically create my DCPs. I am using an older MacBook Pro (2012 model) with Mojave. (I updated to Catalina (I believe) today).

First of all, I do my video editing in Adobe Premiere Pro CC.  Typically this includes fade ins and fade outs for video and audio and a title or two and adjusting the picture size to 2048px by 858px (for 2k cinemascope) if needed.  I then export the media as H.264 (.MP4 container), 44100 hz stereo sound.  Next I load a new project in DOM and add the MP4.  I set the content type (usually "Short") and audio language (en-US).  Then I allow the hints to run.  Typically I will go back even if no problems are detected to look at the audio levels graph and double check the loudness is consistent with all my content.  Once that looks good, I choose to "Make DCP".  I wrote the DCPs directly to my transfer drive this last time.

Once they were all finished I headed to the cinema for delivery.  Now, I will admit that I made a rookie mistake and had my drive formatted to exFAT.  As a result I had to move the DCPs to another drive that was on NTFS.  This was recognized by the Doremi 1000S and ingested one of the DCPs that was about 2 and a half minutes long successfully.  I moved two others, a three minute long DCP and an 18 minute long DCP to the same drive and both of those failed to ingest at about 92% (something about hashes not matching).  I also moved one to the laptop the cinema uses for controlling the projector.  The projectionist decided to try that one, also about 18 minutes long, out of curiosity and it failed the same as the other two.


As of right now, my game plan is to reformat my transfer drive to ext2, after reading some replies Carsten made to another post about filesystems.  Other than that, I'm not sure what else I could be doing to make sure my DCPs work every time.  1 out of 4 is unacceptable to me. 

Do any of you have a process that is dependable?  Could you possibly give me some insight as to what I should be striving for to reach my goal of consistent success?  I want to nail this down to the last detail so every scrap of information is going to be helpful.

Thank you so very much.  I love this community.

[jdent02]

Writing the DCP directly to the drive could conceivably cause an issue with the file hashes. Being on a Mac what are you using to do that? I was under the impression that Mac can read NTFS but not write it.

I always write the DCP to my local hard drive and then copy to the transfer drive. I use NTFS pretty consistently as I'm on Windows. Our IMS3000 always throws up an initial error with an NTFS ingest but then it automatically retries with some other access mode and it works fine.

Also, you'll want to export your audio from Premiere at 48KHz. As that's the DCP standard, DOM is having to do an extra conversion with your current workflow.

[Liberated Images]

Good info!

So in that instance, I was already at the cinema and he uses a PC and had an NTFS drive on hand so I did it that way. You are correct, MAC doesn't natively support writing for NTFS. Only reading. I wrote on an exFAT drive and had to move it to his NTFS.

I will make a note of the audio. And I will go back to writing on the computer then moving it to the transfer drive.

How often do you have ingest failures with your method?

[jdent02]

Only when I too accidentally tried to feed it an exFAT drive.

NTFS has never failed for me so far, even with our previous GDC servers. The only time I've had ingest failures is when I messed around with the DCP files and didn't update the hash and length values in the ASSETMAP, CPL and PKL files. Like you noticed, DCI servers really really really don't like that.

My workflow that I've been using for a few years is to export SMPTE DCPs from either Davinci Resolve or DCP-o-Matic (depending on what I'm putting together) to my computer, copy them onto an NTFS drive (I've used both flash and mechanical CRU drives) and ingest. I don't try to do anything fancy with the DCP (one single reel for video/audio and no VF versions) so they're about as simple as can be.

[Alex Asp]

I am using Paragon Concepts utilities.
They are reasonably priced and work great for me.
https://www.paragon-software.com/

[Carsten]

We get loads of reports about problems with Paragon drivers. They do work to a certain extent, but it's simply not solid enough. Unfortunately, no one seems to bother nowadays to maintain a solid open source ext driver for OS X. Which is weird, since the OS X groundwork is so close to Linux, that it shouldn't be a real issue. At the same time, Apple raises security bars with every new release for software that aims to operate close to the hardware.

Under windows, simply forget ext2/ext3 and use native NTFS. On the Mac so far I had solid results with Tuxera NTFS over many years. It is inexpensive, but, being a commercial software, there is ongoing development and support, unlike fuse-ext2 which is a hit and miss on different Mac OS versions. Some version may read, but not write to ext2/ext3.

As some have mentioned it - the safest way is to create the DCP on a native filesystem and fixed disk. Try to avoid running a DCP-o-matic conversion onto a USB connected disc with an alien filesystem. It worked for me with Tuxera, but, USB-SATA bridges are all different and some may cause weird issues. After the DCP has been created, copy it to an external or removable disk in one go, and let the machine alone during the transfer. DCPs obtained from wonky sources or from the internet/cloud services, I usually copy to a (local) native filesystem and perform a hash/integrity check using DCP-o-matic player, before I ingest them onto the server.

In general, I rarely have transfer related issues with DCPs.

No matter which file system you use, stick with disks max 2TB size and only use a single MBR partition. Some people report issues with NTFS, sometimes ext2/3, but when you look closely, it is NOT an NTFS issue, but one with disk size or partition table format. Many people are not able to differentiate NTFS/filesystem format from partition system.

My preferred way of creating feature length DCPs is to use my desktop computer, a windows HP Z600 with an internal CRU frame using SATA. I can create the DCP directly onto the NTFS disc mounted through a hardware SATA interface. I eject that disc afterwards and mount it on one of our servers, and have a real fast ingest. Never failed for me.


Again: Your only useful choices are ext2/ext3 or NTFS. Even some DCI servers running a real Linux may not be able to read DCPs from an ext4 disk - even if it uses ext4 e.g. for it's internal OS disk or RAID. Yes, FAT32 DOES work on all servers to, but it limits single file sizes to max 4GB, which is only goof for trailers or very short short. Or if you set DCP-o-matic to use very small reels.

- Carsten

[Alex Asp]

I really have no intention to enter a p***ng contest on this one. Please read my post again. It says: It works great for me.
Your mileage and preferences may vary. What's wrong with extFS 2 or extFS 3?
If it works, don't change it. And it has been working great for me the last 5 years at least.
Inexpensive as it was, it returned the investment long time ago.
Loads of reports about Paragon drivers are probably from people who can't distinguish between partitioning scheme and disc format.
And to add insult to injury Paragon updates their software and their Hard Disk Manager for Mac let you see if your disk has partitions that you actually don't need.

Just my 5 cents

[Liberated Images]

Thank you all for your replies and thorough information. Much of what I have been reading is helping by confirming what I was already suspecting. I am still working on deciding which way to go -NTFS or ext2/3. I am sure I will come to a decision shortly.

Guys, I am one of those people who don't quite grasp what a partition is. I understand in vague terms the idea of a filesystem...Sort of, anyway. But I am not educated on the difference between that and the partition. I do think that would be a useful bit of information.

So, I understand that different filesystems are native to different OS. I understand that some have limitations such as file size transfer, read and write compatibility with different systems and such, but I don't understand on a deeper level why the filesystems are different. As far as I have believed, such information was probably beyond the scope of what I've been doing. Would I be correct?

I will amend my procedure as per the suggestions I've gotten here. I am thinking this is going to give me the rate of success to failure that I've been hoping for.

If I'm sending DCPs to theaters all over the place, should I be recommending that they verify the DCPs with DCP Player? I have mixed feelings on this. On one hand it would seem like good practice to verify before attempting an ingest. However, I can see the extra step of downloading a program, learning to navigate it, and verifying ever piece of content being a headache for (especially older) projectionists.

[jdent02]

If you're sending the DCP to several theaters, ext2 would be the best option (it's the only format GUARANTEED to work with all DCI servers). If you copy the DCP onto the hard drive and verify it with the Player app on your computer, then it should work without any issue on a theater server. When ingesting, the server uses the hash values in the .xml files to verify that the files are intact. If they are, then they should be good to go, so long as you don't use an odd audio or video format. Stay with standard 2k/4k resolutions at 24fps and 5.1 audio if you can.

As far as the filesystems, they are different because they are used by different operating systems and were developed by different people. A DCP video track will always be an .mxf file regardless of the filesystem, but the way the OS tracks that file and indexes it will be different.

A partition is like a 'slice' of a hard drive, and it usually shows up as a separate drive to the OS. So for example you can have one hard drive split into three partitions, and the OS will show that as three separate drive letters, even through all three reside on one physical device. DCI servers don't like ingesting from partitioned hard drives, so its always best practice to make sure the entire hard drive is one partition.

Here's a website that takes you through a few different options for creating ext2 drives for DCP delivery:

https://www.knuterikevensen.com/2011/03 ... ckage-dcp/

DCP-o-Matic also has a disk writer that'll do it for you. Be careful with it though, as you can accidentally wreck your computer if you use it wrong.

[Carsten]

We have seen hash checking fail when testing DCPs off an ext2/ext3 disc using alien file system drivers (like Paragon). Shouldn't happen, but does happen.

While NTFS is not a formally accepted format (actually, even ext2/ext3 is only a defacto standard in the DCI world), it is factually guaranteed to work on all DCI servers as well.


Some festivals have formal requirements (e.g. ext2/3 on a CRU disk), but I have seen it happen that even they accept NTFS because they do not even know what format a disk is once it mounts on a server or TMS.

The whole ext2/3 thing is unnecessarily complicated if you don't work in a Linux environment. Of course, you are free to use DCP-o-matic disk writer as well. It's not an ext2/3 driver, but I will give you an ext2/3 disk with a DCP on it.