Here's some HOWTO/tutorial material that has been created by DCP-o-matic users. Enjoy! If you've made a tutorial, video, or anything else do let me know and I will add it.
HOWTOs and tutorials
-
carl
- Site Admin
- Posts: 2548
- Joined: Thu Nov 14, 2013 2:53 pm
HOWTOs and tutorials
You do not have the required permissions to view the files attached to this post.
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: HOWTOs and tutorials
Thom - would you mind correcting some small inaccuracies in your document? I'd send some remarks to you about this later.
- Carsten
- Carsten
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: HOWTOs and tutorials
Okay, I acknowledge your statement 'This is by no means intended to be an authoritative or exhaustive account', but, since this document is meant to stay on the forum/DCP-o-matic site, we should at least try to keep some potential pitfalls out.
- USB 3.0 discs are compatible with USB 2.0 only computers. As a matter of fact, while indeed most cinema servers installed today only support USB 2.0, they have no trouble to ingest from the many USB 3.0 drives that are shipped around (of course, only at USB 2.0 speed). I haven't seen a USB 2.0 (-only) DCP disc in years.
- Creating the render output file in Resolve should always happen at the project native resolution. That will make sure there is no stretching or pillar/letterboxing applied. Formatting to special aspect ratios like full container, flat, or scope, is a cinema specific thing, and should only happen in DCP-o-matic. This may not be so important when creating DPX files (as these files do not qualify for other purposes than creating a DCP), but if you create ProRes or DNxHD/DNxHR masters, they should be 'standard' master files that can be played back normally. So, if your project resolution is 1080p, or UHD, then create your final master at 1920*1080, or 3840*2160. Try not to introduce stretching or letter-/pillarboxing, as that will make it unnecessary complicated to apply the right scaling in DCP-o-matic. All conversion to cinema specific picture and audio parameters can, and should, if possible, be done in DCP-o-matic.
- Ideally, put the audio within the video file. Some people mention that it is mandatory to have separate audio files for DCP creation - that is not the case. While some may argue about wether or not to use compressed video vs. still images for quality reasons, the audio that is interleaved with the video carries exactly the same quality as audio that has been created in separate files. The same is true for possible multichannel audio - If you put 6 channels of multichannel audio into a ProRes file, vs. outputting them into six separate files, will create a bit-identical audiotrack in the DCP - but it means much more work and is a potential source of channel assignment, level, and sync problems if separate files are used. DCP-o-matic allows to use either combined/interleaved, or separate audio files - but it doesn't force you into using one of these formats. If for some reason the files you created or received for DCP creation are split multichannel, then they can be used. Another reason to use e.g. a fully self contained video/audio (interleaved) file in ProRes, DnxHD, MP4, etc. is that you can easily quality check the file before/during DCP conversion. A folder with thousands of DPX files and 6 separate WAV files can not be played back for qc.
- Choosing the full frame container is not recommended, as hardly any cinema have display presets for this. In reality, full frame is only used for technical reasons, testing, etc. The best choice for most type of content is flat (that's why DCP-o-matic uses 'flat' as the default after installation).
- DCP drives must be in either ext2/ext3 or NTFS, with ext2 or ext3 being the official choice, and NTFS being the practical choice. To my knowledge, there is not a single server around currently that does not support NTFS, and as creating ext2/ext3 can be problematic to create (good hint at 'Mini Tool Partition Wizard') and write. For most people without access to Linux or special disc formatting knowledge, NTFS is the safest choice. ext4 and especially ExFAT and HFS(+) should be avoided, as only very few servers actually support it for ingest.
Discs should be created with a single MBR partition table (MBR to exclude GPT/GUID). Most modern OSs choose the partition table automatically to be GUID/GPT. A simple, often successful way to avoid this is to use a smaller disc, e.g. 512Gbyte, as GUID/GPT is only mandatory for discs larger than 2TB. Discs larger than 2TB can not be read by many servers in the field (even if smaller partitions are used on them). So, buying a cheap 512GB USB 3.0 discs at around 50-60€ is perfectly ok. Many professional DCP duplicators use them today.
- Maybe you should limit your recommendation for a fast graphics card towards Davinci Resolve, as only Resolve actually needs a fast GPU for editing/grading. DCP-o-matic will not use any specific graphic cards features. Many people using fast GPUs are disappointed when they notice how slow DCP-o-matic is during encoding. Currently, there is no easy way to fix this other than using encoding servers.
- Carsten
- USB 3.0 discs are compatible with USB 2.0 only computers. As a matter of fact, while indeed most cinema servers installed today only support USB 2.0, they have no trouble to ingest from the many USB 3.0 drives that are shipped around (of course, only at USB 2.0 speed). I haven't seen a USB 2.0 (-only) DCP disc in years.
- Creating the render output file in Resolve should always happen at the project native resolution. That will make sure there is no stretching or pillar/letterboxing applied. Formatting to special aspect ratios like full container, flat, or scope, is a cinema specific thing, and should only happen in DCP-o-matic. This may not be so important when creating DPX files (as these files do not qualify for other purposes than creating a DCP), but if you create ProRes or DNxHD/DNxHR masters, they should be 'standard' master files that can be played back normally. So, if your project resolution is 1080p, or UHD, then create your final master at 1920*1080, or 3840*2160. Try not to introduce stretching or letter-/pillarboxing, as that will make it unnecessary complicated to apply the right scaling in DCP-o-matic. All conversion to cinema specific picture and audio parameters can, and should, if possible, be done in DCP-o-matic.
- Ideally, put the audio within the video file. Some people mention that it is mandatory to have separate audio files for DCP creation - that is not the case. While some may argue about wether or not to use compressed video vs. still images for quality reasons, the audio that is interleaved with the video carries exactly the same quality as audio that has been created in separate files. The same is true for possible multichannel audio - If you put 6 channels of multichannel audio into a ProRes file, vs. outputting them into six separate files, will create a bit-identical audiotrack in the DCP - but it means much more work and is a potential source of channel assignment, level, and sync problems if separate files are used. DCP-o-matic allows to use either combined/interleaved, or separate audio files - but it doesn't force you into using one of these formats. If for some reason the files you created or received for DCP creation are split multichannel, then they can be used. Another reason to use e.g. a fully self contained video/audio (interleaved) file in ProRes, DnxHD, MP4, etc. is that you can easily quality check the file before/during DCP conversion. A folder with thousands of DPX files and 6 separate WAV files can not be played back for qc.
- Choosing the full frame container is not recommended, as hardly any cinema have display presets for this. In reality, full frame is only used for technical reasons, testing, etc. The best choice for most type of content is flat (that's why DCP-o-matic uses 'flat' as the default after installation).
- DCP drives must be in either ext2/ext3 or NTFS, with ext2 or ext3 being the official choice, and NTFS being the practical choice. To my knowledge, there is not a single server around currently that does not support NTFS, and as creating ext2/ext3 can be problematic to create (good hint at 'Mini Tool Partition Wizard') and write. For most people without access to Linux or special disc formatting knowledge, NTFS is the safest choice. ext4 and especially ExFAT and HFS(+) should be avoided, as only very few servers actually support it for ingest.
Discs should be created with a single MBR partition table (MBR to exclude GPT/GUID). Most modern OSs choose the partition table automatically to be GUID/GPT. A simple, often successful way to avoid this is to use a smaller disc, e.g. 512Gbyte, as GUID/GPT is only mandatory for discs larger than 2TB. Discs larger than 2TB can not be read by many servers in the field (even if smaller partitions are used on them). So, buying a cheap 512GB USB 3.0 discs at around 50-60€ is perfectly ok. Many professional DCP duplicators use them today.
- Maybe you should limit your recommendation for a fast graphics card towards Davinci Resolve, as only Resolve actually needs a fast GPU for editing/grading. DCP-o-matic will not use any specific graphic cards features. Many people using fast GPUs are disappointed when they notice how slow DCP-o-matic is during encoding. Currently, there is no easy way to fix this other than using encoding servers.
- Carsten
-
tba
- Posts: 30
- Joined: Mon Jan 15, 2018 4:20 pm
Re: HOWTOs and tutorials
Carsten,
>>USB 2.0
I ran into a USB 2.0 only projector the other day, maybe it's the only one left on the planet but I have no way of knowing that. No-one would be happier than me if all DCP machines could read a NTFS disk but I have stumbled upon 2 that don't.
>>Output render
I get what you mean and I would dearly like to render in the native 4k that the project was filmed in. But as it was edited in a 2k timeline in Resolve there seems to be no 'safe' way to render in UDH. Yes you can select that but I have yet to receive confirmation from the BM guys that the render will always use the raw footage from which to render. On several forums I have read that it first downsizes to 2 k and then back up to 4 k which seems uneccessary and a waste of disk space. However I will continue to test different solutions.
>>Encoding
Happy to encode in DR to Jpg2000 but hasve yet to recieve confirmation that it is DCI compliant. Anyway it's not all that much faster as it turns out. What language is Dcp-o-matic written in, surely it can adress the GPU?
>>USB 2.0
I ran into a USB 2.0 only projector the other day, maybe it's the only one left on the planet but I have no way of knowing that. No-one would be happier than me if all DCP machines could read a NTFS disk but I have stumbled upon 2 that don't.
>>Output render
I get what you mean and I would dearly like to render in the native 4k that the project was filmed in. But as it was edited in a 2k timeline in Resolve there seems to be no 'safe' way to render in UDH. Yes you can select that but I have yet to receive confirmation from the BM guys that the render will always use the raw footage from which to render. On several forums I have read that it first downsizes to 2 k and then back up to 4 k which seems uneccessary and a waste of disk space. However I will continue to test different solutions.
>>Encoding
Happy to encode in DR to Jpg2000 but hasve yet to recieve confirmation that it is DCI compliant. Anyway it's not all that much faster as it turns out. What language is Dcp-o-matic written in, surely it can adress the GPU?
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: HOWTOs and tutorials
Again - USB 2.0 ports will allow USB 3.0 discs to operate at USB 2.0 speeds. Yes, the majority of installed DCI servers are USB 2.0 only, but none of them have problems with USB 3.0 discs. If there were issues, the large commercial DCP duplicators wouldn't ship USB 3.0 discs (they would have a hard time sourcing enough USB 2.0 drives as well). There are some issues with drives that consume too much power in spin-up, but that is not a real compatibility issue - it can be overcome by using a separate USB power supply, or a Y-USB split power cord. However, current 2.5" drives nowadays should all operate fine with the 500mA USB 2.0 ports allow. Those sites using servers with USB 2.0 only will usually know about that issue and have suitable power supplies or Y cables around.
I don't know any DCI server that is not NTFS compatible, the trouble is, it may not have been an NTFS, but GUID/GPT issue, but the operator is not able to recognize why a specific drive has been rejected. Try again with a properly formatted MBR NTFS disc, and it will work. There are some sites that may fail even on specific ext2/ext3 drives. It is important to request all available information from the site in order to track down the reasons.
This is one of the voodoo things about proper disc formats - you may do one thing wrong, but another parameter is blamed, because the server doesn't indicate the issue clearly. It will simply not see the disc. So, NTFS is blamed, but in fact, it was the partition table, or something else.
Also, I am not promoting UHD - as most cinemas still use 2k projectors, there is little need in promoting UHD. But I am promoting to use the native Resolve project resolution for the final render. As you would do for all other target formats as well. In most non-professional projects using standard HD or UHD cameras, that would be 1080p/1920*1080, and avoid all frame rates above 30fps (though DCP-o-matic has no problem to create a 25fps DCP from a 50p source if that went wrong).
As you say, currently I would avoid using the J2C created from Resolve, as it has not been recognized as being DCI compatible and may create playback issues (it has in the past).
DCP-o-matic would have no issue to address GPUs - but there is no free/open source GPU aided J2C encoder that is known to create DCI compatible J2C. It's useless to have a speedy J2C encoder that fails on servers. And J2C is far too complicated for Carl to write his own in Cuda or OpenCL
BTW - which software/driver did you actually use to write to the ext2/ext3 drive?
- Carsten
I don't know any DCI server that is not NTFS compatible, the trouble is, it may not have been an NTFS, but GUID/GPT issue, but the operator is not able to recognize why a specific drive has been rejected. Try again with a properly formatted MBR NTFS disc, and it will work. There are some sites that may fail even on specific ext2/ext3 drives. It is important to request all available information from the site in order to track down the reasons.
This is one of the voodoo things about proper disc formats - you may do one thing wrong, but another parameter is blamed, because the server doesn't indicate the issue clearly. It will simply not see the disc. So, NTFS is blamed, but in fact, it was the partition table, or something else.
Also, I am not promoting UHD - as most cinemas still use 2k projectors, there is little need in promoting UHD. But I am promoting to use the native Resolve project resolution for the final render. As you would do for all other target formats as well. In most non-professional projects using standard HD or UHD cameras, that would be 1080p/1920*1080, and avoid all frame rates above 30fps (though DCP-o-matic has no problem to create a 25fps DCP from a 50p source if that went wrong).
As you say, currently I would avoid using the J2C created from Resolve, as it has not been recognized as being DCI compatible and may create playback issues (it has in the past).
DCP-o-matic would have no issue to address GPUs - but there is no free/open source GPU aided J2C encoder that is known to create DCI compatible J2C. It's useless to have a speedy J2C encoder that fails on servers. And J2C is far too complicated for Carl to write his own in Cuda or OpenCL
BTW - which software/driver did you actually use to write to the ext2/ext3 drive?
- Carsten
-
tba
- Posts: 30
- Joined: Mon Jan 15, 2018 4:20 pm
Re: HOWTOs and tutorials
USB 3.0 requires more amps than USB 2.0 so although the protocol is ok the disk won't start as there isn't enough juice.
>>Linux
To create a Linux partition = Mini tool partition manager
To format / mount the partition = Paragon exFS
>>Linux
To create a Linux partition = Mini tool partition manager
To format / mount the partition = Paragon exFS
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: HOWTOs and tutorials
USB 3.0 ports ALLOW to pull more current, it is not a requirement for USB 3.0 discs to draw more current. As USB 3.0 portable drives need and want to be USB 2.0 compatible, they usually stay below the 500mA rated for USB 2.0.
Believe me, commercial DCP duplicators ship thousands of USB 3.0 drives to cinemas each week. They wouldn't do that if it was an issue.
- Carsten
Believe me, commercial DCP duplicators ship thousands of USB 3.0 drives to cinemas each week. They wouldn't do that if it was an issue.
- Carsten
-
cloud06
- Posts: 52
- Joined: Mon Feb 22, 2016 3:02 pm
Re: HOWTOs and tutorials
I can confirm USB 3 disks work without problems... I use them all the time for commercial cinemas and festivals. Never had any problems. 
Also, it is not good idea to render to DCI Full container in Resolve. I would render to active image area resolution that would be used in DCP. And I would create special master for this. There are many reasons for that. For example, when grading film shot on Alexa @3.2K or on RED @4K/6K, I would set my timeline resolution to my master resolution/aspect. If my final aspect is 2.39 I would use DCI Scope (2048x858) for my timeline and render to this resolution. When rendering HD master I would set output scaling to 1080p. This way both my cinema and HD master are made by downscaling from original source resolution. This is very important, because it is bad idea to shoot at 2K and then export 1080p only to upscale to 2K in DOM... I would do all my scaling in grading application. If my master was 16:9 I would work in 1080p (since that is active image area of 16:9 frame in DCI flat container. If it was 1.85, then I would work in 1998x1080 (DCI Flat) AND export in this same resolution for DCP. For HD master, again, I would set output scaling to 1080p (and have black letterbox). Also, one more reason to have separate exports for HD and DCP is that you can put subtitles for DCP a bit lower in image than for standard HD delivery.
It is good idea to have timeline resolution set by you final aspect, not your output standard. That way you are automatically cropping all your footage to right dimensions, and it makes all downscaling much easier later on... Not to mention that is is easier for computer to process lower resolution without unused pixels around image.
Regrding sound: for HD master, it is good idea to have stereo and 5.1 sound in same QT file (2 tracks, 8 channels)... And to identify channel mapping in filename. When making DCP, mostly I use ProRes444 with audio provided by sound studio. It can be added to QT with Resolve, but for me it was always easier to do it in Premiere Pro (or some other NLE) just because I also need to add logos, titles etc... Sometimes it is waste of time to add audio to exported film. If audio is not in perfect sync with the video, I would import everything to Premiere Pro and check sync, then just enter nubers (timecode) to DOM (in points and position of sound files).
EDIT:
- in your screenshot, you are suggesting checking the audio levels in Content tab. It is much better to do it while in DCP tab. That way your are looking at the levels in DCP after applied gain and channel mapping (for example, sometimes you could import QT with stereo sound and 5.1 sound as separate files - if you forget to disable stereo sync sound in QT, you would have double sound playing at too hight level... This is something you would spot right away looking at final sound in DCP tab).
- Also, Rec709 is not universal option. For example, I do my grading on calibrated 709 monitor using 2.4 gamma. In this case, right preset is BT.1886 (Rec709 with 2.4 gamma)
- You can copy the DCP folder or files that are inside to the root of the drive... You can even have 5 DCPs in different folders of the same HDD. It works.
.... these are just mine experiences and best practices.
Also, it is not good idea to render to DCI Full container in Resolve. I would render to active image area resolution that would be used in DCP. And I would create special master for this. There are many reasons for that. For example, when grading film shot on Alexa @3.2K or on RED @4K/6K, I would set my timeline resolution to my master resolution/aspect. If my final aspect is 2.39 I would use DCI Scope (2048x858) for my timeline and render to this resolution. When rendering HD master I would set output scaling to 1080p. This way both my cinema and HD master are made by downscaling from original source resolution. This is very important, because it is bad idea to shoot at 2K and then export 1080p only to upscale to 2K in DOM... I would do all my scaling in grading application. If my master was 16:9 I would work in 1080p (since that is active image area of 16:9 frame in DCI flat container. If it was 1.85, then I would work in 1998x1080 (DCI Flat) AND export in this same resolution for DCP. For HD master, again, I would set output scaling to 1080p (and have black letterbox). Also, one more reason to have separate exports for HD and DCP is that you can put subtitles for DCP a bit lower in image than for standard HD delivery.
It is good idea to have timeline resolution set by you final aspect, not your output standard. That way you are automatically cropping all your footage to right dimensions, and it makes all downscaling much easier later on... Not to mention that is is easier for computer to process lower resolution without unused pixels around image.
Regrding sound: for HD master, it is good idea to have stereo and 5.1 sound in same QT file (2 tracks, 8 channels)... And to identify channel mapping in filename. When making DCP, mostly I use ProRes444 with audio provided by sound studio. It can be added to QT with Resolve, but for me it was always easier to do it in Premiere Pro (or some other NLE) just because I also need to add logos, titles etc... Sometimes it is waste of time to add audio to exported film. If audio is not in perfect sync with the video, I would import everything to Premiere Pro and check sync, then just enter nubers (timecode) to DOM (in points and position of sound files).
EDIT:
- in your screenshot, you are suggesting checking the audio levels in Content tab. It is much better to do it while in DCP tab. That way your are looking at the levels in DCP after applied gain and channel mapping (for example, sometimes you could import QT with stereo sound and 5.1 sound as separate files - if you forget to disable stereo sync sound in QT, you would have double sound playing at too hight level... This is something you would spot right away looking at final sound in DCP tab).
- Also, Rec709 is not universal option. For example, I do my grading on calibrated 709 monitor using 2.4 gamma. In this case, right preset is BT.1886 (Rec709 with 2.4 gamma)
- You can copy the DCP folder or files that are inside to the root of the drive... You can even have 5 DCPs in different folders of the same HDD. It works.
.... these are just mine experiences and best practices.
-
DCPForum
- Posts: 1
- Joined: Mon Aug 06, 2018 2:34 pm
Re: HOWTOs and tutorials
Hello,
I'm new to all this and trying to learn.
So, thank you for allowing me in to the forum - and thank you for creating software like this that allows someone like be to learn about this in the real world.
Thom's document "How to ....in 9 steps" - looked like it could be a valuable tool for me.
But I can see there are several points that need changing and have been picked up on by others.
I stopped reading when I saw it recommend using full frame which stretches the image - because obviously the document itself points out the picture will be distorted and that can't be a good objective I wouldn't say. And (DCP aside) USB 3 drives are fully compliant with USB2 computers - when used in a USB2 device they behave like USB 2.
So Thom, I was wondering if that document is going to be changed at source so that it is accurate with the points that have been mentioned because - from a newbie point of view - a document like that would be invaluable if the content was all agreed and validated etc. It would be a lot better to have one resource rather than having to determine modifications oneself.
But thanks Thom, and I hope there will be a V2.
I'm new to all this and trying to learn.
So, thank you for allowing me in to the forum - and thank you for creating software like this that allows someone like be to learn about this in the real world.
Thom's document "How to ....in 9 steps" - looked like it could be a valuable tool for me.
But I can see there are several points that need changing and have been picked up on by others.
I stopped reading when I saw it recommend using full frame which stretches the image - because obviously the document itself points out the picture will be distorted and that can't be a good objective I wouldn't say. And (DCP aside) USB 3 drives are fully compliant with USB2 computers - when used in a USB2 device they behave like USB 2.
So Thom, I was wondering if that document is going to be changed at source so that it is accurate with the points that have been mentioned because - from a newbie point of view - a document like that would be invaluable if the content was all agreed and validated etc. It would be a lot better to have one resource rather than having to determine modifications oneself.
But thanks Thom, and I hope there will be a V2.