Creating a DCP-o-matic Workstation

[Cinematheque_Van]

Hello,

I'm using this shutdown period to research building a new dedicated workstation for converting digital film files to dcps using DCP-o-matic. We've been using DCP-o-matic for years, including recommending it to all of our rental clients. As a cinema, we've never wanted to be in the 'business' of DCP creation but we're finding the burden increasingly being placed on our shoulders and at the last minute. Our current average transcode time is about 13 hours for a typical feature film. Ideally we'd like to bring that down to 30 minutes or less, as we still need 30-60 minutes to ingest on to server.

The broad strokes of what I have in mind would be an Ubuntu OS, Ryzen Threadripper 3970X, 32 GG of RAM, and a 1 TB SSD. Does anyone here have experience with a similar system and can give an estimate for DCP-o-matic creation rates? Is Cinebench the closest benchmark I should be looking at for performance, or is there something closer out there? I'm also planning on putting a blu-ray optical drive in, to enable ripping of optical media and converting to DCP, and the planned installation of USB 4.0 when it comes to market. Will it be more efficient to write to internal SSD and then transfer to a portable HD Dor write directly to portable HDD? Any important considerations when selecting a motherboard for this application other than the correct socket?

Thanks,

Al

[Carsten]

There is a range of benchmarks here:

https://dcpomatic.com/benchmarks/input.php?id=1

They are a bit optimistic against real life footage in flat (the Sintel is CGI in scope=fewer pixels), and, times go up considerably in 4k.

In general - clock speed or other high performance features will be less important than number of cores/threads, as DCP-o-matic is scaling very good with the number of cores/threads.

In general, Threadrippers or Ryzen make up the best value machines currently. However, you may need to network two of these machines to get closer to your 30min goal. I think the best value for money currently is the Ryzen 9 3950X, and especially the Ryzen 9 3900X.

A threadripper would only have some benefit in a dual CPU mainboard. But altogether, that will be much more expensive than two networked Ryzen 9.

https://www.cpubenchmark.net/high_end_cpus.html



- Carsten

[scozz76]

When you crack the 30mins DCP creation time, please post your machine specs

[berikey]

Intel Xeon Gold 6132 2,6 -single.
64 Gb Ram.
16 -17 fps!!!
This is normal?

[Carsten]

Not too far away, I guess, but it could be better. I guess you should have settled on AMD?

https://dcpomatic.com/benchmarks/input.php?id=1

- Carsten

[rlsound]

Intel Xeon Gold 6132 2,6 -single.
64 Gb Ram.
16 -17 fps!!!
This is normal?

We have the dual Xeon 6132 listed in the benchmark page. Converting a Bluray MKV or Prores to DCP runs at 22-26FPS. That's also with 192GB of ram, not that Dcpomatic really uses it.

Converting MP4 though, converts it at around 45-50fps.

[Carsten]

Hmm, that's an interesting difference. One would think that still the J2K encoding is the heavy task, and ProRes or MKV/AVC should be decoded just as fast as MP4 (25fps Prores or AVC is nothing for modern CPUs). Is that for the same resolution (1080p)? Maybe a specific FFMPEG/LIBAV weakness? Maybe one that hit's only on CPUs with many cores?


If you think you perform under par on a CPU with many logical cores, try to increase the number of encoding threads. That was more effective with earlier versions of DCP-o-matic, but, it is always worth a try. On a Xeon Gold (single) with 14/28 cores, e.g. try 36 or 48 threads. Another solution that sometimes worked in the past is to start the encoding server on the same machine and split the number of encoding threads between local and encoding server - e.g. 18/18.


Note that encoding performance varies a lot with content and resolution - e.g. the benchmark Sintel is animation at 2k scope resolution. Real life footage in 2k flat or 16:9 may encode quite a bit slower on the same machine.

- Carsten

[rlsound]

I never did look the differences beyond MP4 vs Prores. 1080P trailers from Youtube convert at 45-50 fps as well, so it must be related to compression rather than resolution.

Curious to know what OS the poster with the single 6132 is running. Hopefully it's Ubuntu and then they can run htop and see all the threads and where they are at during a convert. Ours were running between 90-100% and were moving around a bit. Changing the Linux governor to performance from ondemand made them stay at 98-100%, so that's a tweak I recommend anyone with Ubuntu or Linux based machines do.