Is an iMac Pro a viable option for encoding?
3.2 GhZ 8-Core Xeon
3.0 GhZ 10-Core Xeon
2.5 GhZ 14-Core Xeon
2.3 GhZ 18-Core Xeon
RAM 32, 64 or 128 GB DDR4
Since I have to purchase an iMac Pro anyway for my work, I would rather take a more expensive one (as money is not an issue), than building something along this viewtopic.php?t=776#p2610.
The goal would be to convert a 4K movie in more or less realtime.
iMac Pro hardware viable option for encoding
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: iMac Pro hardware viable option for encoding
The more cores, the better. You won't get realtime 4k with any of those machines, but you may get a second cheap machine as an encode server. However, remote encoding in 4k is limited, because the data sent over the network quickly saturates it.
The iMac Pro has 10Gig Ethernet and may benefit more from networked encoding, connected to an encode server with 10GE as well, but so far, we haven't seen performance numbers.
As far as I know, Macs support IP based networking through their very fast thunderbolt ports, so, that could be another option.
Depending on your workflow, you may resort to encoding J2K with e.g. Resolve (Resolve 15.x uses the fast Kakadu encoder), but do the DCP finishing with DCP-o-matic.
The iMac Pro is a very expensive machine. You would get a much faster machine based on plain vanilla PC Components. I am a Mac user myself, but when it comes to encoding fps, I resort to other options.
The best start for very quick encodings is a dual CPU Xeon machine. Currently, XEONS are the only fast CPUs that can be paired within a single machine.
- Carsten
The iMac Pro has 10Gig Ethernet and may benefit more from networked encoding, connected to an encode server with 10GE as well, but so far, we haven't seen performance numbers.
As far as I know, Macs support IP based networking through their very fast thunderbolt ports, so, that could be another option.
Depending on your workflow, you may resort to encoding J2K with e.g. Resolve (Resolve 15.x uses the fast Kakadu encoder), but do the DCP finishing with DCP-o-matic.
The iMac Pro is a very expensive machine. You would get a much faster machine based on plain vanilla PC Components. I am a Mac user myself, but when it comes to encoding fps, I resort to other options.
The best start for very quick encodings is a dual CPU Xeon machine. Currently, XEONS are the only fast CPUs that can be paired within a single machine.
- Carsten
-
Eli Ott
- Posts: 2
- Joined: Tue Oct 16, 2018 9:07 am
Re: iMac Pro hardware viable option for encoding
Carsten, many thanks for your prompt – and very helpful – response. I'll check with a colleague of mine who has offered to built a specialized machine (he did build encoding workstations for several arthouse cinemas here in Germany).
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: iMac Pro hardware viable option for encoding
The problem really is the size of the raw data sent over the network to the encoding servers. 1GigE maxes out around 100-120MBytes per second. There are DCP-o-matic benchmarks of network setups reaching close to 50fps - but that is with BigBug Bunny, which sends out tiny 8bit 853*480 images to the encode servers. That is roughly 1.2MByte per frame, a lot of these fit into the 120MByte/s bandwidth of 1GigE. Go 4k, and maybe 12Bit, that grows to 30-40MByte per frame - and so only two or three of these frames make it through 1GigE per second. So, even if your remote encoding server is able to encode at 24fps, it would be starved by only two to three new frames to arrive each second. With a second internal CPU, that bottleneck doesn't exist, as it is a very fast memory transfer. We had some people considering 10GigE to accelerate network encoding on this forum, but as far as I remember, none of them returned with numbers. At the time, 10GigE was also quite expensive. I would love to see wether Thunderbolt neworking works at all with DCP-o-matic (it should, as it seems to work IP based as well), and all you need is a thunderbolt cable between two machines (not cheap as well, but at least simple).
I understand the new Mac Pro will have dual CPU configurations, but won't be available before 2019. And with Apple's current price politics, I can easily see the dual CPU machines going beyond 10k €.
One could think of other strategies to accelerate multi-machine encoding (e.g. distributing source content to remote machines initially, not during encoding), but that can be very complex for different types of source content, and it would only resolve the bottleneck for certain situations. It would be rather useless to have a fast network encoding, if at first you have to copy 500Gigybytes of source footage to another machine, taking 2 hrs on it's own.
Unfortunately, I guess GPU encoding is still far away, there is not enough incentive for open source people to work on that.
Another option would be to offer Carl some money to build a Kakadu encoding module. Don't know if he is open to that. Kakadu licensing may impose restrictions on that.
- Carsten
I understand the new Mac Pro will have dual CPU configurations, but won't be available before 2019. And with Apple's current price politics, I can easily see the dual CPU machines going beyond 10k €.
One could think of other strategies to accelerate multi-machine encoding (e.g. distributing source content to remote machines initially, not during encoding), but that can be very complex for different types of source content, and it would only resolve the bottleneck for certain situations. It would be rather useless to have a fast network encoding, if at first you have to copy 500Gigybytes of source footage to another machine, taking 2 hrs on it's own.
Unfortunately, I guess GPU encoding is still far away, there is not enough incentive for open source people to work on that.
Another option would be to offer Carl some money to build a Kakadu encoding module. Don't know if he is open to that. Kakadu licensing may impose restrictions on that.
- Carsten