Hi,
I'm fairly new to dcpom and working on setting up a system to encode DCP for a small film festival.
I have a Ryzen 1700x w/16M Ram on Windows 10 overclocked to 4ghz. The system appears to be stable with this config and temp never gets above 65c.
Small files seem to go fine at 24fps. However, several of the feature films start encoding around 24fps and then drop to around 14fps. After appx 5 mins, when I monitor the CPU, I see that the CPU cycles from 4ghz to .5ghz appx every 20 seconds with an effective rate of appx 10fps. There is minimal disk or RAM usage through this process. I'm converting an .mp4 24fps to 24fps with default settings.
Just trying to determine if this CPU cycling is normal behavior. Previously I was testing on an iMac. It only ran at 6fps and didn't seem to show any cycling.
Thanks for any feedback, new to all of this and trying to understand normal operations.
Regards,
Jim
CPU Cycling during encoding
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: CPU Cycling during encoding
Do you promise to supply some official benchmark data for that Machine? Because, we have no Ryzen data so far 
http://dcpomatic.com/benchmarks/
How many encoding threads have you set in DOM? There is plenty of experience with Intel CPUs, but Ryzen, and how the code deals with them...
It is pretty normal that you see a rather high number at the beginning, then dropping. That is because J2K encoding is content dependent, and most movies start with black, or screen graphics, etc. that compress better and faster. So, a drop from 24 to 14 looks 'normal' to me. There will probably be parts in the movie where it will rise back to above 20 fps again for a short time.
Why your system drops to 0.5GHz (is that really 4GHz down TO 0.5GHz?) occasionally, I have no idea. For how long does it stay at 0.5GHz then?
Are you sure about the real temps of your CPU? I can't believe it stays at 65degree Celsius when clocked at 4GHz and all threads running - I would believe, the throttle to 0.5GHz is due to an overtemperature situation.
Other than that, I could think of situations when something in the singlethreaded pre- or postprocess pipeline starves the encoding threads, so the CPU has not enough to do and thus throttles down. However, within these parameters, this should not happen, unless maybe your footage would be on very slow storage, but that's all very unlikely. What does your CPU clock monitor say, when the system is clocked to 4GHz, but is idling? Does it stay at 4GHz, or will it clock down as well?
This certainly does not happen with other CPUs. I use a dual XEON system and once it is running, it stays at the max allowed clock until the CPU temp reaches nearly 100C, then the clock drops somewhat, but not much, it finds a balance of clock and healthy temperature.
Can you watch task manager/CPU utilization during these parts and tell us what you see? Maybe the code doesn't deal with Ryzen too well, or there is still some CPU/microcode/OS anomaly. You probably heard about some issues with Ryzen CPUs before.
When I did testing on various aspects of the encoding, I created test footage that would allow me to test encoding performance without being dependent of image complexity variations e.g. I created noise patterns of different detail to have a fixed complexity. These need to be videofiles, not stills, though, since DCP-o-matic will not recompress stills for slides, but reuse compressed images. You may of course use numbered sequences of the same image. I created my tests with Photoshop using an 'add noise' filter.
In DOM, you should probably go to prefs - advanced and tick all logging options. Then let it run through an encoding and send this log to Carl. He may be able to find out what's going on.
I'm really interested to see your benchmarks for Bunny and Sintel.
- Carsten
http://dcpomatic.com/benchmarks/
How many encoding threads have you set in DOM? There is plenty of experience with Intel CPUs, but Ryzen, and how the code deals with them...
It is pretty normal that you see a rather high number at the beginning, then dropping. That is because J2K encoding is content dependent, and most movies start with black, or screen graphics, etc. that compress better and faster. So, a drop from 24 to 14 looks 'normal' to me. There will probably be parts in the movie where it will rise back to above 20 fps again for a short time.
Why your system drops to 0.5GHz (is that really 4GHz down TO 0.5GHz?) occasionally, I have no idea. For how long does it stay at 0.5GHz then?
Are you sure about the real temps of your CPU? I can't believe it stays at 65degree Celsius when clocked at 4GHz and all threads running - I would believe, the throttle to 0.5GHz is due to an overtemperature situation.
Other than that, I could think of situations when something in the singlethreaded pre- or postprocess pipeline starves the encoding threads, so the CPU has not enough to do and thus throttles down. However, within these parameters, this should not happen, unless maybe your footage would be on very slow storage, but that's all very unlikely. What does your CPU clock monitor say, when the system is clocked to 4GHz, but is idling? Does it stay at 4GHz, or will it clock down as well?
This certainly does not happen with other CPUs. I use a dual XEON system and once it is running, it stays at the max allowed clock until the CPU temp reaches nearly 100C, then the clock drops somewhat, but not much, it finds a balance of clock and healthy temperature.
Can you watch task manager/CPU utilization during these parts and tell us what you see? Maybe the code doesn't deal with Ryzen too well, or there is still some CPU/microcode/OS anomaly. You probably heard about some issues with Ryzen CPUs before.
When I did testing on various aspects of the encoding, I created test footage that would allow me to test encoding performance without being dependent of image complexity variations e.g. I created noise patterns of different detail to have a fixed complexity. These need to be videofiles, not stills, though, since DCP-o-matic will not recompress stills for slides, but reuse compressed images. You may of course use numbered sequences of the same image. I created my tests with Photoshop using an 'add noise' filter.
In DOM, you should probably go to prefs - advanced and tick all logging options. Then let it run through an encoding and send this log to Carl. He may be able to find out what's going on.
I'm really interested to see your benchmarks for Bunny and Sintel.
- Carsten
Last edited by Carsten on Fri Oct 06, 2017 7:22 pm, edited 1 time in total.
-
fredjonze
- Posts: 23
- Joined: Thu Mar 02, 2017 6:28 pm
Re: CPU Cycling during encoding
Carsten,
Thanks for the feedback. Some notes per your questions:
1. In DOM I'm running at the default 16 threads
2. With the CPU cycling it will run at full 4ghz for appx 15 secs and then cycle down to .5ghz for appx 3 secs. It only starts doing this after appx 5 mins of encoding. It will do this continually during image encoding, or until a subsequent process kicks in. Smaller encodes don't show this behavior and run at a solid 20fps for the entire encode.
3. I ran the benchmarks for both sources and emailed you the logs. I also included a screenshot illustrating the cpu cycling behavior.
4. Sintel took appx 29 mins to encode and buck bunny was appx 44 seconds.
5. When no processes are running on the Ryzen clock speed is 4ghz, but performance shows 4% usage.
6. I have 2 RAM sticks currently clocked at 2800mhz (turned down from 3200 while testing)
7. I flashed the latest Ryzen bios firmware before starting
8. Out of curiosity I ran cinebencz and scored around 1600 on CPU. I just have a stock video card.
I appreciate the help!
Jim
Thanks for the feedback. Some notes per your questions:
1. In DOM I'm running at the default 16 threads
2. With the CPU cycling it will run at full 4ghz for appx 15 secs and then cycle down to .5ghz for appx 3 secs. It only starts doing this after appx 5 mins of encoding. It will do this continually during image encoding, or until a subsequent process kicks in. Smaller encodes don't show this behavior and run at a solid 20fps for the entire encode.
3. I ran the benchmarks for both sources and emailed you the logs. I also included a screenshot illustrating the cpu cycling behavior.
4. Sintel took appx 29 mins to encode and buck bunny was appx 44 seconds.
5. When no processes are running on the Ryzen clock speed is 4ghz, but performance shows 4% usage.
6. I have 2 RAM sticks currently clocked at 2800mhz (turned down from 3200 while testing)
7. I flashed the latest Ryzen bios firmware before starting
8. Out of curiosity I ran cinebencz and scored around 1600 on CPU. I just have a stock video card.
I appreciate the help!
Jim
-
fredjonze
- Posts: 23
- Joined: Thu Mar 02, 2017 6:28 pm
Re: CPU Cycling during encoding
Carsten,
Also, i just ran passmark against the CPU and it scored 16,446...in the 99th percentile. Still trying to figure out why disk passmark is low. I have a Seagate Ironwolf 6TB/7200rpm. Will disk problems cause CPU cycling?
Jim
Also, i just ran passmark against the CPU and it scored 16,446...in the 99th percentile. Still trying to figure out why disk passmark is low. I have a Seagate Ironwolf 6TB/7200rpm. Will disk problems cause CPU cycling?
Jim
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: CPU Cycling during encoding
Your numbers for the Ryzen are close to my z600 with dual xeon 5660 (43s Bunny, 24:21 Sintel). The CPU passmark is also close. (2*7766).
So, in general, your machine does a good job, and DOM performance scales with CPU passmark numbers (which is a known fact for Intel CPUs, but wasn't clear for AMDs). The noticeably lower performance for Sintel compared to Bunny is probably caused by the throttling, which doesn't kick in for Bunny, but for Sintel.
In theory, a delayed disc transaction could cause CPU starving and then the CPU goes to idle - but I don't think this happens here. It probably is a stupid OS strategy to clock the CPU at 4GHz and then brake it down to 0.5GHz for a few seconds, but maybe that is exactly what happens here. Maybe there is something wrong with the OS/monitoring routines for this new CPU. The fact that the throttling happens only after a longer encoding phase seems to suggest that, because I wouldn't know why other reasons like slow disc would only have an impact after a while.
You could try to analyze this by reducing the number of compute threads, or clocking the CPU lower.
But as I said, please send a log to Carl, he may be able to find other reasons. To rule out the disc, you could try to connect a separate disc (SSD/sata if possible) and put the content and project folder on it.
Oh BTW, what version of DOM are you running?
- Carsten
So, in general, your machine does a good job, and DOM performance scales with CPU passmark numbers (which is a known fact for Intel CPUs, but wasn't clear for AMDs). The noticeably lower performance for Sintel compared to Bunny is probably caused by the throttling, which doesn't kick in for Bunny, but for Sintel.
In theory, a delayed disc transaction could cause CPU starving and then the CPU goes to idle - but I don't think this happens here. It probably is a stupid OS strategy to clock the CPU at 4GHz and then brake it down to 0.5GHz for a few seconds, but maybe that is exactly what happens here. Maybe there is something wrong with the OS/monitoring routines for this new CPU. The fact that the throttling happens only after a longer encoding phase seems to suggest that, because I wouldn't know why other reasons like slow disc would only have an impact after a while.
You could try to analyze this by reducing the number of compute threads, or clocking the CPU lower.
But as I said, please send a log to Carl, he may be able to find other reasons. To rule out the disc, you could try to connect a separate disc (SSD/sata if possible) and put the content and project folder on it.
Oh BTW, what version of DOM are you running?
- Carsten
-
fredjonze
- Posts: 23
- Joined: Thu Mar 02, 2017 6:28 pm
Re: CPU Cycling during encoding
Carsten,
I uploaded the sintel and buck bunny logs yesterday.
I think I've found the problem and I think it is related to chip characteristics and heat. When I first tuned the system I adjusted cpu speed and voltage to get rid if crashes. I referred to several of the overclock leaderboards to set initial settings. i have a Corsair liquid cooler. However, it appears that the cpu begins cycling when the temp reaches appx 61C. Far lower than the temps my friends regularly use. It may be I have a temperamental core, but the individual core temp utilities don't appear to support Ryzen just yet.
So I backed off the CPU voltage quite a bit and that seems to have stabilized the CPU. At 1.35 volts and 3.95ghz the Sintel encode completed in 25 mins. The effective fps during encoding was appx 17fps. Towards the end of the encode the temp was hovering around 60C and I started to get some CPU cycling, so I'm going to try and drop the voltage a little bit more without crashing the system. When I get there I'll upload another set of logs for Sintel and BB.
Most folks seem to be using voltages at or above 1.5v, so in the CPU lottery I seem to have won a more temperamental CPU.
I'm running DOM v 2.10.5 git 61b5d4ad08
Thanks for your feedback.
Jim
I uploaded the sintel and buck bunny logs yesterday.
I think I've found the problem and I think it is related to chip characteristics and heat. When I first tuned the system I adjusted cpu speed and voltage to get rid if crashes. I referred to several of the overclock leaderboards to set initial settings. i have a Corsair liquid cooler. However, it appears that the cpu begins cycling when the temp reaches appx 61C. Far lower than the temps my friends regularly use. It may be I have a temperamental core, but the individual core temp utilities don't appear to support Ryzen just yet.
So I backed off the CPU voltage quite a bit and that seems to have stabilized the CPU. At 1.35 volts and 3.95ghz the Sintel encode completed in 25 mins. The effective fps during encoding was appx 17fps. Towards the end of the encode the temp was hovering around 60C and I started to get some CPU cycling, so I'm going to try and drop the voltage a little bit more without crashing the system. When I get there I'll upload another set of logs for Sintel and BB.
Most folks seem to be using voltages at or above 1.5v, so in the CPU lottery I seem to have won a more temperamental CPU.
I'm running DOM v 2.10.5 git 61b5d4ad08
Thanks for your feedback.
Jim
-
fredjonze
- Posts: 23
- Joined: Thu Mar 02, 2017 6:28 pm
Re: CPU Cycling during encoding
Carsten,
I think I have a stable system now.
Ryzen 1700x overclocked to 3.95ghz with a Corsair liquid cooler and operating around 59C. I had to tweak the fan profile to operate under DOM load. It got very cranky at higher speeds and voltages.
Total encode time for Sintel was appx 22 mins. Encode speed averaged around 18fps. I emailed the log to Carl. There was no visible CPU cycling.
The passmark CPU score is appx 16.2k. Cinebench is 1746.
Thanks for your help and feedback...it was very useful while tracking down gremlins!
Now, it's time to get to work on the film festival. We have a small non-profit festival in Polson, MT. I've asked the FF board to donate some money to the DOM project when the dust settles.
Jim
I think I have a stable system now.
Ryzen 1700x overclocked to 3.95ghz with a Corsair liquid cooler and operating around 59C. I had to tweak the fan profile to operate under DOM load. It got very cranky at higher speeds and voltages.
Total encode time for Sintel was appx 22 mins. Encode speed averaged around 18fps. I emailed the log to Carl. There was no visible CPU cycling.
The passmark CPU score is appx 16.2k. Cinebench is 1746.
Thanks for your help and feedback...it was very useful while tracking down gremlins!
Now, it's time to get to work on the film festival. We have a small non-profit festival in Polson, MT. I've asked the FF board to donate some money to the DOM project when the dust settles.
Jim
-
Carsten
- Posts: 2804
- Joined: Tue Apr 15, 2014 9:11 pm
- Location: Germany
Re: CPU Cycling during encoding
Good to finally see Ryzen Benchmarks. 22min for sintel is very good for a single 'desktop grade' CPU machine.
- Carsten
- Carsten
-
Anders M Olsson
- Posts: 62
- Joined: Thu Aug 04, 2016 12:24 pm
Re: CPU Cycling during encoding
So it was basically an issue with the CPU overheating then?
That's good to know. I was thinking of building a computer with a Ryzen 7 CPU, but I haven't decided yet. The new Intel Coffee Lake i7 looks interesting as well...
That's good to know. I was thinking of building a computer with a Ryzen 7 CPU, but I haven't decided yet. The new Intel Coffee Lake i7 looks interesting as well...