Problem when transfering DCPs to a external hard drive

[accord]

Hello! Sorry for the long post in advance and english is not my first language. I'll try to make a TL;DR at the end so you can check and see if you want to read it all.

I am a film student who recently got a internship on my college movie theater. In my college movie theather, we have weekly film screenings that ranges from outside (out of the college/not organized by the employees) festivals to ones that we, who work in the theather, curate.

In our projection room, we have a computer that is connected to a system composed of a 50tb server, a Christie CP4220 that has a Dolby IMS3000 attatched to it.

When I started working, my supervisor taught me and other students the basics on how to make DCPs on DCP-o-Matic so we could make it on our own and ingest and project a movie by ourselves.

We always try to exhibit whatever movie we are trying to exhibit in a DCP format, using DCP-o-Matic on our projection computer to make a DCP of the movie. After it's done we ingest it on the Dolby server and play. It has worked perfectly fine for the whole time I have been here.

The thing is: we have from 2 to 4 movie sessions a day and with lack of communication/hardware limitation, sometimes it is impossible to make DCPs of all the movies in a screening, so we have to resort to play it on VLC.

So, knowing the basics on how to make a DCP (and since none of the DCP's I made on the projection computer had any problem playing) I took upon myself to take one of the external HDD that we have to make a DCP in my personal computer, transfer it to the hard drive, transfer it from the hard drive to the server that the computer is plugged into and then ingest on the Dolby system (the last two steps being the ones we usually do when a outside festival gives us a HDD with their DCPs on it).

The problem arised a couple of times, when I made the DCP on my computer, checked the DCP on the DCP-o-Matic player (I usually check by jumping to a frame and pressing play to see if it runs smoothly for a bit then go to other parts of the file and try the same), transfered to the HDD, and checked on the player once again but with the file running from the external drive. Some of them played back from the HDD just fine but some of the DCPs I made, when put on the hard drive, seem to have trouble playing it back on the player, usually by getting stuck on a frame or dropping FPS by a lot. The ones who were fine didn't have any trouble transfering to the server, ingesting and have played smoothly throughout its runtime. The ones who had a problem playing back from the HDD weren't able to transfer to the server at all. When I tried to copy from the hard drive to the computer, the transfer speed drops dramatically after a while (from 100 MB/s to 700 kBs/s), so after 10 or more minutes waiting I just give up on the transfering.

I don't know what I'm doing wrong, if it's a me problem when copying the files to the hard drive (I do the basic drag and release on the drive), if it's the hard drive fault (cause the server USB connection had some trouble recognizing some of the HDDs, to the point I had to format them for it to work), if it's a DCP problem (but the original files on my computer seem to play back just fine). But I feel like the file gets corrupted in the PC > HDD transfer process because these transfer speed drops seems really odd to me.

Sorry for the long post, it's that I really just want to exceed in this temporary internship so when they consider hiring for the full internship I'm one of the people chosen.

TL;DR: I got a internship in my college and tried to make my own DCP at home so I could speed up the workflow of the screenings but some DCPs seems to get corrupted when transfered to the external hard drive and aren't able to transfer from the HDD back to the projection computer.

[carl]

It's hard to be sure but it sounds to me like a problem with the hard drive. Do you have another that you could try?

[accord]

Thank you for the answer!
I have tried with 2 different hard drives, now going for the third. This third one I also had to format it twice so it would actually show on the projection PC when I plugged onto the server USB-3 entrance.
With the first HDD I made and transferred three different DCPs and, from those three, one didn't work.
The second one I also made three different DCPs and one didn't work.
I thinking if maybe I should zip my DCPs before putting them on the Hard Drive but I'm afraid it has even more chance to corrupt the DCP.

[carl]

Do your hard drives have their own power supplies, or are they powered by the USB bus? It's possible the USB port is not providing enough power.

[accord]

They do not have their own power supply. They are one of those that are kinda cheap. But like, once on a outside festival we received 5/6 different hard drives (of the cheap kind as well) to ingest and I don't remember a single one of them having some kinda of problem.
Maybe it's on my end, since this time the one I checked home and knew it was malfunctioning was the one who caused trouble back in the projection computer? Anyways I'm trying once again right now, hope everything goes smoothly.

If it doesn't work I'm going to try and zip the DCP folder and put on the HDD, to see if that works.
Just a quick question, unrelated question, does the whole folder need to go or the only one necessary is the big folder (the one with the DCP naming conventions) inside the folder?

Edit: the one I just made worked just fine in the playback test! It seems to be happening to every third one I made, but I cannot be sure for now since today's Halloween screening is finally over and next week the movies come from an outside source. I'm not even able to test transferring yesterday DCP (the one that didn't work) to this new HDD because I had to delete it to make space to the new one.

[Carsten]

Maybe you need to tell us a bit more about the specific hardware you use. USB versions, do you use a powered USB hub, etc. Which format do you use on the discs? Do you generate the DCPs on an internal disc/SSD and then transfer to an external drive?

The IMS3000 is able to play DCPs directly off the hard disk or a NAS device. So at least the ingest time could be spared on a tight schedule. This is only recommended if the drive is stable. Do you use spinning drives or SSDs?

[accord]

For sure!

All the HDDs I got from the projection room uses USB 3.0 connection/cable and I always plug them on the USB 3.0 entrance in my personal computer when transfering the DCPs I made home. I also connect them into the USB 3.0 entrance of the server that's attatched to the computer system. In both cases I connect them directly into the machine (although a case could be made that I connect them into the server USB port not the ACTUAL computer, but i have no idea where its located. and I also think the problem may be on my end, not the server)

I've read that the universally accepted disk format of DCPs players is the ext2 format, but since we plug the HDD in a Windows OS to transfer the files to the computer and then into the IMS3000 server, they are usually formatted into de NTFS format (one of them was in exFAT, but the server was not able to recognize it so I used another, smaller, computer we also have who was able to read it and formatted the drive into NTFS).

I generate the DCPs into my internal HDD (since the SSD my OS is installed into is basically full) and then I just drop and drag the whole project folder into the external drive.

I have never tried to play it directly from the external HDD, because our supervisor advised that it is quite unreliable and more prone to errors, but I tried to ingest the faulty DCPs directly into the system by plugging the external HDD into one of the ports on the IMS3000 hardware. The problem is that, with those DCPs, the ingestion process usually halts into the 15, 20 percent mark and doesn't go through with the ingestion.

If you are talking about the external HDDs none of them are a SSD, and if you're talking about the server, I'm pretty sure all the disks in there are spinning drives. We don't play it directly from it, though, as the ingest time are not really the problem. The main problem is that we only have one PC that's really able to make a DCP in a faster time, we have to coordinate really well and prepare the DCPs beforehand. Sadly, that's not what happens. That's why I took upon myself to do it home, even though my CPU is not faster than the projector computer CPU I can at least leave it running while I'm asleep and it's pretty much done by the time I'm awake.

Sorry for the block of text, hope is understandable and clear enough/not too convoluted. I'm on the projection room rn and the the DCP I made yesterday transferred just okay to the computer and it's now on the process of ingesting into the IMS3000.

[Carsten]

Maybe there's an issue with your computers USB port. If it is a desktop type computer, it should have multiple USB ports and maybe you can try another one (at the back?).

NTFS is the right choice for you. If you come across this issue again, please bring back the disk to your computer, start DCP-o-matic player, and do a verification (hash check) in player. It will take a while on a USB connected disk (for full lenght features at least), but you will probably notice errors that hint to transfer issues. Sometimes these issues have been caused not by USB or disk issues, but by defective RAM (or RAM running to fast/hot).
Sometimes, Antivirus software also causes issues.

Also, you should not drag the whole project folder to the transfer drive, it usually makes no sense and may sometimes confuse the server, who is only looking for DCP files, not general computer files. Restrict yourself to the DCP folder.


Note that currently Ryzen 7/9 CPUs give you the best price/performance ratio when encoding DCPs.

https://dcpomatic.com/benchmarks/input/1

[accord]

Thank you, I will try to plug into the other ports at my computer! I usually plug at one specific on the back, because the one in the front has some kind of malfunction (bad contact? idk how its called in english) and it's more prone to me knocking it and ending the connection.

I'll try the check If i notice the problem once again and I'll try to uptade this (or maybe make a new post?) once the check is done. I don't know how much it is going to take, since next screenings are from an outside source, so maybe 1 to 1 and a half weeks until I try to make some more DCPs. Maybe you're right on the RAM thing, my PC is due to a cleaning for a looooong time (I have 2 8gb RAMs, by the way).

When you refer to the DCP folder you mean the one with the DCP naming convention right? Sorry if it's an obvious question, but I just wanted to make sure.

My CPU is on the lowest end possible. Is an old, 2020, Ryzen 5 3350G that I first bought when building a pc for the first time; It doesn't seem to cause any problems aside of the long waiting (up to 6, 7 hours for a 2 hour long movie). While on the topic of CPUs, I wanted to ask this: I've been wanting to upgrade my CPU not only because of the DCPs, but because I want to get into editing and stuff. But on the DCP topic (although opinions on the other aspects would be appreciated), do you think a i5-12600K would good enough? Or at least on par with the projector CPU (an Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz - it usually runs on 20-22 FPS for a 2k DCP and maybe 6 or so FPS on a 4k one).

Thank you a lot for taking your time to answer me!

[Carsten]

Yes, the folder that follows the DCP/ISDCF naming convention, that is sufficient.

If your PC uses a Ryzen 5, chances are high that you may be able to change the CPU to a Ryzen 7 - that would be the easiest and cheapest way to improve encoding speed.

You may also buy a second hand workstation or server that has many CPU HT cores on ebay. Like e.g. a E5-2620 v3, or a dual CPU server. That will increase encoding speed drastically. Are you encoding in 2k or 4k?