This is of course for optimized performances. Allocating a lot of RAM could in fact lower performances. But this is a programming question that does not have a place here. This is all about virtual memory and caching. I will just give a simple analogy.
Think of it like reading a giant book: you don’t memorize every page before you start reading (that’s loading into RAM). Instead, you keep the book on the table and just look at the specific page your eyes are on. The computer ‘maps’ the file so it can grab exactly what it needs, exactly when it needs it, without filling up and slowing down your system.
We came to this issue, while converting several terabytes of wav into flac.
It seems to be more efficient, to have as much memory in use as posssible when using 8 or more tasks in batch processing.
We had some files of over 30 GB and those would fit easily in memory at once, instead of picking of chunks again and again.
(Besides just converting, there were some other tasks like normalization et al in the process queue)
This is the first point where loading a complete file in memory slows down, instead of increasing speed.
If the processing task has to wait until the full file is in memory it takes a lot of time, even with SSDs and fast memory. Loading by pieces means the data processing can start immediately when the first piece is loaded. When this piece has been converted it can be written back to disk.
In the meantime the task has loaded more pieces of the file and continues also immediately with processing. So there are parallel tasks (CPU and I/O) working on the file, without one waiting for the other.
Because this leaves room for more files and the conversion process is a sequential task anyway, it is possible to load several files at once to take advantage of multiple CPU cores.
As @PG1 wrote, there is more to this (CPU Cache, Memory Mapping, etc.) than just loading everything in memory and thinking it will speed it all up.
And do not compare this to what is known these days as In-Memory Databases, that is a complete different world.
I thought that something like this would come back from you, now you also start insulting.
You have not understood a single bit of what I tried to explain, basically the same as you did not understand what parallel data processing is in the Cubase thread.
In the future you are now on my mute list, it is wasted time to answer any of your postings.
Best prove that you do not know anything about operating system technology.