It's the same as having "open with.." in Firefox, navigating to /usr/bin and then you get a similar amount of time to load everything. (there it's GTK doing it, not Unity or Mate or XFCE).
In that "file/open" dialog, stuff does appear gradually, gets inserted into the sorted list. Nice if the stuff you want comes up early, but if not it's a bit frustrating and you have to wait it out.. It could be confusing in a file manager if everything populated at random.
It's all hard disk limited while file attributes, icons are loaded and then thumbnails are generated if you have media, pdf files etc.
With big enough icons you even have thumbnails for text files.
I went to /usr/bin with the "caja" file manager, it took about five seconds for 2232 files to show up. Like your system It hides everything till all is ready.
In fact if the thing is really braindead and nothing got cached, displaying 5500 files would be at least 5500 random accesses and if you consider a hard drive does about 150 IOPS, the time to read everything would be at least 36.67 seconds.
Hard drives are just that slow at reading random stuff. A friend has long delays dealing with his local stored mails (many years of it), I told him it's normal, HDD can't keep up because the heads have to physically travel to the data. So he needs a SSD if he wants faster operation there. Sensible stuff would even be to keep the / partition (main OS files) on HDD and have the /home (same as C:\Users) on SSD.
Here on Ubuntu I think you have to take the matter on your own, like configuring the thumbnailer or try another file manager (pcmanfm, Norton Commander clone?) or use the terminal to do whatever you wanted to do.