Little different hard disk benchmark

Benchmark screenshots for various boosters & machines.
Post Reply
Petari
Software Moderator
Software Moderator
Posts: 560
Joined: Tue Nov 28, 2017 1:32 pm

Little different hard disk benchmark

Post by Petari » Tue Aug 07, 2018 2:46 pm

Usually transfer speed is what is measured in case of hard disks/Flash card adapters, and sometimes access time - what is little more complicated case, but certainly is much better with Flash cards, since no mechanic - in range of 1mS by faster solutions - on Atari STs .

I will now give some real work test results with different settings, different filled partitions, different driver/TOS, with very simple test. All is made on same computer - Mega STE, with same UltraSatan, with 2 SD cards with practically same speed.
Test is just copying SUBDIR with 80 very short files to some other SUBDIR on partition.

For DOS/TOS compatibles of 500 MB minimum buffer size is 32 KB, what hard disk driver must create.
TOS 2.06, TOS/DOS compatible partition of 500 MB, filled 80% :
At 8 MHz: 58 sec , at 16 MHz 40 sec

Almost empty 500 MB partition:
At 8 MHz: 39 sec , at 16 MHz 27 sec

TOS 1.04, 8 MHz for same as above: 38 sec - marginally faster.


Improved TOS 1.04 (without large sectors), DOS FAT16 partition of 1000 MB, 80% filled:
No added buffers, only base 2 KB, what TOS creates.
At 8 MHz: 67 sec , at 16 MHz: 46 sec
With 32 KB added buffers:
8 - 66 sec, 16 - 46 sec

Improved TOS 1.04 (without large sectors), DOS FAT16 partition of 1000 MB, almost empty:
No added buffers, only base 2 KB, what TOS creates.
At 8 MHz: 26 sec , at 16 MHz: 17 sec
With 32 KB added buffers:
8 - 25 sec, 16 - 17 sec - marginal improvement, or just in range of test error.

So, what affect most speed during copy of short files ? It is how much partition is filled. The reason is how TOS FAT16 filesystem work: it needs to search by every write through FAT, from start for empty clusters, and when finds, it will copy there. That search takes much more time when partition is filled, of course.
Could make if faster with really large buffers, like over 300 KB . That may be for some of next tests. Smaller cache, like 32 KB here helps almost nothing.
Then, CPU speed is very important - even if disk transfer and access speed is same, there is lot of it to do for CPU during file access and copy.

Improved TOS is faster too - see 26 secs vs. 39 secs for copy on almost empty partition (8 MHz) , The reason is that no large sectors, so it will transfer only 1 sector instead 16 (that's minimal block size (8 KB) in case of 500 MB TOS partition) in case of short files (or short file chunks) access .
There is 2 kind of people: one thinking about moving to Mars after here becomes too bad, the others thinking about how to keep this planet habitable.

stephen_usher
Posts: 167
Joined: Mon Nov 13, 2017 7:19 pm
Location: Oxford, UK.
Contact:

Re: Little different hard disk benchmark

Post by stephen_usher » Tue Aug 07, 2018 7:29 pm

It's not just a FAT filesystem that slows with the number of files, pretty well every filesystem I've come across does, though with most it's the number of files in a single directory.

At work we have very large RAID arrays used for scientific computing. It's amazing how many scientists think writing one file per data entry in a single directory is a good idea. In one case we had a directory on a Linux ext4 filesystem with about half a million, I think, (few bytes each) files which almost filled the RAID array with a combination of unused space and directory file. Just trying to open a file to read would take a good 5 minutes of CPU time on a fast Xeon multi-gigahertz processor!
Intro retro computers since before they were retro...
ZX81->Spectrum->Memotech MTX->Sinclair QL->520STM->BBC Micro->TT030->PCs & Sun Workstations.
Added code to the MiNT kernel (still there the last time I checked) + put together MiNTOS.
Collection now with added Macs, Amigas, Suns and Acorns.

Petari
Software Moderator
Software Moderator
Posts: 560
Joined: Tue Nov 28, 2017 1:32 pm

Re: Little different hard disk benchmark

Post by Petari » Tue Aug 07, 2018 8:10 pm

Yes, you are very right with that many files in 1 DIR slowdown. On Atari, it is enough to have 1000 files in 1 SUBDIR, and it may need minutes to open.
Actually, all this started with 4200 files, what I needed to check, and remove those with improper value in it. I used speed up of Steem, of course.
Made some simple program to make list of valid files - WEB host seems to making lot of errors in DL counting.
So, I got at the end some 1135 valid files, and then came idea to make benchmark - because I knew that there must be differences between when partition is filled or is empty. So large file count of course takes lot of time - even writing out to SD card from PC is not much faster than copy with Atari . So, I went to 80 files, because it takes about 1 minute (in slower case).

It was first test of this kind, and I think that it is very good to get picture which factors are relevant for speed. Of course, there is more. But plenty of files in 1 SUBDIR is not typical. User can take care to place frequently used files to start of partition - what means simply that write them there first.

Surely FAT32 is not for ST machines. That could be extremely slow, considering much more FAT records.

I would not say that FAT filesystem slows down. Some more advanced would be even slower - on Atari. FAT is still the simplest. I used intentionally many short files, as some kind of extremity.
There is 2 kind of people: one thinking about moving to Mars after here becomes too bad, the others thinking about how to keep this planet habitable.

Petari
Software Moderator
Software Moderator
Posts: 560
Joined: Tue Nov 28, 2017 1:32 pm

Re: Little different hard disk benchmark

Post by Petari » Wed Aug 08, 2018 9:30 am

If someone want to make this test on his config, here are files:
80.zip
(8.73 KiB) Downloaded 13 times
Depack it on your Atari disk, media , Then create some new SUBDIR, or use existing one, different from source SUBDIR, and copy there DIR 80 . Start clock after copy dialog appears, in same moment when clicking OK or press ENTER. Stop when copy is finished. With Satandisk it will be for sure much slower.
May give your config parameters: which Atari machine, TOS v. , used mass storage adapter, card type, used hard disk driver SW and how much cache it installed - later is not so easy to check, usually need to go in installing SW. In case of my PP TOS/DOS drivers it is 32 KB .
This test may look trivial for some, and yes, it is. But who wants, can really learn some things about how all it works, and what is worth to care about.
There is 2 kind of people: one thinking about moving to Mars after here becomes too bad, the others thinking about how to keep this planet habitable.

Post Reply