speed populating file browser window - MX 23 vs MX 19

[psychlone]

I've come across something I'd like information about, but I'm not specifically looking for help with fixing the issue.

I have several external 4 TB NTFS HDDs containing mainly flac and wav files. A typical example would be a drive with several folders in the root, including one main folder containing 100 000 or so flac and wav files in about 7500 subfolders, with a combined size of about 3.5 TiB. Most of the subfolders in this main folder also have one or two subsubfolders.

When I (double) click on the main folder, the time it takes to populate the window that action opens with the subfolders contained in the main folder varies significantly with the MX version I'm using. In MX19 XFCE, it takes maybe 3 or 4 seconds to populate the window, and I'm then able to open any of the subfolders contained in the main folder and start listening to music. In MX19 Fluxbox it takes about a second. In MX 21 and MX 23, both XFCE and Fluxbox, it takes about 2.5 minutes (I tried several different file browsers in MX 23 XFCE: no difference to the time taken to populate the window fully). I've tried this on a NUC12 i7 16 Gb RAM, a Mac Mini 2012 i7 16 Gb RAM and a PC built in 2020ish with an i9 and 32 Gb RAM. I've tried it with the OS installed to an internal SATA SSD, an internal NVME SSD and Live USB installs (no persistence). I've tried most but not all combinations (no NVME drive in the Mac Mini, for example) - enough to be confident that the PC used and whether the install is a live USB or to an internal SATA or NVME SSD makes a comparatively insignificant difference to the time taken to open the main folder. But MX 19 is hugely faster than MX 21 and MX 23. I also tried the same exercise on the i9 PC with live USBs of the latest Fedora and Mint: Fedora took about 2.5 minutes and Mint about 1.5 minutes.

So my question is: Can anyone tell me what changed between MX 19 and MX 21 to produce this difference? As I said, I'm not looking for help with tweaking MX 23 - I'm quite happy to continue using MX 19 for music, and I don't need to go online with it so updating is not an issue for me - but any info about this topic could be useful to me in the future. I hope I've posted this in the right section: I figured that as the issue I'm encountering is with the current version rather than the old version, this was the right place.

Thanks.

[oops]

Good question.

Maybe due to the kernel version (and parameters)

And the file system ext4 too, and some other parameters :
ex into MX19 here (fast_commit activated): https://lwn.net/Articles/842385/ (...show 20-200% performance improvements with filesystem benchmarks for local filesystems)

Code: Select all

$ lsblk -fa
$ cat /proc/fs/ext4/sda5/fc_info

$ sudo tune2fs -l /dev/sda5 | grep -E "Default mount options|Filesystem features"
Filesystem features:      has_journal ext_attr resize_inode dir_index fast_commit filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Default mount options:    user_xattr acl

MX23 here:

Code: Select all

chroot> tune2fs -l /dev/sda9 | grep -E "Default mount options|Filesystem features"
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Default mount options:    user_xattr acl

[CharlesV]

I am not positive on this, but I believe the "big difference" is due to NTFS and the kernel differences.

I have noticed significant speed differences on USB drives that were NTFS, and then significant differences in various kernels as well. I only run liquorix kernels, but some have definitely been faster than others here.

[m_pav]

Agree with CharlesV. I'm always connecting multiple filesystem types to my machines as I work on customers PCs'. It is very clear that non-native file systems that require additional driver support are slower in every aspect as compared to native Linux Filesystem types, especially where writing is concerned.

Like CharlesV, I too use mostly Liquorix kernels which are marginally less effective in extended throughput as compared to stock Debian Kernels, but the improvements overall make the Liquorix worth it. I always keep a couple regular non-ahs MX Live ISO's around though to retain boot capabilities offered by their signed kernels to be able to boot with machines with secure boot enabled.

It took me a while to "trust" the Unix type filesystems before I migrated much of my storage over to Linux native types, but now, I simply will not go back. If there's no compelling reason to keep using a non-native format type, then you'd be mad not to make the switch because using NTFS as a full-time R/W will result in accumulative errors that can not be fixed without a Windows system present, and even with a Windows system, repair is not guaranteed when the partitions have been used with a feature limited translator.

Keep in mind the Liquorix kernels use the ntfs3 driver from Paragon, which is faster to a point, but they are more sensitive to minor inconsistencies leading to greater recurrence of failures that require a repairs through a Windows system. I typically blacklist the ntfs3 driver so the Liquorix kernels have to use the ntfs-3g driver, offering greater overall stability.

As to the time it takes to "see" the files, that's the indexer and the thumb-nailer at work. Less files = less work, so using root level folders with less in each folder will save you a ton of time. You could turn off the thumbnailer and the speed would increase, but you'd lose the useful representative icons.

[BitterTruth]

I'm quite happy to continue using MX 19 for music, and I don't need to go online with it so updating is not an issue for me

You can get FREE extended support for another 5 years if you disable your old debian repos and just add this single to a new file in the /etc/apt/sources.list.d:

deb http://deb.freexian.com/extended-lts buster main contrib non-free

you can call the file what you like. Afterwards just run mx repo manager -------> Individual Sources Tab --------> untick all the debian repos.

run

sudo apt update

smile as you receive about 500+ security and app updates

you can read more here:

https://www.freexian.com/lts/extended/d ... ended-lts/

[dreamer]

We don't know what kernel you are using in MX-23 and MX-19. If you are using the ntfs-3g driver all I can say is that in 15 years time I haven't encountered problems with it. It's a solid read/write driver IMO, but not as fast as native Linux drivers (I hope maintainers keep it sane). The problem is people use it to access Windows C drives, that typically are hibernated by default and that can cause problems to say the least. I haven't been looking for NTFS related problems, but if Windows (and Windows partition tools) doesn't complain about external or internal NTFS partitions I simply assume they are in a decent shape.

Regarding speed: It could be that you have more file previews (thumbnails) enabled in MX-23. Some file managers also support sound previews. So you could turn off all previews or use a file manager that doesn't support previews. Many audio files have images attached to them.

Another speed factor is your icon theme. Make sure to use an icon theme from the repositories that have a valid icon cache. If you use XFCE, icon themes without a valid cache file will be marked with a warning in XFCE Settings Manager.

[psychlone]

Many thanks to @oops, @CharlesV, @m_pav, @BitterTruth and @dreamer for your replies to my post.

As I said in my initial post, I'm not looking for a solution to the issue right now as I already have a workaround (continuing to use MX19) that I find satisfactory (so although if anyone already knows the exact cause of the issue the information would be very welcome, it's not really worthwhile anyone's spending a lot of time investigating and trying to figure it out). But for information:

1. The kernels I'm using in MX19, MX21 and MX23 are 5.6, 5.10 and 6.1 respectively (all stock Debian). So if the kernel is what accounts for the difference, the change occurred somewhere between 5.6 and 5.10.

2. I have the same icon theme in MX19 and MX21, with no warnings showing, and no warnings for the theme in MX23.

3. After reading the posts from @m_pav and @dreamer I tried turning off previews in MX21 and MX23 (they're enabled in MX19, but clearly not causing any significant slowdown there), and interestingly it made no difference whatever to the time taken to open the main folder (I was expecting it to make at least a bit of a difference).

@m_pav: you got me thinking. I have lots of external drives, and for years I've been formatting them to either NTFS or exfat, because I use Linux, MacOS and Windows for various different purposes. But these days, for playing audio I only use Linux and I think I'll be sticking with that; and as I'm (slowly) assembling all my audio files onto drives that contain audio and nothing else, it would entirely make sense to start formatting audio drives using Ext. So thanks for the nudge.

@BitterTruth thanks very much indeed for the information. That's massively useful, not because I want to update MX19 (I don't) but because I don't really like MX23 and I use MX21 as my daily driver. So the news that I'll be able to carry on using MX21 for years to come is extremely welcome.

I've got some really useful stuff out the replies here, so again a big thank you to all who replied.

[BitterTruth]

I've just had a thought. Could it be:

1) you are using encryption in mx21/mx23
2) turned on some form of zram/compression in mx21/mx23