Page 1 of 1

Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 4:47 am
by tobyo
Hi, I was recently backing up some files in MX Linux and encountered some strange errors from my backup software.

I have several external 'source' drives using NTFS, these are then backed up to other various drives, also using NTFS.
I generally don't store files on the same drive as my OS.

I suspected the error with the backup software was related to the NTFS file system.
To check these errors I had to connect the drive to a Windows PC to run chkdsk; sure enough it found the errors and repaired them.

I don't want to connect the drive to Windows to repair it, and I am now ready to switch these drives to a Linux friendly format anyway.

I have read about a few file systems and have decided ext4 should be suitable.
I don't care about compatibility with Windows or Mac OS etc. I will have a separate drive for anything specific to those.

As I understand ext4 is quite similar to NTFS in some regards (journaling etc.) so I am wondering if anyone has any advice or anything I should worry about regarding the moving of the files?

Also, I am used to running chkdsk on NTFS drives on occasion. Is there an equivalent for ext4?
I vaguely remember using fsck in Mac OS X, but trying to find a definitive explanation of the commands available is quite difficult with the internet in the state that it is currently in.

I am open to any advice :-)

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 5:21 am
by kc1di
Hello tobyo,
You can use fsck on most any linux system, but the partition must be un mounted first here is a tutorial that will guide you in it's usage.
https://phoenixnap.com/kb/fsck-command-linux

I think in your use case Ext4 would be the best format. Since it's been around awhile and is stable. Good luck and enjoy!

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 5:35 am
by tobyo
Thank you kc1di that link is great.

Assuming you are using ext4, do you scan your drives often?

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 6:08 am
by kc1di
Not too often hardly ever find an error. As I said ext4 is very stable. Of course drives can wear out. Especially spinners.(0ld ones) All my drives are now SSD's .They give me very little trouble. They also all have auto trim enabled also so that keeps them somewhat in line also.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 6:14 am
by tobyo
Thanks, as long as I have backups I have piece of mind :-)

Interesting you mentioned auto trim, do you know if MX Linux does this by default?
Is it something you have to enable per drive?

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 6:18 am
by kc1di
Only if your booting in systemd. If not you can run it manually with this command.

Code: Select all

sudo fstrim -av
if that one does not work use this one

Code: Select all

 sudo fstrim -v /

on the second command replace the / with /dev/sdxxx (deriver desigantor)

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 6:28 am
by tobyo
Thank you again kc1di!

I am starting the process today.

I am keeping some specific drives as exFAT for compatibility.
I have been using F2FS with some USB sticks etc. which seems to be quite efficient.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 11:00 am
by kc1di
tobyo wrote: Wed Oct 23, 2024 6:28 am Thank you again kc1di!

I am starting the process today.

I am keeping some specific drives as exFAT for compatibility.
I have been using F2FS with some USB sticks etc. which seems to be quite efficient.
Let us know how it works out.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 11:38 am
by tobyo
Thanks kc1di

Phew, just finished backing my drives.

First I formatted my first ext4 drive using GParted; except once completed I realised the owner is then set to root.

I found this article: 'Format ext4 filesystem to be owned by regular user'
https://mxlinux.org/wiki/system/format- ... ular-user/

I ran the command but it complained that the drive was mounted so first I added: umount /dev/sda1
Then I ran the command again, this appeared to work but the drive was not being read by the OS and it couldn't determine the size.

So then I discovered the Format USB Drives tool and ran this; it failed with an unknown error.

I ran GParted again and wiped the drive again; with the owner as root (as expected).

Then I ran Format USB Drives again, this time it worked! But then I noticed it defaulted the partition type to msdos (not gpt).

So I ran it again, this time changing the partition style from default to gpt, and success!

However I now have a question: is the drive owner now specifically associated with my particular user, or is it to a group e.g. any admin?

In other words if I connect the drive to a different computer with a different user or even a different distro like Fedora etc. can I still copy files to the drive without some messing about?

I am currently copying my files from my NTFS backup to the ext4 drive.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 1:13 pm
by DukeComposed
tobyo wrote: Wed Oct 23, 2024 11:38 am However I now have a question: is the drive owner now specifically associated with my particular user, or is it to a group e.g. any admin?

In other words if I connect the drive to a different computer with a different user or even a different distro like Fedora etc. can I still copy files to the drive without some messing about?
Disk drives don't have a concept of ownership. Some file systems, including ext4, preserve permissions as a combination of user ID, group ID, and mode.

On Linux the first user account is usually given UID 1000. So if I create a whole file tree on an external USB drive formatted with an ext4 partition, those files will have my default UID of 1000 unless I specifically change it.

When I export that drive and attach it to a different machine, I will typically need root permissions to mount it. Let's say I mount it under /mnt. That machine can now read the files on that ext4 partition, including the UIDs, GIDs, and modes of those files.

The UIDs and GIDs will all still be the same. So if my UID on the first machine was 1000, the user with UID 1000 will be the owner of those files on the second machine whether that's me or not. This isn't a problem if and only if my user account has UID 1000 on both machines. If my UID on one machine is 1234 and on the other it's 5678, the machines will enforce standard UNIX user and file mode permissions, even though I'm the only individual creating or reading those files.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 23, 2024 1:37 pm
by entropyfoe
You said it brother ! For linux ext4 is true and tested.
Agree with all your comments. MX just takes care of that fstrim, there is a log. [hmmm, haven't checked mine lately?]
Even with unexpected power failures, the fs seems to recover with no errors.
kc1di
Re: Switching drive from NTFS to ext4 (or something else)
#4 Post by kc1di » Wed Oct 23, 2024 6:08 am
Not too often hardly ever find an error. As I said ext4 is very stable. Of course drives can wear out. Especially spinners.(0ld ones) All my drives are now SSD's .They give me very little trouble. They also all have auto trim enabled also so that keeps them somewhat in line also.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 3:48 am
by tobyo
Thank you DukeComposed; I wondered why the drive was only accessible via root access, that is very interesting.

I am still in the process of moving the files to ext4, I have several terabytes to go!
I will try connecting an ext4 drive to another computer once the backup transfers are complete and see how it works.

Also, kc1di, I ran 'sudo fstrim -av' - my internal NVME reported 250MB and a 2TB USB-connected SSD reported 390GB trimmed! I think this was long overdue!

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 3:55 am
by tobyo
entropyfoe wrote: Wed Oct 23, 2024 1:37 pm You said it brother ! For linux ext4 is true and tested.
Agree with all your comments. MX just takes care of that fstrim, there is a log. [hmmm, haven't checked mine lately?]
Even with unexpected power failures, the fs seems to recover with no errors.
Hi entropyfoe, do you know where that log would be, or how I could view it?

I found this article https://mxlinux.org/wiki/hardware/trim/
and ran the command:

Code: Select all

$ tail /var/log/trim.log 
But I received: command not found

Edit: I answered my own question, I just opened the .log file in a text editor: /var/log/trim.log

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 4:01 am
by artytux
try
sudo tail /var/log/trim.log
or goto
Root then var > log >trim.log

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 4:27 am
by tobyo
Thank you artytux; I realised how simple it was 😅

I recently had an SSD fail on a Windows machine.

I have used CrystalDiskInfo in Windows and gsmartcontrol in Linux to view the SMART info details in the past, which is useful.

But I would also check Event Viewer in Windows for any hard drive errors in the admin log; this would often log a bad block or read error etc.
I don't know if that was an NTFS specific log, but does anyone know if there is something similar for ext4 (or in general), for Linux? Something that might log a disk read error or hardware failure?

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 4:38 am
by artytux
If you are booting into systemd it has enough tools for what you want
on systemV Ive not had to go down that road yet,
Seems it's time for you and DDG quality time, unless some-one here can chip in with an answer.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 5:10 am
by Dearth

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Thu Oct 24, 2024 10:05 pm
by entropyfoe
artytux,

Yes, that is the log.
/var/log/trim.log

And mine is working wonderfully. The last trim was yesterday.

Code: Select all

*** Wed, 09 Oct 2024 07:59:08 -0500 ***
/home: 81.8 GiB (87858819072 bytes) trimmed on /dev/nvme0n1p3
/media/Data: 307.4 GiB (330021101568 bytes) trimmed on /dev/sda1
/: 3.2 GiB (3432132608 bytes) trimmed on /dev/nvme0n1p1
*** Wed, 16 Oct 2024 13:57:45 -0500 ***
/home: 0 B (0 bytes) trimmed on /dev/nvme0n1p3
/media/Data: 0 B (0 bytes) trimmed on /dev/sda1
/: 0 B (0 bytes) trimmed on /dev/nvme0n1p1
*** Wed, 23 Oct 2024 08:20:14 -0500 ***
/home: 0 B (0 bytes) trimmed on /dev/nvme0n1p3
/media/Data: 27.5 MiB (28880896 bytes) trimmed on /dev/sda1
/: 0 B (0 bytes) trimmed on /dev/nvme0n1p1

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Fri Oct 25, 2024 4:36 am
by tobyo
Hi all, I am still moving various drives from NTFS to ext4; since I have switched some of the drives to ext4 I have noticed they no longer have the eject icon in the Devices menu of Thunar (on the side).

All the drives are connected via USB.

I can right-click to Unmount a drive and this greys the icon out; if I then right-click the unmounted drive I have two options: Mount or Safely Remove.
Safely remove is greyed out, but as the drive is unmounted I would assume it is ok to unplug the USB cable?

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Fri Oct 25, 2024 5:06 am
by davidy
Yes. Just like when you create a bootable flash drive with the built-in tool, when it's done it leaves it unmounted and ready to unplug. But if an external usb nvme is exfat will the fstrim cmds work? I tried the ones suggested (sudo fstrim -v /media/lapiz/65FC-5999/) and it just gives me "the discard command is not supported". I'm assuming this is an exfat thing and trim cmds only apply to journaled file systems?
F2FS sounds great but unfortunately most devices such as phones and tv's require fat32.
I used exfat on my ext nvme only because of my single windows pc or I would have used ext4.

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Fri Oct 25, 2024 5:26 am
by tobyo
Thanks davidy I also noticed when the drives were NTFS if I clicked the eject icon I would see a notification:

Writing data to device. There is data that needs to be written...
(in other words do not unplug yet).

then another notification would show, something along the lines of:

It is now safe to remove the device.

Now the drives are ext 4 I see the first notification that data needs to be written but there is no notification that it is safe to remove.
I guess as long as the icon is greyed out it should be safe to remove (despite no 'safe to remove' notification)?


re. your trim question, I recently read that you can run the following to see if a drive supports trim:

Code: Select all

lsblk -D
in the results check the DISC-MAX column, if the value is 0B then it does not support trim.

Also, I read that and SATA SSD drives (2.5") connected via USB will not support trim, although I did read that an NVME drive connected via USB may suport trim ¯\_(ツ)_/¯

Re: Switching drive from NTFS to ext4 (or something else)

Posted: Wed Oct 30, 2024 8:10 pm
by Danathar
tobyo wrote: Wed Oct 23, 2024 4:47 am Hi, I was recently backing up some files in MX Linux and encountered some strange errors from my backup software.

I have several external 'source' drives using NTFS, these are then backed up to other various drives, also using NTFS.
I generally don't store files on the same drive as my OS.

I suspected the error with the backup software was related to the NTFS file system.
To check these errors I had to connect the drive to a Windows PC to run chkdsk; sure enough it found the errors and repaired them.

I don't want to connect the drive to Windows to repair it, and I am now ready to switch these drives to a Linux friendly format anyway.

I have read about a few file systems and have decided ext4 should be suitable.
I don't care about compatibility with Windows or Mac OS etc. I will have a separate drive for anything specific to those.

As I understand ext4 is quite similar to NTFS in some regards (journaling etc.) so I am wondering if anyone has any advice or anything I should worry about regarding the moving of the files?

Also, I am used to running chkdsk on NTFS drives on occasion. Is there an equivalent for ext4?
I vaguely remember using fsck in Mac OS X, but trying to find a definitive explanation of the commands available is quite difficult with the internet in the state that it is currently in.

I am open to any advice :-)
openZFS.