MX-Boot repair & encrypted root

[andy]

Hi @Adrian,
for now, just one quick question: is MX-Boot repair supposed to work with an encrypted installation? (BTRFS, subvolumes).

If yes, I have a bug report/ asking for help.

Regards,
Andy

[Adrian]

Yep, but it was not extensively tested, please post the relevant info.

[m_pav]

No bugs here, it's working as it should, otherwise what good would it be to encrypt in the first place?

The only thing that "works" on an encrypted drive is the password to decrypt it, without that, you'll go exactly nowhere.

Start by firing up gparted from your Live Boot media, right click the encrypted partition that contains your /boot folder, then choose open encryption. Issue your encryption password, see that the graphic of the drive now shows how much empty space exists on the partition, close gparted and then, with the partition unlocked try fixing boot, unless, after unlocking the encryption you see your drive has no available space. A filled / is a real good way to get a non-booter.
It's also a good idea to run a partition check while in gparted with the right click before you exit because / data corruption is another cause of non-booters.

[andy]

My situation:
Installed MX-Linux Libretto KDE, I had left some space for Windows partitions.

(formerly had more EFI partitions on other physical drives - currently three SSDs. Because older installations of MX-21 and Windows are existing on other SSDs, I had to remove all these EFI partitions. I was not able to persuade MX-23 installer to properly install to the right EFI partition. So backed up all these EFIs, and removed them all. Only then the installation went well, but this is not important in this context).

Then I have installed Windows 10, which finally (similair to MX-Linux) picked the right EFI, and did not made another one.
But this step broke MX's ability to boot.

But: As I was able to boot MX-23 by Grub rescue menu from the USB flash, to boot right into the installed system.
I have started MX-Boot repair there, not in live system.

Acoording to last comment from @m_pav, I am now unsure - is this supposed to work - repair the actual running system - thus without chroot-ing?

Code: Select all

System:
  Kernel: 6.1.0-10-amd64 [6.1.38-2] arch: x86_64 bits: 64 compiler: gcc v: 12.2.0
    parameters: BOOT_IMAGE=/vmlinuz-6.1.0-10-amd64 root=UUID=<filter> ro rootflags=subvol=@ quiet
    splash
  Desktop: KDE Plasma v: 5.27.5 wm: kwin_x11 vt: 7 dm: SDDM Distro: MX-23_KDE_x64 Libretto July
    31 2023 base: Debian GNU/Linux 12 (bookworm)
Machine:
  Type: Desktop System: ASUS product: N/A v: N/A serial: <superuser required>
  Mobo: ASUSTeK model: TUF GAMING B660M-PLUS WIFI D4 v: Rev 1.xx serial: <superuser required>
    UEFI: American Megatrends v: 2604 date: 07/05/2023
CPU:
  Info: model: 13th Gen Intel Core i5-13600K bits: 64 type: MST AMCP arch: Raptor Lake gen: core 13
    level: v3 note: check built: 2022+ process: Intel 7 (10nm) family: 6 model-id: 0xB7 (183)
    stepping: 1 microcode: 0x119
  Topology: cpus: 1x cores: 14 mt: 6 tpc: 2 st: 8 threads: 20 smt: enabled cache: L1: 1.2 MiB
    desc: d-8x32 KiB, 6x48 KiB; i-6x32 KiB, 8x64 KiB L2: 20 MiB desc: 6x2 MiB, 2x4 MiB L3: 24 MiB
    desc: 1x24 MiB
  Speed (MHz): avg: 2882 high: 3500 min/max: 800/5100:3900 scaling: driver: intel_pstate
    governor: powersave cores: 1: 1059 2: 3500 3: 1100 4: 3500 5: 3500 6: 3500 7: 3500 8: 3500
    9: 1100 10: 3500 11: 1100 12: 3500 13: 3500 14: 3500 15: 3500 16: 3500 17: 3500 18: 3500
    19: 3500 20: 800 bogomips: 139776
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
  Vulnerabilities:
  Type: itlb_multihit status: Not affected
  Type: l1tf status: Not affected
  Type: mds status: Not affected
  Type: meltdown status: Not affected
  Type: mmio_stale_data status: Not affected
  Type: retbleed status: Not affected
  Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl
  Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization
  Type: spectre_v2 mitigation: Enhanced IBRS, IBPB: conditional, RSB filling, PBRSB-eIBRS: SW
    sequence
  Type: srbds status: Not affected
  Type: tsx_async_abort status: Not affected
Graphics:
  Device-1: AMD Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
    vendor: XFX Speedster QICK 319 driver: amdgpu v: kernel arch: RDNA-2 code: Navi-2x
    process: TSMC n7 (7nm) built: 2020-22 pcie: gen: 4 speed: 16 GT/s lanes: 16 ports: active: DP-2
    empty: DP-1,DP-3,HDMI-A-1 bus-ID: 03:00.0 chip-ID: 1002:73df class-ID: 0300
  Display: x11 server: X.Org v: 1.21.1.7 with: Xwayland v: 22.1.9 compositor: kwin_x11 driver: X:
    loaded: amdgpu unloaded: fbdev,modesetting,radeon,vesa dri: radeonsi gpu: amdgpu display-ID: :0
    screens: 1
  Screen-1: 0 s-res: 3840x2160 s-dpi: 96 s-size: 1016x571mm (40.00x22.48")
    s-diag: 1165mm (45.88")
  Monitor-1: DP-2 mapped: DisplayPort-1 model: Idek Iiyama PL3288UH serial: <filter> built: 2021
    res: 3840x2160 hz: 60 dpi: 140 gamma: 1.2 size: 698x393mm (27.48x15.47") diag: 801mm (31.5")
    ratio: 16:9 modes: max: 3840x2160 min: 640x480
  API: OpenGL v: 4.6 Mesa 22.3.6 renderer: AMD Radeon RX 6700 XT (navi22 LLVM 15.0.6 DRM 3.49
    6.1.0-10-amd64) direct-render: Yes
Audio:
  Device-1: Intel Alder Lake-S HD Audio vendor: ASUSTeK driver: snd_hda_intel v: kernel
    alternate: snd_sof_pci_intel_tgl bus-ID: 00:1f.3 chip-ID: 8086:7ad0 class-ID: 0403
  Device-2: AMD Navi 21/23 HDMI/DP Audio driver: snd_hda_intel v: kernel pcie: gen: 4
    speed: 16 GT/s lanes: 16 bus-ID: 03:00.1 chip-ID: 1002:ab28 class-ID: 0403
  API: ALSA v: k6.1.0-10-amd64 status: kernel-api tools: alsamixer,amixer
  Server-1: PipeWire v: 0.3.65 status: active with: 1: pipewire-pulse status: active
    2: wireplumber status: active 3: pipewire-alsa type: plugin 4: pw-jack type: plugin
    tools: pactl,pw-cat,pw-cli,wpctl
Network:
  Device-1: Intel Alder Lake-S PCH CNVi WiFi driver: iwlwifi v: kernel modules: wl bus-ID: 00:14.3
    chip-ID: 8086:7af0 class-ID: 0280
  IF: wlan0 state: down mac: <filter>
  Device-2: Realtek RTL8125 2.5GbE vendor: ASUSTeK driver: r8169 v: kernel pcie: gen: 2
    speed: 5 GT/s lanes: 1 port: 3000 bus-ID: 07:00.0 chip-ID: 10ec:8125 class-ID: 0200
  IF: eth0 state: up speed: 1000 Mbps duplex: full mac: <filter>
Bluetooth:
  Device-1: Intel AX201 Bluetooth type: USB driver: btusb v: 0.8 bus-ID: 1-14:10 chip-ID: 8087:0026
    class-ID: e001
  Report: hciconfig ID: hci0 rfk-id: 4 state: up address: <filter> bt-v: 3.0 lmp-v: 5.2
    sub-v: 3462 hci-v: 5.2 rev: 3462
  Info: acl-mtu: 1021:4 sco-mtu: 96:6 link-policy: rswitch sniff link-mode: peripheral accept
    service-classes: rendering, capturing, object transfer, audio, telephony
RAID:
  Hardware-1: Intel Volume Management Device NVMe RAID Controller Intel driver: vmd v: 0.6
    port: N/A bus-ID: 00:0e.0 chip-ID: 8086:a77f rev: class-ID: 0104
Drives:
  Local Storage: total: 8.26 TiB used: 1.91 TiB (23.2%)
  SMART Message: Unable to run smartctl. Root privileges required.
  ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Gigabyte model: GP-ASM2NE6200TTTD size: 1.82 TiB
    block-size: physical: 512 B logical: 512 B speed: 63.2 Gb/s lanes: 4 type: SSD serial: <filter>
    rev: EGFM15.0 temp: 36.9 C scheme: GPT
  ID-2: /dev/nvme1n1 maj-min: 259:8 vendor: Gigabyte model: GP-AG42TB size: 1.82 TiB block-size:
    physical: 512 B logical: 512 B speed: 63.2 Gb/s lanes: 4 type: SSD serial: <filter> rev: EGFM15.0
    temp: 37.9 C scheme: GPT
  ID-3: /dev/nvme2n1 maj-min: 259:7 vendor: A-Data model: SX8200PNP size: 1.86 TiB block-size:
    physical: 512 B logical: 512 B speed: 31.6 Gb/s lanes: 4 type: SSD serial: <filter> rev: 42AATCTE
    temp: 34.9 C
  ID-4: /dev/sda maj-min: 8:0 vendor: Seagate model: ST3000DM001-1ER166 size: 2.73 TiB
    block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s type: HDD rpm: 7200 serial: <filter>
    rev: CC25 scheme: GPT
  ID-5: /dev/sdb maj-min: 8:16 type: USB vendor: A-Data model: USB Flash Drive size: 28.91 GiB
    block-size: physical: 512 B logical: 512 B type: SSD serial: <filter> rev: 1100 scheme: GPT
  SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
  ID-1: / raw-size: 249.98 GiB size: 249.98 GiB (100.00%) used: 10.49 GiB (4.2%) fs: btrfs
    dev: /dev/dm-0 maj-min: 253:0 mapped: luks-edc3f1e0-0478-4b99-8bb7-0a1b247494b2
  ID-2: /boot raw-size: 1024 MiB size: 973.4 MiB (95.06%) used: 12 KiB (0.0%) fs: ext4
    dev: /dev/nvme0n1p5 maj-min: 259:5
  ID-3: /boot/efi raw-size: 100 MiB size: 98.4 MiB (98.45%) used: 25.3 MiB (25.7%) fs: vfat
    dev: /dev/nvme0n1p1 maj-min: 259:1
  ID-4: /home raw-size: 249.98 GiB size: 249.98 GiB (100.00%) used: 10.49 GiB (4.2%) fs: btrfs
    dev: /dev/dm-0 maj-min: 253:0 mapped: luks-edc3f1e0-0478-4b99-8bb7-0a1b247494b2
Swap:
  Kernel: swappiness: 15 (default 60) cache-pressure: 100 (default)
  ID-1: swap-1 type: file size: 6 GiB used: 0 KiB (0.0%) priority: -2 file: /swap/swap
Sensors:
  System Temperatures: cpu: 36.0 C mobo: N/A gpu: amdgpu temp: 37.0 C mem: 32.0 C
  Fan Speeds (RPM): N/A gpu: amdgpu fan: 0
Repos:
  Packages: pm: dpkg pkgs: 2428 libs: 1338 tools: apt,apt-get,aptitude,nala pm: rpm pkgs: 0
    pm: flatpak pkgs: 0
  No active apt repos in: /etc/apt/sources.list
  Active apt repos in: /etc/apt/sources.list.d/debian-stable-updates.list
    1: deb http://deb.debian.org/debian bookworm-updates main contrib non-free non-free-firmware
  Active apt repos in: /etc/apt/sources.list.d/debian.list
    1: deb http://deb.debian.org/debian bookworm main contrib non-free non-free-firmware
    2: deb http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
  Active apt repos in: /etc/apt/sources.list.d/mx.list
    1: deb http://tux.rainside.sk/mxlinux/mx/repo/ bookworm main non-free
    2: deb http://tux.rainside.sk/mxlinux/mx/repo/ bookworm ahs
Info:
  Processes: 412 Uptime: 8h 9m wakeups: 13 Memory: 31.15 GiB used: 3.18 GiB (10.2%) Init: SysVinit
  v: 3.06 runlevel: 5 default: graphical tool: systemctl Compilers: gcc: 12 Client: shell wrapper
  v: 5.2.15-release inxi: 3.3.26
Boot Mode: UEFI

[andy]

...
Start by firing up gparted from your Live Boot media, right click the encrypted partition that contains your /boot folder, then choose open encryption. Issue your encryption password, see that the graphic of the drive now shows how much empty space exists on the partition, close gparted and then, with the partition unlocked try fixing boot, unless, after unlocking the encryption you see your drive has no available space. A filled / is a real good way to get a non-booter.
It's also a good idea to run a partition check while in gparted with the right click before you exit because / data corruption is another cause of non-booters.

Never ever worked for me.
See this post,
viewtopic.php?p=735018#p735018
and if you are interested in the many EFI partitions thing, also the post above (#13).

[andy]

So, boot Live system again, ran MX-Boot repair, choose EFI, select LUKS where the root is (It did not ask for root subvolume "@"), and - a surprise! - this time the partition was deemed unlocked - for the first time in my life.
The result was: repaired successfully. At least this claim.

But this repair sent my crippled MX-23 install to oblivion.
(It was able to boot by the help of Boot Rescue Menus -> GRUB-EFI bootloader on the bootmenu of the Live media, but only until this surgery).
Now, I end at grub prompt.
The result is the same if I try to boot by the help of Boot Rescue Menus -> GRUB-EFI bootloader on the bootmenu of the Live media (choose boot partition, which worked before)
And if I choose bootloader from the Mainboard's UEFI boot menu, the grub tells me: no such partition.

[andy]

I could first reinstall windows, which I will do anyway,
and only then install MX-23, to finally have the working MX-23.
(and then temporarily restore EFI partitions on another drives, to be able to boot older OSes).
edit: as I plan to delete old MX later, now I can use Boot Rescue Menus -> GRUB-EFI bootloader, to boot the old MX, and not bother with restoring its EFI partition.

What I want to achieve here now: I would like to learn, how to repair my system, while it is blank and ditchable, to have the skill when I will really need to rescue it.
And of course help with the testing
Right now I can afford something turned wrong by the tinkering.
I will also be very happy if there was nice, easy way to repair encrypted system for the non-technical users, as MX Boot repair looks like.

I would like to start to promote MX-Linux, instead of Linux Mint, to family and friends.
And for me, I would be very grateful, if the Boot repair will support also the encrypted boot, not just root.

[pbear]

I would like to start to promote MX-Linux, instead of Linux Mint, to family and friends.

Come now, Ubuntu's boot repair utility is so unreliable that all guru helpers I've seen say to use it only for generating a report in difficult cases.

FWIW, fiddling around, MX's boot repair tool seems to work fine with encryption, but not something I've actually used as I don't like inscrutable black boxes. Are you interested in learning how to do this yourself? Meaning, command line, sometimes even chroot. If so, I probably can help. Disclaimer, though, that while I've tested repairing boot with encryption, I don't use btrfs.

Then, thinking laterally, another suggestion. To my mind, a rEFInd USB drive is more useful than the various emergency boot tools. Importantly, in Linux, not dependent on the EFI partition(s), as it can boot directly from kernel images. Do you have a small flash drive you could use to give this a try? Or do you perchance use Ventoy? If so, there's a trick for using that as a rEFInd USB drive.

Further, I will mention there's no need to delete EFI partitions. If you temporarily remove the boot/esp flags, the installer won't recognize them as EFI partitions. Will say, though, that while Ubuntu's installer (also used by Mint) always has had a bug as regards which EFI partition it uses, I can't recall having ever having run into that problem with MX's installer.

Last but not least, you don't need a separate EFI partition for each system. That's MBR thinking. A single EFI partition can host any number of boot loaders, limited only by available space. Bear in mind, you can only use one Linux boot loader at a time. From which Grub can boot any number of installed operating systems, using either os-prober or custom configfile entries. Or you could switch to rEFInd as your regular boot manager.

[andy]

I would like to start to promote MX-Linux, instead of Linux Mint, to family and friends.

Come now, Ubuntu's boot repair utility is so unreliable that all guru helpers I've seen say to use it only for generating a report in difficult cases.

Hi @pbear,
first warm thanks for your willingness to help and to provide information.

FWIW, fiddling around, MX's boot repair tool seems to work fine with encryption, but not something I've actually used as I don't like inscrutable black boxes. Are you interested in learning how to do this yourself? Meaning, command line, sometimes even chroot. If so, I probably can help. Disclaimer, though, that while I've tested repairing boot with encryption, I don't use btrfs.

Yes, I am interested, though, I primarily opened this topic to help testing and making MX-Boot repair better.
I have made a guide for myself in March 22, as I needed to repair my MX21.
I will post it here, maybe it will help someone.

But I suppose it is not complete for usage with BTRFS+subvolumes. Maybe some config files edit/ kernel parameters?

Code: Select all

System - repair BOOT
Repair boot on system with encrypted boot and root partitons
https://www.bleepingcomputer.com/forums/t/740193/how-to-repair-or-re-install-grub-using-the-chroot-command/
https://www.reddit.com/r/debian/comments/sb736l/how_to_installreinstall_grubefi_for_debian_1011/
https://forum.mxlinux.org/viewtopic.php?t=68831

Boot into recovery environment (e.g. USB Live)
(same Debian base as installation)
Open encrypted containers needed for boot

sudo cryptsetup luksOpen /dev/nvme0n1p8 boot_crypt -v
sudo cryptsetup luksOpen /dev/nvme0n1p6 rootMX -v
Setup chroot environment and do chroot

sudo mount /dev/mapper/rootMX /mnt/
sudo mount /dev/mapper/boot_crypt /mnt/boot
sudo mkdir -p /mnt/boot/efi/
sudo mount /dev/nvme0n1p1 /mnt/boot/efi

sudo mount --bind /dev /mnt/dev
sudo mount --bind /dev/pts /mnt/dev/pts
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys
sudo mount --bind /run /mnt/run

sudo chroot /mnt
Prompt changes to:
root@mx1:/#

Issue boot reinstall command.
Attention! Some guides promt to use these switches for grub-install:
--removable --no-uefi-secure-boot
Use only these as stated below:

grub-install --efi-directory=/boot/efi/ --bootloader-id=mx --target=x86_64-efi
update-grub
Exit chrooted environment,
dismount everything mounted earlier, in reversed order.

exit

sudo umount /mnt/run
sudo umount /mnt/sys
sudo umount /mnt/proc 
sudo umount /mnt/dev/pts 
sudo umount /mnt/dev

sudo umount /mnt/boot/efi/
sudo umount /mnt/boot
sudo umount /mnt
Reboot.

Then, thinking laterally, another suggestion. To my mind, a rEFInd USB drive is more useful than the various emergency boot tools. Importantly, in Linux, not dependent on the EFI partition(s), as it can boot directly from kernel images. Do you have a small flash drive you could use to give this a try? Or do you perchance use Ventoy? If so, there's a trick for using that as a rEFInd USB drive.

Yes, I even installed my MX-Libretto with Ventoy.
Sounds interesting, I'll investigate.
I suppose Boot Rescue Menus -> GRUB-EFI bootloader on the bootmenu of the Live media can do the same, because I have to point it to /boot partition, not EFI, to be able to boot my crippled MX-23.

Further, I will mention there's no need to delete EFI partitions. If you temporarily remove the boot/esp flags, the installer won't recognize them as EFI partitions. Will say, though, that while Ubuntu's installer (also used by Mint) always has had a bug as regards which EFI partition it uses, I can't recall having ever having run into that problem with MX's installer.

Thank for the info, it didn't come to my mind. Certainly more clever than delete

Last but not least, you don't need a separate EFI partition for each system. That's MBR thinking. A single EFI partition can host any number of boot loaders, limited only by available space. Bear in mind, you can only use one Linux boot loader at a time. From which Grub can boot any number of installed operating systems, using either os-prober or custom configfile entries. Or you could switch to rEFInd as your regular boot manager.

I allways had only one EFI partition, the more thing is only due to adding disks and retaining OSes on the old disk for a while.

If MX developers want me to test MX-Boot repair, I will gladly participate and I am willing to do new reinstalls of Windows and MX to simulate real life source of damage to MX/GRUB.
I want MX-Boot repair to be better.
If not now, I will move on, and solve the situation by myself.
(and maybe then I will reach you for help, @pbear. Thank you again.)

[pbear]

Let's do the simplest thing first, rEFInd on Ventoy. Download the image file (comes in a zip package); see also website. Developer's name is Rod Smith, by the way, well known (gdisk is another of his apps). Unzip; open resulting folder; copy refind-flashdrive-0.14.0.2.img to Ventoy drive. Test. Be aware, there was in the past at least one .img file which wouldn't boot this way, but 0.14.0.2 works for me. To use, boot the same as you would any Ventoy ISO. You should get a rEFInd boot menu listing all installed operating systems on the computer, including Windows, if you have it; Linux OSs might be listed twice, once from Grub and again from kernel images. Toggle with arrow keys to select, then Enter to boot. rEFInd will hand you off to whichever option you select and disappear. The USB drive can be ejected at this point, if you like; which is to say, rEFInd doesn't continue running once the system is booted.

Edit: By the way, the image file can be written directly to USB drive, using dd or similar. Indeed, that what it's for; see website link above. I don't recommend this procedure, though. As a practical matter, it uses the whole drive; the space can be recovered, but it's a PITA. What I recommend instead, if one wants a standalone rEFInd drive (not using Ventoy) is to split the drive: an EFI partition for rEFInd and a second partition which can be used for almost anything, e.g., a Timeshift archive. The link above explains how to install rEFInd to the EFI partition.