Regarding Current Kernel Alert

[zimbodel]

Now I am confused.
Why is it so difficult to understand that
1) I upgraded to the defect kernel
2) I dont want to reboot as a non booting system is a horrror to fix, and I am not convinced I wll have another kernel available through grub at boot. I just CANNOT know.
3) I use/need this system 24/7 so I dont want to risk booting it and rather want to make sure I reinstall a kernel that WILL at least boot.
4) I am not sure if the latest kernel update that appeared after the warning on the site was removed, in the MX-Package manager will in fact remove the defunct kernel and install a new corrected working kernel.
So which is it. Only the developers can know that even if I have an IQ of 205.

It is easy............

[asqwerth]

... and I am not convinced I wll have another kernel available through grub at boot. I just CANNOT know.

Zimbodel, read this
viewtopic.php?p=512349#p512349

And respond.

I was trying to ascertain what kernels you have and to provide you with a backup kernel if you don't have others.

4) I am not sure if the latest kernel update that appeared after the warning on the site was removed, in the MX-Package manager will in fact remove the defunct kernel and install a new corrected working kernel.

I don't believe it does. It was not an update but a removal of the bad update but this only helps those who haven't installed the bad update yet. For those who installed the bad, some won't be affected by it if they're not using modules that have to be rebuilt. For those who were affected, the fix that was posted was to trigger the rebuilding of the modules.

We don't know which group you are because you've never rebooted. I believe you tried the fix but are still afraid of putting it to the test.

So pls respond to my post so we can suggest the appropriate backup kernel for you and you can proceed to reboot with the main kernel, and if it doesn't, the backup.

[asqwerth]

For the rest of the posters, pls don't go off-topic again and confuse him.

[zimbodel]

Thank you for trying to help.

I will update this post for the next few hours as I read through what you require.

-------------------------------------------------------------------------
To the question asked in the link:

Code: Select all

:/boot# ls
config-4.19.0-1-amd64  initrd.img-4.19.0-1-amd64	   initrd.img-4.19.0-5-amd64.old-dkms  System.map-4.19.0-1-amd64  vmlinuz-4.19.0-5-amd64
config-4.19.0-5-amd64  initrd.img-4.19.0-1-amd64.old-dkms  memtest86+.bin		       System.map-4.19.0-5-amd64
grub		       initrd.img-4.19.0-5-amd64	   memtest86+_multiboot.bin	       vmlinuz-4.19.0-1-amd64

Also,
1) yes I did install the defect kernel during an upgrade and I DID NOT reboot since I saw the alert luckily before doing so.
2) I am not sure if the previous kernel will be available in the boot grub interface. If not Im likely screwed.
3) What I try to do is to install the previous kernel again, which I already posted and which is still running as I did not boot yet.
I dont know why it is so impossible to install the previous kernel I replaced during the kernel upgrade ?
4) I also did

Code: Select all

sudo ./rebuild_dkms_packages.sh

as suggested and it executed without any error.
I am not sure what it fixed at all and whether it recovered my previous kernel. What did it actually do ?

5) For reference: Again, the kernel I want to roll back to is.

Code: Select all

# cat /proc/version
Linux version 4.19.0-1-amd64 (stevep@mxlinux.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)) #1 SMP Debian 4.19.5-2~mx17+1 (2018-12-12)

Which is currently still running as I did not reboot.

6) Again, I cannot just simply reboot and check. This server is critical, and I am caught bad by the defect kernel. If I have to run this server for 3 years without booting like this it is fine, until I can get a new kernel update without defect later. Is that an option ?

Most of my servers are running 6-8 months without any need for reboot. It is usually powerblackouts and lightning blackouts that exceeds UPS life that forces a reboot.


I hope this makes it clear.

[asqwerth]

Zimbodel, is there a reason you will not even install a fallback kernel? Or post the Quick System Info of your system hardware?

[added] Since you said you have run the fix, I really don't know how to tell you whether it works or not if you don't try a reboot. YOu asked what the fix did, but I already explained in my last post that it triggers the rebuilding of modules, which the bad kernel update did not do. That should fix the update to the kernel, which was actually fine except it did not rebuild modules after updating


I can't help you with the rollback.

More techie users will need address this.

@Stevo? @fehlix?

[zimbodel]

There are many ways to post system info, which do you prefer ?
I never said that I dont want to install a backup kernel. Obviously I want to, but I dont just want to install a random kernel.
Preferably I want to reinstall the following which is the kernel that was replaced by the defect kernel during the update:

Code: Select all

# cat /proc/version
Linux version 4.19.0-1-amd64 (stevep@mxlinux.org) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)) #1 SMP Debian 4.19.5-2~mx17+1 (2018-12-12)

[JayM]

1. As has already been explained, the problem with the 4.19.0-5 kernel was (past tense) that when it first rolled out in the upgrades, it failed to reinstall dkms kernel modules as it was supposed to do, and as all of the other kernels do. This issue was corrected over a week ago.

The problem affected desktop and laptop PCs that have Broadcom wifi drivers, that use ndiswrapper to load Windows wifi drivers, and/or that have Nvidia's GPU drivers installed instead of using the nouveau driver: primarily the non-free drivers provided by particular hardware or software companies are what were affected. (It probably also impacted users who had VirtualBox or Open VM though no one reported problems with these.) You may see a list of dkms modules by opening MX Package Installer on your server, clicking on the MX Stable Repo tab, and entering "dkms" (without the quotation marks) in the search field. You can then examine that list and determine if your server uses any of them (any installed ones will be greyed out, which is how you can tell.) If not, then even if you currently have the buggy version of the kernel from over a week ago installed your server shouldn't be affected when rebooted.

2. Someone already explained how to manually cause the reinstallation of all dkms modules for that kernel or whatever your current kernel is. This can be done without rebooting. Doing this will prevent the possibility of system problems due to dkms modules not being installed in your kernel so it should then survive a reboot if your server uses any dkms modules. There should be no need to roll back the kernel to an earlier version once all dkms kernel modules have been manually forced to (re)install. This was actually the solution that was given to those who had already updated to the buggy kernel then found that their Nvidia graphics or Broadcom wifi no longer worked after rebooting, when the dkms kernel modules for those failed to be automatically reinstalled after the new kernel was installed. Since the modules didn't get automatically reinstalled during the upgrade process it was simply a matter of issuing a manual command, once, to make them do so. Kernel modules only need to be installed into a kernel once, after that kernel is installed.

3. I already explained how you can lock down your current kernel to prevent it from being automatically upgraded, using Synaptic Package Manager. You can do the same thing for any other individual packages that you wish to "pin" at their current versions.

4. IT industry best practices are, among other things, not to enable automatic updating on production servers until those updates have first been tested by system administrators on an identical sandbox or test server to make sure they don't break anything. If your server's uptime is very important I would advise you to follow that practice as well. Another best practice is of course to have redundant backups, not only of any user data on the server but of its entire operating system as well. MX Snapshot is a good tool for backing up the OS with all installed software. To restore the server to the state it was in when the snapshot was created, sans the user data which would need to be backed up and restored separately, one would simply reinstall into existing partitions from the .iso file that MX Snapshot created.

5. If the concern is that another kernel in the future may have similar issues as the 4.19.0-5 one did (past tense) installing an alternate, known-stable LTS kernel so it will be available to boot with if needed is highly recommended. Just select the kernel that you want (the package name will begin with "linux-image") in MX Package Installer, check the checkbox at the left of it, and click Install. It's as simple as that. Should the current kernel fail to boot, simply reboot and select the backup kernel in the grub boot menu under advanced options. Or do what I do: run MX Boot Options, check the checkbox to the left of "Use flat menus (no submenus", then a list of all available kernels will be displayed in the main grub menu. If you also enable "Enable saving last boot choice" then grub will remember which kernel you selected and use it from then on until you select a different one.

6. If you choose not to do any of these things you may, if you wish, continue running as-is without rebooting until such time as a newer kernel becomes available (but how will you know that it won't cause problems on your server until you test it by (re)booting with that new kernel? That's why testing updates on a test server with identical hardware (especially CPUs, network and video cards), and not allowing untested updates to be automatically installed on production servers, is industry best practice as I said.)

7. If your server is highly mission-critical I would strongly suggest reconfiguring it with a commercial Linux server product from a company that offers paid support plans, guaranteed uptimes and optional remote or onsite system administration, such as RedHat and SuSE, rather than using a free desktop-oriented distribution with only volunteer community support available, as good as that support may be. Neither MX Linux nor Debian Stable for the desktop (not Manjaro, Ubuntu, Mint, Arch, etc.) offer any guarantees whatsoever that a future upgrade absolutely will never break your particular machine. If you require that level of uptime support there are companies that sell it. As the saying goes, "you get what you pay for."

I apologize if this may sound sarcastic or condescending to the OP or anyone else. I assure you, that is not my intention. I have a background in computer support in enterprise organizations and I frequently "rubbed elbows" with server, network and even mainframe administrators, through working on their desktop computers or attending meetings together, and I also have had some server administration experience in a professional setting so I know how these things work and am just telling it like it is in the business and government worlds.

I hope this information is useful and I wish you the best of luck.

[Eadwine Rose]

There are many ways to post system info, which do you prefer ?

In MX Tools run Quick System Info, then click paste in a new post (formatting is done for you already).

[asqwerth]

There are many ways to post system info, which do you prefer ?...

If you go back to my earlier post which I am now linking to for the 2nd time: viewtopic.php?p=512349#p512349

and refer to paragraphs 1 and 2, I said:

I think zimbodel just wants a secondary kernel installed so that when he reboots - which he hasn't done since the 4.19 kernel issue - he has something that works if his 4.19 doesn't.

If I'm correct in my understanding, zimbodel, the first step is to paste the output of MX Tools >> Quick system info here so that we know your hardware specs.

[fehlix]

If OP still needs help, "Quick System Info" is a good to start with.