I have been doing my due diligence and searching this question first but nothing is an exact match. In general side discussions it may even (possibly) be a bug in AV Linux. Running the latest edition as of December 2023.
I have keyboard layouts for US and Swedish keyboards (despite doing the majority of my writing in English, I use a Swedish keyboard as I need those characters too when writing in Swedish).
While I can freely switch between icons for the different nationalities from the panel, it changes nothing.
In KB settings I have chosen a generic 105 key as my Corsair K70 is not listed. Restart (Just in case...

BTW, I am loving the AV MX edition. Reaper runs well on it (as long as I remember to have the thoughtfully supplied Jack Audio connection kit running first), it seems stable and looks great. I even figured out the few customizations I preferred to the Conky, got rid of the annoying (to me) top panel and good to go.
So yeah, just the KB layout not switching thing I guess...Any ideas?
EDIT (so you do not have to scroll):
After going into "xfce4-settings - Keyboard" and checking the link to go to the online xfce page (https://docs.xfce.org/xfce/xfce4-settings/keyboard), it advised to first turn OFF the "Use system defaults" button. That may sound obvious but the program enables you to place your language of choice at the top of the cue anyway which makes it a little counter intuitive. Add to that, the numerous places/apps one can adjust KB settings caused some head scratching for a while.
Hopefully, this may serve to help someone else in a similar situation.