Page 1 of 1
Testing live persistence
Posted: Fri Jan 31, 2014 5:40 pm
by anticapitalista
We need to get some feedback about running live persistence (and remastering) in MX-14. We'll stick to running it live from usb at the moment, but it also works (and IMO it works better) as a frugal install to hard drive.
Running from an installed MX-14-beta2 or live from cd/dvd/usb/frugal
1. Plugin the usb device you want to run MX-14 live with persistence on.
2. Thunar will automatically mount the device. Unmount it in Thunar (just clicking the pop up doesn't work)
3. Run the live-usb application
4. Find the MX-14-beta2.iso, use full disk (4GB is more than enough though), choose FAT or ext2. DO NOT CHOOSE GRUB!
Click on persistence options and keep to default (for testing) for root and home.
5. Once finished, reboot
6. At usb boot menu screen choose root persistence
7. You will be prompted for a new root and user password
8. In desktop, use the meta-installer to install a couple of apps eg Clementine, minitube (nothing too big)
9. DO NOT REBOT FROM THE MENU. For changes to save on next boot, you MUST run the Remastercc application -> Save root persistence.
10. Once that has finished, reboot as usual and again choose root persistence in the menu.
11. You should now have the apps you installed in 8 in the menu.
Re: Testing live persistence
Posted: Fri Jan 31, 2014 7:09 pm
by KBD
Yes Sir, this works. It even remembered my wireless password. Installed Clementine and it was there at next boot.
I take it I could have set a gig or more persistence and it would have had much more room? I just used the defaults.
Good job!
Re: Testing live persistence
Posted: Fri Jan 31, 2014 8:31 pm
by Paul..
That is really, really cool. Installed to 16G flash drive, used metapackage-installer for chromium, qBittorrent, minitube...all working as advertised. Could have install a lot more...
Do I need to always run the RemasterCC app to save all state (browser changes, blah blah), or just to save the installed apps?
Re: Testing live persistence
Posted: Fri Jan 31, 2014 8:34 pm
by BitJam
KBD wrote: I take it I could have set a gig or more persistence and it would have had much more room? I just used the defaults.
The answer is a little tricky. There are 3 different answers depending on how you do things.
Dynamic Persistence
With normal (dynamic) persistence, filesystem changes are stored in RAM. When persist-save is run these changes in RAM get copied to the rootfs persistence file. When you boot with persistence enabled, the changes saved in the rootfs file get copied back to RAM. In this mode the amount of free RAM you have will affect the maximize size of the file system changes that can be stored. If you only use this mode then it makes no sense to have the rootfs persistence file any larger than your RAM (minus what RAM you need for the system). There are two upsides to this approach: 1) it is very fast, and 2) it reduces wear if you are saving the rootfs persistence file on a flash drive or flash usb stick or card.
Static Persistence
In this mode (accessible via the bootloader menu) file system changes are stored directly in the rootfs persistence file. No extra RAM is used. Here, the size of the changes are only limited by the size of the rootfs persistence file. The downside is that it can be a lot slower and it cause more wear on the device that holds the rootfs persistence file. You get to choose static versus dynamic every time you boot. So one approach is to normally use the fast Dynamic Persistence but when you know you are going to do a system upgrade or a bunch of installs then boot into the Static Persistence mode.
Live Remaster (via the RemasterCC)
Live remaster takes all the file system changes whether persistence is enabled or not (and regardless of which type of persistence is enabled) and creates a new squashfs file (called linuxfs.new) that combines all the changes with the existing squashfs file. Since you are compressing the entire file system this can take a little while and max out your CPU(s) while it is running. Also you need enough space on the persistence device to hold the new squashfs file as well as the existing one. The next time you boot the new squashfs file will be used automatically. You can use the "noremaster" boot parameter to keep this from happening. After you boot with the new squashfs file you can use the "rollback" boot parameter to go back to using the original squashfs file.
The great benefit of a Live Remaster is that it frees up the RAM that was being used to hold filesystem changes. If you used Static Persistence to do a system upgrade then you should really do a remaster-live so that you can go back to using Dynamic Persistence or even no persistence at all. But you don't need to enable persistence in order to do a Live Remaster. For example, you can just use it to make a LiveUSB (or LiveHD (frugal)) that has the programs *you* want on it. Although if you want to install a bunch of things in one go then you might need to use Static Persistence until you do the remaster-live so the size of your changes is not limited by your RAM.
Re: Testing live persistence
Posted: Fri Jan 31, 2014 9:21 pm
by BitJam
pcallahan80 wrote:Do I need to always run the RemasterCC app to save all state (browser changes, blah blah), or just to save the installed apps?
To save installed apps you need to use either root persistence (either Dynamic or Static) or RemasterCC. My comment above explains this in more detail.
If you just want to save user preferences and bookmarks and such then any of the above will work and you also have the option of using Home Persistence. This only comes in one form which is similar to Static Root Persistence in that changes are saved immediately to disk and are not stored in RAM. Home Persistence is not fancy. We just mount the homefs persistence file at /home. This is just like mounting a partition at /home but we only use a single file (called homefs) to hold the filesystem instead of an entire partition.
If you want to download a lot of iso files or a lot of movies on your LiveUSB then you should really use Home Persistence and store all of the big files under the /home directory (usually under your own home directory). Otherwise you will run out of RAM.
Summary- If you want to store large files use Home Persistence.
- If you want to install a few small packages you can just use RemasterCC without using persistence.
- If you want to track system changes every time you boot then you need to use either Dynamic or Static Root Persistence.
- Use Static Persistence if you want to install a lot of packages and/or if you don't have much free RAM.
- Use Dynamic Persistence if you want the system to be very fast and are not installing a bunch of packages.
- Finally, use RemasterCC to reset your RAM usage (for filesystem storage) back to near zero. You can do this with or without Root or Home Persistence.
Remember that the LiveUSB is still designed to run on almost any system you plug it into. You get to select which language and which timezone you want to use in the bootloader along with a bunch of other options. Another feature (that I think is) available on the MX LiveUSB is the
F7 Save bootloader menu. This will
automatically save all of the other menu options you have selected as well as any custom boot parameters you have added. This should only appear on LiveUSBs. On LiveCDs you will get an
F7 DPI menu instead. You don't need persistence or RemasterCC for this to work. You just need to have the Live system installed on a device that we can write to. This is probably the easiest and most intuitive way to set the language and timezone on
any Linux system, Live or installed.
Re: Testing live persistence
Posted: Fri Jan 31, 2014 10:36 pm
by Paul..
@BitJam, you're the best! This is a great explanation...and a very cool implementation. We could use both of your posts as documentation 'whole-cloth'...no editing required.
Re: Testing live persistence
Posted: Fri Jan 31, 2014 10:43 pm
by arjaybe
pcallahan80 wrote:@BitJam, you're the best! This is a great explanation...and a very cool implementation. We could use both of your posts as documentation 'whole-cloth'...no editing required.
I agree. +1
Re: Testing live persistence
Posted: Fri Jan 31, 2014 11:25 pm
by Adrian
Yes, we need to add that in manual in the remastering section. Very cool, I didn't even know it can all these options.
Re: Testing live persistence
Posted: Fri Jan 31, 2014 11:49 pm
by KBD
Yes indeed, and thanks BitJam!
Re: Testing live persistence
Posted: Sat Feb 01, 2014 1:32 am
by BitJam
pcallahan80 wrote:This is a great explanation...and a very cool implementation. We could use both of your posts as documentation 'whole-cloth'...no editing required.
If someone could make it so, that would be great. Forum posts tend to get buried and lost over time. There is also this detailed
LiveUSB guide but some if it needs to be updated. For example it doesn't mention Static Root Persistence. Part of it was a guide to implement user programs which have already been created. There are some nice html tables on that page but they don't include Static Root Persistence.
There are many more options in addition to the ones I gave above. For example you can use persistence even when you boot from a LiveCD by simply putting the persistence files on a read-write partition somewhere. You can specify this partition with the pdev= plabel= or puuid= boot parameters. Or you can specify the device more generally by adding either "usb" or "hd" to the persist= boot parameter. But when the device we are booting from is read-only and just general persistence was specified (for example via the persistence entries in the main bootloader menu) then we will automatically look for the persistence files on a device with the label "MX-14-Persist". This way, you can have persistence on with a stock LiveCD and you don't have type in any boot parameters, just make a main menu selection!
There is also a somewhat comprehensive, somewhat outdated
list of Live boot options. Although it is probably best not to overwhelm users up front with all the possible options. Best to keep it simple, at least at first.
If someone does want to document this, they should feel free to PM me and I will glad to let them know what the ground truth is (or is supposed to be) for the MX-14 release. I've been on vacation but now I feel it is time for me to help with the push to get MX-14 out the door (and fix a few remaining bugs before the final release).
Re: Testing live persistence
Posted: Sat Feb 01, 2014 10:19 am
by dolphin_oracle
Persistence seems to be working fine. I do miss the auto-save (for dynamic persistence) from antix, but the use case is a different for mx so not that big a deal. Just got to remember to save...
Re: Testing live persistence
Posted: Sat Feb 01, 2014 10:22 am
by Adrian
dolphin_oracle wrote:Persistence seems to be working fine. I do miss the auto-save (for dynamic persistence) from antix, but the use case is a different for mx so not that big a deal. Just got to remember to save...
Can we hack the shutdown scripts to save automatically?
Re: Testing live persistence
Posted: Sat Feb 01, 2014 10:25 am
by Jerry3904
Adrian wrote:dolphin_oracle wrote:Persistence seems to be working fine. I do miss the auto-save (for dynamic persistence) from antix, but the use case is a different for mx so not that big a deal. Just got to remember to save...
Can we hack the shutdown scripts to save automatically?
If not, needs to be added to the Doc on "Persistence" (not that anybody reads them...)
Re: Testing live persistence
Posted: Sat Feb 01, 2014 10:27 am
by dolphin_oracle
Adrian wrote:dolphin_oracle wrote:Persistence seems to be working fine. I do miss the auto-save (for dynamic persistence) from antix, but the use case is a different for mx so not that big a deal. Just got to remember to save...
Can we hack the shutdown scripts to save automatically?
it only is an issue with the "dynamic" persistence option. the static version saves as you go.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 2:40 pm
by BitJam
dolphin_oracle wrote:Persistence seems to be working fine. I do miss the auto-save (for dynamic persistence) from antix, but the use case is a different for mx so not that big a deal. Just got to remember to save...
Good catch.
I think there may be an easy way to do this. Running MX-14-beta-2 live in virtualbox I see that /usr/local/bin is the first directory on the path. Also, the xfce4-session-logout.desktop file does not specify an absolute path to xfce4-session-logout so you can intercept the logout process with a script at /usr/local/bin/xfce4-session-logout.
For example, (as root) edit a file at that location and add:
Code: Select all
#!/usr/bin
yad --text="No logout for you\!"
Then (as root) run:
Code: Select all
chmod a+x /usr/local/bin/xfce4-session-logout
Then when you hit the logout button, you will get our silly dialog instead of the logout dialog. If you want to really logout then either delete the script or from a terminal run /usr/bin/xfce4-session-logout.
This will be slightly different from what happens in antiX because here you will do the persist-save before you choose which logout option you want. I don't know which way will be better for users but I think they both will work. If this works then we should disable the /usr/local/bin script upon install.
D.O. I will get you a script to test ASAP (or at least real soon now).
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:22 pm
by loco
thanks guys
,
4gig cruzer here, installed opera along with gimp-basic 2.8.10
.
found that i couldn't use fat32? it wouldn't copy all the files/folders. i tried three times twice with toram and once from cd/dvd only, ext2 worked perfectly.
easy...
i've been laid up for a couple of weeks. rehabbing from a supposedly minimal surgery and it feels good to be semi mobile again
.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:37 pm
by BitJam
loco wrote:found that i couldn't use fat32? it wouldn't copy all the files/folders. i tried three times twice with toram and once from cd/dvd only, ext2 worked perfectly.
I'm sorry for being dense. What exactly could you not use fat32 for?
I hope your recovering continues apace.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:44 pm
by loco
thank you sir,
as one of the options to format the usb drive.
thanks again
no, me cave man :).
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:48 pm
by anticapitalista
loco is that using unetbootin or the antix2usb app?
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:56 pm
by loco
big guy,
i tried to follow your post/instructions to the letter. found the iso on my sdb5 partition with live-usb, yes antiX.
thanks
Re: Testing live persistence
Posted: Sat Feb 01, 2014 3:58 pm
by anticapitalista
Ok. It might be that the stick needed to be already partitioned as fat32 for it to work. In that case there is a bug in the app.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 4:02 pm
by loco
:deleted text..
thanks
.. more:
Re: Testing live persistence
Posted: Sat Feb 01, 2014 4:17 pm
by BitJam
I'm made a quick version of xfce4-session-logout that triggers persist-save. Installation:
Code: Select all
gunzip xfce4-session-logout.gz
sudo cp xfce4-session-logout /usr/local/bin
DO NOT overwrite /usr/bin/xfce4-session-logout!
This is alpha code. I wanted to just PM it to D.O. but I didn't see an option to attach files on the send-PM page.
Once it is installed in /usr/local/bin, the program should run when you try to logout from XFCE. If you are not on a Live system or if dynamic root persistence was not enabled then it should fallback to the normal xfce4-session-logout. If you are on a Live system with dynamic persistence enabled then it should obey the settings made by persist-config. If persist-config has not be run then we fall back to semi-automatic mode which means you are prompted before persistence is saved. In all cases (unless there was an error saving persistence) you should end up with the normal XFCE logout screen.
If things don't work, there might be a few more clues available if you run it from the command line:
Code: Select all
/usr/local/bin/xfce4-session-logout
Re: Testing live persistence
Posted: Sat Feb 01, 2014 4:31 pm
by dolphin_oracle
BitJam wrote:I'm made a quick version of xfce4-session-logout that triggers persist-save. Installation:
Code: Select all
gunzip xfce4-session-logout.gz
sudo cp xfce4-session-logout /usr/local/bin
DO NOT overwrite /usr/bin/xfce4-session-logout!
This is alpha code. I wanted to just PM it to D.O. but I didn't see an option to attach files on the send-PM page.
Once it is installed in /usr/local/bin, the program should run when you try to logout from XFCE. If you are not on a Live system or if dynamic root persistence was not enabled then it should fallback to the normal xfce4-session-logout. If you are on a Live system with dynamic persistence enabled then it should obey the settings made by persist-config. If persist-config has not be run then we fall back to semi-automatic mode which means you are prompted before persistence is saved. In all cases (unless there was an error saving persistence) you should end up with the normal XFCE logout screen.
If things don't work, there might be a few more clues available if you run it from the command line:
Code: Select all
/usr/local/bin/xfce4-session-logout
OK, I'll check it out.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 4:53 pm
by anticapitalista
I tried this Bj with success. Cool!
Re: Testing live persistence
Posted: Sat Feb 01, 2014 4:59 pm
by dolphin_oracle
Yep, I agree with anti. As long as you log out from xfce4-session-logout, it works great! No problems going back and forth with static persist (no save dialog, which is good) and dynamic (save dialog, again which is good). And this way if you don't like your changes from that session, you can still use the gui-logout while after canceling the save.
Perhaps a special bash prompt that warns folks not to reboot or shutdown from the cli without running persist-save?
Re: Testing live persistence
Posted: Sat Feb 01, 2014 5:42 pm
by BitJam
We could intercept reboot and shutdown commands like we intercept xfce4-session-logout. Do you want to give that a try? It might be a few hours before I can get to it.
Re: Testing live persistence
Posted: Sat Feb 01, 2014 7:24 pm
by dolphin_oracle
BitJam wrote:We could intercept reboot and shutdown commands like we intercept xfce4-session-logout. Do you want to give that a try? It might be a few hours before I can get to it.
I'm game. You could almost do it with a bash alias I would think.
Re: Testing live persistence
Posted: Sun Feb 02, 2014 12:46 am
by BitJam
Okay I made a little tarball that contains 3 commands and a little Makefile. Unpack the tarball and then:
Code: Select all
cd persist-logout-0.01
sudo make install
You can use "sudo make uninstall" to remove the the programs it installed. The halt and reboot commands are just symlinks to the xfce4-session-logout command. It figures out what to do based on the name of the program. Please check and make sure xfce4-session-logout still works because the program has changed a lot and a bug may have crept in.
Since testing programs that turn off the system can be a royal pain you can give it a "-t" or "--test" parameter and then it will just print out the name of the program it would run instead of running it. But even in test mode, persist-save does get called.
I've made the program flexible so you can create a link to it with the name of almost any program and if we can find the full path to that program then we will run persist-save and then run that program. We might want to lock out this feature before we release the program to the public. For example:
Code: Select all
cd /usr/local/bin
sudo ln -s xfce4-session-logout ls
Then whenever you run the "ls" command, we would first run persist-save. BTW: both "halt" and "reboot" are in /sbin so they are not on the PATH of the normal user but they are on root's path. By putting dopple-gangers in /usr/local/bin, our versions will be on the PATH of normal users. So if a user is used to running /sbin/reboot or /sbin/halt then they will bypass our code and persist-save won't be run.
This might not be the best approach for "halt" and "reboot". I'm open to other suggestions. Also, I was thinking that we might want to bypass calling persist-save if our program is called by root. That way if some system software calls "halt" or "reboot" without specifying a path then we will just mimic the program they are trying to call.
Oh yes. If you are running Live then the "db+" boot parameter automatically changes the Bash prompt to something a bit more fancy. There will be 12 different fancy prompts available, each with a slew of options. Use "prompt-usage" to get instructions. If you forgot to use the "db+" boot parameter then you can get fancy prompts for the current session with "source /usr/local/bin/fancy-prompts.bash" Then use "prompt-fancy" to get the basic fancy prompt. You can also experiment with something like:
Code: Select all
prompt-fancy --title "Remember to run persist-save"
I think that constant reminder would get annoying after a while and it would likely either get removed or ignored.
Re: Testing live persistence
Posted: Sun Feb 02, 2014 9:32 am
by dolphin_oracle
Works pretty well. If you run halt as a regular user, the save still happens before the warning that you have to be superuser to run halt.
reboot function also works.
regarding bypassing with /sbin/halt and /sbin/reboot ---I don't know how big a problem that is...I'm not sure I would worry about it.
One other point...if you want to talk about old-school commands, you might want to also link shutdown (as in shutdown -h now).
Re: Testing live persistence
Posted: Sun Feb 02, 2014 9:36 am
by Jerry3904
If we need to add something to the doc on "Persistence," please post revised/additional copy.
TIA
Re: Testing live persistence
Posted: Sun Feb 02, 2014 3:08 pm
by BitJam
dolphin_oracle wrote:Works pretty well. If you run halt as a regular user, the save still happens before the warning that you have to be superuser to run halt.
Ah. That's a bug in my program. I will fix it. Thanks.
One other point...if you want to talk about old-school commands, you might want to also link shutdown (as in shutdown -h now).
My thought was to leave "shutdown" alone so it gives people a way of shutting down from the CLI that bypasses my bypasses. I figured that if a program was going to call a program to shutdown or reboot then it was more likely to call "shutdown" rather than "halt" or "reboot". Also, both /sbin/reboot and /sbin/halt are in /etc/sudoers.d/antixers so normal users are allowed to run those commands without entering a password but /sbin/shutdown is not. OTOH, please feel free to try this:
Code: Select all
ln -s xfce4-session-logout /usr/local/bin/shutdown
You can then see if it interferes with anything. It will work better in the next version that adds "sudo". That was the bug above, I forgot to use "sudo".
Jerry3904 wrote:If we need to add something to the doc on "Persistence," please post revised/additional copy.
I will post a blurb about intercepting commands. I will leave it to you (or others) to decide if and where it should go in the docs. PLMK if this is not what you meant.
Re: Testing live persistence
Posted: Sun Feb 02, 2014 4:16 pm
by BitJam
I've attached version 0.02. It adds "sudo" when the real path to the command contains "/sbin". It also passes on command line parameters. For example "reboot -t x y z" eventually results in this output:
Re: Testing live persistence
Posted: Sun Feb 02, 2014 7:12 pm
by dolphin_oracle
Nice work! Works really well.
Re: Testing live persistence
Posted: Mon Feb 03, 2014 5:04 am
by BitJam
Jerry3904 wrote:If we need to add something to the doc on "Persistence," please post revised/additional copy.
Larry Wall wrote:The following two statements are usually both true:
- There's not enough documentation.
- There's too much documentation.
IOW, please feel free to heavily edit this to make it more suitable for users (from your perspective). Or ask me to trim it down. I often tend to go into too much technical detail. I think you want a users manual, not a technical reference manual. It is possible that users don't need to know any of the information I give below.
BTW: I think users/testers/anti should decide exactly which commands we highjack for persist-save. Certainly we should highjack xfce4-session-logout. I'm not certain about halt, reboot, shutdown, or poweroff.
Dynamic Root Persistence and Persist-Save
When Dynamic Root Persistence (DRP) is enabled, filesystem changes are stored in RAM just like on a Live system without persistence. In order for your changes to be saved, they must be copied to a disk-based filesystem. That is what the persist-save program does. If DRP is enabled then you are always free to run persist-save via the RemasterCC menu. You can also configure persist-save to run every time the system shuts down. If you select automatic-mode then the saving will happen without first asking if you want to proceed. In semi-automatic mode, you will be asked if you want to save or not. In manual mode you have to run persist-save yourself in order for your changes to be saved.
In order for the automatic and semi-automatic modes to work, we created a xfce4-session-logout program in /usr/local/bin. The actual xfce4-session-logout program is in /usr/bin. Since /usr/local/bin has higher priority in the $PATH variable, our version gets used when xfce4-session-logout is called without an absolute path specified ("xfce4-session-logout" instead of "/usr/bin/xfce4-session-logout"). This is exactly how it is called from within the XFCE GUI. If DRP is not enabled then our program just calls the real xfce4-session-logout and the system behaves normally. If DRP is enabled then we will first call persist-save (if the save mode is automatic or semi-automatic) before we call the real logout program.
If the $PATH variable gets changed so /usr/bin has higher priority than /usr/local/bin or if you call the real program directly as /usr/bin/xfce4-session-logout then you will bypass our version of the program and persist-save will not get run automatically at shutdown. Our mechanism might also get broken if the /usr/share/applications/xfce4-session-logout.desktop file gets changed to use the full path to the real xfce4-session-logout.
We have done the same thing with the "reboot" and "halt" programs. If DRP is enabled and you run either "reboot" or "halt" from the command line then we will first call persist-save before shutting down or rebooting. If you use the full path to the commands: /sbin/root, /sbin/halt then you will bypass our versions and persist-save won't be run automatically.
Static Root Persistence and Live Shutdown
One of the beauties of Dynamic Root Persistence is that persist-save is run while the system is fully up and running so it is trivial to make sure the rootfs file and the partition it resides on are cleanly unmounted. We don't have this luxury with Static Root Persistence (SRP) because the root file system must be unmounted before we unmount the rootfs file and the partition it lives on. We solve this problem by running pivot_root in the /init script in the initrd (technically, the initramfs) of the Live system and then running switch_root during shutdown. On the Live system there is a small set of tools based on busybox that is stored entirely in RAM. It lives in the /live directory. The switch_root command turns the filesystem inside-out and makes what was originally /live into the root filesystem. This gives us the fulcrum we need to unmount the normal root filesystem and still have access to common Linux/Unix commands for cleanly unmounting the rootfs file and the partition it lives on. We always run the Live shutdown this way even if there is no persistence enabled. Because of the /live fulcrum, this method of shutting down is more robust than the standard Debian shutdown procedure. Perhaps someday this more robust way of shutting down will become common practice.
We have our own Live versions of the sendsigs and umountfs shutdown scripts that initiate the process described above. They both call the program /live/bin/umount-live. You can see it in action in the first screen shot below. More detail is available by using the
uverb= boot parameter. The second screen shot shows the output with
uverb=7. The default value is 6. You can use the
ubp= boot parameter to set breakpoints during the Live shutdown. setting
ub=a (lowercase "a" for almost all). This can even allow you to run a shell after all the filesystems have been unmounted.
Re: Testing live persistence
Posted: Sun Feb 09, 2014 10:47 pm
by taydui
I found that persistent feature works well for both static and dynamic but remaster one terminates early with null error and nothing created.
Re: Testing live persistence
Posted: Sun Feb 09, 2014 11:21 pm
by BitJam
taydui wrote:I found that persistent feature works well for both static and dynamic but remaster one terminates early with null error and nothing created.
Thanks! There should be a log file at /var/log/live/remaster.log (or something like that). Could you post it please? Maybe we should move this to a new thread?
If you can't get the log file, then I'd like to know more specifically when/where in the remaster-live process it terminated early. You can try running "sudo remaster-live -c -v" in a terminal window. The -c option tells it to stay in the terminal (--cli) and the -v option (--verbose) tells it to print a little more. Still, most of the info I want should be in the log file.
Has anyone else tried to remaster the Live system?
If a non-colorized version of the log file is not available then a simple trick to get rid of the colors is to cat the file in a terminal window and then copy and paste from there. gzipping the log file and attaching it is another good solution. If you need to post from a different machine then you can copy the log file to your usb stick (assuming you are running a LiveUSB) with:
Code: Select all
sudo cp /var/log/live/remaster-live.log /live/boot-dev
IOW, the LiveUSB you booted from is mounted at /live/boot-dev.
Thanks again.
Re: Testing live persistence
Posted: Mon Feb 10, 2014 12:08 am
by taydui
Here you are!
Thank you for your respond, i tried it again.
1st try using remaster gui: failed!
2nd try using command line: WOW it worked!
I attached 2 log files so you can check.
Re: Testing live persistence
Posted: Mon Feb 10, 2014 3:18 am
by BitJam
@taydui, there is a bug in the antiX-common.sh library that had hard-coded "roxterm" as the terminal emulator. The crash happened because roxterm does not exist on MX-14. The fix is to change line 21 of that file to and replace "roxterm" with "/etc/alternatives/x-terminal-emulator". Here is a patch which also fixes the GUI_FILER:
Code: Select all
--- /live/linux/usr/local/lib/antiX/antiX-common.sh 2012-12-13 05:52:01.000000000 -0500
+++ /live/aufs/usr/local/lib/antiX/antiX-common.sh 2014-02-10 03:12:43.308476250 -0500
@@ -18,11 +18,11 @@
# This is needed for restarting
CMDLINE_ARGS=("$@")
- GUI_TERM="roxterm"
+ GUI_TERM="/etc/alternatives/x-terminal-emulator"
TERM_OPTS="--geometry=+50+50 -e"
TERM_TITLE_OPT="--title"
- GUI_FILER="rox"
- FILER_OPTS="--new -d"
+ GUI_FILER="thunar"
+ FILER_OPTS=""
ARCHIVE="archive"
I've also attached the patch as a file.
Again, thanks for your help, taydui.
Re: Testing live persistence
Posted: Mon Feb 10, 2014 4:28 am
by taydui
Everything works well now.
Thank you !
Re: Testing live persistence
Posted: Mon Mar 03, 2014 10:20 am
by malspa
In the documentation, under "How do I Create Persistence Files?", it says:
If using UNetbootin you will have to set up persistence once running from the stick/device.
I have MX-14 already on a flash drive, done with UNetbootin. I'd like to set up home persistence, which I think should be fairly simple, but it isn't clear to me how to do this. The info under "Using UNetbootin" describes how to get the system onto the flash drive with UNetbootin (although it mentions "antiX live" instead of "MX-14"), but doesn't go on to explain how to set up home persistence once running from the flash drive. Are the steps for doing this explained elsewhere?
Re: Testing live persistence
Posted: Mon Mar 03, 2014 1:08 pm
by BitJam
I think the documentation is not clear on this important point.
You need to go the the Remaster Control Centre (RemasterCC) and then select "Setup Live Persistence". In the menus you can get there via Applications --> Accessories --> RemasterCC.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 1:41 pm
by malspa
Thanks, Bitjam. Do I need to boot with the "Home Persistence" boot option first?
Re: Testing live persistence
Posted: Mon Mar 03, 2014 1:49 pm
by BitJam
No, you need to first create the home persistence file, then reboot and enable home persistence. The file needs to exist before home persistence can be enabled.
If you want to normally boot with home persistence enabled then when you select "Home Persistence" in the main bootloader menu, also select "F7 Save" --> both. This should adjust the bootloader default to whatever is currently selected. If "F7 Save" does not appear near the bottom-right corner of the screen then this is a minor bug which we can fix (if you report it). In that case, manually enter the boot parameter "gfxsave=both".
Re: Testing live persistence
Posted: Mon Mar 03, 2014 2:26 pm
by malspa
BitJam wrote:No, you need to first create the home persistence file, then reboot and enable home persistence. The file needs to exist before home persistence can be enabled.
After homefs has been created, reboot using the "Home Persistence" boot option, or the default MX-14 boot option? And I notice that if I boot with the "Home Persistence" boot option I see the message "Could not mount file /live/boot-dev/antix/homefs..." or something like that. And, once back into the system, how exactly do I enable home persistence?
Re: Testing live persistence
Posted: Mon Mar 03, 2014 2:55 pm
by BitJam
malspa wrote:I notice that if I boot with the "Home Persistence" boot option I see the message "Could not mount file /live/boot-dev/antix/homefs..." or something like that. And, once back into the system, how exactly do I enable home persistence?
The problem is sometimes explained in the "..." part. Please post the /var/log/live/bootloader.log file. If you change the extension to .txt then you can attach it to a post. I'd also like to see the output of the command:
If you have a LiveCD, make sure it is not inserted in a drive when you boot from the LiveUSB.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 3:23 pm
by malspa
BitJam wrote:The problem is sometimes explained in the "..." part. Please post the /var/log/live/bootloader.log file.
/var/log/live/bootstrap.log, maybe?
Code: Select all
demo@mx1:/var/log/live
$ ls -al
total 16
drwxr-xr-x 2 root root 100 Mar 3 14:23 .
drwxr-xr-x 14 root root 220 Mar 3 14:23 ..
-rw-r--r-- 1 root root 932 Mar 3 14:23 antiX-init.log
-rw-r--r-- 1 root root 2917 Mar 3 14:23 bootstrap.log
-rw-r--r-- 1 root root 4682 Mar 3 14:23 bootstrap.log.color
Here's the last part of /var/log/live/bootstrap.log:
Code: Select all
Start to mount persist file homefs at /live/aufs/home
Could not mount file: /live/boot-dev/antiX/homefs for home persistence
Not enabling home persistence.
Run custom code after mounting homefs: /live/custom/MX-14/8.sh
Creating /media tmpfs at: /live/aufs/media (size=1 MB)
/live/bin/mount -t tmpfs -o size=1m tmpfs /live/aufs/media
Creating /live tmpfs at: /live/aufs/live (size=10 MB)
/live/bin/mount -t tmpfs -o size=10m tmpfs /live/aufs/live
The /init bootstrap took 6.36 seconds.
BitJam wrote:If you change the extension to .txt then you can attach it to a post.
Contents of the entire file attached.
BitJam wrote:I'd also like to see the output of the command:
Code: Select all
$ df -Th /live/boot-dev
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 vfat 7.5G 1.7G 5.8G 23% /live/boot-dev
Re: Testing live persistence
Posted: Mon Mar 03, 2014 3:28 pm
by anticapitalista
I can replicate the error.
Code: Select all
Mounting aufs at /live/aufs
/bin/mount -t aufs -o br:/live/aufs-ram:/live/linux=ro /live/aufs /live/aufs
symlink: /bin /boot /etc /lib /root /run /sbin /tmp /usr /var
Device: /live/boot-dev/antiX/homefs
Mount count: 0
Max mount count: 65535
Last checked: Mon Mar 3 15:19:03 EST 2014
Magic number: 0xEF53
Check interval: 0 (days)
----------------------------------------------------------------------
Start to mount persist file homefs at /live/aufs/home
Could not mount file: /live/boot-dev/antiX/homefs for home persistence
Not enabling home persistence.
Run custom code before prepare switch_root: /live/custom/MX-14/8.sh
Creating /media tmpfs at: /live/aufs/media (size=1 MB)
/live/bin/mount -t tmpfs -o size=1m tmpfs /live/aufs/media
Creating /live tmpfs at: /live/aufs/live (size=10 MB)
/live/bin/mount -t tmpfs -o size=10m tmpfs /live/aufs/live
Code: Select all
df -Th /live/boot-dev
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdc1 ext2 3.8G 703M 3.1G 19% /live/boot-dev
Re: Testing live persistence
Posted: Mon Mar 03, 2014 5:15 pm
by anticapitalista
malspa are you using beta2?
This might work for the version you are using. If it doesn't we know how to fix it for the next build.
At boot menu, choose home persistence and then type bp=2
This will quickly dump you to a shell.
Then type /bin/load-live-modules, press Enter and you might get a load of text scrolling. Then type exit.
If you get an error about load-live-modules not present, then you will have to wait a little longer for a fix.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 5:46 pm
by malspa
Yes, beta2.
Okay, I followed those instructions. Now I'm back into the system. In /var/log/live/bootstrap.log, it says:
Code: Select all
Start to mount persist file homefs at /live/aufs/home
Enabling home persistence.
Run custom code after mounting homefs: /live/custom/MX-14/8.sh
So, home persistence is enabled, right?
Re: Testing live persistence
Posted: Mon Mar 03, 2014 5:51 pm
by anticapitalista
Yes.
Unfortunately, you will have to keep using bp=2 /bin/load-live-modules hack until next release.
Alternatively, use root persistence. (It also saves changes to home)
Re: Testing live persistence
Posted: Mon Mar 03, 2014 6:14 pm
by malspa
Ah, yes. Okay, so here are my notes on creating home persistence for MX-14 beta2 on a Unetbootin flash drive:
Boot into the system with the default boot option > Menu > RemasterCC > enter password ("root") > click the "Set up live persistence" button > select action "Home: Create Home Persistence" > click "OK" > select size > File System Type (ext2 is recommended) > click "OK" > "About to create homefs file... Shall we proceed?" Clicked "Yes".
There is no confirmation message showing that this completes successfully.
Reboot and select the Home Persistence boot option. At the end of the boot code line, type bp=2, then press Enter. At the shell prompt, type /bin/load-live-modules, then press Enter, then type exit. Boot persistence is enabled.
Note: You'll then have to use bp=2 and /bin/load-live-modules each time you want to boot into home persistence mode.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 7:50 pm
by BitJam
@malspa, I believe that all you need do at bp=2 is to wait a second or two. In fact, I tested it here and it seemed to work if I just add the ALOAD boot parameter. Could you please see if this works for you? ALOAD must be in all uppercase. There is a lowercase "aload" but that should not affect what happens.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 8:40 pm
by malspa
BitJam wrote:@malspa, I believe that all you need do at bp=2 is to wait a second or two. In fact, I tested it here and it seemed to work if I just add the ALOAD boot parameter. Could you please see if this works for you? ALOAD must be in all uppercase. There is a lowercase "aload" but that should not affect what happens.
Not sure that I got this right. I added bp=2 ALOAD at the end of the line, so I used:
Code: Select all
quiet persist=home! antiX=MLX bp=2 ALOAD
Leaves me at the shell prompt.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 9:11 pm
by BitJam
Thanks. I have a fix in the works but it involves a change in the code.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 9:55 pm
by BitJam
@malspa, would you please try using load=loop and no other extra boot parameters. No bp=2 boot parameter.
One trick I use is to set bp=8. This will drop you to a shell after the home persistence file has been mounted but before we've started MX. Then you can either Ctrl-d to continue and start MX or use the command "safe-shutdown" to unmount everything cleanly and then shut down. The only reason to use this trick is to save time. If you want to start up MX anyway then there is no reason to use it. Since breakpoint 8 happens after we try to mount the home persistence file, it won't interfere with the test.
Re: Testing live persistence
Posted: Mon Mar 03, 2014 10:30 pm
by malspa
BitJam wrote:@malspa, would you please try using load=loop and no other extra boot parameters. No bp=2 boot parameter.
Nice! That worked fine!
Re: Testing live persistence
Posted: Mon Mar 03, 2014 11:47 pm
by BitJam
malspa wrote:There is no confirmation message showing that this completes successfully.
I just made a rootfs (and a homefs) using the GUI version of persist-makefs. I've attached a screen-shot of what is shown after I create a file which does show that the file got created as well as other useful information.
Your comment indicates that this feedback is insufficient for letting the user know the operation was a success. I can add an additional window for each operation that says "xyzzy completed successfully" with an "okay" button or something like that. But I draw the line at dancing paper clips!
What do other people think? Do you think we should add confirmation windows for all the operations?
Edit: BTW the persist home bug was due to a change in how the kernel handles /dev devices and was only triggered if someone enabled home persistence without enabling root persistence. Kudos to
malspa for finding it!
Re: Testing live persistence
Posted: Tue Mar 04, 2014 6:19 am
by Jerry3904
What do other people think? Do you think we should add confirmation windows for all the operations?
An "OK" box is pretty nice for the end user.
Re: Testing live persistence
Posted: Tue Mar 04, 2014 6:24 am
by BitJam
Okay!
Re: Testing live persistence
Posted: Thu Mar 06, 2014 10:25 pm
by BitJam
Thanks Jerry and malspa! I've added more feedback after you create a file in persist-makefs. I've attached the file persist-makefs.txt which contains these changes. If you want to have it replace the current version then do this:
Code: Select all
sudo cp persist-makefs.txt /usr/local/bin/persist-makefs
sudo chmod a+x /usr/local/bin/persist-makefs
It would be great if you (or someone else) could test it and give me feedback.
BTW: you can also run it in CLI mode if you start it from a terminal window and use the -c (or --cli) option:
I noticed that the list of devices was sometimes broken. I think I fixed it. The changes are in the library file antiX-common.sh. You can install this file with the command:
Code: Select all
sudo cp antiX-common.sh.txt /usr/local/lib/antiX/antiX-common.sh
You get to the device selection menu from the " Device: Change Persistence Device" entry in the main menu. Sometimes it will be the first thing you see if you are running from a LiveCD or from an installed system.
Re: Testing live persistence
Posted: Sat Mar 08, 2014 8:18 pm
by malspa
BitJam wrote:Thanks Jerry and malspa! I've added more feedback after you create a file in persist-makefs. I've attached the file persist-makefs.txt which contains these changes. If you want to have it replace the current version then do this:
Code: Select all
sudo cp persist-makefs.txt /usr/local/bin/persist-makefs
sudo chmod a+x /usr/local/bin/persist-makefs
It would be great if you (or someone else) could test it and give me feedback.
I ran these two commands in MX-14 default live session. After running each one, I was not returned to the command prompt, so after waiting awhile, I simply closed the terminal window each time. Probably not good.
Anyway, then I followed the same steps I followed before:
Menu > RemasterCC > entered password ("root") > clicked the "Set up live persistence" button > selected action "Home: Resize Home Persistence" > clicked "OK" > selected size > File System Type (ext2 is recommended) > clicked "OK" > "About to create homefs file... Shall we proceed?" Clicked "Yes".
The only difference from before was that instead of the action "Home: Create Home Persistence" there was the option "Home: Resize Home Persistence", so I chose that option.
Again, no confirmation message showing that this completed successfully.
Reboot and select the Home Persistence boot option. At the end of the boot code line, type bp=2, then press Enter. At the shell prompt, type /bin/load-live-modules, then press Enter, then type exit.
However:
Code: Select all
Could not mount file: /live/boot-dev/antiX/homefs for home persistence
Not enabling home persistence.
I am also unable to boot into home persistence using load=loop at the end of the boot code, or with nothing added to the end of the boot code.
Hm. I renamed the homefs.old file as homefs and tried to boot into home persistence again, no luck. And RemasterCC shows homefs as "broken." So put the "new" homefs back, still no luck. I think I've made a mess of things, so I guess I'll wipe this flash drive and run Unetbootin again and put the MX-14 beta2 .iso back on it and start all over.
Re: Testing live persistence
Posted: Sat Mar 08, 2014 8:25 pm
by malspa
I'll just hold off and check out the RC1, I guess.
Re:persistence... "anti," "mr. jammer"
Posted: Thu Mar 13, 2014 12:40 pm
by loco
anticapitalista wrote:We need to get some feedback about running live persistence (and remastering) in MX-14. We'll stick to running it live from usb at the moment, but it also works (and IMO it works better) as a frugal install to hard drive.
Running from an installed MX-14-beta2 or live from cd/dvd/usb/frugal
1. Plugin the usb device you want to run MX-14 live with persistence on.
2. Thunar will automatically mount the device. Unmount it in Thunar (just clicking the pop up doesn't work)
3. Run the live-usb application
4. Find the MX-14-beta2.iso, use full disk (4GB is more than enough though), choose FAT or ext2. DO NOT CHOOSE GRUB!
Click on persistence options and keep to default (for testing) for root and home.
5. Once finished, reboot
6. At usb boot menu screen choose root persistence
7. You will be prompted for a new root and user password
8. In desktop, use the meta-installer to install a couple of apps eg Clementine, minitube (nothing too big)
9. DO NOT REBOT FROM THE MENU. For changes to save on next boot, you MUST run the Remastercc application -> Save root persistence.
10. Once that has finished, reboot as usual and again choose root persistence in the menu.
11. You should now have the apps you installed in 8 in the menu.
a little help?
i used my 64bit antix 'jessie' to create the live usb with 1.5g for root.
followed the directions and all worked pretty swell, which brings us to current #8. i used the meta installer for chromium, gimp and 'drum roll' ... kde-light ... i think it worked, but am a little skiddish with the rebooting thing, ahem.
much gratitude to all involved
usb1.jpg
any tips on how to select kde.
ps why kde would seem so important has to do with the situation of using ati opensource drivers (live) and hdmi connector, whoo...
pps having personal passwords seems natural and a nice feature imo, once i got used to it
.
Re: Testing live persistence
Posted: Thu Mar 13, 2014 1:02 pm
by malspa
I'll add that home persistence worked fine for me in RC1. Haven't downloaded RC2, but I'm assuming that it's fine there as well.
Re: Re:persistence... "anti," "mr. jammer"
Posted: Thu Mar 13, 2014 1:24 pm
by BitJam
loco wrote:followed the directions and all worked pretty swell, which brings us to current #8. i used the meta installer for chromium, gimp and 'drum roll' ... kde-light ... i think it worked, but am a little skiddish with the rebooting thing, ahem.
The persist-save window in your screen shot shows that about 750 Meg are used in your rootfs. This sounds about right. How much RAM do you have? I ask because this will use up 750 Meg of RAM until you remaster.
If you have plenty of free space on your LiveUSB, I suggest that you go to RemasterCC and remaster. This takes a little while because it compresses your entire file system to make a new linuxfs file. Then when you reboot, the 750 Meg of RAM will be freed. If you run "free -m" now, you will see that "cached" is over 750. That is where the RAM used for storing file system changes is accounted for. If you reboot with Dynamic persistence enabled then over 750 Meg will still be cached. But if you reboot with "Static root persistence" then that 750 Meg of RAM won't be used (but your system might run very slowly). If you do the remaster-live before rebooting then the 750 Meg won't be used *and* your system will still be fast.
Recap:
Code: Select all
Plain LiveCD/USB: Runs fast, but fs changes use RAM, no saving
Dynamic Persistence: Runs fast, but fs chagnes use RAM, changes saved
Static Persistence: Rusn slow, fs changes don't use RAM, changes saved
Remaster Live: Runs fast and fs don't use RAM, but the reamaster is expensive
So after installing a bunch of big packages, I suggest that you do a remaster-live to move the changes out of RAM and into the linuxfs file.
If you want to get familiar with this stuff, you can try the following exercises before you remaster:
- Reboot without persistence. Not much RAM is cached but all your changes are gone
- Reboot with Dynamic persistence. Your changes are back but lots of RAM is used
- Reboot with Static persistence. Your changes are back, not much RAM is cached but system is slow.
- Run remaster-live then reboot. Your changes are there, not much RAM is used, and your system is fast.