Rsearch, a rofi files search tool

[Girafenaine]

Hello MX Community,

I had long wanted a tool to search files that fits my needs : snappy, with full text search, and sorting files by access time. Albert was nice, but not perfect and took some RAM. Searchmonkey and catfish are efficient, but are in their own window, and don't sort recent files.

Thanks to MX forum and Antix forums I read about rofi and have used it as my app launcher for a few months and it's all that I want, and can be customized. I learnt about find command options, then sort command... then I wrote two tools that could be helpful for anyone on linux :

1. This one-line command, which you can customize to fit your needs. My main addition was to combine "find" command, "sort" command to make recent files appear top of the list, and "rofi" to display them with plain text search :

Code: Select all

 choice=$(find ~ -type f ! -ipath "/home/*/.*" ! -ipath "/home/*/*.default/*" -printf "%A@ %Tx %P\n" | sort -rn | cut -c23- | rofi -dmenu -i -width 70 -l 22 -p rsearch: | cut -c12-) && if [[ $choice != "" ]] ; then thunar ~/"$choice" ; fi 

2. A more complete (but quite simple) script for the same goal. The main point compared to the one-line command is that it uses database files to save "find" command results, and thus it is much snappier if you have got a lot of files. I added some options to modify the default behaviour. You can make the script to regenerate the database each time it is called (as for the one-line command actually). WIth "rsearch -i 120 -hf" you can use a database which is updated only if older than 2 hours, and include hidden files. With "rsearch -i 0 -d -p "~/Documents" you can search only in your Documents folder, and include directories, and generate the database each time you call the script (and leave no database file).

I guess it could be nice for those wanting a tool that you launch with a shortcut (or several shortcuts, eg one for "normal" home files, one to include hidden files), hands on keyboard. Those alreading using rofi will find this tool very easy. Others could be pleased to discover rofi ! If you are used to dmenu, you can use it instead of rofi to display and search files - There are two commented lines in the script to use dmenu.

Here is the full script. If you are interested, please read the included "help section" which I tried to make clear about default behaviour and options (you can copy the script, save it as a file, then launch it with -h option). Please let me know if you have ideas to make it faster or better, if there are bugs, and if it is useful for you. I could make some other versions in the future :

EDITED 04/25/2021 : add "umask 022" in openfile function to let rsearch open files needing a "lock file".
EDITED 18/06/2021 : change the default "open file" function because "thunar name-of-the-file" does not open the file anymore with preferred app, it just opens the containing folder in thunar

Code: Select all

#!/bin/bash

### RSEARCH V1.1 : ROFI FILES SEARCH TOOL
### Sorts files by access time. Displays modification time for each file.
### This script combines find command, sort command and rofi "dmenu" text searching abilities
### It can be used for "recent files" finding and for file searching by path name.
### A dmenu version (instead of rofi -dmenu) is available in commented lines.
### Written by Girafenaine for use on fluxbox and especially MX-Fluxbox 10/04/21
### Released under GNU General Public License V3

### to initialize options and vars-i 0 -d -p "~/Documents
#########################################################################
#to create database files with restricted permissions (rw-------)
umask 177
#default intial path to search in
inpath=$HOME
#default command to open files without any "xdg" rules
editor=geany
#default suffix for database file ("h" added for database including hidden files)
dbpathhid=""
dbpathhid2="h"
#default nb of files to display (modified with -l option)
lines=22
#default width for rofi window (modified with -w option)
width=65
#default way to display files (find command option), with no "initial path" (/home/username/ is not displayed)
print="%P"
print2="%p"
#default excluded patterns in find search, to exclude hidden files and thunderbird profile folders
exclu='! -ipath \/home\/*\/\.* ! -ipath \/home\/*\.default\/* '
exclu2='! -ipath *cache\/* ! -ipath *\/cache* ! -ipath \/home\/*\/\.cache* '
#default file type that are searched for. Find options. f for "real" files, d for directories
type="f"
type2="f,d"
#default indicator for displaying file type or not
dtype=""
dtype2="%y  "
#default mode indicator (1=normal: sorted home files, 2=sorted home files including hidden files, 3=whole filesystem, without any sorting)
mode=1
#default update indicator
update=0
#default interval between two database updates in minutes. "O" means : update at each call.
interval=180

### to catch options
#########################################################################
while [ $# -gt 0 ] ; do

if [[ $1 = "-h" || $1 = "--help" ]] ; then
echo '#########################################################
RSEARCH - ROFI RECENT FILES SEARCH TOOL 
#########################################################

rsearch generates a "database" file listing your files from the initial path of your choice. Your home is the default initial path. It sorts them by access times, and displays modification times (the access time is used for sorting but not displayed), and lets you search through your files thanks to rofi -dmenu.

Thanks to rofi abilities, it is well suited for those wanting a snappy tool that you call with a chosen key and then you directly hit first letters of the wanted file to make it to appear, then return to open it with the appropriate program.

You can use it with "-i 0" option to generate the database each time the script is called. If the database takes too much time to be generated when you call the script, you can separate database generation (at startup, and/or when you call the script but only if the database is too old, and/or every x min/h with a cron task) and database searching. Thus your searching is snappier and easier. Default behaviour is to update if database is older than 3 hours.

rsearch generates a database file for each initial path. Thus you can use it to search in different folders or media, each with its own database file making rofi interface to jump quickly on your screen.

With default settings only your home files are listed, and the hidden files are... hidden. You can include hidden files with -hf option (except for cache folders), or even search though you whole file system with -a (--all) option. In this last mode the script uses "locate" command instead of find, to be quick enough, and files are not sorted by access time.

If you prefer you could use this one-line command that does quite the same as this script, without some options and without the ability to use a database file :
choice=$(find ~ -type f ! -ipath "/home/*/.*" ! -ipath "/home/*/*.default/*" -printf "%A@ %Tx %P\n" | sort -rn | cut -c23- | rofi -dmenu -i -width 70 -l 22 -p rsearch: | cut -c12-) && if [[ $choice != "" ]] ; then thunar ~/"$choice" ; fi

#################################
YOU MAY USE FOLLOWING OPTIONS :

	-u (or --update) : the script only updates the files database and quit. Useful to separate database generation and searching if database generation is long and makes the script not snappy enough.
	-d (or --directories) : directories are searched for as well as files. "f" and "d" are displayed in the files list.
	-hf (or --hidden-files) : home files including hidden files. Only cache folders are hidden. Files are sorted by access time.
	-a (or -all) : global search : initial path is / and hidden files are included. So you can access all files on you system file. This mode uses "locate" command, which comes with its own database management. Files are not sorted by access time.
	-i (or --interval) interval-in-seconds : max time interval between two database update (default : 3 hours). I you use -i 0, then the database will be generated each time you call the script. Furthermore with -i 0 option, the script removes the database files so you do not keep any list of your home files. It could be useful if you have privacy concerns. You can set any other time interval you want.
	-p (or --path) initial-path : default initial path to search from (default is your home). With this option, files are displayed with their short path, without initial path, which is easier to read.
	-mp (or --multiple-paths) " "path1" "path2"..." : several initial paths to search from. With this option, files are displayed with their full path, including initial path. May work with only one path of course. Paths should be quoted together.
	-l (or --lines) number-of-displayed-files : modify number of displayed files in rofi -dmenu (default is 22). One file by line.
	-w (or --width) rofi-window-width : modify rofi window width. It is a rofi option, under 100 it is a percentage of your screen, above 100 it is a number of pixels. Default is 65 (which is 65% of your screen width)
	-h (or --help) : displays this help text

#################################
TO USE THIS SCRIPT :

	0. Install rofi (sudo apt install rofi) if not already installed. You can customize its appearance with rofi-theme-selector.

	1. Copy this script in a file in your chosen place (eg ~/.local/bin/) with the name rsearch, and give it execution permission.
		
	2. Add a shortcut to call the script easily. In fluxbox for example, add in your keys file such a line :
		none F2 :Exec ~/.local/bin/rsearch -i 60 -l 20 (update if older than 1 hour, display 20 lines)
		and you could add for full search including hidden files :
		Shift F2 :Exec ~/.local/bin/rsearch -hf - i 240 -d -l 20 (include hidden files, update if older than 4 hour, display 20 lines and include directories)

In addition you may :

	3.a Add the script with --update option to your startup file, for your files database to be updated at the session beginning. Just add the line : ~/.local/scripts/rsearch --update -your-desired-options
	
	3.b Adapt the interval between two updates. Default behaviour is following : when the script is called it checks how old is the database file. If older than 3 hours it updates the database, else it use the existing one. You can modify this interval with -i option. If you use -i 0, the script updates the database each time it is called, and no database file is kept.
 
	3.b Use cron (needs cron service to be on) to execute it every eg 1 hour : "crontab -e", then add a line such as "*/60 * * * * ~/.local/scripts/rsearch -d -l 18 --update"
' | less
	exit

	elif [[ $1 = "-u" || $1 = "--update" ]] ; then update=1 ; shift
	elif [[ $1 = "-i" || $1 = "--interval" ]] ; then interval=$2 ; shift 2
	elif [[ $1 = "-d" || $1 = "--directories" ]] ; then type=$type2 ; dtype=$dtype2 ; shift
	elif [[ $1 = "-hf" || $1 = "--hidden-files" ]] ; then mode=2 ; exclu=$exclu2 ; dbpathhid=$dbpathhid2 ; shift
	elif [[ $1 = "-a" || $1 = "--all" ]] ; then mode=3 ; shift
	elif [[ $1 = "-p" || $1 = "--path" ]] ; then inpath=$2 ; shift 2
	elif [[ $1 = "-mp" || $1 = "--multiple-paths" ]] ; then inpath=$2 ; print=$print2 ; shift 2
	elif [[ $1 = "-w" || $1 = "--width" ]] ; then width=$2 ; shift 2
	elif [[ $1 = "-l" || $1 = "--lines" ]] ; then lines=$2 ; shift 2
#	elif [[ $1 = "-e" || $1 = "--editor" ]] ; then editor=$2 ; shift 2
	else echo "unkown option \"$1\" ignored" ; shift
fi

done

### management of database files
#########################################################################
inode=$( ls -id $inpath | cut -d" " -f1 )
dbpath="$HOME/.local/share/rsdb_$inode$dbpathhid"

### if interval=0 then it means that you want to regenerate database each time you call the script. So you don't need to keep the database. Furthermore this could be a good thing for privacy and security. So we remove database files, as for temporary files, as soon as exit signal is caught.
if [[ $interval == 0 ]] ; then
	trap "rm -f "$HOME/.local/share/rsdb_$inode" "$HOME/.local/share/rsdb_${inode}h" " EXIT
fi

### function to open files
#########################################################################
function openfile () {
	umask 022
	mimetype=$( xdg-mime query filetype "$1" )
	mimeapp=$( xdg-mime query default "$mimetype" )
	if [[ $mimeapp != "" ]] ; then
	echo "echo "opening $1 through xdg-open with ${mimeapp%\.desktop}" "
	xdg-open "$1" 2>/dev/null &
	else
	echo "echo "opening $fullname with $editor \(no associated app for this mimetype\)" "
	$editor "$1" 2>/dev/null &
	fi
}

#other function for the same goal, using your file manager instead of xdg-open
#function openfile () {
#		umask 022
#		filemgrdesktop=$( xdg-mime query default inode/directory )
#		filemgr=${filemgrdesktop%\.desktop}
#		echo "echo "opening $1 through $filemgr" "
#		$filemgr "$1" &
#}

### mode 3 : whole filesystem with locate command, without any sorting
#########################################################################
if [[ $mode == 3 ]] ; then
	choice=$(locate / | rofi -dmenu -i -width $width -l $lines -p rsearch -keep-right -no-custom )
	#dmenu version
	#choice=$( locate -e / | dmenu -f -b -i -l $lines -p dsearch )
	
	if [[ $choice != "" ]] ; then $(openfile "$choice") ; fi
	exit 0
fi

### modes 1 and 2 : update check, and update if needed (if given interval is over)
#########################################################################
if [[ $update == 0 ]] && [[ -e $dbpath ]] && [[ $(date -r $dbpath +%s) -gt $(($(date +%s)-$interval*60)) ]] ; then
	echo "database $dbpath updated at $(date -r $dbpath +%c), no required update, search with rofi is going to begin"

else
	find $inpath $exclu -type $type -printf "%A@ %Tx  ${dtype}${print}\n" | sort -nr | cut -d" " -f2- > $dbpath
	
	if [[ $update == 1 ]] ; then echo "database $dbpath updated, end of script" ; exit 0 ; fi
	
	echo "database $dbpath updated, search with rofi is going to begin"
fi

### mode 1 and 2 : rofi for searching through database
#########################################################################
choice=$( rofi -dmenu -input $dbpath -i -width $width -l $lines -p "rsearch" -keep-right -no-custom )
#dmenu version
#choice=$( cat $dbpath | dmenu -f -b -i -l $lines -p dsearch )

#remove useless part of the chosen line and call "openfile" function
file=${choice#*\ \ }
fullname=$inpath/${file#[fd]\ \ }
if [[ $choice != "" ]] ; then sudo -u $USER $(openfile "$fullname") ; fi

#if you just used "hidden files" database, use this database to update "home" database to be ready for future calls by sorting out hidden files (containing " \." in their path, and *.default/ files)
#if [[ $mode == 2 ]] && [[ $interval -gt 0 ]] ; then
#	grep -Eiv "[[:space:]\/][\.].*|.*\.default\/.*"  $dbpath > "$HOME/.local/share/rsearch_$inode" &
#fi
	
exit 0

[Helen-Earth]

I just use rofi locate script it's not fast it's instant.

bindsym $mod+shift+s exec --no-startup-id ~/.config/rofi/scripts/rofi-locate

Script

Code: Select all

#!/bin/bash

#       https://www.youtube.com/user/gotbletu
#       https://github.com/gotbletu
# info: rofi-locate is a script to search local files and folders on your computer using the locate command and the updatedb database
# requirements: rofi mlocate
# playlist: https://www.youtube.com/playlist?list=PLqv94xWU9zZ0LVP1SEFQsLEYjZC_SUB3m

xdg-open "$(locate home system | rofi -threads 0 -width 100 -dmenu -i -p "locate:")"

[Girafenaine]

Hello Helen-Earth,

Thanks for your post. This rofi-locate script was one that inspired this rsearch script. rofi-locate has following weaknesses :
- it's not so fast (for me it takes about one second before rofi pops up. rsearch with default settings pops up instantly because it does not generate the database if not too old).
- you cannot easily see if the line is about a file or a folder (especially for file without extensions)
- and files are not sorted ! that was the main reason why I wanted to find something else, and it ends up in this rsearch script. In find it nice to have recent files first, it helps a lot to find the right file.
- if your file has not any xdg-mime type associated, xdg-open will fail to open it (probably will launch your browser, as if it was a URL).

EDIT : - furthermore the "locate" command use its own database, which you have to configure with "updatedb" if you want to have up-to-date data. Without any configuration, you often got removed files. You can get a better result with "locate -e" option, that check for files existence. But here the command is much slower. rsearch can be called directly with an option to give the desired max age of the database file (eg rsearch -i 120 for 2 hours old max).

I tried to overcome these points with rsearch. However I used the same "locate" + rofi combination for a special mode "rsearch --all" when you want to search your full files system, because you can't use "find" on root files as a simple user (and because "find" is too slow on the whole filesystem).

Girafenaine

[manyroads]

rofi is nearly instant for me... if you want to speed up to instant use it with j4-dmenu-desktop

EDIT: Here's a couple of reference links (it's pretty simple to do):
https://classicforum.manjaro.org/index. ... ic=23882.0
https://www.reddit.com/r/i3wm/comments/ ... d_j4dmenu/

[Girafenaine]

Hello manyroads,

I had read your previous post about j4-dmenu-desktop combined with rofi on the forum. Thanks for sharing ! Generally speaking I learnt a lot of things by your posts through antiX and MX forums.

Actually, as I wrote I have already been using rofi as my main app launcher. It is instant on my laptop and I like it (I don't feel any need for j4-dmenu).

The rsearch script I put forward is a tool fo another goal : it is to search files, not app. I guess j4-dmenu can't help here.

And as far as files searching is involved, snappiness is more challenging. rsearch could be a nice solution.

[cedardoc]

I really like your script Girafenaine, but I'd like to customise it by adding my own exclude folders/ filename

The rofi I was using before discovering yours was this:

Code: Select all

choice="$(find ~/SyncthingFolders/allScripts -type f -name "*.sh" | grep -v ".*\/\..*" | grep -v ".*stversions.*" | grep -v ".*Backups.*" | grep -v ".*iddlers.*" | grep -v ".*/temp.*" | grep -v ".*/BibleWiki.*"| grep -v ".*/\.vscode.*" | grep -v ".*/Dendron.*" | grep -v ".*\.tid.*" | rofi -combi -filter "$last" -threads 0 -width 80 -dmenu -i -p "locate:(ctrl-w 2 clear)")"

how would I reproduce the "| grep -v ".*Backups.*" | " parts of mine to work with yours?

I assume it would involve this part, but just not sure how to implement that:

Code: Select all

#default excluded patterns in find search, to exclude hidden files and thunderbird profile folders
exclu='! -ipath \/home\/*\/\.* ! -ipath \/home\/*\.default\/* '
exclu2='! -ipath *cache\/* ! -ipath *\/cache* ! -ipath \/home\/*\/\.cache* '

Thanks,
- Dave

[cedardoc]

Nevermind, figured it out :)

Code: Select all

exclu='! -ipath \/home\/*\/\.* ! -ipath \/home\/*\.default\/* ! -ipath *\/myBackups\/* ! -ipath *\/nas\/* '

[Girafenaine]

Hello cedardoc,

I believe you met success alone

Glad it can be useful to you.

I have to update the script I put forward, because you can run into script not being able to open some doc. Because of the restrictive "umask 177" I set to have "protected" database files, the script creates some "lock files" with same restrictive rights. That prevents the script to effectively open libreoffice docs, for instance. So in the "openfile" function I set a new, more usual 022 mask. And now it works well. I will update the script above, but if you already copy it, you can just add a line as first line in the "openfile" function :

Code: Select all

umask 022

Just in case : grep can handle several banned pattern, which seems much better than to call several times the grep command (especially on big files). Just use the -E option to use extended regular patters (and -i for grep not to be case sensitive). That should work as well :

Code: Select all

 grep -Evi ".*\/\..*|.*stversions.*.*Backups.*|.*iddlers.*|.*/temp.*|.*/BibleWiki.*|.*/\.vscode.*|.*/Dendron.*|.*\.tid.*" 

But the modified "rsearch" script as you did should work well, and snappier, and with recent files fist !

Girafenaine

[Girafenaine]

Hello,

I have just updated the above rsearch script (changed the default "open file" function because "thunar name-of-the-file" does not open the file anymore with preferred app, it just opens the containing folder in thunar).

I keep using it every day as my main file search tool (on a F2 key shortcut).

[josevidal]

Hello, after updating MXLinux Xfce from version 21 to version 23 I see that it is not possible to install Albert through repositories and the version of downloadable binaries that are now available to debian brings telemetry enabled. Rofi and your script work very well, I have only modified the part of excluding folders so that it does not analyze the music. I haven't found a way to copy the file path, something I could do with Albert, but I barely use that function. Thanks for sharing it.