For a while, I've been borrowing, editing, and appending those extra lines from my MX PC to the /etc/hosts on my non-MX desktop computer. Curiosity got the better of me, so I studied the antiX Advert Blocker script found at /usr/local/bin/block-advert.sh. As a result, I created a pair of scripts, one to retrieve four files that contain lists of sites to block, then a second script to combine, normalize and sort those four source files.
I'm posting the code for the two scripts here for your use and enjoyment. They should run on any version of Linux or Unix. Neither of these files make any changes to your /etc/hosts, but you may use the output as a file to append to your /etc/hosts if you wanted to do so to block ads. Edit to suit your tastes.
Rather than sending ad servers to 127.0.0.1, I've been using 0.0.0.0, which you will find in the second script.
Code: Select all
#!/bin/sh
# /etc/hosts advert blockers used in antiX-advert-blocker
# Runs as ordinary user and requires directories /home/username/hosts and
# /home/username/hosts/hosts.bak/ to pre-exist as working directories.
# Adjust script to conform to your personal layout differences or changes.
# lines beginning with mv command are just backing up previously retried files
#
# Also: https://github.com/Ultimate-Hosts-Blacklist/Ultimate.Hosts.Blacklist
#
# https://hosts-file.net/
#mv ~/hosts/hosts-hphosts-file.txt ~/hosts/hosts.bak/
#wget http://hosts-file.net/ad_servers.txt -O ~/hosts/hosts-hphosts-file.txt
#https://pgl.yoyo.org/adservers/
mv ~/hosts/hosts-yoyo.txt ~/hosts/hosts.bak/
wget 'https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext' -O ~/hosts/hosts-yoyo.txt
#
#http://winhelp2002.mvps.org/hosts.htm
#mv ~/hosts/hosts-mvps.txt ~/hosts/hosts.bak/
#wget http://winhelp2002.mvps.org/hosts.txt -O ~/hosts/hosts-mvps.txt
#
#https://someonewhocares.org/hosts/
mv ~/hosts/hosts-someonewhocares.src ~/hosts/hosts.bak/
wget http://someonewhocares.org/hosts/zero/hosts -O ~/hosts/hosts-someonewhocares.src
#Extract from hosts-someonewhocares.src only those lines beginning 0.0.0.0
sed -n '/^0\.0\.0\.0/p' hosts-someonewhocares.src > hosts-someonewhocares.txt
#http://stevenblack.com/
#https://github.com/StevenBlack/hosts
mv ~/hosts/hosts-stevenblack.src ~/hosts/hosts.bak/
wget https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts -O ~/hosts/hosts-stevenblack.src
#Extract from hosts-stevenblack.src only those lines beginning 0.0.0.0
sed -n '/^0\.0\.0\.0/p' hosts-stevenblack.src > hosts-stevenblack.txt
#https://github.com/AdAway/adaway.github.io/
mv ~/hosts/hosts-adaway.txt ~/hosts/hosts.bak/
wget https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt -O ~/hosts/hosts-adaway.txt
#https://blocklist.site
mv ~/hosts/hosts-ads.txt hosts.bak/
wget https://blocklistproject.github.io/Lists/ads.txt -O ~/hosts/hosts-ads.txt
#https://blocklist.site
mv ~/hosts/hosts-tracking.txt hosts.bak/
wget https://blocklistproject.github.io/Lists/tracking.txt -O ~/hosts/hosts-tracking.txt
Code: Select all
#!/bin/sh
# Compansion script to hosts-wget.scr to automatically concatenate files
# retrieved by that script, normalizing them, and finally doing a sort unique
# into an output file named adlist-all, an ordinary text file.
# Runs as ordinary user and requires directories /home/username/hosts and
# /home/username/hosts/hosts.bak/ to pre-exist as working directories.
# Adjust script to account for personal layout differences or changes.
# The output file may be APPENDED to your /etc/hosts file. Don't overwrite it!
# Comments follow:
# 0. Starts with copy (cp) to backup previous adlist-all file, then
# concatenate (cat) all files ending with .txt and pipe through ...
# 1. print only lines beginning with number 0 or 1,
# sed -n '/^[01]/p' | \
# This step was moved to top of the list since it also removes
# both comments that begin with a hashtag and blank lines.
# Originally, it was the final instruction, sed -n '/^[0-9]/p' | \
# 2. deleting lines that contain the string localhost,
# sed '/localhost/d' | \
# 3. (removed) suppress comments, # sed '/^#/d' | \
# 4. (removed) suppress empty lines, # sed '/^$/d' | \
# 5. replaces tabs with spaces, sed 's/[\t]/ /g' | \
# 6. replaces double spaces with single spaces, sed 's/ / /g' | \
# 7. In lines beginning with 127.0.0.1 substitute 0.0.0.0,
# sed 's/^127\.0\.0\.1/0\.0\.0\.0/g' | \
# 8. suppress \r at end of line, tr -d '\015' | \
# 9. then sort unique by field 2 (url) into adlist-all,
# sort -u -k 2 > ~/hosts/adlist-all
# begin script:
cp ~/hosts/adlist-all ~/hosts/hosts.bak/
cat ~/hosts/*.txt | \
sed -n '/^[01]/p' | \
sed '/localhost/d' | \
sed 's/[\t]/ /g' | \
sed 's/ / /g' | \
sed 's/^127\.0\.0\.1/0\.0\.0\.0/g' | \
tr -d '\015' | \
sort -u -k 2 > ~/hosts/adlist-all
