Page 1 of 1
A European mirror for MX-14 / MEPIS 12 repositories
Posted: Fri Apr 24, 2015 7:14 pm
by timkb4cq
Community Repository traffic hit an all-time record last month at over twice the prior years average. Only 1/3 of this traffic was from the US, while Germany, France, U.K, & Italy together accounted for another 1/3.
Accordingly, I've set up a repository mirror in the Netherlands that should serve our European users better, as well as provide some desirable redundancy.
The mirror is located at
http://nl.mxrepo.com and these URLs may be used in place of the corresponding
main.mepis-deb entries you are currently using.
deb
http://nl.mxrepo.com/mepiscr/xfce/ xfce-4.10 main
deb
http://nl.mxrepo.com/mepiscr/repo/ mepis12cr main non-free
# deb
http://nl.mxrepo.com/mepiscr/mx-test/ mx-14 test
# deb
http://nl.mxrepo.com/mepiscr/testrepo/ mepis12cr test
BTW, mxrepo.com also points to the old US server and may be used interchangeably with main.mepis-deb.org
Enjoy!
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Fri Apr 24, 2015 9:22 pm
by Jerry3904
I've set up a repository mirror in the Netherlands
How did you do that, exactly?
<Well done, BTW!>
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Fri Apr 24, 2015 9:55 pm
by timkb4cq
It's a self-managed VPS, running debian and using apt-mirror to sync. Whenever I upload packages to the old server I take a few seconds to trigger apt-mirror on the new one. Unless there are some massive packages it takes less than a minute to sync.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 4:31 am
by Gaer Boy
A good development, Tim. For the information of those who might benefit, I've been using the new mirror since last weekend and it definitely gives a better speed for me. In particular, reloading the repos is about twice as fast as before.
Phil
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 7:42 am
by Jerry3904
How is this going to be handled systemically?
I have trouble seeing a scenario where every user finds out about this in some haphazard fashion, tries to figure out how to change from the default setup, makes errors, etc. Seems like a headache both for the user and for maintenance.
Would it be best to get this done during installation? Maybe a single question about which server to use, or does the installer do it automatically by location? Or does a button go in MX Tools? Etc.
It seems to me that the Dev Team might want to mull this over a bit sometime.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 8:40 am
by timkb4cq
Debian uses a question during install, and maintains a list of hundreds of mirrors. Users wishing to change mirrors after installation are left to edit their sources manually.
I don't think we'll have too much trouble. Most users don't change sources after installation unless they have a problem.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 8:43 am
by Jerry3904
Can't say as I think having users do manual edits is optimal for our mission and audience.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 12:24 pm
by lucky9
I've thought that the migration instructions have been reasonably functional. Perhaps a note would be useful in the Docs or here at the forum. Adding a note and link to instructions on the forum might be easiest?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 12:30 pm
by ko
EU users of Mepis 12 and MX: replace (as root) the contents of /etc/apt/sources.list.d/mepis.list as follows to point to the dutch mirror:
# Dutch Mirror of Mepis Community Main and Test Repos for Mepis 12 and MX
deb
http://nl.mxrepo.com/mepiscr/xfce/ xfce-4.10 main
deb
http://nl.mxrepo.com/mepiscr/repo/ mepis12cr main non-free
deb
http://nl.mxrepo.com/mepiscr/mx-test/ mx-14 test
deb
http://nl.mxrepo.com/mepiscr/testrepo/ mepis12cr test
Disable test repos if deemed necessary. Mine are enabled.
btw: in the debian.list I always use european ( .nl. or .de.) debian repos, replacing the default .us. ones
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 12:44 pm
by Gaer Boy
On a Live boot, using F2 to select English (UK) the UK debian mirrors are listed in sources. I don't know where this magic happens, but could something similar be done for the MX mirrors?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 2:06 pm
by kmathern
Gaer Boy wrote:On a Live boot, using F2 to select English (UK) the UK debian mirrors are listed in sources. I don't know where this magic happens, but could something similar be done for the MX mirrors?
I think that magic happens in the "set_mirror" function of the /etc/init.d/antiX-init script:
Code: Select all
set_mirror() {
local mirror=$1
case "$mirror" in
[a-zA-Z][a-zA-Z]) ;;
*)
error "$_Bad_mirror_code_X_expected_two_letters_" $(pquote $mirror)
return;;
esac
source_list=/etc/apt/sources.list.d/debian.list
first_write $source_list || [ "$CMD_MIRROR" ] || return
echo_live "$_Localizing_X_to_Y_mirror_" $(pquote $source_list) $(pquote $MIRROR)
sed -i "s=/ftp\.\(..\.\)\\?debian\.org/=/ftp.$mirror.debian.org/=" $source_list
}
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 2:23 pm
by Stevo
So would the the debian.list file be possibly reset at every boot? If so, it would be frustrating for someone that edited it to get a custom mirror, only to have the customizations vanish. If that's how it does work, we should add a comment in the file documenting that.
It seems you could have something the same for the mepis.list file, but we'd have to decide which locales use the EU server and which the US one. That looks tough to implement.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 2:44 pm
by kmathern
Stevo wrote:So would the the debian.list file be possibly reset at every boot? If so, it would be frustrating for someone that edited it to get a custom mirror, only to have the customizations vanish. ...
The /etc/init.d/antiX-init script is only present and only gets run when booting the live session, so I think the sed edit of the debian.list file only happens once.
From what I can see, the /etc/init.d/antiX-init script isn't present on a harddrive install. Either it doesn't get copied over to harddrive partition during the install process, or it gets copied over and then removed by the installer.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 3:24 pm
by BitJam
Stevo wrote:So would the the debian.list file be possibly reset at every boot? If so, it would be frustrating for someone that edited it to get a custom mirror, only to have the customizations vanish. If that's how it does work, we should add a comment in the file documenting that.
That is why the call to first_write() comes first. We only update the mirror once, even on a live system unless a user specifically types in a mirror=xx cheat code. The mirror is also automatically set if you select a language in the bootloader but that only happens once. The CMD_MIRROR variable is set by the mirror= cheat while the MIRROR variable is set by both the mirror= cheat and the lang= cheat.
For systems without persistence this makes no difference because you always start out with a clean slate on every live boot (so we always update the mirror in that case if a language is set). The first_write() call is designed to protect live systems with persistence. Since the antiX-init script (now called live-init) only runs on live systems, people with installed systems have nothing to worry about.
This system has been a place for a number of years. We tried to strike the right balance between automation and customization without clobbering changes made by users.
BTW: the files we use to record which files we've written to are at /etc/live/protect/persist and /etc/live/protect/remaster. The 2nd one is generally not used. The first one could/should be optionally deleted when doing a snapshot depending on whether it is to be general purpose or personal.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sat Apr 25, 2015 8:13 pm
by timkb4cq
I should have thought of this earlier, but I finally realized that the antiX server, located in Ontario, must also be experiencing the increased traffic. So I asked Anticapitalista if I should host a mirror of the antiX repos as well. That made sense to him as well so:
antix.daveserver.info/stable &
antix.daveserver.info/testing
are now being mirrored at
nl.mxrepo.com/antix/stable &
nl.mxrepo.com/antix/testing
Specifically for MX-14 that means that
deb http://antix.daveserver.info/stable/ stable main
can be replaced in /etc/apt/sources.d/antix.list by
deb http://nl.mxrepo.com/antix/stable/ stable main
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 12:29 am
by antiX-Dave
Well there was nothing really to out of the ordinary. My server only really complained (sent me a message) about a large set of connections coming from an address in the Netherlands recently... go figure why that may have been :P
Anyway for the somewhat curious there has always been a log set...
http://antix.daveserver.info/log/
Edit:
May I ask what times have you set the sync for? Then I can add it to the ignore / trust list and I should not get unwanted messages warning me of possible issues :)
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 3:15 am
by Pierre
timkb4cq wrote:Community Repository traffic hit an all-time record last month at over twice the prior years average. Only 1/3 of this traffic was from the US, while Germany, France, U.K, & Italy together accounted for another 1/3.
Accordingly, I've set up a repository mirror in the Netherlands that should serve our European users better, as well as provide some desirable redundancy.
BTW, mxrepo.com also points to the old US server and may be used interchangeably with main.mepis-deb.org
Enjoy!
as an item of curiosity:
what is the traffic from Oceania?. - if that is available. ..
- from here, the US may be faster, but the EU can be more stable.
& there is now also the issue, now that Netflix has arrived locally,
of congestion issue, at peak times.
- some ISPs are reporting that Netflix is ~25% of their traffic. ..
that alone, is causing congestion.
- is is indeed an issue, that so much traffic is derived from NA sources.
it's a serious bottle-neck, to deal with.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 6:07 am
by timkb4cq
antiX-Dave wrote:May I ask what times have you set the sync for? Then I can add it to the ignore / trust list and I should not get unwanted messages warning me of possible issues :)
I have it set to update hourly at 4 minutes after the hour. I can easily adjust that if you would prefer a different frequency and/or time .
Now that the initial mirror is complete that will only retrieve index files and any newly added files, so it shouldn't really be noticeable.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 6:37 am
by timkb4cq
Pierre wrote:as an item of curiosity:
what is the traffic from Oceania?. - if that is available.
After a quick tally, it looks like about 4% of the traffic is from Oceania. Another 2.5% is from eastern Asia (Japan ,China, Taiwan, Hong Kong, Singapore, Thailand, Vietnam, Cambodia, S. Korea).
Over 100 countries are represented.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 11:30 am
by Gaer Boy
I made the change from the daveserver repo to nl.mxrepo this morning, but had a 'not found' on Reload in Synaptic. I've tried intermittently during the day but still no joy.
Phil
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 11:45 am
by timkb4cq
It's loading here, and I can install packages from it.
Could you copy & paste the source line that's failing?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 12:07 pm
by kmathern
Are the instructions on how to add the repos, at the toplevel of the mirror here:
http://nl.mxrepo.com/, going to confuse some people?
They're telling how to add them for the original 'main.mepis-deb.org' repositories.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 4:55 pm
by Gaer Boy
timkb4cq wrote:It's loading here, and I can install packages from it.
Could you copy & paste the source line that's failing?
Sorry - I've been out this evening.
Code: Select all
# Use with Debian Wheezy repository. Set as default for MX-14.
deb http://nl.mxrepo.com/stable/ stable main
Edited in Synaptic - this is the contents of antix.list
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 4:59 pm
by kmathern
Gaer Boy wrote:timkb4cq wrote:It's loading here, and I can install packages from it.
Could you copy & paste the source line that's failing?
Sorry - I've been out this evening.
Code: Select all
# Use with Debian Wheezy repository. Set as default for MX-14.
deb http://nl.mxrepo.com/stable/ stable main
Edited in Synaptic - this is the contents of antix.list
You didn't include the 'antix' part.
This loads okay
Code: Select all
Repos: Active apt sources in file: /etc/apt/sources.list.d/antix.list
deb http://nl.mxrepo.com/antix/stable/ stable main
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Sun Apr 26, 2015 5:00 pm
by anticapitalista
Code: Select all
# Europen mirror
deb http://nl.mxrepo.com/antix/stable stable main
is what you want.
kmathern beat me to it.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 3:30 am
by Gaer Boy
Thanks, both - it had to be something obvious. I'd checked it a few times, but there's none so blind as those who won't see!
Synaptic reloads blindingly fast now.
Phil
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 5:24 am
by GuiGuy
Slightly off-topic: "stable" is now jessie, so the sources.list is not really correct.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 8:37 am
by kmathern
GuiGuy wrote:Slightly off-topic: "stable" is now jessie, so the sources.list is not really correct.
In the case of the antiX repo, 'deb http
://antix.daveserver.info/stable stable main', I don't think the "stable" necessarily equates to "wheezy" or "jessie".
Both MX-14 and antiX-15-beta2-V_386-full have it enabled by default. MX-14 is based on wheezy, antiX-15.... is based on jessie. I didn't check some of the antiX-13.... and antiX-14.... isos I have, but I think they also have it enabled by default.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 9:13 am
by dolphin_oracle
kmathern wrote:GuiGuy wrote:Slightly off-topic: "stable" is now jessie, so the sources.list is not really correct.
In the case of the antiX repo, 'deb http
://antix.daveserver.info/stable stable main', I don't think the "stable" necessarily equates to "wheezy" or "jessie".
Both MX-14 and antiX-15-beta2-V_386-full have it enabled by default. MX-14 is based on wheezy, antiX-15.... is based on jessie. I didn't check some of the antiX-13.... and antiX-14.... isos I have, but I think they also have it enabled by default.
kmathern is correct. the stable in the antiX repo definition refers to the antiX stable, vs. antiX testing. The antiX installer will allow you to choose to follow the stable debian series (wheezy for 13.2 series or jessie for 15 series, explicitly called out as wheezy and jessie btw). If you choose "testing" during the installer setup, the repo definition will changes to antiX testing, as well as enable the debian testing sources.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 10:17 am
by GuiGuy
Thanks for the replies, gentlemen.
I find this very confusing, and just wish everybody would stop using the word "stable" in connection
with Debian repositories.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 10:24 am
by dolphin_oracle
GuiGuy wrote:Thanks for the replies, gentlemen.
I find this very confusing, and just wish everybody would stop using the word "stable" in connection
with Debian repositories.
well, that is ONE of the ways debian defines they repo. "stable" is always "stable". so if you were running debian proper and had "stable" instead of wheezy in your source definitions, this morning you probably updated to jessie-based stuff.
In this age of "rolling" distributions like arch, et. al., if easy to forget that debian is/can be a rolling distribution...it just rolls really really slowly (at least on the stable side).
from debian.org:
Upgrading Debian
Upgrades to Debian 8 from the previous release, Debian 7 (codenamed Wheezy), are automatically handled by the apt-get package management tool for most configurations. As always, Debian systems may be upgraded painlessly, in place, without any forced downtime, but it is strongly recommended to read the release notes as well as the installation guide for possible issues, and for detailed instructions on installing and upgrading. The release notes will be further improved and translated to additional languages in the weeks after the release.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 10:33 am
by GuiGuy
I do understand all that, but for our repos can we not use the codename instead of "stable"?
Edit: Sorry if I am incorrect in thinking that antix.daveserver is "ours" - I suppose that is cheeky!
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 11:59 am
by ko
Thanks for the speedy European Mirror. Things (reloads and downloads) occur pretty fast now.....;-)
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 1:27 pm
by Stevo
Will this page be updated?
http://main.mepis-deb.org/
And is the choice of which server to use basically an Eastern/Western Hemisphere sort of thing, or should the Pacific Rim nations use the US server?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 1:35 pm
by lucky9
What's the name of the debian package that will test repository speeds and pick the fastest?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 1:40 pm
by kmathern
lucky9 wrote:What's the name of the debian package that will test repository speeds and pick the fastest?
You might be thinking of apt-spy
Code: Select all
$ apt-cache show apt-spy
Package: apt-spy
Version: 3.2.2-1
Installed-Size: 101
Maintainer: Stefano Canepa <sc@linux.it>
Architecture: i386
Depends: libc6 (>= 2.3), libcurl3 (>= 7.16.2)
Description-en: writes a sources.list file based on bandwidth tests
Parses the list of mirrors downloaded from ftp.debian.org and then based on
the region specified by the user each of testes mirrors for bandwidth, at the
end it writes /etc/apt/sources.list.d/apt-spy.list using the best mirror it
found out.
But I think it only finds the best Debian mirror to use, not mirrors of other repos.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 2:02 pm
by Jerry3904
I have no access to that, but I believe Tim does though, since the current revision came from him within the last 6 months IIRC.
And is the choice of which server to use basically an Eastern/Western Hemisphere sort of thing, or should the Pacific Rim nations use the US server?
That was why I proposed at the beginning of this thread the idea of a "Servers" button on MX Tools, so users could easily switch to the better one--something that might vary from day to day for some locations.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Mon Apr 27, 2015 2:03 pm
by timkb4cq
Yes
Stevo wrote:And is the choice of which server to use basically an Eastern/Western Hemisphere sort of thing, or should the Pacific Rim nations use the US server?
That's kind of hard to tell. Theoretically the U.S server should be faster to the Pacific Rim - but those circuits can also be more congested. People outside Europe, North America, Western Asia and Northern Africa will just have to try each & see.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Thu Jun 18, 2015 7:05 am
by anticapitalista
*bump*
Is the nl mirror up to date for MX and antiX?
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Thu Jun 18, 2015 2:29 pm
by timkb4cq
Yes, indeed. It runs an automatic update at 4 minutes after each hour, and I check the logs every day that I update anything to make sure I have no download errors.
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Thu Jun 18, 2015 2:40 pm
by kmathern
If you have the M12 testrepo enabled, some packages are being held back
(ffmpeg, libavcodec-ffmpeg56, libavformat-ffmpeg56)Code: Select all
$ apt-get dist-upgrade -s
NOTE: This is only a simulation!
apt-get needs root privileges for real execution.
Keep also in mind that locking is deactivated,
so don't depend on the relevance to the real current situation!
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
ffmpeg libavcodec-ffmpeg56 libavformat-ffmpeg56
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Code: Select all
$ apt-get install ffmpeg libavcodec-ffmpeg56 libavformat-ffmpeg56 -s
NOTE: This is only a simulation!
apt-get needs root privileges for real execution.
Keep also in mind that locking is deactivated,
so don't depend on the relevance to the real current situation!
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libavcodec-ffmpeg56 : Depends: libx265-59 (>= 1.7) but it is not installable
E: Unable to correct problems, you have held broken packages.
Code: Select all
$ apt-cache policy libx265-59
libx265-59:
Installed: (none)
Candidate: (none)
Version table:
It looks like libx265-59 isn't available, in a 32bit package anyway
Re: A European mirror for MX-14 / MEPIS 12 repositories
Posted: Thu Jun 18, 2015 4:08 pm
by timkb4cq
I just PM'd Stevo. I didn't notice when he sent libx265 that it only had the 64bit debs.