git repository mirrors

[freghar]

I've set-up several git mirrors of the github repository, partly because of the DDoSes, partly to balance the github load.

They are updated every full hour, you can use them instead of git://github.com/mangos/mangos.git , they are in fact faster most of the time (I got 4MB/s download from gitorious).

git://git.assembla.com/mangos-mirror.git
git://gitorious.org/mangos-mirror/mangos-mirror.git
git://mangos-mirror.git.sourceforge.net/gitroot/mangos-mirror/mangos-mirror

... and to help some people behind firewalls ...

http://git.gitorious.org/mangos-mirror/mangos-mirror.git

And that's it! Well, some bonus stuff below.

----------------------------------------------------------------------------------

Checking mirror's valididy

You shouldn't have any need to do this, it's just one of the options that multiple mirrors allow you to do. Since git was designed as distributed SCM, it has some of these checks by design. The most visible one is SHA1 hash.

You can clone (for example) two mirrors and then compare their HEAD. If both hashes are the same, you either have a valid repo .. or both mirrors were compromised. Note that smart people don't actually need to clone another copy, they just fetch from the other URI and if nothing gets fetched, they know both repos are basically the same.

You can do this check with github.com itself, but bear in mind that a new commit might have been pushed there and mirrors are a few minutes behind (which makes you receive two different HEADs) .. so do it by hand (git log -1 origin/master and search github.com website for that hash).

Please note that you REALLY don't need to do this unless you are 110% paranoid or "just want to give it a shot".

Github down, but I want to update

That's fairly easy, if you are the "git pull" person, instead of using

git pull
or
git pull origin master

replace the "origin" by mirror URI:

git pull git://some.mirror.org/mangos-mirror.git master

(feel free to replace "master" by "mangos-0.12" or anything like that)

If you are the "git fetch + merge/rebase" person, instead of

git fetch origin
git rebase origin/master

try this:

git fetch git://some.mirror.org/mangos-mirror.git master
git rebase FETCH_HEAD

(there's no FETCH_HEAD/master, if you want both master and mangos-0.12, you need to fetch them separately, FETCH_HEAD is in fact .git/FETCH_HEAD file and it always contains last fetched HEAD, which is master in this case)

Once github.com (or your favorite mirror) is back online, you can update as usual.

[xeross155]

or one could add the mirrors as seperate remote

git remote add NAME git://url/to/repo

Then when master is down they just have to do

git pull NAME

and then

git merge NAME/master

[freghar]

or one could add the mirrors as seperate remote

git remote add NAME git://url/to/repo

Then when master is down they just have to do

git pull NAME

and then

git merge NAME/master

First, yes, it's possible to add them as remotes, preferably editing the fetch line in .git/config, so they're able to share references, but it's often really not needed, only if some mirror is offline, which isn't going to happen that often.

Secondly, please read something about git-pull, it somewhat differs from fetch, making the merge in your solution useless.

[xeross155]

hmm someone could use git pull NAME/master to automerge I guess

[przemratajczak]

thank you for foreseeing complications : (

can forum itself can be mirrored to? i know it is much more difficult and sorry for offtop

[freghar]

can forum itself can be mirrored to? i know it is much more difficult and sorry for offtop

Yes, it's possible, but not that easy. Search how wikileaks server is "mirrored" to get the idea.

Ie. http://wikileaks.org/wiki/Wikileaks:About#What_is_Wikileaks.3F_How_does_Wikileaks_operate.3F and http://wikileaks.org/wiki/Wikileaks:About#Is_Wikileaks_accessible_across_the_globe_or_do_oppressive_regimes_in_certain_countries_block_the_site.3F for starters.

[The_Game_Master]

Just a quick question.

How can you reverse a pull?

[xeross155]

I would want to mirror the forums but that would mean I need a database of all accounts, threads and posts. However that would mean the entire forum is readable without registration.

[Neo2003]

Hello,

I still cannot get source behind a http proxy which requires auth.

I export the http_proxy value successfully but these do not support http:

git://git.assembla.com/mangos-mirror.git

git://gitorious.org/mangos-mirror/mangos-mirror.git

git://mangos-mirror.git.sourceforge.net/gitroot/mangos-mirror/mangos-mirror

I always get something like:

fatal: [url]http://gitorious.org/mangos-mirror/mangos-mirror.git/info/refs[/url] not found: did you run git update-server-info on the server?

And http://github.com/mangos/mangos.git has the following problem:

Initialized empty Git repository in /home/stephane/mangos/.git/
got 23ddaba9a863cbabf25281fc55940e94d6e3b1ff
walk 23ddaba9a863cbabf25281fc55940e94d6e3b1ff
got eee42e2f5821775412d70f8ad0c5fb19d3f71126
got 815000938eee183aad4ba731cc31f0096909b325
walk eee42e2f5821775412d70f8ad0c5fb19d3f71126
walk 815000938eee183aad4ba731cc31f0096909b325
Getting alternates list for [url]http://github.com/mangos/mangos.git[/url]
Getting pack list for [url]http://github.com/mangos/mangos.git[/url]
got 098de341b62b4cb9b2f3c8eb062e2e4aebd21c88
got a894943d091f527e6c494f1811ba35cdd50796e1
Getting index for pack 69e69e44777b1b7ead3e67e7c9437425a66f241b
got 1d2a4c43a9db140417adec733cfb61dc258504c6
got 1e38a1788b9de8a6d1cd39fa12ee2e7e333b3b34
got 270ba19c2ba116b12c0316645a9b2918ab4dc449
got 37521422f0f69a8b2c49bdc1f0993c881f9046c1
Getting pack 69e69e44777b1b7ead3e67e7c9437425a66f241b
which contains 7367b1e2d9179b9969979c10f8fa739bc126669a
error: Unable to get pack file [url]http://github.com/mangos/mangos.git/objects/pack/pack-69e69e44777b1b7ead3e67e7c9437425a66f241b.pack[/url]
The requested URL returned error: 502
error: Unable to find 7367b1e2d9179b9969979c10f8fa739bc126669a under [url]http://github.com/mangos/mangos.git[/url]
Cannot obtain needed object 7367b1e2d9179b9969979c10f8fa739bc126669a
while processing commit 815000938eee183aad4ba731cc31f0096909b325.
fatal: Fetch failed.

Seams like git and http don't like each other.

[freghar]

I'm sorry for the downtime of all those mirrors + the SD2 git mirror, I have - however - major problems setting up my new machine. First, there was an unnoticed known_hosts error, which caused the update script to fail for the last ~7days before I recognized it.

Since all the services were moving to a new machine, I decided to solve this problem once I have all the things moved (it was supposed to happen in ~5 days after move). I managed to set-up the new machine fine, everything went okay, hardware tests as well, then I noticed a strange "silent data corruption" on both hard drives in RAID1. Simply said almost no file could be read twice with the same md5 hash. I'm still investigating it.

This said, I'm going back to the old hardware - for a while. The repo updating script should be up-and-running today/tomorrow.

[Hw2]

thanks freghar, i hope you'll bring it to life again

[freghar]

Good news - all the mirrors are back online. The issue was most likely in motherboard's SATA interface.

[freghar]

And sad news; I'll be shutting down the mirroring service in about a month. It shouldn't impact anything really, all those mangos mirrors are probably not used by anybody and github stability has increased over time, so ..

[silvermoo]

y am i not able to clone repo is som tng wrong with srvr or my git clinet ??

plz help

Getting pack c6f3b96e3574d69be91b934aa2e135bb2e14697b

which contains 7367b1e2d9179b9969979c10f8fa739bc126669a

error: Unable to get pack file http://git.gitorious.org/mangos-mirror/mangos-mirror.git/objects/pack/pack-c6f3b96e3574d69be91b934aa2e135bb2e14697b.pack

The requested URL returned error: 403

error: Unable to find 7367b1e2d9179b9969979c10f8fa739bc126669a under http://git.gitorious.org/mangos-mirror/mangos-mirror.git

Cannot obtain needed object 7367b1e2d9179b9969979c10f8fa739bc126669a

while processing commit 421b2459a5c62c032c2a61f22ff6c52030dc9038.

fatal: Fetch failed.

[ichijinkn]

git pull git://github.com/mangos/mangos.git master I get:

github.com[0: 207.97.227.239]: errno=No such file or directory

fatal: unable to connect a socket (No such file or directory)

Happens even if I try to create a fresh one (git clone, etc) Even with the newly listed ones on this page. Hasnt happened before.

- Edit--

Seems I just needed to uninstall that Git and install a newer version. They must have just updated it in the last day or so.

[Auntie Mangos]

After all the services were moving to a new machine, I decided to solve this problem once I have all the things moved (it was supposed to happen in ~5 days after move). I managed to set-up the new machine fine, everything went okay, hardware tests as well, then I noticed a strange "silent data corruption" on both hard drives in RAID1. Simply said almost no file could be read twice with the same md5 hash. I'm still investigating it.