Migration Complete, everything is working....post here if you think it isn't

[shovenose]

Hello everybody! The website has been moved to the new server. The only way we could get this done in a reasonable amount of time was to move everything except the attachments and worry about them later. It's going to be days to a week before those are done, so we appreciate your patience. In the mean time, no attachments will load, and we have disabled the uploading of attachments. While they're typically annoying because they slow down page loads, use of an image host like http://imgur.com/ is suggested at this time. Sorry for any frustrations, but if we had not made that choice you'd be sitting here staring at an error message for another decade.

Please report any problems you experience with the forum (besides the lack of attachments, obviously), or give your feedback for performance, or anything else, in this thread. We have done minimal optimization and wanted to get it up as far as possible so there is room for improvement, but your preliminary thoughts are much appreciated.

[mariushm]

Re: Site is back, attachments aren't yet. Read this please!

somewhat off topic... this whole upgrade/migration looks like it was handled very poorly but it's free so I shouldn't really complain.

Seriously, how slow is the server's internet connection?

Even with just 10mbps upload, it would have taken less than 24 hours to upload those 70 GB.. unless rsync or some other crap like that was used.

I typically use 7zip with ultra fast compression or rar with none/very fast to compress tons of small files (like the thumbnails and images) into 100MB - 1 GB files which I can then transfer through ftp to other servers (downloading several files concurrently) to saturate the uplink.

The packing + simultaneous transfers usually saves a huge amount of time.

I guess I'm just too used to European datacenters with good peerings that allow me to saturate 100mbps upload to US or anywhere in Europe.

[shovenose]

Re: Site is back, attachments aren't yet. Read this please!

somewhat off topic... this whole upgrade/migration looks like it was handled very poorly but it's free so I shouldn't really complain.

Seriously, how slow is the server's internet connection?

Even with just 10mbps upload, it would have taken less than 24 hours to upload those 70 GB.. unless rsync or some other crap like that was used.

I typically use 7zip with ultra fast compression or rar with none/very fast to compress tons of small files (like the thumbnails and images) into 100MB - 1 GB files which I can then transfer through ftp to other servers (downloading several files concurrently) to saturate the uplink.

The packing + simultaneous transfers usually saves a huge amount of time.

I guess I'm just too used to European datacenters with good peerings that allow me to saturate 100mbps upload to US or anywhere in Europe.

The old server was moving data at about 70KB/s. There was not sufficient space on the old servers (disks were almost completely full) so zipping stuff up was not an option. Also, the old server went down twice for what was apparently RAID array rebuilds during the download process. Quite frustrating, and made the migration infinitely more complex.

[diif]

Re: Site is back, attachments aren't yet. Read this please!

I'm definitely noticing the speed increase.

[ReeceyBurger123]

Re: Site is back, attachments aren't yet. Read this please!

I'm definitely noticing the speed increase.

Same here instant loading

[TechGeek]

Re: Site is back, attachments aren't yet. Read this please!

WOOHOO! How's the store?

[ratdude747]

Re: Site is back, attachments aren't yet. Read this please!

Hell at this rate, if anybody has access to the servers directly, I wonder if copying them to an external HDD and mailing it to the new server would be faster. No joke, even if it was a backup plan alone.

[TechGeek]

Re: Site is back, attachments aren't yet. Read this please!

What happened to the epic thread "the gutless, bloated power supply hall of fame"? I can't find it.

[ratdude747]

Re: Site is back, attachments aren't yet. Read this please!

The old server was moving data at about 70KB/s. There was not sufficient space on the old servers (disks were almost completely full) so zipping stuff up was not an option. Also, the old server went down twice for what was apparently RAID array rebuilds during the download process. Quite frustrating, and made the migration infinitely more complex.

For those who want a little more on this, here was some test data Shovenose had me run on the old server (since I have a different ISP and happened to be on Skype at the right time, I was the guinea pig):

 [larry@larry-e6430 ~]$ traceroute badcaps.net
traceroute to badcaps.net (199.68.189.233), 30 hops max, 60 byte packets
 1 DD-WRT (192.168.1.1) 4.746 ms 4.726 ms 4.726 ms
 2 192.168.0.1 (192.168.0.1) 7.007 ms 7.011 ms 7.083 ms
 3 * * *
 4 xe-0-0-6.nwalinco02h.midwest.rr.com (74.128.45.41) 38.191 ms 38.189 ms 40.395 ms
 5 be29.lsvmkyzo01r.midwest.rr.com (65.29.27.156) 30.105 ms 30.110 ms 30.108 ms
 6 be24.clmkohpe01r.midwest.rr.com (65.189.140.162) 38.133 ms 28.129 ms 31.754 ms
 7 bu-ether35.chctilwc00w-bcr00.tbone.rr.com (107.14.19.60) 43.095 ms 43.104 ms 43.101 ms
 8 0.ae1.pr0.chi30.tbone.rr.com (66.109.1.78) 43.055 ms 0.ae0.pr0.chi30.tbone.rr.com (66.109.1.76) 43.058 ms 0.ae1.pr0.chi30.tbone.rr.com (66.109.1.78) 43.048 ms
 9 * * *
10 ae-2-3513.edge1.Atlanta4.Level3.net (4.69.150.161) 61.664 ms 61.675 ms 61.672 ms
11 64.209.110.102 (64.209.110.102) 53.001 ms 55.660 ms 57.842 ms
12 r03.crtga.11.fogodc.net (199.68.188.2) 57.870 ms 57.859 ms 57.861 ms
13 r04.crtga.13.fogodc.net (199.68.188.22) 57.851 ms 59.655 ms 59.649 ms
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *

My DL speed from the old server about 20 minutes before ^ was run was 72Kb/s. Poor server

[kevin!]

Re: Site is back, attachments aren't yet. Read this please!

Thank you for moving the forum to a new server.
We are grateful, because to load attachments took more than 5 minutes.
Now go faster to load.
Bye

[mariushm]

Re: Site is back, attachments aren't yet. Read this please!

The old server was moving data at about 70KB/s. There was not sufficient space on the old servers (disks were almost completely full) so zipping stuff up was not an option. Also, the old server went down twice for what was apparently RAID array rebuilds during the download process. Quite frustrating, and made the migration infinitely more complex.

Well, I think it's just sad that the server got so bad, got to this point.

In any case, any sane person would have backups.. if it was my forum, i would have made a backup of the static files (attachments etc) every 2-3 months or so, and a backup of the database at least every month. Honestly, it only takes a few minutes to perform such a task.

New server? migration? messed up server? .... I'd upload the latest backup from home computer to the new server, then ftp from old server to new server with option to skip/resume existing files (to refresh and get the more recently uploaded files) and job's done.

Alternatively, torrents can do wonders (as a sort of sync mechanism between servers) but in this particular case the messed up raid would have caused huge delays due to the hashing required to build the torrent .

[Topcat]

Re: Site is back, attachments aren't yet. Read this please!

Well, I think it's just sad that the server got so bad, got to this point.

In any case, any sane person would have backups.. if it was my forum, i would have made a backup of the static files (attachments etc) every 2-3 months or so, and a backup of the database at least every month. Honestly, it only takes a few minutes to perform such a task.

New server? migration? messed up server? .... I'd upload the latest backup from home computer to the new server, then ftp from old server to new server with option to skip/resume existing files (to refresh and get the more recently uploaded files) and job's done.

Alternatively, torrents can do wonders (as a sort of sync mechanism between servers) but in this particular case the messed up raid would have caused huge delays due to the hashing required to build the torrent .

Glad you have such wisdom.....but unfortunately you have no knowledge of the very bad situation that I didn't have a lot of control over....so those things were a lot easier monday-morning-quarterbacked than actually done.....I'll just leave it at that.

[Stefan Payne]

Re: Site is back, attachments aren't yet. Read this please!

Any chance the attachments will be back?

[whiz62]

Re: Site is back, attachments aren't yet. Read this please!

So glad your back! I know it all will get sorted out. Thanks for hanging there.

[kazia]

Re: Site is back, attachments aren't yet. Read this please!

Any chance the attachments will be back?

yea, does any chance?

[shovenose]

Re: Site is back, attachments aren't yet. Read this please!

Attachments will be back within a few days to week.

[kazia]

Re: Site is back, attachments aren't yet. Read this please!

Attachments will be back within a few days to week.

Thanks, glad to hear that..

[zx8401]

Re: Site is back, attachments aren't yet. Read this please!

Thank you for all the trouble its caused you, it must have been a bloody pain :-(
People dont appreciate the work that goes into maintaining a server.

[goodpsusearch]

Re: Site is back, attachments aren't yet. Read this please!

I would like to thank Topcat, shovenose, ratdude747 and anybody else who took part at this project!

I hope that attachments will be restored soon!

[TELVM]

Re: Site is back, attachments aren't yet. Read this please!

Thanks for your work preserving and improving this great forum!