June 01, 2007
Planned Outages - June 2nd, 3rd and 4th, 10AM GMT
This weekend, I'll be rolling out the update to version 1.0, and switching to the production servers, so expect half an hour or so of downtime each day.
Posted by: Pixy Misa at
11:59 AM
| No Comments
| Add Comment
Post contains 38 words, total size 1 kb.
May 27, 2007
Someone Let The Magic Smoke Out
Not, on the whole, a fun weekend.
We've had a drive failure on the beta server, which isn't causing problems with the application yet, but could do so without warning. I'm preparing everything for a move to the production servers right now.
Not, on the whole, a fun weekend.
Update
I backed everything up to the production server, and then my PC died. After repairing my PC (a task that wasn't made easier when I found that my Windows install disk had gone missing), I rebooted the beta server to see if that would help with the disk problem.
The beta server didn't reboot.
So I went to the backup on the production server, and discovered that it didn't exist - I had logged in to the wrong system, and backed everything up to the beta server again. Leaving the only up-to-date backup on a dead server.
So I started up the hardware monitoring tools, and tried to reach the server, and couldn't see any signs of life at all. And then, then it came back, working just fine. I've migrated everything off the faulty disk, re-run the backup to the right server, and it should all be good to go for the next 48 hours, after which we will migrate across to the production cluster and I can have the friendly tech support folk rebuild this box.
Whew.
I backed everything up to the production server, and then my PC died. After repairing my PC (a task that wasn't made easier when I found that my Windows install disk had gone missing), I rebooted the beta server to see if that would help with the disk problem.
The beta server didn't reboot.
So I went to the backup on the production server, and discovered that it didn't exist - I had logged in to the wrong system, and backed everything up to the beta server again. Leaving the only up-to-date backup on a dead server.
So I started up the hardware monitoring tools, and tried to reach the server, and couldn't see any signs of life at all. And then, then it came back, working just fine. I've migrated everything off the faulty disk, re-run the backup to the right server, and it should all be good to go for the next 48 hours, after which we will migrate across to the production cluster and I can have the friendly tech support folk rebuild this box.
Whew.
Posted by: Pixy Misa at
03:00 PM
| Comments (1)
| Add Comment
Post contains 229 words, total size 2 kb.
May 03, 2007
Planned Outage - 3rd May 2007 - 8 AM to 10 AM GMT
Hi everyone.
We're having an outage to upgrade our server hardware; this will be taking place between 8 AM and 10 AM GMT (2 AM to 4 AM CDT) on Thursday May 3rd - which is today! (At least for me.)
We'll be moving from those slow old Core 2 Duo chips* to shiny new Core 2 Quads.**
Yay!
Update: This server has been upgraded. The production servers are still being worked on, but the beta test is back in action.
* Xeon 3060's to be precise, and they're actually very fast.
** Xeon 3220's, and they're very very fast.
We're having an outage to upgrade our server hardware; this will be taking place between 8 AM and 10 AM GMT (2 AM to 4 AM CDT) on Thursday May 3rd - which is today! (At least for me.)
We'll be moving from those slow old Core 2 Duo chips* to shiny new Core 2 Quads.**
Yay!
Update: This server has been upgraded. The production servers are still being worked on, but the beta test is back in action.
* Xeon 3060's to be precise, and they're actually very fast.
** Xeon 3220's, and they're very very fast.
Posted by: Pixy Misa at
04:03 AM
| Comments (7)
| Add Comment
Post contains 113 words, total size 1 kb.
April 16, 2007
Update Schedule
I'll be applying a number of smaller updates this week, beginning with Tuesday 17th, again at 6AM GMT. None of these updates should cause any more than a minute or so of downtime.
Update: April 17th update done. This puts in place several minor fixes, and the new sorting and selection options for the [posts] tag.
Update: April 17th update done. This puts in place several minor fixes, and the new sorting and selection options for the [posts] tag.
Posted by: Pixy Misa at
07:21 AM
| Comments (7)
| Add Comment
Post contains 57 words, total size 1 kb.
ops:April 15, 2007
That Was NOT The Scheduled Outage
Sorry about that, folks.
Our main server cluster is hosted at the Softlayer datacenter in Dallas. Yesterday (as I understand it), the DFW area was suffering severe thunderstorms, and our hosting company switched to their own generators in preparation for this, in case the power went out.
Well, the power didn't go out, the thunderstorms passed, and they switched back to utility power. And then a 2500 amp circuit breaker blew, taking down most of their operation.
We've recovered now; I had to repair the database, but that's completed, so we should be back to normal.
However, because the unplanned outage ran over my planned outage, I'll have to reschedule that. It's now set for the same time tomorrow: Monday April 16th 6AM GMT. Downtime should be less than 30 minutes, assuming we don't get any more tornadoes.
Our main server cluster is hosted at the Softlayer datacenter in Dallas. Yesterday (as I understand it), the DFW area was suffering severe thunderstorms, and our hosting company switched to their own generators in preparation for this, in case the power went out.
Well, the power didn't go out, the thunderstorms passed, and they switched back to utility power. And then a 2500 amp circuit breaker blew, taking down most of their operation.
We've recovered now; I had to repair the database, but that's completed, so we should be back to normal.
However, because the unplanned outage ran over my planned outage, I'll have to reschedule that. It's now set for the same time tomorrow: Monday April 16th 6AM GMT. Downtime should be less than 30 minutes, assuming we don't get any more tornadoes.
Posted by: Pixy Misa at
06:23 AM
| Comments (1)
| Add Comment
Post contains 144 words, total size 1 kb.
April 13, 2007
Scheduled Outage - April 16th 6AM GMT
I'll be applying another code update this Sunday, April 15th 16th, at 6AM GMT. This also involves updating a number of the standard templates. This should only take five to ten minutes, but please allow for an outage of up to half an hour.
This update will fix a number of minor issues like the timeout problem when uploading large files, add some features including archive pages, and bring the post and comment tags properly in line with the documentation (rather important, that one).
This update will fix a number of minor issues like the timeout problem when uploading large files, add some features including archive pages, and bring the post and comment tags properly in line with the documentation (rather important, that one).
Posted by: Pixy Misa at
06:07 AM
| Comments (2)
| Add Comment
Post contains 91 words, total size 1 kb.
