Log in

No account? Create an account

Previous Entry | Next Entry

Avoidable Office Crisis of the Day

Our Car Service division is a mess of disorganization. It has been run as a separate division of our firm since we acquired them about 4-5 years ago and as such, I've had little interaction with them.

Three senior people from Car Service left the firm in the past couple of months so there's been movement to merge the support of their Windows servers & infrastructure into the Operations group of the rest of the firm. This makes sense and I'm happy to be part of it.

Except that now I get to see how the sausage is being made. Or rather, I now see that the sausage isn't always sausage, it isn't necessarily being made or made on time, or isn't made out of meat or even something organic, or isn't even made on the equipment we thought it was.

What has been hidden from view is that the Windows Servers are creaking under their own weight and have frequent undetected/unreported hardware errors. We have also discovered the apps need to be touched or helped by a human being every few hours or the app will simply die. Nothing is automated, monitoring is sparse at best, there are no performance monitors at all, and nearly nothing was documented. This is a classic case study in how not to do business.

But all of this isn't news: we've been repairing & cleaning up this mess for much of October and have made great progress.

Today was another surprise. A domain I've never heard of (transponet.com) apparently expired last night. Huh? A look into the Whois records doesn't help: the registration info is just a privacy proxy firm in Florida, shielding the actual registration holder & contact information. The domain isn't listed in our portfolio of domain names and we never received a renewal notice.

In the massive email thread which ensued, we have a strong suspicion that the domain name was registered by one of the software engineers in their personal account, but was never transferred to the corporate account as they should have. They probably did this as a convenience many years ago and it slipped through the cracks like so many other things. Unfortunately, the engineer we suspect did this died two years ago. Because of that, he's not available to renew the domain or even transfer it to our corporate account for renewal. *sigh*

We're now working legal channels to see about getting this transferred as needed. The people who have access to various mailboxes & files of departed employees aren't in the office yet so I can't check for any renewal notices or even confirmation of our suspicions on how the domain was registered. At this moment, I'm at a dead stop.

Things like this aren't supposed to happen. It's a profound embarrassment & shame upon the clowns who created this situation, and the rest of us for not catching it in time. Needless to say, the balance of my day is going to be spent scanning all of the source code in that division to look for any other domain surprises and ensuring all of them are in our corporate portfolio where they can be properly managed.

Update: We've renewed the problem domain but are still trying to transfer it and several others to our regular portfolio. The snag we're finding in many of these domains is that the registrant contact information is incorrect: there's a typo in the email address! We're now in the process of getting around this by registering the typoed domain name, then creating MX records to point it to our corporate email server. *sigh*