?

Log in

Damage Control

The initial shock of the election results has largely faded for me, so now it's time to take some action on limiting the coming damage.

The GOP has attempted for years to remove all federal spending for women's health. To counter this, I've been donating monthly to Planned Parenthood, and you can too!
https://www.plannedparenthoodaction.org/

The young are most vulnerable, lacking adult legal status to own their own lives and lacking resources to protect themselves from the whims of the powerful. I'm donating to Supporting and Mentoring Youth Advocates and Leaders (SMYAL) in DC.
http://www.smyal.org/

There are going to be a lot of hungry people in the coming years. Please donate food, money or your time to your local food bank. In my area, it's the Capital Area Food Bank.
https://www.capitalareafoodbank.org/

The president-elect is a self-proclaimed sexual predator. Please donate to the rape crisis support line/support center in your area. In the DC area, I'm donating to the DC Rape Crisis Center.
http://dcrcc.org

The white supremicist & alt-right hate campaigns which supported the new administration need to be watched carefully. I'm donating to the Southern Poverty Law Center to monitor and report on hate crimes.
https://www.splcenter.org/

The GOP's track record of voter suppression should not go unchallenged. Hate crimes must be prosecuted, existing laws & regulations need to be followed, conflicts of interest should be exposed and corruption needs to be shown for what it is. I'm donating to the ACLU to fund court actions to preserve the Constitution.
https://www.aclu.org/

Mourn for the future as long as you need, but taking action will help you as you help others.

Tags:

The Hole in My Head

At this moment, I literally do have a hole in my head. My mother was right, finally & briefly, after all these years.

For as long as I can remember, I've had a relatively small soft bump just behind & above my right ear. Imagine having 2-3 toonies embedded in your scalp: it felt fluid-filled and slid easily against with the scalp against the skull so I knew it wasn't attached to or protruding from my skull. Since it was hidden by my hair, wasn't painful in the slightess and was otherwise irrelevant, I never worried about it.

About 3-4 weeks ago, it began growing. It's diameter grew another 50% or so, and it thickened to the point where it was a visible bump despite my hair. Indeed, after my last haircut, it was so visible and apparently disturbing that one person asked me if I needed help because I had been clearly beat up. Soon, it also began scaling over and began sporadic bleeding.

In the three days before my regular doctor could fit me in, the swelling reduced dramatically. In the end, it was about the size & thickness of a quarter, rough to the touch but still painless.

My regular doctor said it looked like a common wart, but referred me to a dermatologist for a proper exam and possible remediation. Monday, I visited Dr Stolar who repeated Dr Ward's wart theory, adding it could also be a keratosis. In any case, we agreed to remove it.

Within ten minutes, the area of my scalp had been anaesthetized, the wart/keratosis had been surgically removed, the area cauterized, some antibiotic ointment was applied to the wound and I was on my way home while the excised tissue was on its way to the pathology lab. Dr Stolar took a look at a few other skin irregularities across my forehead as a precautionary measure but said there was nothing of concern; being fair-skinned, he recommended either the SPF 2 billion or wearing a burqua when out in the sun, but otherwise carry on my life per usual.

At this moment, the area is still pain-free and I can feel the wound has scabbed over. I can wash my hair without concern. The healing process will take another week or so, but it feels odd not having a bump in that area any longer.

Tags:

The Morning After

This morning is the day after the US elections. I voted for Hillary; she lost fairly, and Donald Trump won. We can argue about voter suppression attempts, gerrymandering of districts, and the validity or not of the electoral college but in the end, Trump is the president-elect and the Republicans have control of the House of Representatives and the Senate.

I'm disappointed my prefered slate of candidates did not win, but I've been voting for 30 years in various contests and am accustomed to winning some and losing others. That's not a problem.

For the first time, however, I am genuinely fearful of the outcome of an election. Never before has someone so blatantly campaigned on demonizing portions of the population for not being the correct religion, for not being white enough, for not being personally loyal enough, for not being straight enough, for not being male enough --and won. And never has someone campaigned with plans with of how he's going to deal with those people who do not meet his standards --and won. I don't remember another candidate for the presidency who bragged about being a sexual predator --and won. The man endorsed by the KKK has won the most important election in the most powerful nation on the planet.

And now this man will be in the White House, backed & protected by the power of the executive branch of the US federal government. The House and Senate are of his own party and will be only too happy to fulfill his whims --he did win an election so he has the mandate-- and the Supreme Court will soon be stacked with like-minded men rubber stamped by his friends in the Senate. There will be no checks & balances.

I fear what will happen to my friends & relatives on disability or retired with Social Security, those who get their medical care via Medicare or via the Affordable Care Act, and for my LGBT friends whose marriages are a specific target of the new administration. My own employment prospects may go very dark if various trade agreements are dissolved and trade wars are encouraged. My retirement plans are on indefinite hold until we see whether a recession results, and whether my savings will survive.

Until today, I had no particular reason to fear the future. Now I do.

Tags:

Avoidable Office Crisis of the Day

Our Car Service division is a mess of disorganization. It has been run as a separate division of our firm since we acquired them about 4-5 years ago and as such, I've had little interaction with them.

Three senior people from Car Service left the firm in the past couple of months so there's been movement to merge the support of their Windows servers & infrastructure into the Operations group of the rest of the firm. This makes sense and I'm happy to be part of it.

Except that now I get to see how the sausage is being made. Or rather, I now see that the sausage isn't always sausage, it isn't necessarily being made or made on time, or isn't made out of meat or even something organic, or isn't even made on the equipment we thought it was.

What has been hidden from view is that the Windows Servers are creaking under their own weight and have frequent undetected/unreported hardware errors. We have also discovered the apps need to be touched or helped by a human being every few hours or the app will simply die. Nothing is automated, monitoring is sparse at best, there are no performance monitors at all, and nearly nothing was documented. This is a classic case study in how not to do business.

But all of this isn't news: we've been repairing & cleaning up this mess for much of October and have made great progress.

Today was another surprise. A domain I've never heard of (transponet.com) apparently expired last night. Huh? A look into the Whois records doesn't help: the registration info is just a privacy proxy firm in Florida, shielding the actual registration holder & contact information. The domain isn't listed in our portfolio of domain names and we never received a renewal notice.

In the massive email thread which ensued, we have a strong suspicion that the domain name was registered by one of the software engineers in their personal account, but was never transferred to the corporate account as they should have. They probably did this as a convenience many years ago and it slipped through the cracks like so many other things. Unfortunately, the engineer we suspect did this died two years ago. Because of that, he's not available to renew the domain or even transfer it to our corporate account for renewal. *sigh*

We're now working legal channels to see about getting this transferred as needed. The people who have access to various mailboxes & files of departed employees aren't in the office yet so I can't check for any renewal notices or even confirmation of our suspicions on how the domain was registered. At this moment, I'm at a dead stop.

Things like this aren't supposed to happen. It's a profound embarrassment & shame upon the clowns who created this situation, and the rest of us for not catching it in time. Needless to say, the balance of my day is going to be spent scanning all of the source code in that division to look for any other domain surprises and ensuring all of them are in our corporate portfolio where they can be properly managed.

Update: We've renewed the problem domain but are still trying to transfer it and several others to our regular portfolio. The snag we're finding in many of these domains is that the registrant contact information is incorrect: there's a typo in the email address! We're now in the process of getting around this by registering the typoed domain name, then creating MX records to point it to our corporate email server. *sigh*

Tags:

OMFG, Akamai!

We use Akamai as a content distribution network. For the non-geeky crowd, this means that some of the items in our firm's web applications are distributed to the planet by Akamai: when a user in, say, the UK, uses our application, the graphics, images & logos in our app are retrieved from a local server in the UK close to the user rather than reaching across the planet to get a copy from our server in Virginia. This gives the user a faster web experience, and it eases the load on our web servers in Virginia.

Naturally, we use web certificates so users can be certain (a) they are getting data from an authorized source instead of Someone Evil, and (b) it is encrypted for privacy, (c) it is secured for consistency to ensure the data wasn't manipulated in transit.

We have several web certificates for many domains in Akamai. Three of the certs will expire this week so Akamai contacted us for approval to renew them. I've been trying to tell them that we don't need the certs and to let them go, but they just don't believe me.

Akamai: There's three three certs which are gonna expire. Let's coordinate a time to talk to the cert vendor to renew them!
Me: Not needed. We don't use those domains so we can let the certs expire and save money.
A: But there's customer traffic!
Me: Not from us. We don't use those domains.
A: But there's customer traffic!
Me: If any, it's probably web bots and spiders. Don't care.
A: But there's customer traffic! Look, one domain name is CNAMEd traffic to our caches!
Me: Yes, the data is in the zone files, but we don't use those domains.
A: But there's customer traffic! You need to renew immediately!
Me: No, that's not us. We don't care. *Logs into the Akamai control panel, removes the certs. Updates DNS to remove any reference to Akamai in those zones* Look, I've wiped out the references to Akamai. We don't use the domains, we don't care about the certs, we want to them to expire.
A: But there's customer traffic! It will all fail now!
Me: What can I do to persuade you we aren't using these services and can let them expire?
A: *silence*

Tags:

Why I Voted for Hillary

Why did I vote for Hillary in the early polls in Maryland? The easiest answer goes back to Ronald Reagan: Are you better off now than you were eight years ago? And my answer is a resounding yes. At this moment, I'm healthier, stronger, wealthier, and generally happier than I've ever been in my life. The economy is doing well, my employment & income are stable, my basic human needs are well covered, LGBT rights are stronger than ever before, and I see no significant threats to my wellbeing on the horizon. Of course, not all of these can be linked back to the president or politics in general and none are perfect, but if Hillary represents a continuation of the bulk of Obama's policies, I'm all in favour. More of the same, please! I don't want this particular train of progress to stall or rewind.

Apart from obvious self-interest, I think Hillary has an amazing resume. Her past political experience shows she can be a shrewd operator, a dignified representative, a forceful advocate and a balanced negotiator as needed by the situation. True, not every decision was a good one and lord knows she's made some mistakes, but that's the price one pays for actually doing something instead of being a passive by-stander. Show me a perfect track record and I'll show you an under-performer who can't handle risks or leadership. "To avoid criticism, say nothing, do nothing, be nothing."

For all of the noise made about how untrustworthy she is, all I can say is that the accusations are being made by those whom I deem to be even more untrustworthy. Years of investigations by people who have dedicated their lives & careers to attacking her still haven't found the evidence they are convinced exists somewhere. So many of the stories are clearly fakes, distributed among the true believers in her being Satan incarnate: she isn't dying, she isn't sick, she isn't hosting drunked orgies, she doesn't have a half-dozen love children, she isn't having a lesbian affair, she isn't hosting satanic rituals, she isn't high on drugs, etc., etc.. She's simply a good target for conspiracy theorists who profit from selling tabloid fiction to the tinfoil hats. ProTip: If you have a stack of assumptions, dig for years for the evidence to confirm them and come up with nothing, perhaps your assumptions are wrong.

She is a woman, which in the eyes of many is enough to disqualify her for elected office, not to mention employment anywhere or even treatment as an equal. By contrast, I think it's high time a woman was elected into office. I checked the constitution: there is no requirement of a penis for the office of president, although there are some people who clearly think otherwise.

I could go on about how horrible Trump is, but it's not enough to vote against someone. I would vastly prefer to vote for a candidate than against their opponent. It's not good enough to simply give my vote to the least evil among evil people: if that was my choice, I'd skip that particular question on the ballot or write in another person if there is an option. This election however, I chosen to affirmatively vote for Hillary.

On Trump, I've only talked to a few of his supporters as my area is highly Democratic. Most of them complain about how horrible Hillary is, and how they're loyal Republicans, but few every speak openly of their love of Trump. They seem to be voting against a candidate rather than in support of their own. I have no patience for those who blindly vote their party without regard to the candidates. This isn't a sports team you may love in good times & bad with no consequence: the choices on the ballot will affect the future of everyone and requires consideration of the candidates more than the party.

Those who did voice support for Trump loved his anger, his promise to destroy convention, his blather, his ego and his unpredictability. These may be fine on reality television, but I absolutely do not want that in a manager let alone a president. I can see this as being attractive to voters who perceive themselves as having nothing left to lose: it's a roll of the dice which can't make things worse but have a chance at making something different and perhaps a faint hope of something better. I disagree: I think Trump can make things much worse for everyone, including himself. I would much prefer to fix the things which are broken than throw out everything, good & bad, to begin again from scratch. There is much wrong with the country, but there is also a great deal which is working well and should be preserved.

My vote was cast last week and we are mercifully in the final days of the election campaign. In my dreams, Hillary wins the White House, and the Democrats take both the House and the Senate. Trump concedes graciously, then the lame duck session of Congress proceeds to work on the stalled legislation & appointments in compliance with the declared will of the people, including the Supreme Court nominee on hold.

More likely, Clinton takes the White House, Dems take the Senate, Republicans keep the House, and Trump concedes grudgingly as a sop to history even as his ego urges him to scorch the earth with wrath about stolen & rigged voting. (FFS, Donald, just read the prepared speech on the teleprompter as written then step away from the microphone!) There is noise already that the Republicans will do all they can to block all judicial appointments forever until a Repubican regains the White House, but I hope no one actually thinks this is a good idea or sensible policy. I also hope that a Democratic majority in the Senate will push these appointments along the pipeline to conclusion.

We'll see how it all shakes out next Tuesday evening.

Shopping for new Glasses

I've been wearing the same frames for about eight years now, I think. For many years, my eyes had the same prescription but about two years ago, I had to go to multifocal lenses. Not finding any frames I liked at that time, I simply had the lenses replaced in my old frames. Those frames are showing their wear & tear however and I'm unsure how long they will last.

After making my appointment this morning, I looked at some of the frames on offer, quickly narrowing the selection down to about three. For laughs, I took pics of myself wearing each, as well as one with my current glasses. Combining those into a single photo, I posted it to FaceBook and Instagram to get feedback & reactions.

My favourite of the new frames received favourable reactions but to my surprise, my existing frames were the runaway favourite. Possible reasons:
  1. My friends are accustomed to seeing my existing glasses and are therefore unconsciously biased to the familiar;
  2. My current glasses are already an excellent choice for my face, my complexion, and facial hair colour, so the other candidates have a high bar to reach;
  3. My friends have the same taste, good or bad, as I do.


In any case, my eye exam is tomorrow morning. If my prescription hasn't changed, I may skip new glasses entirely. If it has changed and new glasses are warranted, I may defer the purchase for a couple of weeks to consider other frames.

Tags:

How Not to be a Sysadmin

My dear employer purchased another company about six years ago, adding ground & car bookings to our business travel portfolio. Of all of the acquisitions we've done over the years, this was practically the only one which made sense, the only one which has been financially worthwhile and the only one still operating, but that's another story.

This particular car service division has been largely independent of the larger firm: our travel systems makes system calls into the car service systems, but we haven't tried doing a full integration of their services or their staff. Our core travel systems are all based on Linux with Solaris/Oracle handling the backend databases, while the car service machines are all Windows Server based with Microsoft products and some cloud-based services to supplement.

In the past month, two of the primary people from the car services division have left the firm, and because we have no other staffing, care & feeding of their systems have fallen to my systems engineering team. And now we're seeing the true nature of the nightmare...

These car service systems require constant care. Constant. We've learned that the core database has been receiving manual maintenance daily for the past eight years it has been in service. We've learned the logging system has been manually restarted every 48 hours or so for the past number of years. There's a stack of little things like this which have been consuming the full attention of two fulltime staff on a daily basis.

I'm horrified by the amount of work that has been required daily if not hourly to maintain uptime for these systems. I'm horrified that no one in management seems to have noticed and thought it odd. I'm horrified that no one has seen fit to fix any of these problems, especially the guys who have been doing the work. And I'm horrified that even if the guys couldn't correct the root problem, that they didn't even attempt to automate the required recovery steps. Seriously?!

My team is now trying to pick up the pieces but I have little Windows experience and the training hand-off occurred while I was on vacation so I'm missing huge chunks of knowledge about their architecture, single points of failure, and other gems one could collect from those who built & maintained these things. It doesn't take great knowledge though to know that This Isn't Right.

Remember your training, young padawan:
1. Automate everything.
2. Automate recoveries as much as possible.
3. If something breaks daily, fix it.
4. Document everything so the people coming after you have a guide.

Transitions, Part II

In various telephone calls today with people at our San Francisco mothership, I have learned that two more senior staff from our Car Service division in Secaucus, NJ, have left the firm. By default, my team is now caring for their servers & services, although at this moment, I don't even have login credentials for most of their machines, not to mention an inventory. This is a sad and dangerous situation.

On the good side, my boss is going through the formality of interviewing a candidate for the recently departed Car Service director. It's a guy that the boss has already worked with, and with whom I worked with on a recent visit to California. If we can make him an offer, I will be a very, very happy camper.

In a vastly bigger bit of news, the board of directors has approved a budget of $1.2 million for support & upgrades of our data center facilities. At long last, we have approval to proceed with the hardware refresh in the data center where I work, after nearly a year of false starts. The boss is making some calls to confirm numbers and then we start pushing out the purchase orders, probably before the end of this week. My October is going to be very, very busy but it will be doing the work I love.

Tags:

Transitions

I've been away from work for two weeks, the longest single vacation I've taken in years, possibly ever.

I've been checking up on office email via my mobile from time to time, sometimes to see if there's any emergency which concerns me, but mostly to trim out the crap so I'm not overwhelmed with 1500 emails the moment I step back into work. I dislike surprises so I wanted to keep abreast of any developments. A lot of peace of mind can be bought for only ten minutes per day.

So far today, I've just been taking care of some outstanding tickets and minor issues which have been ignored in my absence. There is one server issue which has been generating an email message every two minutes for a week now... I'm unsure how that one was ignored all this time but I'll have that fixed shortly.

The HR folks have been busy. The official unlimited vacation policy is now in place, along with the steps for implementation on Sept 15, 2017. To clean up the bank of outstanding vacation days before that start date, the firm is officially shutting down for the week of Thanksgiving and the week of Christmas this year, and the week of Labour Day next year. That wipes out 15 vacation days, whether I want to or not.

I have no concerns though about the compelled vacation. Even after taking two weeks off, I have eight vacation days left in the bank. And with my seniority, I get another 20 vacation days before the policy starts. Even with the forced burn, I'll still have 13 vacation days leftover. Further bonus: the company shutdown days are exactly the date ranges I was planing for anyway.

Since my extra vacation days won't be cashed out, I'll find other uses. I still need to set aside days for my grandmother's upcoming 90th birthday in April, 2017, as well as for the IAGSDC convention in Palm Springs around July 4, 2017.

Tags: