Category: T-SQL Tuesday

Closing out 2019’s T-SQL Tuesday, Mala (blog | twitter) is asking us to reflect on the past twelve months:

This is a time for material gift giving, for many of us. It might also be a time to consider the many gifts we have received through the year, and perhaps use this opportunity to appreciate people or situations that we were blessed with. So my question would be – what are a few things would you consider as gifts, and why?

I’m publishing this a couple hours late and I’ll admit, I’ve seen a few other folks’ posts which have inspired a few thoughts of my own.

• An amazing job opportunity came along at the end of the summer. I wasn’t looking to make a change, but amazing things happen when you least expect them. The jump was completed in mid-October and every day, I’m discovering more things to learn and do.
• I know a few others have written this, but meeting more people in the SQL community and deepening the connections with people I already know has been wonderful. Y’all know who you are, and I am so grateful to know you 🙂
• To celebrate a milestone anniversary, my wife & I enjoyed a short vacation. This was the first vacation we’ve had where it was just the two of us since our honeymoon. It was pretty nice.
• I got a lot more comfortable with saying no. This applies to several contexts, but the most important one is saying no to myself. Over-committing is a habit I’m slowly breaking out of. I’ve learned that it’s OK to not force something into my life just because people expect me to do it. It’s OK to skip a weekly activity every now and then if you decide you just need a break. In a way, 2019 was about the things that I didn’t do as well as the things I did do.

For T-SQL Tuesday this month, Alex Yates (blog | twitter) gives us this challenge:

I would like you to write about something in your IT career that you have changed your mind about. What was your original opinion? Why did you believe that? What do you believe now? Why did you change your mind?

You are welcome to discuss technical or non-technical topics. Feel free to go as deeply technical or as personal and human as you like. Brain-melting technical posts about the inner workings of the SQL engine or effective machine learning architectures in Azure are great. SQL 101 posts or perspectives on age old debates such as tabs and spaces or where to put your commas are great too. Human posts about effective teamwork or diversity or wellbeing in tech are also great.

I hope that if we think hard about the ways we have changed our minds in the past, and if we read about how and why other people have changed their minds, it will help us to have better conversations in the future. I hope this will help us to collaborate more effectively at work – and maybe in other parts of our lives as well.

The Only Constant is Change

I’ve changed my mind on a number of topics over the years. It’s only natural. We learn more, we get more information, circumstances change. But the first and biggest one that comes to mind for me is my position regarding Microsoft.

Say What?

“But Andy,” you say, “haven’t you spent your entire career working with Microsoft technology?” Yes, yes I have! But I didn’t always feel good about it.

My formative geek years were spent in the 1990s, living through the meteoric rise of Microsoft, their missteps around the early days of the WWW, and then their total dominance of the market. You might have read about it in the newspaper. I studied Computer Science, surrounded by UNIX and Linux systems day in and day out and steeped in the idealistic dogma of “Open Source is good, Microsoft is evil. Embrace, extend, and extinguish! EVIL!!!!!!”

Sure, I still used Windows. I needed it for the dial-up connection I had at home over the summer and to play games (it always comes back to games, doesn’t it?). At the time, it was Windows NT 4.0 Workstation because I had (thought I had) higher computing demands than I could entrust to “consumer” Windows. Except…Microsoft changed the video driver architecture and the driver for my video card was buggy in such a way that some JPEGs loaded in Netscape Navigator could crash the whole OS. Gee, how good can Microsoft be if I can’t even load a JPEG without crashing Windows?

As time went on, I found myself trying avoid Windows more and more. My final semester of college, I don’t think I used Windows for more than 15 minutes total. Linux all the way!

But I Need a Paycheck

In the middle of my final semester of college, I accepted a job that used Windows everywhere – desktop and server. Some of my friends shrugged. Others sneered. I wasn’t thrilled about feeding into the “Evil Microsoft Empire” but I needed to eat. As I wasn’t the best student, major companies like IBM and Red Hat weren’t exactly lining up to interview me on campus.

I begrudgingly used Windows at work. And I got pretty damn good at it. Within months, I was fixing major outstanding bugs in our ASP 2.0/VBScript apps and moved on to building new apps from scratch. But at home I was still running Linux all the time, holding on to that idealism. Until…it broke.

I had an distro upgrade go bad and decided I was over my Linux phase. Microsoft was still kind of “ick” but at the same time, I was done tinkering w/ my OS and just wanted something that was left in a 100% working state after the installation was finished.

OS/2 on the desktop was a flop so it was back to Windows. Windows 2000, then Windows XP fit the bill at home. And my employer was still all-Windows.

The next computer I bought for myself was a MacBook. I had been fascinated by macOS since seeing the first public preview of Mac OS X way back in early 2000. It was UNIX with a candy-coated shell! I installed Windows in a VM for the one or two pieces of software that I still needed that for, but I was going all-in on the Mac at home.

Did the Suits Take Over?

By then, Steve Ballmer’s version of Microsoft was in full effect. Vista finally happened, leading me to I question whether Microsoft could get anything right. Everything seemed to be built around “how can we make you a Windows customer, no matter how bad Windows may be?”

The Pendulum Swings

Microsoft’s OS group redeemed themselves with Windows 7. Hey, they can make a decent OS after all. Things started moving forward (let’s try to forget Windows 8 happened). Azure started spinning up, taking on Amazon.

More importantly, software releases became more frequent and more regular. Security improved, quality improved, and you could start seeing more influence from the engineering side of the company emerging. The business drivers started to take a backseat to delivering products to the people who influenced technical decisions. Build good products, and the business will follow.

The tipping point for me was when Satya Nadella took the reigns in 2014. Microsoft seemingly was returned to the engineers. Open Source was embraced and encouraged. Microsoft was even releasing things as Open Source without being asked about it! They moved their focus to “we want to host the platform you run your business on, regardless of the OS and toolkit – but here’s a whole bunch of awesome tools that you can build with if you want to.”

The Microsoft of today is not the company I grew up watching and fearing (that position is now jointly held by Google and Amazon). They’re building exciting, compelling products and driving the industry forward. It was a rough few years, but I have done a U-turn on Microsoft. While I’m dabbling in other environments because it’s the smart thing to do for my career, I’m still working primarily with Microsoft technologies and it’s a decision that’s served me well for over twenty years.

This month’s T-SQL Tuesday comes from Shane O’Neill (blog | twitter). He starts us off with this:

Dipping into the Cookie Jar is about when the going gets tough and you don’t think you can handle anymore, then you think back about your accomplishments and take some sustenance from them. You dip back into that cookie jar and use whatever energy that provides to keep going.

So tell me about a time when you had an accomplishment that can keep you going.

I. Love. Cookies.

I feel like this could describe my entire career. I cannot tell you how many times I’ve found myself staring looking at a new project, thinking “how on earth am I going to pull this off? This is nuts!”

I do exactly what Shane describes above – I just didn’t know there was a name for it. I have to remind myself “hey, you’ve said this same thing before, and it worked out. Breathe, slow down, and you will figure it out. You always do.”

For the really gnarly problems in the past, I’ve hidden away for a few days to just hammer away at code. So, what do I do when I get to the next one? I remind myself that if I do the same thing, I’ll get through.

Here’s the thing

In this business, you should never be doing the same thing over and over. Those are the things you automate so you can move on to more interesting problems.

There should be something coming along new every week. If there isn’t, make something new happen. If all you’re doing is things you’ve done before, things that you already know exactly how to do, you’re going to stagnate. You’re not going to learn. You’re not going to grow.

For me, it’s all about breaking the problem down in two ways:

• Find parts that you know you know how to do
• Find ways to experiment with the parts that you don’t know how to do

The first one is your cookie jar. Break that project down and tackle the stuff you have seen before. Maybe you’ve even got a time-tested script or checklist to get you through it.

On to the stuff that’s new. This is the exciting part! You won’t get it right on the first try. Everyone falls the first time. That’s how we learn. That stuff that’s old hat to you in the paragraph above? There was a time you didn’t know how to do that, either. And now look at you – you’ve mastered those parts.

An Example

Right, Shane wanted real examples, didn’t he? Time for me to go back to the well that is my 8000-database migration. Let’s break that down:

• Upgrading from SQL Server 2008R2 to a newer version? High-level, I’ve done that before. Had a sysadmin doing most of the heavy lifting, but I was right there with him for most of it. 🍪
• Oh, but wait! I worked on an upgrade from 2000 to 2008R2 before that. We kept the whole workflow document and checklists around from that so we built on that. 🍪
• I migrated from 2008R2 to 2008R2 before using dbatools. That went pretty well. 🍪
• When I did that previous migration, I successfully copied things in groups, I didn’t have to do it all at once. Just like the big migration to 2016. 🍪
• As an organization, we’d previously moved our 2008R2 setup to new server hardware, with minimal issues. We learned from the breakage we had and documented those lessons. 🍪
• When I stood up the new test 2016 instance, I copied everything but the databases from the 2008R2 test instance and that worked out OK. 🍪

So, what was I left with here for the big migration? I’ve been through version upgrades. I’ve been through scripted migrations. I’ve copied everything except databases from 2008R2 to 2016. We’ve changed hardware before. So many cookies!

The only part that I was really worried about was attaching the databases themselves. As a result, I was able to test that out a couple dozen times to make sure it would work properly. And no, I didn’t get it right on the first attempt – it took probably a dozen attempts before I got the attach script working properly. But I knew it would be OK, because I’d worked on plenty of other projects where the first attempts were unsuccessful as well.

Keep at it

You didn’t give up in the past, and you found success. So keep plugging away and you’ll keep succeeding.

This month’s T-SQL Tuesday is hosted by Jeff Mlakar and he asks us to write about a project that went horribly wrong. My story isn’t really worthy of the name “death march” but it was a pretty rough project.

The project started sometime in mid-2003. I was working as a web developer (Classic ASP) for an insurance company and they wanted to modernize the underwriting process with a web-based “workflow” application.

A platform was selected, the team was picked, and we set about customizing the devil out of it (it was more of a “framework” than turnkey application) and integrating with our existing systems. Of course, being an insurance company everything ultimately had to land in the mainframe.

The tech team was pretty large – upwards of a dozen and a half programmers across various disciplines. Four of us were kind of off on our own building the ASP front end and COM-based middleware – myself and Ned on the ASP side, Tony & Phil (all names changed) on the other side of the cube walls working on COM objects. The four of us worked really well together, with Phil & I each taking the lead in our respective disciplines. Our PM was great; we got along well and she knew the right questions to ask me to nudge me in the right direction.

As Phil & I dug into the platform API, we realized we were going to be implementing more features with our own code than expected as they weren’t actually built into the API yet.

We brought a few consultants from a large company you’ve certainly heard of to help us work out a UI design. After 5 weeks, we presented our design proposals. Every – I mean every – proposal put forth by the consultants and me was shot down. The response was “those are very nice, but we want what we described to you 2 months ago, so build that.” I was flabbergasted. I asked the consultants if they’d ever seen anything like that and they admitted that they hadn’t. We just spent five weeks and a bunch of money to produce nothing.

February was approaching and while Phil, Tony, Ned and myself were firing on all cylinders, others weren’t doing as well. We were supposed to launch in March, but it was becoming apparent that we’d miss that date. The release date slipped to mid-April.

The four of us spent the last half of February and all of March tightening up our code. We had some performance concerns but hadn’t figured out where they were coming from yet. Acceptance testing hadn’t started yet; the project was expected to be “nearly complete” before that’d start.

Acceptance testing started in late March. Ned & I knew our code inside and out, and I could suss out pretty easily if a reported issue was there or in the COM objects so I could throw those bugs over the wall to Tony & Phil as necessary.

I returned from lunch on the last Friday of March and Ned said to me “I’m leaving at the end of the day.” Um…OK, thanks Ned, I am too. Then he clarified. See, Ned was a contractor and he he was ending his contract at the end of the day – and starting a new gig in Miami on Monday! We were unable to bring in a replacement, so I was flying solo for the rest of the project.

I spent April fixing bugs as fast as I could, re-implementing some features because they sounded good on paper but once seen in action, they weren’t acceptable. Mid-April came around and the project release date was pushed back by another month.

I went to the PM and expressed my concerns about the release date changes. I reminded her that I was getting married Memorial Day weekend and I’d be unavailable for 2 weeks due to that and the honeymoon. She assured me that this was the last reschedule, and even if that turned out to not be true, I wouldn’t be asked to change my plans.

Early in May was the final blow for me. We had an all-project meeting with several people from the highest levels in the company in attendance. The missed deadlines, the incredible strain placed on not only the development team but their families, the importance of the project to the company, all brought up. People laid some very personal feelings out. The response from one of the top-level people was cold, harsh, uncaring, impersonal, tone-deaf, and completely disheartening.

The project went live 10 days before I left for my wedding and honeymoon.

Epilogue

No project implementation is perfect out of the gate. Right before we went live, Phil figured out what was causing our performance issues and over the next month or two, more or less re-implemented that portion of the vendor’s API himself. We pushed out an updated version as quickly as we could; users were complaining about performance from day one. We got a lackluster “yeah, I guess it’s better now” but no real acknowledgement of the improvement Phil had made so quickly. It was too late; the perception was that this system is slow and that wasn’t going to change, now matter how fast we made it.

The project was started in the summer or early Fall of 2003 and went live in the last third of May 2004. By the end of the first quarter of 2005, the company had decided to go in a different direction and the system was on track to be mothballed; no new business was being scanned into it, and once the last piece of business that was in it was processed, everything was shut down.

The system spent more time under development than it did in use.

This month’s T-SQL Tuesday from Aaron Bertrand gives us a choice:

• Life beyond the technical in a search for what Drew with the Burdensome Name calls #sqlibrium
• Your own T-SQL bad habits

And I’ve got one, maybe two posts in progress on the first topic. Alas,

Thanks to Eugene Meidinger (b|t) for nudging me in the direction of posting this. Ever since this month’s T-SQL Tuesday was announced, work and non-work has been a complete whirlwind and I have failed miserably at reaching sqlibrium.

I’ll expand on this in later posts and link back. Really.