I’ve learned two important rules since becoming a CSO. One: You can’t argue with a technology expert. And two: You’ve got to argue with a technology expert.
I remembered this particular lesson on a Tuesday. It hadn’t even been such a bad Tuesday. That is, until Technology Guy came charging into my office.
“Forget it,” he says to me, responding to an e-mail I had just sent him. “No way am I going to do that.”
“Why?” I ask blankly.
“Because it doesn’t make sense,” he says. “It’s stupid security. And when it fails, I’ll end up taking the blame for it.”
“Look, it’s not your decision,” I remind him, “nor is it mine. Big Boss has asked us to open the port in the firewall to support a new business application. You remember Big Boss,” I say to him in my most controlled management voice. “He’s the one with the big, cushy chair, approves the budget, signs your paycheque…”
“It’s still not happening,” he tells me, defiantly. “They’re using Report Procedure Calls with that application, which require me to open up not one port but 2,000 ports, which makes my firewall worthless.”
“Nevertheless…,” I struggle to inject my own defiance.
“No way,” he interrupts. “Not happening.” He leans emphatically across my desk and looks me straight in the eye. “Tell them to rewrite their damn application, and tell them to use real software for a change.” Then he storms out of my office before I can say another word.
Hoo boy. Technology Guy should walk a mile in my wing tips. He doesn’t get that this issue is nonnegotiable. The ports need to be opened so that the app can run. Period. Firewall or no, we need to solve the app problem first. And there’s no way they are going to re-code the application – the company has a lot riding on this. I really need Technology Guy to help me out. But he doesn’t seem to get the big picture: The business needs to make money to survive. This app will make money. Ergo, we need this app.
I give Technology Guy time to cool off and then walk down to his office. He’s still fuming, however. He says there comes a time when every good technology guy has to put his foot down on bad security. Apparently he has chosen this moment. He calls me a wimp for not standing up to corporate. “Nothing personal,” he says.
He thinks I should explain to Big Boss that we are all going to die because of security stupidity. I make one last attempt to explain, but he refuses to discuss it any longer. Technically speaking, of course, he’s right. But Big Boss doesn’t want to hear about security problems – they only cost him time and money.
A conversation with the programming team doesn’t help matters either. I explain to them about opening the 2,000 ports and why that might be a problem. Don’t they care about my problems with security?
“Not so much,” Project Boy tells me. “You should have said something much earlier in the process.”
“Really? When?” I wonder to myself. “You’ve been working on this project for a year. I never heard anything about it until two weeks ago. Where was the security review of the project?” I ask impatiently.
“We didn’t do one,” he retorts, knowing I’m trapped by a technicality: “This isn’t a security product, so no security review. You were supposed to be kept in the loop by Big Boss. Didn’t he tell you about it?” He smiles again. It’s obvious that I hadn’t heard a word from Big Boss. “Guess you need to take this up with him then,” says Project Boy. “But if you don’t get those ports open, we’ll both end up in his office. And my project has more priority,” he says. And he’s right.
Nothing like being the meat in a crap sandwich. So it’s back down to talk to Technology Guy.
“Is there any solution we could use that will solve the problem of the RPCs through the firewall?” I ask politely.
“Absolutely not,” he says. “They’ll just have to re-code the app. It’s the only solution.”
Secretly, I find it hard to believe. But I persevere. “I’m sure someone has solved this,” I say, not knowing anything for sure at this point.
“Nope,” he says. “And I’m an expert when it comes to this sort of thing.”
Technology Guy may be an expert, but I’m a manager. Not every CSO may be up to snuff when it comes to technology, but we know other managers and their own technical people. And it’s a good way to check out the truth from time to time.
So I call my friend, Manager Maven, at a company across town and explain the situation. He says they had the same problem, but one of his guys came up with a firewall that could deal with RPC calls. Seems that applications using RPCs have to negotiate whichever of the 2,000 ports they’re going to use on Port 135, and then they use the negotiated port. RPC firewalls that understand how RPCs work shut down all ports except for the ones where the apps have negotiated a common port between the two. That way, there are no open ports without an actual app attached to them. The other ports aren’t available to scanners or hackers that come calling.
I also call Vendor Professional. He has a product available for servers, so we could use the existing firewall and park that software on the host server without disturbing the firewall we have in use. Pretty slick. All we need to do is open the 2,000 ports and then fix them to the IP address for the RPC box, which would not allow the ports to be used with any other server. Problem solved.
Vendor Professional agrees to come by for a demo. Everyone – Big Boss, Technology Guy, Project Boy – are gathered in the conference room. Vendor Professional shows us how his RPC-savvy firewall product would work and offers to install it right away.
Predictably, Technology Guy asks a lot of tech questions, but Vendor Professional is prepared and answers them all. Also predictably, Technology Guy leaves the conference room in a huff. Oh well.
Vendor Professional installs the product on the server and, sure enough, it protects the server properly, it deals with RPC strangeness, and it works with the existing firewall. Nice job, says Big Boss. Let’s make this happen. Happy to do so, I think to myself.
Except for one thing: I need to get Technology Guy to open up the ports on the connection point firewall to talk to the Internet. When I appear in his doorway, he looks up smugly and says, “Told you it wouldn’t work.”
“Wrong,” I say, even more smugly. “It’s up and working, and everyone is happy but you. So you need to open up the 2,000 ports and Port 135 and set them to go to the server’s IP address only.” Smugness aside, I think I should be commended for my good mood given all the grief I had put up with from Technology Guy about the subject.
And that’s when it happens. A dark cloud appears over Vesuvius, and…it…blows. “People who don’t know anything about security should not be messing in security stuff,” Technology Guy rants. I know he means me, even though I am not completely without a clue when it comes to security technology. “This has completely violated corporate policy,” he says. “No one understands the dangers this will unleash.” It was almost tragic.
Still, he sticks to his guns and refuses to open the ports in the firewall. I ask him politely – one more time – and still he refuses. As a manager, I know that, in a deadlocked situation, a leader has to make a decision. Yes or no. A non-decision becomes a decision, and the factors will spin off in an uncontrollable way. In this case, I decide yes, we will make the changes and no, I don’t need Technology Guy to do it. I track down his backup and ask him to give me the password to the firewall. I may not be an expert, but I know how to open a damn firewall.
“Can’t,” he tells me. “Technology Guy changed the password on the firewall and won’t give it to me, so I can’t make the changes you want. He said we have to put our foot down on bad security practices.”
That so, huh? I call the firewall vendor and ask how to get the password out of the firewall if the security manager won’t give it up. It won’t be easy, they tell me, but it can be done. I have them back up and look over everything to make sure there are no back doors or other issues. “Nope, none,” they confirm, and even offer that the firewall looks as if it has been meticulously maintained.
When Technology Guy comes back from lunch, he loses it again. “I told you we could not make those changes,” he shouts.
“I know,” I say quietly. It’s amazing how easy it is to keep your cool when you’re in control. “You already said that. You also said there was no solution to the RPC problem. You said the solution wouldn’t work, and it did. Then you refused to help out with alternative solutions. Why?” I ask.
“Someone has to put their foot down and keep the company from killing itself,” he says. “You’re not the only one who can backdoor a firewall. So if you changed the password on the firewall, I’ll just change it back.”
And that’s when I remember the third important rule. When dealing with difficult, uncompromising, domineering, pig-headed people, sometimes you just gotta do what you gotta do.
“If that’s the way you feel about it, I am truly sorry,” I tell Technology Guy. “You’re fired.” Nothing personal.
