by Daniel Chawner
Megs tried not to throw up on her keyboard, zero-early o’clock on Labor Day weekend, the morning after Margarita Madness. She swallowed hard and tried to focus. Her logins weren’t working, not the regular account, not the super-all-powerful admin account, nor the test accounts. She couldn’t get into the company’s main app.
Bobby, from over her shoulder, said, “See? See? We’re totally screwed. How did this happen?” He pounded the desk, sending Megs keyboard flying.
“Bobby, calm down. Now that I’m here, walk me through it again. You weren’t making sense on the phone.” Bobby had sounded hysterical, said she needed to come to work right now, and it didn’t matter she was at her friend’s lake house, two hours away. Hung over.
“It’s like I told you, I got this weird email. Then I tried to get onto our app and couldn’t.”
“Show me the mail.”
“I’ll send it to you.”
“No,” said Megs. “Let’s assume your emails are toxic.” More toxic than usual. “Show it to me, on your phone.”
Bobby thrust his oversized iPhone in Meg’s face. She pushed it away until the text came into focus and read it twice to comprehend the broken English.
“Robert Ugnaught, You have been pwned. App at Ugnaught Construction and Engineering are mine. Follow instruction below to buy and transfer $100,000 Bitcoin. You get 48 hours. Or delete everything.” Below the text were step-by-step instructions for buying Bitcoin.
“How did they do this? How did they get into UFA? I thought you and Ravi had locked everything up.”
“We secured what we could, but you wouldn’t let us lock down our system, remember? I wanted to buy those tools and restrict access to non-work-related sites?” Bobby loved to spend his afternoons on SnapChat, Among Us and other non-work sites.
“So is there something we can buy now?”
“No.” Megs rubbed her temples. Explaining technology to Bobby, the youngest brother of the family business, was hard under normal conditions. Talking to him through a pounding headache and dry mouth seemed impossible. “It’s too late. We should contact the FBI, or a firm that specializes in emergencies like this.”
Bobby stopped pacing and leaned on Meg’s desk. “We can’t let this get out. I mean, if we talk to the FBI or whoever, it will be public. And it will take too long to hire someone else, right? We basically have two days; we need everything perfect for Tuesday morning.”
“Or you could, you know, pay them and hope this all goes away.”
“I’m not paying ransom to a bunch of Russian kids. No way. And besides,” he said in a lower voice, “I don’t have that kind of money.”
Outside the window of their suburban office, Bobby’s Model S blocked the fire lane. In his reserved spot was his other car, a fully loaded Cadillac SUV. Bobby’s house, which Megs was forced to visit every Fourth of July for the big company party, was at least 5,000 square feet and had two pools.
“You hide it very well,” she said.
“What’s that?”
“Bobby, they aren’t looking for you to pay, right? It’s the company that got hacked, not you personally.” As long as we ignore you’re in charge of technology and security, and fired the CIO for disagreeing with you last year and never replaced him. “We should call Brad.”
“Listen,” Bobby said, “I think this is best if we just keep this between us. Don’t involve my brother. So, what do we do now?”
“Coffee. Why don’t you get me some? In the meantime, I’ll call Ravi and see if he has any ideas.”
“You think you can get him?”
“I don’t think they celebrate Labor Day in Bangalore. So, yea. Make it large with a splash of cream.”
Bobby raised his eyebrow. This may be the first time anyone asked him to run an errand.
“By the time you get back, I’ll have some ideas. Go, there’s a Starbucks a few minutes away.”
“Yeah, I could use some coffee. Be right back,” he said and pushed through the heavy door separating IT from the rest of the office.
The caffeine would help but getting Bobby out of her hair would help more. Ugnaught Construction and Engineering had moved their most critical app, Ugnaught Field App (UFA) to the cloud, servers that someone else owned and maintained. It let them lower costs and move faster. It also meant if their accounts didn’t work, they couldn’t use their cloud-based app and data. The entire firm, field engineers, back office, and the CEO, needed UFA to do their jobs. Megs tried her logins again, just to do something. Same result.
Megs felt helpless; she usually solved problems and kept the app running. Now some hacker from thousands of miles away was threatening their company and potentially their livelihood. Did they target Ugnaught, or cast a wide net? Did the hacker need money to pay bills and feed a family? Was it worth ruining our weekend, our career, our lives over this? When she was with Microsoft entire teams stood ready to handle this type of crisis. Here, we didn’t even have a CIO or proper network engineer. These hackers had Ugnaught’s fate in their hands. Megs clenched her fists, then called Ravi.
“Hey Ravi, do you have a few minutes?”
“Hi Megs, thought you guys were off this weekend.”
“We were supposed to be.” Megs filled him in on the details.
“I just tried my credentials as well. No luck. Do we know how this happened?” said Ravi, polite and helpful as always.
“Could be phishing, or we left something open on a server and they found it during a scan, or one of the software—”
“It’s almost always phishing, someone clicking on a malicious link, right? I can run a scan on emails from the last week or so and see if there are any likely sources. Luckily for us, we still have our network and email local, not in the cloud.”
“Will a scan help?”
“At least we’ll know how the hackers got in,” said Ravi.
“Alright, let me know.”
Megs stood, stretched, and shuffled over to a blank whiteboard. If they couldn’t get into their system and Bobby wouldn’t pay, they needed to rebuild. From scratch.
She wrote “Source Control”, “Data”, and “Backups” across the top of the whiteboard. The best choice is the backups, copies of their entire server made on regular intervals. Megs hurriedly re-dialed Ravi.
“Ravi, my mind is mush this morning. We didn’t talk about the backups.”
Silence.
“Ravi, did you hear me?”
“Yes, I heard you. Do you remember where we put the backups?”
Oh shit. “We back them up to the same servers, don’t we?”
“Yeah. We can’t get to the backups either. We intended to fix it over the summer, but…”
But there were issues over the summer. Instead of working on the backlog of work projects, like backups, cleaning up directories, auditing accounts and killing old, legacy jobs, she spent most of July and August with Mom and Dad. Megs had moved back to New Jersey from Seattle five years ago and gave up a nice career at Microsoft; she was on a Managing Partner track. But the weekly phone calls with Mom became more strained, and Rodger, her brother, cracked under the stress. So, she left the fast-paced world of high-tech and became a jack-of-all trades, part sysadmin, part programmer for a mid-sized, family-run firm.
In July, Rodger announced he couldn’t handle the daily visits to Mom and Dad anymore and was broke. Two days later, he and his family moved to Philadelphia, leaving Megs to buy supplies, drive to doctors, pick up medicine and arrange nurse visits. After two weeks of continual care, Megs found a local nursing home. Which solved one problem and created another: a $15,000 per month bill, payable in advance. In cash.
“Oh crap. But we have the off-site tape backups, where we write all the database information to a tape and mail it somewhere secure.”
“Let me check,” said Ravi, followed by the sound of typing on his keyboard. “Yes, but we only do full backups to tape once a month. On the fifteenth.”
“So, we didn’t lose everything, then. How soon can we get the tape sent here?”
More typing. “Thursday. Assuming they pull the tapes and mail them Tuesday morning.”
That’s better than nothing; that’s most of the data. “But the source code, that’s in a separate place. In GitHub, a different cloud system.”
“Yes,” Ravi said, “that’s true. But we do our builds, where we assemble and deploy the source code, on the main server.”
“So? That doesn’t change the… oh. We store our logins to GitHub there, don’t we?”
“Yup. In—”
“In clear text. Not encrypted, which was another thing on the summer list. If these hackers looked in there…”
“They could get to the source code and lock us out. Or wipe it. Or both,” said Ravi.
Megs jumped back in her seat and opened GitHub, furiously entered her login and password and held her breath while the icon spun. Then she exhaled; their source code was still there, filed neatly in branches. “I’m in Ravi, it’s all here.”
“Change your password, now. And disable the account that does the build.”
“Yep. You should login and do the same,” said Megs, smiling for the first time this morning.
“I may have found the phishing email,” said Ravi. “I’ll let you know when I’m sure.”
Megs leaned back in her chair. Like the tapes, something else tickled the back of her brain. Maybe closing her eyes would help.
The bang from the IT door jolted Megs out of her nap. Luckily, she fell asleep upright up in her chair, not face down on her keyboard.
“Here you go. Any luck?” said Bobby. He placed a large white paper coffee cup on her desk, light brown liquid escaping from the cap and rolling down the side.
“Ravi and I are working on it. Good news is we still have access to the source code.”
Bobby smiled. “We’re good, then? I meant to tell you, our app on my phone is fine.”
“They can’t lock that down. But, if you tried to connect to the server, you wouldn’t be able to. Remember how we have that set up?”
“Yeah, to let the engineers work offline. So, they have their own little databases on their devices.”
“That’s right. And, if they didn’t update anything, they could work for a day or two.”
“We could have some more time to repair this?”
“Kinda, it only solves the engineer part. The rest of the company, the execs, accounting, they all login to the portal which is on the cloud.”
“If I told Brad we were doing maintenance, and it went long, but the techs could still work… and we got it fixed by like Wednesday… then maybe they wouldn’t have to know.”
“Only if you hid the truth from them.” Megs shouldn’t have been so blunt. The best way to deal with this man-child was to nudge him in her chosen direction. She hastily added, “We can build the software, err, apps again. And set them up on a new server. But they won’t have any data. For that, we’d need to instruct all the engineers to upload their field data to the server and set up some rules for putting that stuff back in the database. If we get lucky, maybe we could restore half of the data. At least the most recent stuff. And then get the tape back by Thursday.”
“So that doesn’t help us at all. Shit.”
“It helps a little, it—”
“When the company comes into work Tuesday, when the engineers fire up their apps, when Brad sits his ass down at his desk and tries to pull up the monthly numbers, will any of that work?”
“The apps would be there, but no data.”
“So, they won’t work. Useless. I thought we had something in place to get backups. I see them on our monthly bills.”
“They got them, too.” No need to explain why they got them, at least not yet.
“Dammit,” said Bobby, flopping into a chair around the small conference table in the center of the room. “What else?”
“Ravi thinks he knows how they got in. He’ll let me know soon.”
Bobby pulled out his phone, slurped his mocha-colored iced drink through a straw, and turned away from Megs. Great, he’s going to stay.
“Are you sure we can’t call Brad and talk to him? Maybe he can negotiate with these guys?”
Bobby flinched at the mention of his older brother’s name. “No.”
“But we may not —”.
“No, we can’t let Brad or anyone else know. In fact, send out an email letting people know you took the system down for maintenance and they can’t access it today. In case some eager beaver logs in on their day off. Like Brad.”
An instant message from Ravi flashed on Megs screen.
Found out how they got it. Bobby clicked a fake FedEx link, and they got his admin credentials. Keyboard logger.
Megs opened her mouth to tell Bobby, then stopped. He was quiet. Better to keep him that way.
Great. Any other ideas?
No, will keep looking
Let me know, thanks.
What were the remaining options? Pay and get everything back. Maybe. Don’t pay and rebuild the system from scratch on the sly. Or come out with the truth and restore the apps with the company’s help.
Coming clean and telling Brad, CEO of the firm, made sense. He was an engineer; he’d see this for what it was, a puzzle.
“Bobby, this is where we are: we can rebuild the code. Probably take most of the day, but we could get the app stood up pretty quickly. We need to tell the users to connect to a different server… which isn’t hard.”
Bobby stared at Megs, expressionless. His hair looked matted, his eyes bloodshot and face puffy. Puffier than usual.
“And then we need to tell them to go into the settings of the app and do a one-way, err, push all the data they have to the new, empty database. Ravi and I could write some rules for what data to keep, what is most recent, that kinda stuff. But it won’t be 100%, and it may take us a few days to get the data straight. By then, we should have the tape backup. As long as it’s okay, we can piece together most of the data, probably with a few gaps from last month.”
“How long?”
“For which part?”
“Until everything is back to normal.”
“If everything goes like it should, and all the engineers followed the instructions… Friday, or early the following week?”
Bobby hung his head. “Not good enough. If I have to tell Brad about this… he’ll ask a lot of questions. And we’ll be screwed.”
“He might just choose to—”
“No chance. He’d rather shut down the company then pay a hacker. You know what he did the last time something like this happened? When a few computers and printers went missing a few years ago?”
“Faintly.”
“He ran a full investigation that cost more than the missing equipment. Questioned people for hours, put together a report, hired someone to help him. And got deep into everyone’s business. It was awful. Cost me a bonus that year also, ‘cus I run the office team as well as IT”
Megs stomach lurched again. Brad was intimidating; if he questioned how this happened, they’d have to tell him about the shitty backup plan.
“What else have you got?” asked Bobby.
“Right now, nothing,” said Megs.
He slammed his open palm on the table, rocking his latte. “Useless, you are useless. There has to be another way to fix this.”
“I’ll keep looking,” muttered Megs. She focused her attention on her monitor and randomly moused through her open windows. And tried not to shake.
This was Bobby’s fault. He clicked on the link and forced them to keep the system open. She could make Brad understand. But not having backups available was on her. Could she hide it? Brad, while analytical and fair, wouldn’t hesitate to fire someone over this. And he couldn’t fire his brother. Working at UCE wasn’t perfect, but the pay was alright and the hours, for IT, pretty manageable. And there weren’t a lot of other places she could work and still be near her parents. Megs now felt doubly trapped, by Ugnaught and these hackers.
Megs clicked on the window with the source code. Nothing new there. Clicked on the window with UFA. Her password still didn’t work. The only other system she had access to was email. Megs clicked on her inbox, her deleted mail, then on the separate folders she had in her inbox for something to do. Or at least to look like she was doing something. Out of the corner of her eye, she saw Bobby drop his head into his hands.
One of Megs’ folders had 700 unread mails, named LegacyJobs. Great, another reminder of the work we didn’t do this summer. She clicked in the folder and gasped. There it was: “FullDBJob09032020-Complete”, the answer to her hung-over prayers, the thing tickling at the back of her skull. One of the legacy items, a job she set up when they first moved their data to the cloud, took a full backup of their database, compressed it, and sent it to their old server. The old server, the thing running in the almost-empty closet next to her desk. Here, all morning. The company’s data backed up as of two days ago sitting on a piece of hardware she meant to turn off weeks ago.
They could stand up a server in a few hours, quickly restore the database, and have a busy Tuesday morning tackling minor items. No one at the firm had to know. Bobby would get off scot-free again.
Megs sat up straighter in her chair. The buzzing headache receded and she felt whole, strong. They were going to get out of this mess. Or, at least, she was. Now Megs held the fate of the app, Bobby’s career and the health of the company in her hands. No one else knew about this legacy backup, not even Ravi.
If the hackers were going to extract a pound of flesh, why not Megs? The nursing home bills would ruin her finances for years, maybe for the rest of her life. But not if someone else picked up the tab. An extra $15k a month would change her life, for the better.
“Hey Bobby, we know how the hackers got in.”
Bobby raised his head out of his hands and looked over at Megs.
“You clicked on a bogus FedEx link, and they stole your password. You handed them the keys.”
Bobby, who was pale already, looked deathly as blood rushed out of his face. “I, uh, how…shit.”
“And they really got us. Too bad we didn’t have a CIO anymore to pin in on, eh?”
Bobby squinted at Megs.
“Maybe someone like that could get us out of this jam. And would have forced us to buy the right tools, earlier, and get serious about blocking phishing links. Probably pretty pricey, though.”
“What good does that do us? Oh my God, I’m so screwed. We’re all screwed.”
“Yea,” said Megs. She paused to pick her words carefully. Days later, while she admired her new office and nameplate, she’d wonder if the conviction of her next line came from deep inside or that fourth margarita.
