S1E3: Spam, Scams, and Sabotage: Tales of Adventure from the IT Helpdesk
S1E3: Spam, Scams, and Sabotage: Tales of Adventure from the IT Helpdesk
The basement-dwelling nerd isn't just a boring stereotype. And while it's never been accurate, it's less accurate than ever in today's interactive and fast-paced world of IT and data security. Jim Timberman and Chadd Wheat discuss safety tips, horror stories, and the daily challenges IT Helpdesks face and overcome.
Chadd WheatPrincipal Consultant and Service Desk Manager at Moser Consulting
Jim TimbermanManaged Services Managing Director at Moser Consulting
Announcer: ASCII Anything, a podcast presented by Moser Consulting. Join us every Wednesday to find out who for Moser's more than 200 resident experts we'll be talking to and what they're focused on at the moment. Trends, security, setup, ASCII anything and we'll give you our best answer.
Angel Leon: Hello, everyone. Welcome to another episode of ASCII Anything presented by Moser Consulting. Today we have a very special episode for you where we are talking about IT help desk. What does that look like today? A lot of people still believe that an IT help desk is some guy in a dark room on a computer just typing away solving all our problems. But that's not really what it is. It's a lot more involved than that, so we have two of our resident experts here at Moser Consulting to talk about that. Our guests for today will be Moser managed services managing director, Jim Timberman, and principal consultant and team manager, Chadd Wheat. Jim has been at Moser Consulting for six years and is a results- oriented business consultant with an outstanding track record of delivering cost effective technology solutions. Jim has led the Moser managed services practice in becoming a trusted resource for IT help desk and managed services solutions for dozens of companies from small to large and in many industry verticals. Chadd has been with Moser Consulting for the last 10 years. He's currently a principal consultant with a background in infrastructure services, enterprise monitoring, and team leadership. He was recently promoted to the position of service desk manager for Moser and is responsible for managing our ticketing system and processes, and ensuring that Moser managed services meets and exceeds customer expectations on a daily basis. Gentlemen, it is a pleasure to have you here with me today. Thank you very much for joining us.
Jim Timberman: Thanks, glad to be here.
Angel Leon: So we're going to jump right into these questions because I think it's very important that we discuss how people look at IT help desk. So when we imagine what an IT help desk is, most people think about a guy sitting at a desk waiting for the phone to ring, but I don't think that's the reality, is it, guys?
Jim Timberman: Yes and no on that. A lot of your larger managed service providers do have a staff kind of sitting in call centers waiting for that phone to ring and their model is based on that. So it's call us, we'll take care of the issue. But the downside of that is or I should say the upside first on that is that they're readily available. The downside is that your issue or your request isn't resolved quickly. Their model really is to take the call, capture the information, and then escalate that to someone who can resolve your problem. Pending a minor issue of a password change or so forth, so a lot of those are scripted out for them. Our model within Moser is a lot different. We're really kind of based on customer service and establishing our relationships. And I'll kind of let Chadd talk a little bit more about our approach.
Chadd Wheat: Yeah. Like Jim said, our approach basically is while we have the customer online or on the phone, we try to understand the request and/ or the issue, capture as many details as we can, and work to resolve it while the customer is online or near line with the agent handling that call. And 95% of the time, we're able to resolve that issue and complete the request while we're talking to the customer. We have numerous ways our customers can get ahold of us. We have obviously the help desk line. We have an online portal where customers can submit their tickets in writing and sort of collect their thoughts better and even put screenshots. We have email submission, of course, and a newer tool we have for PC is called Connect Moser which sits in people's system tray icon where they can basically right- click on that and be attached lifetime to one of our technicians. And we find that that interaction, that right away interaction helps not only sort of de- escalate the customer because typically when somebody calls a help desk or a service desk, they're not in a very good mood because something's gone wrong. So that helps de- escalate that. It helps them know that our people are actively working and they haven't just called any answering service as it will that the actual person they're talking to is working on their issue right now.
Angel Leon: I was just going to say that I think it's very interesting to see the amount of solutions that we have available to us right now. I mean, I remember when I joined the workforce 20 some odd years ago like Jim was saying, back then, you probably could just get on a phone call with somebody and try to have that person basically just fix your issue over the phone if at all. And like Jim was saying, sometimes the person would call angrily because something wasn't working and then they wouldn't get their issue resolved right away. So I'm very curious to know about all these different methods that we have nowadays to have those solutions readily available.
Chadd Wheat: Yeah. Well, let me reiterate the goal for our managed services team is to resolve the issue as expediently as possible.
Jim Timberman: Our goal is essentially to resolve the issue while we're in contact with our client. Not to, " Hey, we'll get back to you." Escalate it to somebody who can. 95% of our interactions with our customers are resolved at first point of contact. Usually, if we're escalating, it's something that's a major issue that are possibly outside of our realm of experience. Maybe it's a ISP issue that there's a connection with the internet provider that we just don't have access to or it's an application issue that we need to get in touch with the vendor. Because of the way our staff is built and the way the managed services teams are built, we have the skill set to usually take care of that right away. And the team camaraderie that we have, a lot of these guys are sitting right next to each other that they're able to just pop up and say, " Hey, can you help me?" Boom, that individual comes over and helps.
Chadd Wheat: I was going to say for expediency sake too, our goal is to solve that issue right there online. We don't want to send a technician out to a company site or a user site if we can help but we want to help them right then and there without delays or without having to schedule further downtime for a user or a company. So yeah, like Jim said, 95% of the time, we can do that through our online tools, everything from server side manipulation of accounts and issues to actually going into the user's desktop with some remote tools and solving the issue right there right then with the user actually guiding us and showing us what the problem is.
Angel Leon: I was going to follow up with Chadd, was just saying with a good segue into what we're living in right now with the pandemic. Chadd, you were mentioning that the focus is to fix everything right away, right? And not necessarily have to send somebody into a company, but how has the pandemic changed how companies need to interact with their users especially for things like onboarding problems, things that happen through that initial setup? Because with the pandemic, obviously, if we can work it out remotely, then I guess that's probably better, right?
Chadd Wheat: Right. Well, Moser, we've all sort of been geared toward that model. So when this whole thing broke out in the beginning of 2020, we were already on board with doing remote work and assisting customers remotely. Now, the shift has been on the customer side where now, they have remote work forces. And so we've had an opportunity to educate a lot of different clients as to what working from home looks like, what their support's going to look like. And so I think, really, we and our clients have been highly successful in adapting that because frankly, we were prepared and we were doing this before we had all those stay- at- home orders and all the pandemic madness that we've had. So we were very well equipped to handle this.
Jim Timberman: To kind of expand on that too in that when we kind of saw the signs happening, we actually took a very proactive approach with our clients to kind of talk to them about, " Hey, how are you going to virtualize your work force?" A high percentage of our clients, they're bricks and mortar type companies. A lot of the folks that work there, that's where all their work gets done. They're not used to or have never actually connected remotely. So not only implementing the tools to get them to be virtual, we have to look at, how are we going to train your users and how are we going to be available to them? So we took a very proactive approach and said, " Hey, we're going to kind of step back for a second and focus on nothing but being readily available to our clients." And that means expanding kind of our service desk offering saying, " Hey, we're going to offer this number to everyone. We're going to be readily available, be able to answer the phones and talk through that." We've also provided educational tools to them on how to connect VPN, videos, et cetera, so that they're prepared so that when this does happen, it's a smooth transition. I did not say there were hiccups, but a lot of it went a lot smoother than we anticipated.
Chadd Wheat: During this time, it's especially important to remember that the safety of our team and the safety of our clients is the utmost priority. So early on, we adopted some protocols for our team so when they do indeed have to go on to a client site, we have things like self- assessments, personal protective gear, and those kind of things. So again, we're trying to minimize any possible exposure to anyone who's been exposed to COVID. And like I said, our self- assessment tool, every morning whether our team is coming in to a Moser office or a client site, each individual has to self- assess and that assessment is sent to their management and to our HR department for review and recommendation on whether indeed they should go on site. So safety is very, very important to us.
Angel Leon: Speaking of electronic help desk, companies that didn't have these right now set up, how have you heard or have you seen the way people have scrambled to kind of catch up to this new normal of being able to do something from that space away from the office?
Jim Timberman: To answer that, yes. There's the managed services providers out there. There's what we would call the giant larger companies that have the call centers throughout the US or even locally who really kind of have the infrastructure in place to support that. But a number of smaller mid- tier managed service providers that really focus a lot on field work where they're providing field services where they're deploying individuals on site to go fix things really did scramble a lot and really struggled through this because they didn't have the infrastructure or tools to support these clients. Talked about this a lot because during the whole COVID, we had a lot of new clients onboarded just due to the fact that they felt that their current providers couldn't support them in this environment and we were able to actually thrive from that. And our concern internally was, " Yeah, we have the tools but do we have the capacity in order to meet this?" Even though you are taking the personal side out of it, you still have to have the resources available to support all the inquiries and requests coming in because again, as we mentioned before, you're taking a staff of people that have never worked from home. When they go home, they shut down. Now you're asking them to do that and you're asking them to also balance their life. A lot of these are families with young children that their kids are home. They're having to balance between when they do their work versus when they're helping their children in virtual school. So our availability had to expand out too. So having the ability to not just be nine to five support, we had to go above and beyond 24/7 to be available to our clients' needs. So the typical call that comes in at one o'clock in the afternoon now comes in at seven o'clock at night or nine o'clock at night where that individual has time to sit down and start working. So in a lot of that too as it relates to not just worker side of things and also the network and infrastructure side, is the network performing? So a lot of things had to be changed as far as making systems and networks available and changing the time when reports are being run, so performance was optimal. So a lot of that planning and so forth there. A lot of companies just didn't have the capacity to do that.
Chadd Wheat: Yeah. And I think one of the keys to Moser managed services too is we've always been customer service oriented. So sort of referring to your previous question about what people perceive of a service desk or a help desk, our team has never been just people sitting in logging calls. We've always been focused on connecting with the customer. And I think during COVID, that brings a certain amount of comfort and relief to people who are basically trapped at home. When they call our team, they're talking to a live person who's going to solve their problem right then and there. Like Jim has said, we're focused on solving their problem right then and there and our escalation model while we have it in place. We hardly ever have to utilize it because we're getting things solved right there and then and the end user is talking to the person who's going to be servicing their ticket.
Angel Leon: And that's very important. I kind of like the way you phrase it, Chadd, about people under the circumstances that we're living in just having that someone to talk to because at the end of the day, like you said, it's customer service driven. We're always looking out for the customer as an organization, so I really like the way you put it there.
Chadd Wheat: I was telling one of the people we just onboarded, I said, " Really gone are the days, sort of the'80s,'90s where people pictured an IT professional sitting in his mom's basement in a dark room tapping on a keyboard with Mountain Dew." The job is customer relationships and customer service and that's where IT in general has shifted focus from, the nerd in the dark to somebody who you can relate to and communicate to.
Jim Timberman: And I want to elaborate a little bit on that too and a lot of the feedback we got from our clients was the amount of compassion and understanding our guys and our team put out there. So it didn't matter how simple the question was or how difficult the situation was because it can be a little bit frustrating when you're trying to communicate your issue over the phone or trying to recreate what happened and there's that level of frustration that a lot of people have with IT that we were able to address. " Hey, slow down. I got all the time in the world to take care of this for you. I'm not trying to get you on and off the phone. We're walking through you with that." And a lot of our model and our approach isn't just it's customer service facing at the user level, it's also that communication and planning at the management level too. So we're constantly providing feedback to our clients. Here's the incidents. Here's the requests we're getting. Here's the type of things we're seeing. And we're always looking to try to improve that for them to say, " Hey, we're seeing a number of these types of tickets coming in. Hey, we need to put something in place to correct this." Whether that would be implementing a new technology, or making specific changes to that environment, or educating the user and making those things available so that they're like, " Oh, yeah, I remember now. Okay. Now that I've seen how to do that, I can do it myself." A lot of that communication and planning was ongoing throughout this whole pandemic and still continues to go on.
Angel Leon: So going off of what you were saying, Jim, I think it's interesting you talk about the frustration of the clients because you're right. I mean, I consider myself to be technologically savvy kind of guy, not necessarily the most knowledgeable but it does feel frustrating when you don't know all the IT jargon that somebody else might know when you're trying to explain something that's going on and you can't really just kind of say it outright. So it's interesting that you mentioned that our guys, the passion that they have for the business and the passion that they have for following through for our clients. I think that just speaks volumes to their knowledge, their skills, and their abilities.
Jim Timberman: Yep, I would agree. And when we look to bring individuals on to a managed services team, we do look at their skill sets. We do look at what they know technology wise, but we also do look at kind of the personality because I do believe that you can teach the technology skills, you can't teach the interpersonal skills.
Chadd Wheat: That's sort of how we approach hiring also because, for example, you take a typical college student who's been to university for four, five years and learned all the technical skills. We usually tell them in the interview, " We're looking for somebody who's going to fit culturally and chemically with our team and be able to work as a good team member." Because frankly, with enough exposure, we can train anybody to do about anything. So we really look for good communicators and people with good customer skills.
Angel Leon: Yeah. Those soft skills are very important. So I commend you guys for doing that because the technological skills, like you said Chadd, you can teach them, you can show them the way but those soft skills, the way to treat people, the way to show empathy, that's very important. So I want to shift gears here a little bit to a different topic. So we got a question here about, what should you be doing or not doing about email viruses, scams, and spam both as a person using an email system and as an IT leader responsible for the email system of a company?
Jim Timberman: That's a great question. That seems to be a lot of discussion today and more so in this pandemic situation we're in. The COVID environment that's been established is that phishing, email attacks, hacking have been on the rise. And it's just been a little bit more prevalent because everybody is now in a virtual environment. Companies have kind of opened themselves up a little bit more in order for folks to get in via virtual instead of being kind of contained in their environment and in their own network. So we've seen a rise of, " Hey, is this is a spam email? Oh, wait a minute, they stole my email," typical things. And we've really kind of combated that. I mean, you really can't stop everyone from getting in, but what we strive to do is build a tighter wall, make it more difficult. We have a number of tools that we have in place that we offer to our clients. We have email filtering, so it's going to filter out any of the blacklisted domains, poor IPs, IPs or domains specific to countries, the typical virus stuff. And we also allow them to like report, " Hey, I think this is spam." They can report that out. We take a look at it and say, " Yes it is or no it's not." Or if it's something that we feel that isn't spam, we work with them to kind of make sure that possibly their clients get whitelisted instead of blacklisted so that they're able to communicate as well.
Chadd Wheat: Yeah. I was going say on the user level there's... And these are posted all over the internet, but a good healthy dose of common sense. And one of the things I tell people is don't trust anyone. And if it seems suspicious, it probably is. For example, we see a lot of instances where people are getting email from people they know or people in their mailing list. I got one yesterday that said it was an urgent favor and do I have an account with the Amazon company? And could I help them? And I was like, " Well, first of all, this is a guy I barely know. Second of all, does anybody really call it the Amazon company?" So you sort of have to almost treat everything with a little bit of skepticism and that especially goes for things with email attachments or links. As a rule of thumb, I never click a link that's coming out of the blue. And we tell people, " Verify with the person who send it. Send them a personal text message or whatever communication they're using." We use Slack and Microsoft Teams. Contact that person and say, " Did you send this email? Did you send this attachment?" Treat everything with a huge dose of skepticism, is the number one rule because as Jim said, as we get more savvy and trying to block and screen these things, the hackers and the phishers are also getting more savvy and they adapt to what we do and then we adapt to what they do. So it's sort of this vicious cycle but self- protection and a little bit of common sense will help go a long way. And we've had to train users on that because people are already intimidated a little bit in the tech space and we have to make them realize what you read is not necessarily true.
Jim Timberman: Yeah. And through that too, education is the big key of what to look for in these emails. inaudible ones that come in that look perfect. That looks like it's got the signature line in there and everything. And then all of a sudden, you look at the email address and it's a Gmail account and say, " Well, that's the kind of stuff you have to look for." But also, a lot of not just being common sense but putting some practices in place, which is changing your passwords every 30, 45, 60 days, adding multi- factor authentication in. Really from our side of things on the managed services side, we provide quarterly audits where we look at what their environment looks like, where possible vulnerabilities. We run a number of scans within there to see, " Hey, you caught up on your patches." We kind of do a dark web scan too to see if maybe there's something out there and then work to remediate that to continue to build that wall. The other side of that from an email perspective is like I said, we've got some tools in place that do some filtering that kind of prevent those from coming in, as well as even going out. So if we see that in some instances that somebody has kind of gotten in and stole someone's address and they're starting to send emails for them that they can't see because we've seen that happen a number of times, that we can help shut that down quickly and quarantine those off and prevent those from going out and then work with our clients to kind of like, " Okay, hey, here's what you need to do to remediate that and what you need to communicate out to your clients as to what's happened." But on the other side, you look at the antivirus, malware, ransomware, et cetera. We have a tool in place that is really an endpoint protection which is that next generation antivirus that allows us to detect any kind of malware, or viruses, or ransomware on a machine. Whether that'd be a desktop server or anywhere within the network. And once that's captured, we can there isolate and quarantine that device from the network and from the domain. Go in, do our research, do our forensics, determine what the problem is, remove that, clean it, and then bring it back online. And that can usually happen within minutes to hours depending on the severity and what the virus is and how deep it's been penetrated. And that helps prevent a lot of that. A lot of companies today have antivirus and they feel like that's good enough. Sometimes it's not and the hackers today at any kind of infiltration that is happening in an environment usually isn't, " Boom, we got hacked and it happened today." They usually have been in there anywhere from weeks to months just kind of searching and trying to see if you can find them. And if you don't, that's when they hit. and even though they say that they're not, you pay your ransomware, duh, duh, duh, they're still in there. It's not unusual to hear of clients that have been hit two, three, four times within a few months and it's the same person. So education is the best thing and just trying to find the right tools and continue to build a stronger perimeter and make it difficult are really the keys to it because it's not a matter of if. Usually, it's how long or when it's going to be.
Chadd Wheat: Great. And especially again with the pandemic going on with virtually all users remote, you can't simply yell over the cue ball, " Hey, Suzie did you send me this email?" So people are more isolated and, " Oh, I got an email from one of my coworkers. What's this attachment?" So common sense, customer savvy training is a huge piece of it when it comes to especially people being isolated here and being exposed to those kind of things.
Angel Leon: I think it's very interesting to hear the two of you talk about all the... basically the new adjustments and the newer technology that we have available to fight off all these attacks to call it for what they are. And like Chadd and Jim, you guys were mentioning it earlier, I mean, things as simple as a simple email that you get on your inbox and I know that I work in an HR, I've had my fair share of emails come in with the names of the individuals that we have working at Moser where they look like, " Oh, yeah, hey, can I get you to do this for me and give me my bank account information?" I mean, Jim and I, I always share them with Jim because I think they look so genuine, right? They look so real. But then if I click that reply button or any of the links that are associated with the email, then it's all over.
Chadd Wheat: Yeah. And some of them are pretty easy to spot. Like I said, the one I got about the Amazon company account yesterday, that was pretty easy. Things like no punctuation or improper punctuation or bad use of grammar. But some of them are actually pretty savvy.
Jim Timberman: Yeah. And to that too is that you mentioned that clicking that link and so forth, now that they've got your email address, they've kind of got your approach to things. So they have your signature and they'll send that out to your customers. And now that they're in there, they can create rules and so forth that you never know that somebody is sending or receiving anything from that particular email address. We've seen situations where that's happened and it hits a lot of more of like the controllers and finance departments where... It didn't happen to us but heard it at a conference about a client that they got hit pretty hard that it hit their finance department, hit three accounts payable accountants that they had been sending emails out to their clients saying, " Hey, by the way, we're not receiving checks anymore. Everything's got to be done through ACH. Here's the bank account you send it to." This was going on for like three or four months. And the company is like, " Hey we've got about 15 clients that have been late on their payments." And they're like, " We've been sending them. We've been sending them." And then all of a sudden, they realize they've been sending it to the wrong bank account. These weren't$ 2, 000, $3,000 invoices. They were few$ 100,000.
Chadd Wheat: Yeah, it's really corporate identity theft.
Jim Timberman: Yes. And that also brings up the other thing and it's not just money. Larger corporations, a lot of hackers are getting in and stealing intellectual capital, getting into their file servers and stealing design specs or various patterns and so forth that they can get in there.
Angel Leon: Yeah. That's a nice segue to my next question which has to do with data breaches. And that's something that any IT managers should be aware of. We've been living in a world where these can be pretty common but there are ways in which they could be mitigated and managed. First of all, how can someone breach an organization?
Jim Timberman: A lot of it's really just kind of stealing user information. I'll use Chadd as an example. I figured out Chad's email address. I hack his password and then I realize his password is the same for everything. It's not so much the individual user, it's a lot of the service accounts that a lot of internal IT and sys admins use like admin @ company. com. And then they have the simple password of password @. So maintaining that, that's how they get in. Being able to identify and find those simple usernames and the simple passwords, we stress to our clients, " Never use something simple." Complex, at a minimum, eight characters, capitals, numbers, special characters included, up to 14. And in some instances, we like to have them auto- generated. We would force multi- factor authentication. Everything we do now is multi- factor authentication. And that includes not just internally at Moser, everything we do with our customers. When we establish a username and password, we want a way to be able to validate that and no shared accounts.
Chadd Wheat: And like Jim said earlier, sometimes or most times when a company has been hacked, they may not be aware of it for weeks and months potentially. So that's where something like our security audits come in because we can find those earmarks and those forensic clues that show there's a probability that this account or this server has been hacked and take remedial efforts to stop it before it really gets out of hand.
Jim Timberman: And also too with our monitoring tools, we're looking at, who's trying to access what? And it's all based on privileges. Who has the right privileges to get to this environment? Who has the right privileges to get to the database? And if we're seeing that that'd be the secretary trying to log into your corporate data warehouse or your reporting tool or your finance system they shouldn't have had access to it, then that usually triggers a red flag to say, " Okay, what's going on?" So we're kind of tracking back to like, " Hey, Debbie, you need to change your password." Or we need something to figure out and some inaudible a mistake too. She thought she had access to it but we always kind of err on the side of caution on that too.
Chadd Wheat: A real simple user and preventative measure is by administrators forcing password changes every 45 or 90 days or whatever the policy may be because if the passwords never change and somebody's got your password, then they're going to get in. But if you force a password rotation and keep a history log so they can't just change it back and forth to something they've used in past that. A lot of this is a pain for the user, but it's an unfortunate necessity in our world today so safety overrides comfort in this case.
Jim Timberman: Into that point too, as part of our services that we provide, we do quarterly audits on the environment and then report those incidents back to our clients to say, " Hey, here's where we see particular vulnerabilities. Here's where we're seeing where areas that need to be fixed and patched more." And we're not going in and looking at individuals passwords because nobody should have access to that. But we are looking at, hey, we need to make sure that we're changing these on a regular basis, as well as looking at where the holes may be. Staying on top of any patches and updates because Microsoft and a lot of the different providers out there are pushing out code to kind of help prevent some of that and putting some of those security patches in place to make it more difficult for hackers to get in and so forth. So that's the key, is staying on top of that. The big story was I think it was last year with Anthem and a few of the larger corporations all got hacked and a lot of that was because they weren't up to date on their patches. And we work with our clients. We're on a regular cycle.
Chadd Wheat: Yeah, it's sort of like when the engine warning light comes on your car and you think, " Ah, I'll get to it next week or next month." Well, something bad's going to happen. So being proactive, having a normal patching schedule. And then as Jim said, sometimes the vendors, Microsoft or whoever will come out with an emergency security patch because they discovered some kind of exploit. So on the back end, that's where we have to be up on our game and make sure that we're scheduling and we're in constant communication with our customers to tell them what's going on with those.
Angel Leon: Yeah, and it's interesting. So we talk about how to get in, right? What can you suggest for individual IT managers out there how to educate themselves on this topic and keep educating themselves just to be on the forefront of this so that they don't have this happen to them?
Jim Timberman: There's a lot of things like education is the key. There's a number of different sites and newsletters that come out related to security. The biggest suggestion I can make is to at least have an annual security audit. Bring in a third party. We're a managed service provider and we try to do as much as we can as it relates to security, but we're not a SOC or a security operation center, or even a security provider. So our level of expertise only goes so far. We're not doing any kind of penetration testing. We're mostly just looking at what's established today and continuing to try to make it better.
Chadd Wheat: Today's IT manager has to be savvy and really, you have to I think joining user groups or manager groups in their local communities, user groups for certain vendors. And you really have to stay on top of that. Back in the day, everybody used to get info week and that was published every week and you could keep track of trends well. Unfortunately today, things moved in a timeframe of minutes, not weekly. So today's IT manager really has to stay on top of what's going on in the industry and the world as a whole in order to make sure that they're doing the best course action.
Jim Timberman: And Chadd brought a great point up there in that not just you as a company looking at what you're doing security wise, look at your vendors. What are their security policies? Because we've kind of seen it just recently, the big one with the SolarWinds breach in that that came in and they got everywhere. They're in the Department of Treasury. They're in the Pentagon. That Microsoft got hit through this. So these large players and understand what their security policy is because if they have access to your systems, then that may be something that will get attacked eventually. So even using like Salesforce. These SaaS systems are great. However, they're just as vulnerable as everyone else.
Chadd Wheat: If you have any kind of internet- facing application service or server, you are constantly being probed. And I think everybody needs to understand that this is not just, " Hey, they targeted us and something bad happened." No, if you have internet- facing software, hardware, or whatever, you are getting probed on a daily basis. And what they're looking for is that exploit, that sort of hole in the wall. So I think it's good even for users to understand even on a personal basis, your home PC is probably getting probed every day and people are looking for a way in.
Jim Timberman: It's not that too, it's just to make sure that you have tools in place to encrypt local PCs, providing tools to be able to wipe a machine quickly. So having those solutions in place and those tools in place really does help too in the long run.
Angel Leon: Kind of just to extrapolate that a little bit more, boy, this program has been built with nice segues. In the event of an organization falling victim to a breach, how best could they manage that situation?
Jim Timberman: Oh, that's a good one. Honesty is the best thing to do, that you really need to ensure that they are communicating with their clients that there has been a breach. The level of it, they may not know at the time, but to be open and not try to hide it is the key. One of the first things they should do is really shut everything down and try to figure out when to remove where it's at.
Chadd Wheat: Isolate.
Jim Timberman: Yeah, isolate and quarantine off where those pieces are at. It is unfortunate that there's a hiccup in that. And you almost have to look at it as kind of like disaster recovery is that if I get hit, what's my contingency plan for being back up and running? Us personally have not as a managed service provider and even as Moser have been pretty lucky that we have not had a major catastrophe breach with any of our clients. Most of what we've run into is a lot of the email. " Oh, they stole my email. I've been phished." And those are really rather quickly. But the bigger ones where there's a full blown ransomware or a full breach that we can see that there's people in there, we personally haven't run into. But usually, our policy is that we have to treat this like disaster recovery.
Chadd Wheat: Contingency planning and disaster recovery planning is a must. And even small companies need to be aware and need to have some kind of plan. It's almost like the COVID outbreak. We've all heard of or we've all been exposed at one time or another and you take certain steps, right? You isolate, you go get tested, et cetera, et cetera. And then if you actually develop COVID, well, now you've got a different set of circumstances and you have to put your plan in place. What does that look like? So contingency and disaster planning are huge and when we're talking about this topic. And that's a whole another webcast we could probably do.
Jim Timberman: Yeah, the disaster recovery and planning, and execution, and testing on that is something we could spend hours talking about that. But the other side of that too, not only into the disaster recovery is having good backups because a lot of times, you're going to need to restore from those backups and at what level you want because that would be the key too, is that you've got to restore these systems from somewhere. Where are you going to do that? And are those backups compromised? So a lot of that stuff needs to be addressed and it should be looked at on a regular basis. And as Chadd had mentioned, we look at disaster recovery and our backups more so on a monthly basis, even though they're being done daily. We're going through and double- checking everything and confirming that everything's good and putting that back to our clients. But on the disaster recovery, we're looking at on a quarterly. We're going back and saying, " Hey, we need to review what we're going to do." Whether that would be a form of testing, which we've picked a weekend and say, " Hey, we're going to take everything down and spin it back up." Or we do kind of a tabletop exercise to walk through. Get everybody in a room and say, " This happens. What do we do?" I do this. Okay, check, move on. So they know the process so we're prepared that if this does happen, we're available.
Chadd Wheat: Again, this is all systemically related. That's where a good security and system audit can come into play too. Having an outside third party look at your policies because that's really what the auditor is looking at. I mean, yes, they find obvious holes, but a lot of audits will look at your policies, what are your backup policies? What are your contingency plans? What are your disaster recovery? Disaster really doesn't mean getting hit by a tornado at your data center. Yeah, that can happen. Disaster means your servers got ransomware on it and what are we going to do about it? How does that affect our customers? How does that affect our employees?
Angel Leon: Interesting. Yeah. You guys come from preparation. It's sort of like what Chadd was saying. You prepare for a disaster, you go through the disaster, and then you mitigate in the end. So do an after- action report, after- action review after a breach happens. It's very interesting that we take the same approach for a natural disaster or something like that into the IT world. So thank you guys for that explanation.
Chadd Wheat: On that inaudible, doing what we'll call drills too. We see a lot of clients, they'll have a yearly disaster drill basically where they cut over to their backup servers. They find their backups. They walk through the entire process of this as alive as you can get exercise to make sure their policies actually work.
Angel Leon: Would you recommend once a year, maybe twice a year for them to do some of these audits or these preparation risk assessments?
Jim Timberman: Yes. And actually, we do that as part of our service, is that we come in and do quarterly risk assessments. We've got a number of vulnerability scans we do and ports that we run to say, " Hey, here's the things at risk. And here's some of the things that need to be fixed." Because we kind of work with that with our clients to kind of roadmap that out over the course of our contract and our engagements and to say, " Hey, we see that this system's near end of life which could create some risk and vulnerability. What is our plan to fix that?" In looking at where we're at with access control and so forth, we look at those individual pieces of it and say, " Okay, here's what we want to do to remediate this and when we want to get it done." As well as prioritize that because there's some things that are big needs they need to get fixed and others are less important, but we want to keep them out there. As Chadd pointed out, that's some of the stuff that an auditor is probably going to find. And more likely, they'll come in and say, " Do you know about this?" And we'll say, " Yes, we've got that on our roadmap. It's scheduled to happen in Q2 or Q3." They'll be, " Okay, that's good because maybe we want to move that forward." So we're working with them to kind of prioritize those changes that need to happen within that environment.
Angel Leon: Yeah. Auditors are important to have. I guess I should know, I'm married to one. But anyway, thanks. Thank you guys for that explanation about what people should do prior, during, and after a breach. I think those thoughts should be present for any IT manager, but let's end this episode on something... A little bit of a light question that I have here for you guys. So please be honest, what's the most challenging help center tickets you've ever received?
Chadd Wheat: Well, that one's near and dear to my heart as Moser service desk manager. The most frustrating and hard to accomplish tickets are the ones that come in without good knowledge and good data behind them. For example, it's actually happened where we'll get an occasional ticket saying, " I need help." Well, what's that look like? Can your car not start? Have you had a breach? Or what does I need help mean? So for me and I think for my team, it's very frustrating not having the knowledge and the data they need to go in and prosecute a solution. And that's the first thing we have to do. And really, that gives your lifecycle of your issue. It really extends it because now we have to do a lot more investigation whereas if I've got screenshots, user account names, what server we're talking about, things like that, that really cuts out a lot of steps for the forensic investigation that we have to do. So by and far, the largest frustrating tickets we get are the ones that just have no backing data and we have no idea what the customer is talking about.
Jim Timberman: Or the other famous one and I laugh at those is, " Can you call me, please?"
Chadd Wheat: Yeah.
Jim Timberman: We work a lot of our tickets based on severity. What is the urgent need? And through that, we don't know if it is urgent or it's not. So we'll look at that and go, " Okay, do I need to call this person now? Can I wait because there's an outage over here?" So it's a delicate game of balancing what's important versus what's considered a nuisance.
Chadd Wheat: Right. To users, everything's important and we understand that, but when you say, " Hey, can you call me?" Well, most times actually, that turns out to be, " I had a question about this or that. Or how should my password look?" And you're like, " Okay, well, we've had to pull a resource off to work on a different customer issue for just sort of a question," which is fine. We can answer customer questions, but if we understand, and like Jim said, if we can prioritize and direct our assets to where they need to go, that helps the customer, that helps us, that helps everybody.
Angel Leon: Interesting. Well, I know that I've fallen victim to the I need help ticket. Not necessarily with Moser yet, but in the past, I know I have. So I really appreciate the honesty on that one. So folks, Jim, Chadd, it's been a pleasure to talk to you today. I think this topic like Jim was saying earlier, I think this is something that we could probably sit down and talk for hours. Not that we can but we want to limit the listening experience right now. So Jim, Chadd, thank you very much once again. We really appreciate you guys coming on today.
Jim Timberman: Thanks for having us.
Chadd Wheat: Yeah, you're welcome.