Continuing on our conversation from part one of this series, scheduling patching can be quite a challenge. To do it, you must handle a large number of different devices, juggle availability, comply with a range of policies, and balance securing the environment with keeping your customers happy.
In this blog, we’ll cover the following elements of the patching process tasks and how to schedule them:
- approving patches (manually or automatically)
- installing patches
- rebooting after patching
There are a variety of ways to approve patches. While it’s not often a scheduled process, here are some of the more common ways we see MSPs approve patches:
- manually every day/week/month
- automatically as they are detected, and the rest is manual
- automatically with a delay (if your RMM allows it), and the rest is manual
Note: some ways to approve patches may or may not be available in your RMM platform, but the general idea is the same regardless of which platform you use.
Most customers I speak to say approving patches is cumbersome. No one has time to read up on each patch to determine if they want to install it or not. Furthermore, almost no one has time to test patches one by one.
So what should you do? It’s common to approve all patches manually. If that works for you, then do it at least weekly due to the growing number of out-of-band patches Microsoft continues to release. I also recommend you automatically approve and automatically install definition updates for Microsoft Defender (the built-in free antivirus (AV) from Microsoft) in case your customer uses it.
If you want to approve some patches automatically (with or without delay), people typically automatically approve security and critical patches. You’ll likely need to approve those regardless, so you might as well save time and auto approve them. You can then review the rest manually.
Finally, some partners automatically approve everything. This can be aggressive but if it works for you, then great.
Whether you approve patches manually or automatically, try to adhere to these three principles:
- Don’t leave patches unapproved: approve, decline, or set them as ignored to ensure you have a clean report and dashboard
- Do it often: check at least weekly—Wednesday is usually a good day as most patches come out on Tuesday
- Write a standard procedure: a standard procedure allows anyone on your team to do the approval/declines reliably and consistently since the process remains the same
This is another one that has some misconceptions. Lots of people say they can only install patches at night on weekends, but when do you ever see a laptop turned on at night on a weekend? if you’re like me, your laptop is in a bag at home and turned off on Saturday night.
Fortunately, today’s patch installations don’t behave the same as they used to. In the old days, we would see patches being disruptive, crashing programs as they replaced locked files, and generally causing harm in other ways. Recently, patches have become much more stable, almost always requiring a reboot later so the patch can install without disruption since it won’t take effect until the reboot happens.
Because of this, I can recommend installing patches on desktops during the day. If your remote monitoring and management (RMM) solution supports it, you can run patches weekly on a day of your choice and set patches to patch the next time the device reboots in case it misses its window. You can install patches at 11:30 a.m. each day (around lunch time so as to minimize potential disruption), and have it installed later if the user is on the road or offline.
If you’re not comfortable with that, or have to do it at a specific time, then follow your own process and procedure to ensure you’re able to install patches effectively and efficiently.
Since servers are always open, patching them on the weekend is usually best. If you have servers in clusters, or in groups (like having two domain controllers) do one first. When it’s done and rebooted do the other one. This will minimize any downtime.
Rebooting is tricky as it will affect the user. In recent years, Microsoft has introduced things like fast boot and hibernation, which means users think they’ve rebooted their machines when they actually haven’t. Since most patches these days require reboots, the patch won’t take effect until the device fully reboots. This means the security vulnerability it patches will continue to be a vulnerability until a day or a week later (whenever the device reboots next). So it’s important to properly schedule patch reboots.
To counter this issue, you can schedule a forced reboot once a week at night if the devices are typically online. However, this doesn’t work very well in my experience. For that reason, I recommend a couple of other options. First, you can monitor how long it’s been since the last reboot and use an automated popup to remind the user to reboot. Alternatively, you can do a reboot during the day (either early morning, at lunch, or at the end of the user’s work day). Depending on your RMM’s capabilities, you can allow them to delay the reboot, see a popup warning (or not), or cancel it.
As you can see, patching isn’t super complicated to schedule, but partners can do it in a variety of ways. Hopefully this break-down can help you think of how to do it within your own business or how to improve upon your current process.
If you have any comments/question on this topic, or have created an automation policy and would like to share it with the community, please feel free to email me at [email protected].
As always, don’t forget to look in the Automation Cookbook at www.solarwindsmsp.com/cookbook if you’re interested in other automation policies, script checks, and custom services.
Marc-Andre Tanguay is head automation nerd. You can follow him on Twitter at @automation_nerd.