Security Updates Triggering the Need for a Reboot

On the Linux systems, certain security updates require the machine to rebooted. There is an automated process running on the machines watching for such a situation. When a security update is applied that requires a reboot it starts a process of warnings and watches for an appropriate moment to run the reboot.

Reboot trigger process after the security update has been applied

If at any point in the process below the user logs out, then provided no one else is using the machine or no other user processes are still running, the reboot trigger will notice within 5 minutes and reboot the machine after which it is safe to log back in.

• Days 1-3
  • Since 60-80% of machine are not in use at night those machines will shutdown and hence the reboot will have completed the next time they boot up.
  • If the reboot does not occur the first night/weekend then warnings will start to appear for new login attempts and warning emails will be sent to those already logged in.
• At the end of day 3 - A warning email is sent to anyone who has a process running on the machine
• From days 4-6 - warnings will continue for new login attempts
• At the end of day 6 - A further warning email is sent to anyone who has a process running and a forced reboot is scheduled to occur after a further 24 hours
• At the end of day 7 - the forced reboot occurs

The warning emails explain to email @email with any problems or if there is a reason why you cannot logout/terminate all processes but do not want the (forced) reboot to occur.

It is important that security updates are applied in a timely fashion which is why the above process ultimately results in a forced reboot if no user action is taken. To allow the security updates to go unapplied for longer in general would pose an unreasonable risk to the departmental systems but in specific cases when an important process is running and more time is needed if you email @email we will look at the specific risk and delay the forced reboot if possible. Delaying the reboots though can only be done in limited circumstances as overall we need to comply with university policy too that requires security updates to go live in a timely fashion.

Why did the machine not reboot when I logged out?

There are two common situations where the machine does not reboot even though you log out:

1. Even though you have logged out there are still processes running on the machine owned by you
2. Another user is remotely logged into the machine or has left processes running on the machine

In both these cases the reboot trigger detects the presence of user processes and hence will not reboot the machine.

Case 1 can occur if a process has gone rogue and thus does not close/die properly when you log out.

Case 2 is most likely to occur on the departmental compute machines that are used by lots of different people. If the machine is being used by lots of people it is unlikely they will all logout in turn and stay logged out (so ultimately no one is left logged in) and hence the most likely outcome is the machine will go through a forced reboot at the end of the 7th day.

Case 2 may also occur on an ordinary desktop if it is being remotely used by someone else (which is quite rare but does happen). In this case with only a couple of users it is quite likely the remote user will logout after which the local user can logout and the security reboot will occur.

Killing off rogue processes

You can look at the list of processes running on a machine with the command

ps aux

from which you can identify a process that was started prior to your most recent login and then kill it with a command of the form

kill -9 pid

where you replace pid with the process ID number of the relevant process as listed in the output of the previous command.

Killing off all my process/remotely logging myself out

Sometimes you may simply want to kill off all your process on a machine, which also has the effect of logging you out.

To kill off all your processes on a machine run a command of the form

pkill -9 -u username

where you replace username by your username. Please note that this command will kill all your applications on the machine, no matter whether those have unsaved data. All unsaved data will be lost.

To log yourself out remotely you could first use ssh to remotely log in to the machine gate.maths.ox.ac.uk and from there use ssh to remotely log in to the particular machine you need to log out of, after which you would run the above command to kill all your processes on that machine (it will also kill the process representing your remote login too).