This section will deal with how to get early warning, how to be alerted after the fact, and how to clean up from intrusion attempts.
Intrusion Detection Systems (IDS for short) are designed to catch what might have gotten past the firewall. They can either be designed to catch an active break-in attempt in progress, or to detect a successful break-in after the fact. In the latter case, it is too late to prevent any damage, but at least we have early awareness of a problem. There are two basic types of IDS: those protecting networks, and those protecting individual hosts.
For host based IDS, this is done with utilities that monitor the filesystem for changes. System files that have changed in some way, but should not change -- unless we did it -- are a dead give away that something is amiss. Anyone who gets in, and gets root, will presumably make changes to the system somewhere. This is usually the very first thing done. Either so he can get back in through a backdoor, or to launch an attack against someone else. In which case, he has to change or add files to the system.
This is where tools like tripwire (http://www.tripwire.org) play a role. Tripwire is included beginning with Redhat 7.0. Such tools monitor various aspects of the filesystem, and compare them against a stored database. And can be configured to send an alert if any changes are detected. Such tools should only be installed on a known "clean" system.
For home desktops and home LANs, this is probably not an absolutely necessary component of an overall security strategy. But does give peace of mind, and certainly does have its place. So as to priorities, make sure the Steps 1, 2 and 3 above are implemented and verified to be sound, before delving into this.
We can get somewhat the same results with rpm -Va, which will verify all packages, but without all the same functionality. For instance, it will not notice new files added to most directories. For this to be helpful, it needs to be done after a clean install, and then each time any packages are upgraded or added. Example:
# rpm -Va > /root/system.checked |
Then we have a stored system snapshot that we can refer back to.
Another idea is to run chkrootkit (http://www.chkrootkit.org/) as a weekly cron job. This will detect common "rootkits".
Maybe you are reading this because you've noticed something "odd" about your system, and are suspicious that someone was gotten in? This can be a clue.
The first thing an intruder typically does is install a "rootkit". There are many prepackaged rootkits available on the Internet. The rootkit is essentially a script, or set of scripts, that makes quick work of modifying the system so the intruder is in control, and he is well hidden. He does this by installing modified binaries of common system utilities and tampering with log files. Or by using special kernel modules that achieve similar results. So common commands like ls may be modified so as to not show where he has his files stored. Clever!
A well designed rootkit can be quite effective. Nothing on the system can really be trusted to provide accurate feedback. Nothing! But sometimes the modifications are not as smooth as intended and give hints that something is not right. Some things that might be warning signs:
Login acts weird. Maybe no one can login. Or only root can login. Any login weirdness at all should be suspicious. Similarly, any weirdness with adding or changing passwords.
System utilities are slower, or awkward, or show strange and unexpected results. Common utilities to be modified are: ls, find, who, w, last, netstat, login, ps, top.
Files or directories named "..." or ".. ". A sure bet in this case.
Unexplained bandwidth usage.
Logs that are missing completely, or missing large sections. Or a sudden change in syslog behavior.
Mysterious open ports, or processes.
Indications of a "sniffer", such as log messages of an interface entering "promiscuous" mode.
Modifications to /etc/inetd.conf, or /etc/passwd. Especially, any additions. Try using cat or tail to view these files. Additions will most likely be appended to the end. Remember though such changes may not be "visible" to any system tools.
Sometimes the intruder is not so smart and forgets about root's .bash_history, or cleaning up log entries, or even leaves strange, leftover files in /tmp. So these should always be checked. Just don't necessarily expect them to be accurate.
Packet sniffers, like tcpdump (http://www.tcpdump.org), might be useful in finding any uninvited traffic. Interpreting sniffer output is probably beyond the average new user. snort (http://www.snort.org), and ethereal (http://www.ethereal.com), are also good. Ethereal has a GUI.
As mentioned, a compromised system will undoubtedly have altered system binaries, and the output of system utilities is not to be trusted. Nothing on the system can be relied upon to be telling you the truth. Re-installing individual packages may or may not help since it could be system libraries or kernel modules that are doing the dirty work.
We can use rpm -Va |less to attempt to verify the integrity all packages. But again there is no assurance that rpm itself has not been tampered with, or the system components that RPM relies on.
If you have pstree on your system, try this instead of the standard ps. Sometimes the script kiddies forget about this one. No guarantees though that this is accurate either.
Another approach is to visit http://www.chkrootkit.org, download their rootkit checker, and see what it says.
Some interesting discussions on issues surrounding forensics can be found at http://www.fish.com/security/. There is also a collection of tools available, aptly called the "Coroner's Tool Kit" (TC).
Read below for steps on recovering from an intrusion.
So now you've confirmed a break-in, and know that someone else has root access, and quite likely one or more hidden backdoors to your system. You've lost control. How to clean up and regain control?
There is no sure fire way of doing this short of a complete re-install. There is no way to find with assurance all the modified files and backdoors that may have been left. Trying to patch up a compromised system risks a false sense of security and may actually aggravate an already bad situation.
The steps to take, in this order:
Pull the plug and disconnect the machine. You may be unwittingly participating in criminal activity, and doing to others what has been done to you.
Depending on the needs of the situation and time available to restore the system, it is advantageous to learn as much as you can about how the attacker got in, and what was done in order to plug the hole and avoid a recurrence. This could conceivably be time consuming, and is not always feasible. And it may require more expertise than the typical user possesses.
Backup important data. Do not include any system files in the backup, and system configuration files like inetd.conf. Limit the backup to personal data files only! You don't want to backup, then restore something that might open a backdoor or other hole.
Re-install from scratch, and reformat the drive during the installation (mke2fs) to make sure no remnants are hiding.
Restore from backups. After a clean install is the best time to install an IDS (Intrusion Detection System) such as tripwire (http://www.tripewire.org).
Apply all updates from ftp://updates.redhat.com.
Re-examine your system for unnecessary services. Re-examine your firewall and access policies, and tighten all holes. Use new passwords, as these were stolen in all likelihood.
Re-connect system ;-)
At this time, any rootkit cleanup tools that may be available online are not recommended. They probably do work just fine most of the time. But again, how to be absolutely sure that all is well and all vestiges of the intrusion are gone?