The Network Detectives
First Published: SC Magazine July 2003

Given the nature of my security specialization, I often get approached by clients requiring an immediate response to a critical security concern or ‘compelling event.’ These incidents typically range from threats of targeted business interruption, through to investigation of successful host compromises.

When dealing with a system compromise, I have found that placing an experienced forensics expert alongside a skilled ethical hacker yields the best results for the client. While the forensics person is fantastic at dealing with the preservation of evidence and retrieving the hidden trail of the attacker, the skilled ethical hacking consultant understands the tools and exploit material likely to be used by the attacker.

Usually, the forensics expert will take a low-level image of the compromised host’s data drives, recover most of the deleted content and log files, and preserve the chain of evidence.

The ethical hacking consultant adds the ability to interpret the log files. For instance, he might be able to identify that the deleted HTTP log contains entries that are associated with an attacker running a particular web-scanning tool, deduce the specific vulnerability that was used to compromise the host, and in many cases, obtain a copy of the exploit material that would have been used.

To understand how complex some host compromises can be, consider the following real-life example. A client required my team to investigate the suspicious crash of a critical host that had previously been running without fault or rebooting for more than two years.

The client’s initial assessment of the host revealed a lot of user accounts relating to people who did not exist within their organization. Our investigation revealed more extensive compromises.

The host crashed because an attacker had run some popular exploit scripts against the available IRC service and had subsequently tried escalating his permissions locally. But he had got it wrong as the local service had been patched a few months previously.

According to the system administrators for the compromised host, there should only have been an FTP service running. Analysis of the FTP service revealed two things. First, there were dozens of ‘warez’ directories containing hundreds of megabytes of pirated software and MP3s. Secondly, the FTP service was not currently vulnerable to any previously disclosed remote vulnerabilities as it too had been recently patched.

Obviously this was of great concern to the client, but it had still not really solved the fact that there were a lot of user accounts that they had not created themselves. Further investigation of the host finally revealed the whole story.

When the system had been set up originally, the client had hardened the server correctly and only the single FTP service was accessible from the internet. Not long after this, the host was compromised by exploiting the FTP service; new user accounts were created, and the ‘warez’ file sharing began.

At some intermediate date, another attacker compromised the host again, installed the IRC service and, to prevent others from doing the same, changed permissions on various host services and patched the host. It was only when the third attacker blundered his way through the system and crashed it, that the client realized something was amiss.

In this case, the client did not require any legal action to be taken. The emphasis for them was to understand how the host was compromised and ensure that their administrative support team developed better procedures for securing similar hosts in the future.

This is a common course of action - less than 40 percent of my clients wish to proceed with a legal prosecution. This of course assumes that there is enough evidence to identify the attacker. Often the path leads to a hijacked dial-up internet connection and it is more trouble than it is worth to get the ISP to identify a probable attacker at their end.

Certainly, if a client of mine suspects it has been compromised or is likely to be the subject of a targeted attack with specific timescales (government elections, hostile takeover, etc.), I strongly recommend that the incidence response or emergency response team consist of both experienced forensics people and consultants with strong ethical hacking skills. With such a team, the investigation is more thorough, takes less time and costs less.

Don’t compromise your hosts
(1) Ensure that all hosts are hardened to an accepted business standard.
(2) Review your host hardening procedures regularly and adjust as required.
(3) Keep a record for each host of the installed software, versions and accessible services.
(4) Keep track of the latest security patch alerts and apply patches quickly.
(5) Port scan all hosts regularly. For internet-visible hosts, do it daily.
(6) Run various vulnerability scanners regularly. For internet-visible hosts, do this weekly. Don’t forget to update your vulnerability scanner each time.
(7) Conduct detailed security audits regularly (at least quarterly) using an ethical hacking methodology. Aim to conduct these both remotely and on site.
    Copyright 2001-2007 © Gunter Ollmann