Assessing Your Security
Advice on Assessing your IT Security Posture

Security, an ever-changing battlefield

Most people will agree that Information Technology (IT) is changing or altering business processes and work environments at a dizzying pace. Unfortunately for those responsible for maintaining the security posture of these processes and environments, security changes faster. Organisations often fail to realise that even if the technologies, operating systems and environments were to remain static, the mechanisms required to secure those systems against the latest threats would continue adapt and force change. It doesn’t take too much effort to find news articles of the latest computer virus to circulate the world, the number of new vulnerabilities discovered last month, or the critical fixes for your operating systems that need applying today. However, it does take a substantial amount of time for an organisation to develop the security mechanisms to help protect against both last month’s and next month’s threat.

For most organisations, “security” includes all the mechanisms and processes developed to protect or mitigate both perceived and real threats. For this reason, “security” can mean different things to different organisations, and threats will be ranked depending upon business priorities. For instance, a large and respected bank may find the defacement of their corporate web site taking priority over a £1,000,000 internal theft, as reputation and customer confidence in their online offering is most valuable.

The number of digital mechanisms currently available to organisations for conducting business has increased with the proliferation of new electronic communication systems. In the past, IT security departments were able to focus largely upon internal threats and issues, and the mechanism for verifying their security posture was commonly the audit. With the opening of corporate borders and the increase in the sophistication of communication methods at the desktop level, IT security must now identify and respond to both internal and external threats. To evaluate the security posture of an organisation’s critical systems and data, security must now include active testing and assessment phases. It is only through assessment processes that an organisation can identify security holes and configuration issues, while maintaining system integrity and resilience.

Assessing Your Security

The process of assessing an organisation’s IT security posture can often be a daunting task. For many large organisations, the sophistication of their integrated systems coupled with the distribution of skills and responsibilities for maintaining and securing them, often means that no clear owner can be identified. For those saddled with the responsibility of ensuring all key business critical systems are safe, obtaining the necessary confidence in a particular system can be difficult. Thus it is often necessary to regularly audit and assess the systems to ensure that adherence to either a central corporate-wide security policy or agreed “security best practice”, is achieved.

Auditing or Assessing

Assuming a corporate-wide security policy exists, it is important that appropriate monitoring steps are undertaken to ensure a level of consistency. For most organisations, this task of monitoring can be divided into two processes - auditing and assessing. Depending upon the system or policy issue, one may be better than the other.

These two monitoring processes are different, and a definition is required to adequately differentiate between them:

  • Auditing – Any IT system that can be defined by a set of processes, or procedures, can be compared to a master checklist for compliance. Auditing is the mechanism of confirming that the processes or procedures agree with such a checklist. For many organisations, the security policy defines the elements of the audit checklist (e.g. all passwords must be 8 characters long and changed every quarter), and the method of checking for compliance is often clear. In many cases an audit can be seen as a passive, or non-intrusive, method of checking the security of corporate systems.
  • Assessing – Unfortunately, many processes or system components cannot be adequately verified using a checklist or security policy. It may be necessary to take a more active, or intrusive, testing methodology to adequately assess your corporate security. This process, referred to as a security assessment, tests the holes in your auditing processes and evaluates the resilience of your systems to real threats and attacks. Many assessment suppliers will divide an assessment into three categories based upon the threat:
    Remote Infrastructure Security Assessment - External threats
    On-site Infrastructure Security Assessment - Internal threats
    Application Security Assessment - Integrated system threats

Depending upon an organisation’s internal structure and the skill or availability of personnel, it is often possible to conduct most security auditing functions in-house (provided a well-defined security policy exists), without an interruption to service levels or continuity.

While an auditing process can be conducted at any time, the intrusive nature of an assessment usually limits the frequency of such security tests. A security assessment also requires computing skills less prevalent in most organisations and, although various tools such as vulnerability scanners, network sniffers and password crackers are readily available to help with the first phases of an assessment, it is often difficult for security individuals to stay up-to-date with the latest “ethical hacking” methodologies or cover the diverse system architecture within their own organisation. It is for this reason that many organisations consider outsourcing their security assessment requirements.

Understanding Threats and Prioritising Risks

Security means different things to different people and is often dependant upon the environment or the situation they are in. To a typical High Street retail store, security revolves around reinforced doors and windows, physical locks, alarms and CCTV. At home, security typically involves window latches, an alarm system and multiple door locks. To the group responsible for the security of an organisation’s infrastructure, the mechanisms available are much more diverse and complicated. However, in all cases, these security mechanisms are a response to a real or perceived threat. Most IT magazines or newspapers are very good at describing potential threats to your organisation. The threats may take the form of viruses, cyber-hackers, industrial espionage, terrorism, fire, flood and many other things. An important part of any security audit or assessment is to evaluate the risk associated with the threat.

Defining Risks

An important part of any security assessment is to ensure that any risks identified are adequately classified, and that prioritisation for remediation is possible. It is often an easy process to classify most risks associated with vendor-supplied software (including bugs, missing operating system patches, vulnerable services, and insecure choices for default configurations), administration (including options available but not used correctly, insecure requirements for minimum password length or unauthorized changes to the system configuration) and user activity (including risky “shortcuts,” such as sharing directories to unauthorized parties, policy avoidance such as failure to run virus scanning software or using modems to dial in past the corporate firewall, and other, more malicious activities).

Generally, these risks are classified into the following three levels:

  • High - A high-risk vulnerability is defined as one that will allow an intruder to immediately gain privileged access (e.g., administrator or root) to the system or allow an intruder to execute code or alter arbitrary system files. An example of a high-risk vulnerability is one that allows an unauthorized user to send a sequence of instructions to a machine and the machine responds with a command prompt with administrator privileges.
  • Medium – A medium-risk vulnerability is defined as one that will allow an intruder immediate access to a system with less than privileged access. Such vulnerability will allow the intruder the opportunity to continue the attempt to gain privileged access. An example of medium-risk vulnerability is a server configuration error that allows an intruder to capture the password file.
  • Low - A low-risk vulnerability is defined as one that will provide information to an intruder that could lead to further compromise attempts or a Denial of Service (DoS) attack. It should be noted that while the DoS attack is deemed low from a threat potential, the frequency of this type of attack is very high. DoS attacks against mission-critical nodes are not included in this rating and any attack of this nature should instead be considered to be a “High” risk.

Outsourcing Assessment Services

For many organisations, outsourcing security assessments represents the most thorough and cost effective option for ensuring the highest level of confidence in their systems.

There are four key stages in outsourcing your security assessment requirements:

  1. Understanding your business requirements for conducting the assessment.
  2. Selecting an appropriate security partner.
  3. Legal Issues.
  4. Deciding upon the assessment deliverables.

Understanding your requirements

Unfortunately, like in-house software development, security assessments are often prone to “scope-creep”. Thus it is important to define exactly what the business wishes to have achieved at the end of the assessment, and define processes and steps to be taken in reverse order. To help scope the security assessment, consider these basic questions:

  • Is there a clear business case for the security assessment? Can sign-off or support from other business streams and supporters be attained?
  • How does the assessment fit into business security practices? Has the organisation defined any security policies?
  • What is the organisation trying to protect against? Are internal or external threats more important to the organisation?
  • What are the timescales? Is there a window of opportunity for conducting an assessment in the project plan?
  • For how long does the organisation need to maintain this security posture? How often will the security assessment need to be conducted?
  • How much is the organisation prepared to spend in conducting the security assessment?
  • Is there a need to achieve an industry recognised standard or certification?
  • What internal resources does the organisation have available to oversee, shadow, or conduct the security assessments?
  • How good are the organisations internal resources at discovering and/or fixing vulnerabilities?
  • Are there live, business critical systems that should not fail during an assessment? Is it possible to conduct some aspects of the assessment on test systems or out of business hours?

Selecting an Assessment Partner

The most important phase of outsourcing an organisations security assessment requirement is the selection of the supplier. Organisations will often find themselves overwhelmed by the choice of suppliers and the diversification of their services. Thus it is important that a thorough review of potential suppliers is carried out and answers to specific questions sought. As a base to the review, the following questions or processes may be useful:

  • How much experience do they have? – Find out how many assessments the proposed supplier has performed in environments similar to yours. Ask about the number, size and complexity of assessments carried out annually.
  • How likely are they to deliver within your timescales? – Find out how many consultants the supplier has dedicated solely to conducting assessments and what their utilisation is. Additionally, evaluate the likelihood of the supplier being able to deliver these services in one or two years time for security continuity.
  • How good are their consultants? �� Review copies of their résumés and, if possible, meet with the consultant’s most likely to carry out the assessment and evaluate their individual skills.
  • Can the supplier and consultants be trusted? – The supplier and their consultants will be working with your business systems. It is extremely important that they can be trusted and will respect the importance of the systems and the data they contain.
  • How good is the final deliverable? – Typically, the only tangible deliverable from a security assessment is the Final Report. It is therefore vital that you are satisfied that the layout and content level is sufficient for your requirements. In particular, ascertain whether the content was generated primarily from tools and if any further analysis has been carried out.
  • Are they dedicated to security? – There are an ever-increasing number of companies that claim to offer security assessment services. The suppliers can range from individual skilled hackers, through to organisations that have a long history of management and business consultancy services. Your organisation will need to evaluate their commitment to assessing your security in the short and longer terms. In general, it is normally better to select a supplier with a strong history in assessing the security of diverse system configurations and excels in this area, than an organisation that offers the service as a “bolt-on” to other management and business services. Select a provider that has a proven track record of finding known and new threats and whose recommendations provide a holistic approach to security.
  • What steps does the supplier take to secure your data? – Find out what steps the supplier takes to ensure that confidential data about the organisations systems will remain confidential. How long will they retain the data, how is it stored?
  • Are references available? – Although many organisations do not disclose whom they use for their security assessments, it may be possible for the potential supplier to provide a small list of satisfied clients. Evaluate this list and ensure that the references are current and, if possible, ensure that the references include at least one similar organisation.
  • What tools and methodology? – It is important to evaluate how the supplier proposes to assess your security. Ensure that they can explain their security assessment methodology and that it makes sense for your organisation’s environment. Of equal importance are the tools the supplier will utilise in the tests against the systems. Ask about the tools they use, and why they choose them. Ask about their relation with the tools – did they develop them, how up to date are they, what special benefits are gained by using them, are they licensed? Additionally, if they go beyond their tools, ensure that you have evaluated their personal skills. Look for a robust penetration testing methodology, preferably one that provides a multi-layer approach that identifies and leverages small cracks that springboard into identifying major cracks not found by typical security tools.
  • What is the level of risk to your systems while under going testing? – Depending upon the level and sophistication of the security assessment, there may be a possibility of some interruption of service to the systems under going testing. It is important that the supplier can explain the likely risks and elaborate on what steps should be taken to reduce the impact upon the tested systems.

Legal Issues

Before any security assessments can take place, it is important to ensure that several legal documents are mutually agreeable and signed.

  • Non-Disclosure Agreement (NDA) – Reinforcing the issues of trust between an organisation and the security assessment supplier, the NDA ensures that any confidential materials or ideas shared between both parties will not be disclosed to a third-party without prior consent.
  • Terms and Conditions –Due to the relatively intrusive nature of a security assessment, special conditions normally exist within the Terms and Conditions that may not be normally found in other supplier or contractor legal documents - in particular, very specific clauses on liabilities and the disruption to business services. It is important that the Terms and Conditions are agreed in advance of the assessment. Ensure that the legal team responsible for reviewing this documentation, fully understands the nature of the work to be carried out, particularly the testing methods and likelihood of disruption to business services.
  • Authorisation to commence work – Most security assessment suppliers will include some form of “Authorisation to commence work” page within their proposal (or statement of work). This page will normally have to be signed, dated, and sent back to the supplier before any assessment work will be carried out. Without this signed authorisation, some phases of testing could be illegal and the consultants conducting the assessment could potentially be prosecuted under various computer crime or anti-terrorism laws.

Assessment Deliverables

For most organisations, the final deliverable of a security assessment will be the report. However, several other deliverables could be included as part of the security assessment to ensure that maximum benefit is gained, provided sufficient resources and time is available. Consider the following deliverables:

  • Daily reporting – At the conclusion of each day, the security assessment provider should make a daily report available. This daily report should elaborate upon the activities that have been undertaken, what new vulnerabilities and security risks have been discovered, and describe the provisional fixes.
  • Technical briefings – For the duration of the security assessment, it may be advisable to have a morning review of the previous day’s findings between the supplier and the technical staff who are responsible for maintaining the systems undergoing security testing. The organisations technical staff will be able to better assess the impact of the discovered vulnerabilities or risks, and apply the most appropriate fixes.
  • Fix retesting – If it is possible to apply the correct fix to any discovered vulnerability during the security assessment period, it may be possible for the supplier to retest fixes daily and verify that the vulnerability or risk has been removed.
  • Final reporting – It is important that the supplier fully understand the reasons your organisation has for conducting the security assessment. Thus, particular emphasis can be placed upon key findings. For instance, some organisations may wish to emphasize the technical resolutions to vulnerabilities and risks for remediation purposes, while others may wish a non-technical report to reflect how much better their security is than an industry norm for distribution to existing or potential investors. Either way, the supplier should issue a “provisional” final report, whereby tweaking and emphasis of content is possible. Once agreed, a “Final Report” should be released.
  • Follow-up presentation – It may be possible to arrange a presentation following the security assessment. Most suppliers will be pleased to have an opportunity to present their findings and conclusions to your management group. Many technical leaders see the presentation as an opportunity to bring closure to the assessment and stress the key findings and business benefits.


The process of verifying and maintaining confidence in your system’s security level can be achieved through a combination of policy, auditing and assessing. While the first two can, and should, be largely conducted internally with in-house resources, the skills and processes constituting an assessment often mean that an assessment partner must be sought. Due to the nature of security assessments and the prevalence of potential partners, organisations must ensure that they both fully understand their internal requirements for the assessment, and establish a process for carefully evaluating the skills and assessment deliverables of the potential partner.

Security assessments should be seen as one of the main tools in an organisations security arsenal, capable of identifying weaknesses in both processes and systems. The utilisation of a robust “ethical hacking” methodology and real-life hacker tools, is often the only conclusive verification that key systems are capable of withstanding a typical attack.

    Copyright 2001-2007 © Gunter Ollmann