Hacking boxes too early to be beneficial
First Published: SC Magazine

As businesses attempt to improve their development processes by accelerating their release schedules, there is often a detrimental knock-on effect to the security of the application. Whether the application is web-based or compiled, internal or external, this pruning of the development cycle to rush out the latest software solution makes for easy pickings by security consultants. While many of these organisations respect the detailed findings of a “pre-live” security assessment of the application, the rushed development and deployment cycle inevitably means that the detailed security issues discovered by the security consultants have a reduced value on the overall protection of the application and its supporting environment.

The reasons behind this lie with the limited access to “top-drawer” security consultants, and the advanced scheduling of their time. For many penetration testing companies, dates for conducting security work must be agreed a week or two in advance – for some organisations this may even be a month or two. Therefore, any changes or slippage in the development cycle will probably affect when the security work can be carried out, and consequently the “go live” date – and invariably the project managers financial bonus.

Instead of rescheduling the security work for the next available slot (e.g. 2-4 weeks later), an increasing number of organisations choose to carry out the work anyway; against systems and application components that are already in place.

In many cases, security work that was originally scoped for the final “pre-live” environment must now be conducted against a partial or incomplete “test” environment. Consequently the consultants find many bugs and configuration flaws that, while critical or high risk from a standalone vulnerability perspective, do not actually relate back to what will eventually be the live application. Instead of getting valuable security advice and analysis that will help secure the business critical application, the client receives a report detailing configuration errors, lack of functionality, usability bugs, lax permission settings and scalability issues that provide very little insight into the real applications security.

Invariably, should any relevant vulnerabilities be discovered that would be applicable to the live environment, their value (and impact) is diminished amongst the confusion of what the real differences between the environment tested and the actual live environment will be.

In some cases the consultants are forced to evaluate the security of the “pre-live” hosting infrastructure – without even part of the application running in it. While this doesn’t happen too often, I can’t help but be amused when it does. For example, the last time it happened, the client insisted that we carry on with the application security assessment even though no custom application had been deployed within the environment. Instead we were tasked with assessing the security (or lack of it) of the web servers default suite of sample pages and online help functionality – which, obviously, should not and would not ever be found within a live environment. While we could root the environment and compromise back end systems through the existing vulnerabilities, the majority of the findings would not have existed in the real application once it had been deployed.

Apart from having completed a milestone on some middle managers project development schedule, I have to question both the security benefit and financial expenditure of carrying out this particular piece of work. The clients’ security team understood the worthlessness of the application assessment, however their hands were tied – they had a schedule to keep. Unfortunately for them they then had to spend several hours in meetings with system developers and architects reviewing the findings of the final report – probably followed by hours of violent agreement with all concerned that the findings didn’t really apply to what would eventually constitute the final application deployment.

A lot of organisations place too great an emphasis on assessing the security of applications before they are deployed – regardless of their state. Wherever possible, I would sooner be flexible and work with the client to slot the security assessment into a scheduled period where the results would be most beneficial. In almost every case, the client would be better off waiting until the application had gone live, and then testing – rather than testing an incomplete ensemble of code and components.

While am only too familiar with the fact that security consultancy is a “growth area” within IT services, the man-day rates for using penetration testing companies with access the most experienced security consultants is still an expensive affair. Certainly, from my own experience, many small to medium sized companies find it a bit of a shock to discover that access to technical security consultancy skills is more expensive than getting their external accountants done.

However, these consultancy rates shouldn’t be used as the battering ram for carrying on regardless and assessing the security of something that’s incomplete. Most professional penetration testing companies are very flexible in rescheduling and can easily accommodate changes given a few days notice. In the worst case scenario they may seek to charge the client for some of the lost time – but a little negotiation will usually keep this figure very low (or dropped if you hint at extra work in the future). It is certainly better to agree on a timeline that sees the complete application being assessed – rather than paying for a worthless report for an application that still has to be security tested. Believe me, even the penetration testers carrying out the work would sooner work on the real application.

Top Tips

  1. Select a penetration testing company with sufficient resources that can be flexible should you require it.
  2. Give the company a heads up of any delays or changes to the schedule as soon as you can – it may just save you money later.
  3. Testing an incomplete environment or application is almost worthless. It is better to wait until the application is fully deployed – even if this means the application has to go “live” in the meantime – than waste time and expensive resources.
    Copyright 2001-2007 © Gunter Ollmann