Security Assessment Techniques: Code Review v Pen Testing
Web application security testing and assessment should include both security code review and penetration testing techniques. Dave Wichers, an OWASP board member, spoke at the recent AppSec DC 2010 Conference about the pros and cons of code review and pen testing approaches in finding security vulnerabilities in web applications.
Dave said both code review and pen testing can leverage automated analysis tools to assist the security engineers in the process. The code review process should include checking all the custom developed code as well as any configuration files of the application, libraries, frameworks, and the server where the application has been deployed.
He compared the strengths and weaknesses of each approach. The strengths of pen testing are it requires less specialized expertise, easier to setup and perform, exercises the entire application infrastructure and it proves the vulnerabilities. On the other hand, code review approach has the advantages that it’s easier to find all the content, all instances of certain types of flaws, it verifies controls are correct and that the controls are used in all the required places.
Dave also showed how these two techniques stack up to the OWASP Top 10 list of security vulnerabilities. Code review is a better approach in finding vulnerabilities in the following categories:
- Injection flaws
- Cross-Site Scripting (XSS)
- Direct Object References
- Cross-Site Request Forgery (CSRF)
- URL Based Access Control
- Crypto Storage
- Redirect/Forward flaws
And penetration testing is the winner in finding security flaws in the following categories:
- Configuration flaws
- Transport Security flaws
Other issues like authentication vulnerabilities and finding authentication & session management flaws can be discovered using both the techniques. Both provide value in finding authentication related issues such as Account Lockout, Strong Credentials Requirement, Authentication Event Logging, Proper Invalidation of Sessions on Logout, and Sufficiently Random Session Tokens.
Security teams should use both techniques in their security testing efforts but the advantage of code review grows significantly with size of the application portfolio and the level of rigor of the assessment. Dave concluded the discussion saying it's a myth that code review is way more expensive. If you have people with the right skills, it's actually faster and more effective.
Web app scans vs Manual Code Reviews
Re: Web app scans vs Manual Code Reviews
Re: Web app scans vs Manual Code Reviews
I believe that Jeevak is from HP and so he might be a little biased on the capabilities of automated analysis tools. Such tools have their value, but they also have serious limitations as well, and require much more expertise to use effectively than most people understand.
The sarcastic reply from 'HP Rules', whoever that is, is just pointing out that the previous poster is most likely from a product vendor. I'm sure HP isn't promising anything as wildly fantastic as this sarcastic reply, but customers of such tools do need to clearly understand the strengths and weaknesses of such tools and the level of expertise, integration, and customization required to get good value from any investment in automated analysis tools.
Aspect Security has worked with numerous customers who have first invested large amounts of money buying tools, but didn't invest much in training, integration, customization, as well as making sure they had reasonably skilled staff using these tools. It's certainly appealing to imagine tools can handle the bulk of the effort required to discover vulnerabilities and determine the severity of the risks they represent but the amount of false positives and negatives is still so high that a significant amount of expert user involvement is still required to triage what was found, find the flaws the tools can't find, and then triage all this stuff so that the most important risks rise to the top, rather than simply the risks that are easiest to identify with automation. After we worked with these customers to integrate and customize these tools for their environment, they started to receive much more value from their investment in automation than they were getting before.
One other thing I want to point out is that finding flaws, which was the focus of my talk, is still the least expensive part of the application security problem, and yet it is getting the most focus today. The expensive part is building secure applications in the first place, or remediating issues that are found through application security assessments. Training developers, arming them with reusable security controls, introducing security architecture reviews and threat modeling, and secure coding guidelines is where you will get the most bang for your buck, and yet automated analysis is getting the lion's share of many naive organizations attention because it's so appealing to imagine that this will solve their problem, or at least be the best first place to invest.
In summary, I think automation clearly has a useful role in both code analysis and application penetration testing, but it requires far more expert human investment to get significant value out of them than most people realize, and these tools are simply unable to discover some of the highest risk issues like authentication, access control, and context aware input validation and output encoding.