An (aspiring hacker’s) web application penetration testing guide for 2024

Imagine receiving an email from cybercriminals stating that all your personal information has been compromised—name, date of birth, home address, and finances. They are demanding money from you in exchange for not leaking your sensitive information. 

It's a terrifying situation to be in, isn't it?

This is exactly what happened during the MOVEit breach in May 2023. Cybercriminals (the Clop ransomware gang) exploited a vulnerability in a file-transfer service that affected thousands of organizations and at least 60 million individuals.

On a more personal note, my talent team recently performed a penetration test on a highly regarded institution that is very diligent in protecting hundreds of critical records and documents. 

Although their primary concern was to ensure confidentiality, the team discovered a tiny detail in the web application that could delete all records from the outside instantly.

These are glaring testaments to what can go disastrously wrong when web application security isn't respected. 

Enter: Web application penetration testing.

Editor's note💡: One of the biggest data breaches on record emerged at the time of writing this piece.  Named the Mother of All Breaches (MOAB), 26 billion records are reported to have been leaked from apps like Dropbox, LinkedIn, and Twitter. 

What is web application penetration testing?

Web application penetration tests are proactive security assessments that evaluate the security of web applications by simulating real-world attacks. 

The primary goal is to uncover potential weaknesses, misconfigurations, and vulnerabilities that malicious actors could exploit to compromise the confidentiality, integrity, or availability of a web application.

Types of web application pentests 

There are multiple approaches to penetration testing web applications. 

Black box testing: In this approach, the tester operates with no prior knowledge of the system and formulates the test as if they were an uninformed attacker. 

This method closely simulates real-world attacks, demanding a high level of technical expertise.

Note💡: Although this approach resembles a realistic attack, penetration testers usually have a limited amount of time to perform them. Cybercriminals, on the other hand, have all the time in the world. So, there is a possibility that hidden vulnerabilities might not be uncovered during the test.

Grey box testing: With only limited knowledge of the system, the tester adopts this approach. The advantage lies in the focused nature of the web attack, leveraging the restricted information to pinpoint specific vulnerabilities.

The client can use this approach to give testers a “head start” and potentially speed up the process. This helps them uncover more vulnerabilities within the given time frame. 

White box testing: Testers under this approach possess complete knowledge of the system and enjoy unrestricted access. The primary goal is to conduct comprehensive testing, delving deeply into the system to gather as much information as possible. 

The main advantage of this approach is that it allows for an in-depth analysis of the system, leaving no stone unturned. However, this process can be time-consuming as testers have to consider large amounts of data.

Internal vs. external web pentesting

By now it’s clear that a web application penetration test is designed to mimic a real-life attack.

But do all attacks come from an external source? 

The answer is no. Threats lurk everywhere. 

They can even attack from inside.

This is why web applications can be tested in two ways.

Internal web penetration testing

You guessed it. This particular type of penetration test is done from within. Inside the organization’s network. The goal? 

Uncover vulnerabilities that malicious insiders or external attackers who have gained internal access could exploit. (For example, attacks that leverage user privileges or take advantage of unattended workstations.) 

This type of testing helps organizations identify and address potential security risks that may arise from within the network, that otherwise wouldn't be detected from the outside. 

Speaking of which…

External penetration testing

In this type of penetration test, all activities start from the "outside." Testers will conduct simulated attacks knowing only the public-facing information of the web application.

This includes the testing of web applications hosted on the internet.

The web app penetration testing process

 

Throughout the penetration testing procedure, testers mirror the cycle of a conventional malicious threat or "adversary," albeit with a more focused strategy. 

We use various references to guide us through the stages of an app penetration test. These act as a map when navigating the testing process. Different references (or “maps”) serve different purposes. 

When testing web applications for vulnerabilities, we usually refer to the:

• OWASP Web Security Testing Framework. 

• Open Source Security Testing Methodology Manual (OSSTMM). 

Several of these references mention all the steps required in a web app penetest. 

Note💡: Companies and testers often employ these guided steps flexibly. During a recent penetration test, for instance, the client provided my team with all the necessary intelligence and vulnerability scans required for that specific test. In this case (and as requested by the client), we skipped the recon stage. We proceeded directly to exploitation.

The stages of this attack lifecycle can be succinctly summarized as follows:

1. Pre-engagement

Involves direct contact with the client. Here is where the test will be defined and documented via a contract. During a face-to-face meeting or conference call, key arrangements are agreed upon, including:

• Non-Disclosure Agreements.

• Goals.

• Scope.

• Time estimation.

• Rules of engagement.

You can think of this as the "get out of jail card" that protects you and your employer. 

All tests must be legal and mutually agreed—because simulating a real-world attack can have real-world consequences.

 2. Information gathering

“Know your enemy.”

This stage is all about gathering as much information as possible about the target. Specifically involving searching for information about the target company’s: 

• Web infrastructure.

• Software applications it employs.

• Hardware it uses. 

You can gather information in two ways:

Passive information gathering

Passive reconnaissance aims to avoid alerting the target to the probing activities.

Information is obtained without interacting directly with the target. You’ll leverage publicly available resources to gather insights into the target's web presence, technology stack, and potential points of vulnerability.

Google Dorking is one example: If testers are interested in searching for a specific word on a website, they can refine Google search results using the following syntax: site:*.<domain.com> <word>.

(Read Ippsec’s guide on Google Dorking for more info on this recon method.)

 

I first realized the power of a Google Dork when I was looking for the Cisco Anyconnect software used to connect to Cisco VPNs. However, Cisco didn’t allow for downloading the software and googling “Cisco Anyconnect” led to many websites just talking about the software. 

 

I looked at the advanced search operators of Google and noticed I could search the titles of web pages. After seeing I could search by the HTTP Titles, I had an idea that if I searched for "Open Directory Listings", then my results would be just files. I changed my search query to intitle:index.of cisco anyconnect and suddenly I found loads of results. 

 

I ended up downloading the executable and then generating an MD5sum hash of the file then comparing it with what was on Cisco’s site. Once I found a file that matched up, I knew it hadn’t been tampered with and was safe to run.

 

At the time I didn’t know this was called Google Dorking, it wasn’t until I came across the Google Hacking for Penetration Testers BlackHat presentation that I realized the full power of Google.

Active information gathering

Active reconnaissance means direct contact with the target to gather detailed information about its infrastructure. 

A more invasive and thorough analysis reveals potential vulnerabilities and hidden assets that might not be discovered through passive reconnaissance.

The famous vulnerability scanners are put to good use here:

• Nmap

• Nessus

• OpenVAS

• Nikto

We’ll dive into these and other web pentesting tools later.

3. Exploitation

Armed with all the information and identified vulnerabilities in the previous stages, testers are ready to rock and roll. This is where the actual “hacking” takes place.

This stage is what sets penetration testing apart from vulnerability assessments. Vulnerabilities are hands-on tested to determine their true exploitability and impact.

A diligent tester is always up to date with the latest and most prevalent web application vulnerabilities and how to exploit them, such as:

• SQLi

• HTTP Verb Tampering

• Directory Traversal

• SSRF

• File Inclusion

• Arbitrary File Uploads

• Broken Authentication

• DoS

• XSS

• XXE

4. Post-engagement

Testers diligently craft comprehensive documentation intended for delivery to the client. 

The standard deliverables encompass a detailed penetration testing report accompanied by a walkthrough meeting.

The report documents each action, step, command, and tool employed by the testers throughout the assessment. 

This provides the client with a clear understanding of the identified vulnerabilities, their potential impact, and recommended remediation strategies.

Note💡: Hack The Box has a ready-to-use penetration testing template waiting for you. Follow this in depth guide on how to use it.

Tools for web application penetration testing

There are essential tools that any web pentester must be comfortable with to carry out tests.

Web proxies

Web proxies are indispensable for web application penetration testing. They capture the requests and traffic passing between applications and back-end servers and allow for the manipulation of these types of requests. Popular web proxies include:

• Burp Suite

• OWASP Zed Attack Proxy (ZAP)

 

(Get detailed guidance and practice hacking web apps with our Intro To Web Applications Module). 

Vulnerability scanners

While web proxies focus on intercepting and manipulating traffic, vulnerability scanners automate the process of identifying potential weaknesses in web applications providing a baseline for further investigation. 

Examples include:

• Acunetix

• Nessus

• OpenVAS

• Nikto

Penetration testing frameworks

Some tools and frameworks make the whole web penetration testing process smoother. After all, you must hack in style, right?

Metasploit, for instance, stands out as a versatile framework that aids in the development and execution of custom exploits, providing a robust platform for in-depth testing and analysis.

To give you an insight into the potential of Metasploit, consider a test where our team integrated the result from a Nessus vulnerability scan into Metasploit. By leveraging all its different modules we quickly confirmed and automatically exploited a particularly serious vulnerability. 

 

This potentially saved hours of work. 

 

A process that would be extremely slow with several stages was now instant and seamless. Without this solution, the vulnerability and its consequences may not have been discovered within the limited time frame of the test.

Fuzzing tools 

Feeding large amounts of data into a web application can uncover unexpected vulnerabilities. That's called fuzzing.

Fuzzing can uncover weaknesses that could not be found using other tools and methods. 

It can identify intricate bugs that are hidden deep inside the web application's codebase, which cannot be found by vulnerability scans or brute force content discovery methods.

Other tools include: 

• Ffuf: An open-source fuzzing tool written in Go. Helps you find hidden content on web applications.

• Gobuster: For brute-forcing directories and files in web sites.

• WFuzz: A tool used to discover common vulnerabilities on web applications. Written in Python.

You must also keep in mind that it's not all about automated tools! Manual testing is also fundamental. 

It allows for a more granular, in-depth exploitation of web applications. Uncovering what automated tools miss or get wrong (hackers truly shine at this stage).

We've been talking about tools. I bet by now you are thinking "I must learn all these cool tools". Yes, you must. But learn fundamental concepts first; have a clear understanding of why you're using those tools and not how to use them.

Recommended read: 7 powerful pentesting tools for beginners (with learning resources to practice them).

Become a web app pentester:

 

Foundations, foundations

We've all seen the memes:

(Unless you can do the splits like Jean-Claude Van Damme, please do not try this at home.)

How can we effectively test web applications if we don't have a basic understanding of what they are and how they work?

Pause for a moment and evaluate your current grasp of operating systems. Do you have a good handle on both Linux and Windows? Are you ready to rock and roll with both graphical interfaces and command line interfaces?

Do a knowledge check and take both the Windows Fundamentals and Linux Fundamentals on Hack The Box Academy. 

“I don’t need weapons. I am the weapon.”

Web application penetration testing tools often have user-friendly interfaces and can do some of the heavy lifting for you. Knowing programming languages, however, will level you up from a tool user to a tool creator.  

Being proficient with programming languages will allow you to better understand the building blocks of the applications you'll be testing. 

In the long run, this will set you apart from the average penetration tester, developer, or hacker. 

You’ll be able to tailor and extend existing tools, create custom exploits, and truly comprehend the vulnerabilities you aim to exploit. 

You can start with HTB Academy’s Introduction to Python 3 course. A perfect module for beginners to understand programming languages.

Python is highly regarded in cybersecurity. It's easy to learn, amazing for networking, and fast as Lighting McQueen. You can do a lot with it with a minimal amount of code! *chef's kiss* 

Learn Python on the HTB Academy

• Get everything you need to know to create valuable scripts.

• Make your first “homemade” tool for cybersecurity using Python. 

• Enjoy a hands-on guided learning environment to practice Python-related cybersecurity skills for both offensive and defensive security.

Start Learning Python

Learn web application pentesting with HTB Academy 

Drill down with the Web Requests module, and learn the fundamentals of web applications. This module will also introduce you to cURL and the Browser DevTools. These are essentials every penetration tester must have in their arsenal (yay, more tools!).

Follow up with Introduction to Web Applications. Learn how web applications work and have your first contact with the art of web application penetration testing.

This will be your initial encounter with common vulnerabilities found in web applications, and soon enough, you’ll be exploiting them.

After covering all the fundamentals you can start focusing your path on learning web application vulnerabilities and how to exploit them:

• Cross-Site Scripting (XSS)

• SQL Injection Fundamentals

• Command Injections

• Broken Authentication

• HTTP Attacks

And here's the kicker, you don't have to sprint through these courses. Take it slow, enjoy the ride. 

Eventually, you'll develop a mindset that will allow you to correlate concepts, making it easier to further develop your skills. (Trust me, this whole learning process can become quite addictive.)

The path of the righteous: HTB job role paths

Hopefully by now you'll have a more clear understanding of the basics. 

If you are still overwhelmed with the amount of information or you want a more focused approach without "outside noise", consider choosing a Job Role Path on Hack The Box. 

Hack the Box Job Role Paths will guide you systematically through Academy modules, equipping you with practical skills and the mindset required for a specific job role.

Consider following The Bug Bounty Hunter Job Role Path. 

It’s ideal if you’re a security-minded developer, aspiring web pentester, or want to become a good bug bounty hunter. (Despite bug bounty hunting being a separate discipline from web application penetration testing, the skills necessary for both are similar.) 

If you want to become a professional web penetration tester, check out the Penetration Tester job role path. Learn core security assessment concepts and a deep understanding of the specialized tools, attack tactics, and methodology used during penetration testing.