How to Prepare for a Capture the Flag Hacking Competition

Hacking isn't like the movies. You're not facing off against a worthy adversary in real-time, backed by your motley crew.

In fact, most security jobs are boring—and to be honest, you want them that way. Audit, find vulnerabilities, patch, rinse, and repeat. To stay sharp, many security professionals, both new and old, enter Capture the Flag (CTF) competitions or use CTF challenges to learn.

For aspiring white hats, CTF challenges are a great way to learn hacking techniques, strengthen your problem-solving skills, and gain critical hands-on practice. CTF competitions provide just the right amount of pressure to keep things interesting

For the pros, CTF competitions help you assess your skill level, challenge yourself among peers, and maybe even earn some bragging rights.

Here's how CTF competitions work and a few tips on how to prepare for your first competition.

What are Capture the Flag Competitions?

Capture the Flag hacking competitions are exactly like the first-person shooter game mode. One team of players attempts to locate and capture an opposing team's "flag" while also defending their flag. 

In CTF competitions, the flag is typically a snippet of code, a piece of hardware on a network, or perhaps a file. In other cases, the competition may progress through a series of questions, like a race.

They can either be single events or ongoing challenges, typically falling into three main categories: Jeopardy, Attack-Defense, and mixed events.

Attack-Defense

This style of competition is much closer to the backyard capture-the-flag game than the Jeopardy-style. In this type of event, teams defend their host PC while attacking opposing teams' target PCs. 

Each team starts with an allotted time for patching and securing the PC, trying to discover as many vulnerabilities as possible before the opponent's attacking teams can strike. Teams receive points for staving off attacks from opposing teams and successfully infiltrating other teams. The team with the most points wins.

Jeopardy CTF

Jeopardy-style CTFs present competitors with a set of questions that reveal clues that guide them in solving complex tasks in a specific order. By revealing clues, contestants learn the right direction regarding techniques and methodologies that are needed going forward. Teams receive points for each solved task—the more difficult the task, the more points you can earn upon its successful completion.

Ongoing, online CTF competitions are most likely to be Jeopardy-style. It's easier to play solo and requires less coordination among players than an Attack-and-defend competition.

Mixed Events

As the name suggests, mixed competitions combine Jeopardy and Attack-Defend formats. Sometimes, organizers segment the competition into events, and other times, they split teams to compete in concurrent events of different styles.

What's the Difference Between CTF and Hackathons?

Both CTF competitions and hackathons involve teams using their skills in a concerted fashion within a time limit. But that's about where the comparison ends.

CTF competitions encourage teams to earn points by subverting security systems and sidestepping safeguards through known or competitor-created exploits. In other words, it's a game.

Hackathons are more collaborative events that allow developers and programmers to showcase their creative talents by building a working application or program within an allotted time period while following specific criteria. Although they can be security-related, hackathons are a generalized term.

The word 'hack' in a hackathon refers to how an end product is 'hacked together,' a popular phrase in homebrew and DIY enthusiast circles, not as in computer hacking.

How to Prepare for Capture the Flag Competitions

Unlike most technical certifications, CTF competitions are 100 percent practical. There's no multiple-choice. To be successful, you've got to build up a strong knowledge base and then draw from it. 

Though that sounds daunting, it's not that bad. Provided you've learned (or started learning) white hat basics, you'll learn everything else you need from practice, practice, and more practice.

There are some great resources with challenging problem sets available for free.

PicoCTF

Plenty of aspiring white hats start with PicoCTF. It's actually intended for middle and high schoolers. For that reason, it covers the basics very well, provides many hints, and reveals challenges as an interesting storyline. If you're an adult, you can't compete for prizes, but the lessons are still excellent.

Smash the Stack

Among the most popular wargame sites, Smash the Stack hosts several wargames to attack operating systems, networks, and applications. Most wargames are always online, but they also have regular competitions. Due to its popularity, beginners can reference plenty of write-ups on GitHub, personal blogs, or even YouTube.

Over the Wire

Developed by a robust community of "good-looking hackers," OverTheWire has wargames for every skill level. The 34-level Bandit wargame is the perfect starting place for absolute beginners. 

Eventually, you can progress to the Manpage wargame. With each game on its own SSH port, even connecting to the individual games is a learning exercise.

Microcorruption

It's not a pretty website, but Microcorruption shows you how to exploit real-world software flaws with a debugger. Even better, you channel your inner Mission Impossible with a storyline that involves stealing a briefcase of bearer bonds. As they put it, "Should be a milk run. Good luck."

Google CTF

The Google CTF comprises 23 challenges and one "Beginner's Quest." The challenges are available year-round, but the team competition only runs for a weekend in the summer. Google pays $100 for the best 21 write-ups and $500 for the 11 most creative solutions.

Note: These are merely a few popular examples of CTF challenge sites. They are by no means the only resources out there.

Types of Capture the Flag Questions

If even gamified learning sounds daunting, then find solace in the fact that CTF questions typically fall into five categories. You don't have to become an expert in every subject matter area, but you should have a working knowledge of each.

Question Type 1: Binary Exploitation

Binary exploitation comes down to making an application act differently than it was intended to run. By making the application run differently, you're gaining valuable information that you'll use to alter or commandeer the target.

Common binary exploits use a technique known as memory corruption, which can enable an attacker to gain unauthorized privileges to the system running the application or by hijacking the application's control flow and injecting their commands directly into the system.

Question Type 2: Reverse Engineering

Sometimes the flag will be a string hidden inside the application code. Depending on the challenge type and level of difficulty of the task, you might need to use reverse engineering.

Reverse engineering challenges require an intimate knowledge of debugger and disassembler software. The goal: Take a compiled binary, rip it apart, and find out how it works.

You will want to be familiar with how the application uses control flow, loops, and conditionals so that you can figure out how to bend the program to your will and then hopefully capture the flag.

Question Type 3: Web Exploitation

These question types cover a wide range of different methods to exploit web-based resources. While the methods are broad, there are tools commonly associated with web exploitation, including Nmap, Wireshark, and Metasploit.

Some of the easier flags are even accessible through your web browser through "View Page Source" or the equivalent in your browser.

Question Type 4: Cryptography

Cryptography challenges are particularly fun. Even the definition of cryptography sounds fun. "Cryptography is the practice and study of techniques for secure communication in the presence of third parties." 

In practice, however, they can be difficult. Often enough, these questions are based on string conversions from one format to another. For instance, you might be given a file that starts like this:

8a10247f90d0a05538888ad6205882196f5f6d05c21ec8dca0cb0be02c3f8b09e382963f443aa514daa501257b09a36bf8c4c392d8ca1bf4395f0d5f2542148c7e5ff22237969874bf66cb85357ef99956accf13ba1af36ca7a91a50533c4d89b7353f908c5a166774293b0bf6247391df69c87dacc4125a99ec417221b58170e633381e3847c6b1c28dda2913c011e13fc4406f8fe73bbf78e803e1d995ce4dbd20aad820c9e387ea57408566e5844c1e470e9d6fbbdba3a6b4df1dd85bce2208f1944f1827d015df9c46c22803f41d1052acb721977f0ccc13db95c970252091ea5e36e423ee6a2f2d12ef909fcadd42529885d221af1225e32161b85e6dc03465cf398c937846b18bac05e88820a567caac113762753dffbe6ece09823bab5aee947a682bb3156f42df1d8dc320a897ee79981cf937390b4ae93eb8657f6ced9eccbe79394ca0575a81d1fa51443aa3e83e5e3cdb7a4c5897faa6b4ac123c1dde2dff4d7c5b77d2aa3c090cebce70340e7f0e0b754ca26b9c108ca0dbfdcd8aa230eb9420654b252ffc118e830179cc12b64b2819f81edcc2543d759c422c3850051d543c9bc1dcda7c2a6c9896e1161d61c3c13c80ee79c08379abf3e189f2ecf5f997db17db69467bf6dfd485522d084d6e00a329526848bb85414a7e6a

And scrolls forever. Your challenge: "In this file are a bunch of hex-encoded ciphertexts. One of them has been encrypted with ECB. Detect it." And that's an intro Cryptopals challenge.

In other cases, you'll have to encrypt or decrypt messages. You'll need to have a good handle on programming for cryptography. If you don't, it's a lucrative skill to attain.

Question Type 5: Forensics

This type of question in a CTF environment can cover a lot of ground, but it is quite common that you'll be asked to find files or information hidden within other file types. For instance, a simple jpg or png file could be manipulated to hold information such as text or even an executable.

By digging into these files with scripts and tools, competitors can extract data (normally encrypted) and then run it against a series of other tools as they try to decode the coveted flag. There are many useful tutorials and write-ups online that can get you started.

Find Your Favorite Type of CTF Problem

As you learn your trade, you'll likely find that you're strong with, or particularly enjoy, one type of problem. Once you have found your favorite type of problem to solve, then specialize. It's perfectly acceptable to go deep into one subject matter area. In fact, it's recommended.

Write-ups are a key part of CTF competitions. Teams prepare documentation (called write-ups) about the vulnerabilities they found and the processes they used to exploit the vulnerability. 

Judges often use write-ups to evaluate teams, like in the Google CTF. Remember that white hats have a goal in mind, to develop a fix. Write-ups help track vulnerabilities and how to fix them. For that reason, they're great learning material for beginners.

When you're first learning, write-ups are a great resource to check your solution, or even provide a little help when you're stuck.

To reiterate, before you dive in and start looking at solutions, make sure that:

• You use write-ups only after you've solved the problem

• You don't use write-ups as a cheat sheet

To prepare for a CTF competition, you'll want to find and read as many write-ups as possible. They also make for good practice, even if they're way over your head. You can find past event results with questions, and you can try taking them on before you join a live tournament.

These past events are usually well documented, with solutions and problem-solving steps included in most of the write-up resources. Even if answer quality and completeness vary from author to author, you can look at multiple solutions to the same problems. 

You can then compare the best answers to your attempts and find out if you could learn any more about these types of problems by incorporating some of those techniques into your bag of tricks.

Get Started Hacking Today

If you're brand new to hacking, then find a good course to teach you the skills. The best courses, like Keith Barker's new White Hat Hacking v10 course, start by setting up a Kali Linux practice lab so that you can get hands-on experience immediately.

When you're ready, work through the CTF challenges, review the write-ups, and maybe even enter a local competition. By competing in these competitions and following the challenges, you will strengthen your knowledge and understanding of how the technologies work and how to select the appropriate responses to a particular challenge.

These skills quickly accumulate and, over time, teach you valuable, real-world techniques that you can apply in your studies and working environment, allowing you to expand your ethical hacking toolkit.

Not a CBT Nuggets subscriber? Sign up for a free 7-day trial.