How to Assess AppSec Skills Before You Hire
How to assess AppSec skills: four skills, a code review task for each, what strong and weak attempts look like, and how to judge code you cannot read.
6 min read 
To assess AppSec skills, test four things: finding vulnerabilities in unfamiliar code, reasoning about how they would be exploited and how serious they are, giving fixes developers will accept, and reviewing designs before code is written. Use real code in the languages your teams use, with a known set of flaws and convincing decoys, and score what the candidate finds and how they explain it. You do not need to read the language yourself if the task comes with an answer key.
Below, each skill comes with what it looks like on the job, a task that tests it, and what strong and weak attempts look like.
1. Finding vulnerabilities in unfamiliar code
On the job: an AppSec engineer reviews code they did not write, in services they have never seen, often against a deadline. The core skill is reading fluently enough to spot what is wrong.
A task that tests it: a realistic pull request or service with several real vulnerabilities and some plausible decoys. Ask the candidate to find the real ones.
| A strong attempt | A weak attempt |
|---|---|
| Finds the real flaws, including missing authorisation checks | Hunts for SQL injection and stops when there is none |
| Ignores decoys that sound plausible but are not present | Picks every option that sounds dangerous |
| Explains each finding with the exact line and reason | Names vulnerability classes without pointing at the code |
2. Reasoning about exploitation and severity
On the job: not every finding matters equally. The engineer has to work out how a flaw would actually be exploited, what an attacker would gain, and how urgent the fix is.
A task that tests it: take one of the findings from task 1 and ask the candidate to walk through the exploit path and rate its severity, with reasons.
| A strong attempt | A weak attempt |
|---|---|
| Describes a realistic attack, step by step | Says “an attacker could exploit this” with no detail |
| Weighs exposure, authentication needed and data at risk | Uses the vulnerability class alone to set severity |
| Treats CVSS as an input, not the answer | Quotes a score without explaining it |
3. Giving fixes developers will accept
On the job: a finding is only useful if it gets fixed. The engineer has to propose a fix that works, keeps the feature working, and is explained so a developer agrees with it.
A task that tests it: ask the candidate to write up one finding for the developer who has to fix it, including the corrected code.
| A strong attempt | A weak attempt |
|---|---|
| Gives the right fix, such as parameterised queries or an ownership check | Suggests input filtering for everything |
| Includes steps to reproduce and example code | Describes the problem but not the fix |
| Writes respectfully and specifically | Writes as if the developer should have known better |
4. Reviewing designs before code is written
On the job: the cheapest vulnerabilities to fix are the ones caught in design. Senior AppSec engineers threat model new features and APIs before they are built.
A task that tests it: give the candidate a short design for a new feature, with a data flow diagram, and ask what could go wrong and what they would change.
| A strong attempt | A weak attempt |
|---|---|
| Marks the trust boundaries and where data crosses them | Lists generic threats that apply to any system |
| Finds abuse cases, such as skipping a step or changing an ID | Misses business logic flaws |
| Turns findings into specific design changes | Leaves the team with a list of concerns and no actions |
How do you judge code review skill in a language you cannot read?
You do not need to read the language if the assessment is built properly. Use a task with a known set of vulnerabilities and decoys, so findings can be scored against an answer key. Ask the candidate to explain each finding in plain language, which shows whether they understood the code or guessed. And choose the languages your teams actually use: strength in one language does not guarantee fluency in another.
How to calibrate the assessment by level
| Level | What they own | What to test |
|---|---|---|
| Junior | Reviewing code and triaging tool findings | Finding vulnerabilities, writing clear findings |
| Mid-level | Owning security review for several teams | Exploit reasoning, severity, fixes developers accept |
| Senior or lead | Design review and the AppSec programme | Threat modelling, influence, prioritisation across teams |
What an assessment cannot tell you
A code review task shows whether a candidate can read code and find flaws. It cannot fully show how they build relationships with development teams, how they handle pushback on a finding, or how they decide what a security programme should focus on. AppSec is half technical skill and half influence, so leave the influence half for the interview and references.
For questions to use in that interview, see these application security interview questions. For the wider method behind task design and scoring, see how to assess cyber security skills of candidates.
Frequently asked questions
What skills does an AppSec engineer need?
Fluent code reading in the languages their teams use, knowledge of common vulnerability classes and their fixes, exploit reasoning, threat modelling, and the communication skills to get developers to fix what they find.
Should AppSec candidates sit a coding test?
A code review test is more useful than a coding test. The job is mostly reading other people’s code and finding flaws, not writing features, so test reading and reviewing.
How long should an AppSec assessment be?
Long enough to review a realistic piece of code properly. A focused review task works well for screening; a design review or longer exercise suits later stages and senior roles.
How CyberHire assesses AppSec engineers
CyberHire is cyber technical screening that tests these skills on real code before you book an interview. Pick the ready-made application security assessment or paste your job specification and generate one. Candidates review realistic code across languages, separate real vulnerabilities from decoys, explain the exploit path and propose the fix.
Every candidate is scored the same way, with integrity signals next to each score. See how it works for application security hiring. If you would rather not build the assessment yourself, our team will build it with you.
Hiring an AppSec engineer?
Test AppSec skills on real code, free.
Send us the job spec. Within 48 hours we send you a hands-on code review assessment built around it, in your branding. See who can read the diff before you interview anyone.