How to assess a SOC analyst before you interview them
SOC hiring runs on certifications and trivia. Here is what a SOC analyst actually does all day, how to test each part of it, and what to probe at interview.
10 min read 
Ask a SOC analyst candidate what port DNS uses and they will tell you 53. So will anyone with a browser.
Ask them what Kerberoasting is and a good candidate and a well-prepared bad one give you the same answer, because it is a definition and definitions are free.
The short version: work out what the analyst will actually be doing on shift, then make them do a small piece of it. For a SOC role that means triage under ambiguity, log analysis, correlating across sources, and explaining what happened. Those four things are testable in under an hour, and none of them can be bluffed.
Start from the shift, not the job spec
Job specs for SOC roles are mostly a list of tools. Splunk, Sentinel, CrowdStrike, Defender. Then a certification block. Then “excellent communication skills.”
None of that tells you what the person does between nine and five.
Here is what they actually do. An alert fires. They have to decide, quickly, whether it is nothing, whether it needs more digging, or whether it needs waking someone up. They pull logs from two or three places to work out which. Most of the time it is nothing, and the skill is in being confident about that without being careless about it. Occasionally it is something, and then they have to explain it to someone who does not read logs.
That is the job. So that is what to test.
| What they do on shift | What to put in front of them |
|---|---|
| Triage an alert under ambiguity | A case with genuine decoys, where the obvious answer is wrong |
| Read and interpret raw logs | Real log output, unlabelled, with legitimate traffic in it |
| Correlate across sources | Two or three data sets that only make sense together |
| Explain it to a human | A written summary for a non-technical stakeholder |
Everything below is those four, made concrete.
1. Triage under ambiguity
The single most useful thing to test, and the one most assessments miss entirely.
Real triage is not “spot the bad thing”. Real triage is looking at something that is mostly normal, deciding which parts are not, and being right about it often enough to be trusted. An assessment where the malicious event is the only interesting thing on the page is not testing triage. It is testing reading.
Here is what testing it properly looks like.
Look at what this question is doing. The candidate is asked what attack class this is, and why a legitimate Microsoft URL does not make it safe.
Everything a weak analyst reaches for returns green. Authentication passes. The domain is real. The link is Microsoft’s own. One of the offered answers is even “display-name spoofing only, low risk because auth checks pass”, which is exactly the conclusion someone will reach if they have learned a checklist rather than a threat model.
The right answer requires understanding that the attack is OAuth consent phishing: the link really does go to Microsoft, the user really does authenticate with Microsoft, and what they are consenting to is an application with offline_access, Files.ReadWrite.All and Mail.ReadWrite scopes belonging to the attacker. No password is stolen. MFA does not help. The authentication working correctly is the mechanism, not a mitigation.
You cannot bluff that. You either understand the consent model or you do not, and no amount of CV polish gets you there.
The design principle: build the decoys deliberately. A phishing case where the sender domain is obviously wrong tests nothing, because every candidate has been taught to look at the sender domain. A case where every standard check passes tests whether they understand what the checks actually prove.
2. Reading raw logs
The most basic skill and the one candidates most consistently overstate.
“Experienced with log analysis” appears on nearly every SOC CV. What that means in practice varies from “I have read a log” to “I can reconstruct an intrusion from one”. The gap is enormous and a conversation will not surface it, because both candidates use the same words.
Give them logs.
Two things make this a real test rather than a spotting exercise.
First, the volume of legitimate traffic. Requests are coming from a dozen different addresses hitting /index.html, /contact, /dashboard, /api/v1/users, static assets. The attack is in there among normal browsing, which is exactly how it looks in production.
Second, the answer box is free text. No options to eliminate, no shortlist to guess from. They have to actually work out which address is doing it, which in practice means noticing a pattern of repeated /login requests from one source and being able to distinguish that from a person who mistyped their password twice.
That is also a real Linux shell, not a text box with log lines pasted into it. The prompt at the bottom is a root shell in a container. A candidate who knows what they are doing will reach for grep, awk, sort, uniq -c. A candidate who does not will scroll.
What you learn: whether they have a method. Strong candidates attack a log file with tooling. Weak ones read it top to bottom and hope.
3. Correlating across sources
Tier 1 work is often single-source. Anything above it is not. The step up from junior to competent is being able to hold a question across two or three data sets and follow it.
The question here is deliberately upstream. Not “which account was compromised” but which one was the initial point of compromise, which means working backwards through the chain until you find the first thing that happened.
That requires the candidate to know what to look for in sign-in telemetry, what device events would corroborate it, and what an audit log would show afterwards. Answering it from SigninLogs alone is not possible. That is the point.
This is also where the “do you know KQL” question gets settled properly. Someone who has genuinely hunted in Sentinel starts typing. Someone who listed KQL because the advert asked for it opens the schema tree and stalls. You learn that in about ninety seconds and no interview question gets you there as reliably.
4. Explaining it to a human
Consistently undervalued, and it is often the thing that determines whether a good analyst is actually useful to you.
An analyst who can find the intrusion but cannot explain it to a service owner, a manager, or a customer creates work rather than removing it. Ask for a short written summary of what happened, aimed at someone non-technical. You are looking for whether they can separate what is known from what is suspected, whether they lead with impact rather than method, and whether they can resist showing off.
This is the one part of the four that a well-designed interview can also assess. But having it in writing, produced under the same conditions as everyone else’s, gives you a comparison you will not get from four separate conversations.
Calibrating to the tier
The most common way SOC assessments fail is difficulty, and it usually fails in the same direction.
You ask your strongest analyst to write the test. They write something they find interesting. Your strongest analyst finds Tier 3 problems interesting. Now you are screening Tier 1 candidates against a Tier 3 bar, everyone fails, and the conclusion is that there are no good candidates.
There are. The test is wrong.
| Tier | What they own | Assess for |
|---|---|---|
| Tier 1 | First-line triage, escalation, following playbooks | Can they read a log, follow a defined process, and escalate the right things? Depth is not the point. Judgement about when to stop is. |
| Tier 2 | Investigation, correlation, tuning | Can they take an escalation and actually run it down across sources? Can they tell a false positive from a real one and say why? |
| Tier 3 | Threat hunting, detection engineering, deep IR | Can they work without a hypothesis handed to them? Can they design the detection rather than run it? |
A sanity check that takes twenty minutes and saves you a failed hiring round: give the assessment to someone already doing the job at that tier. If they find it uncomfortable, it is too hard. If they finish it without thinking, it is too easy.
The questions that do not work
Worth naming, because these are what most SOC interviews are still built on.
- “What port does X use?” Measures whether they revised. Nobody looks up ports from memory during an incident.
- “Explain the phases of incident response.” They will recite NIST or SANS. So will a candidate who has never worked an incident.
- “What is the difference between IDS and IPS?” Definition. Free.
- “Walk me through how you would investigate a phishing email.” Better, but still a description of work rather than the work. The candidate above who thinks “auth checks pass, therefore low risk” will describe a perfectly reasonable process and still get the case wrong.
- “Tell me about a time you handled a security incident.” Useful for other reasons, but unverifiable and rewards storytelling.
The pattern: if the answer lives in memory or in a browser tab, you are testing recall. Recall is not the job.
What to do with the result
Rank the cohort on what they produced, then use the detail to decide what to ask.
The value of assessing first is not that it replaces the interview. It is that it changes what the interview is for. Instead of spending your hour establishing whether someone can read a log, you spend it on the things an assessment genuinely cannot tell you: how they behave when they are wrong, whether they escalate too early or too late, how they handle a service owner pushing back, whether they will still be here in two years.
Use the assessment to generate the questions. A candidate who got the phishing case right but took a long time is worth asking how confident they were. A candidate who found the brute force but missed the correlation is worth asking how they would have approached it with more time. Those are much better interview questions than anything on a standard list, and they come out of evidence rather than a template.
The wider method, including scoring and how to read banded output, is in how to assess cybersecurity candidates. If you are staring at a large applicant pool, the sequencing problem is covered in how to screen 200 cyber applicants. And if you would rather not build any of this yourself, that is what we do.
One honest sentence
A SOC analyst’s job is deciding what matters in a pile of things that mostly do not, so an assessment that hands them one obviously bad thing on a clean page has tested the opposite of the skill you are hiring for.
Hiring for this role now?
Send us the spec and the shortlist.
CyberHire builds the technical assessment around the role, runs the candidates through it, and returns a report showing who performed and what to probe at interview.