Race Conditions and HackerOne Heartbreaks:Day One or One Day

01The beginning

Day one, with all the false starts.

This is the beginning of a series tracking my journey into AI-assisted pentesting. I’m not claiming to be the best or to have the best methodology. I want to document the real journey from day one, including what did not work, because that feels more useful than posting only the highlights.

We have all read the flashy posts about AI jailbreaking enterprise targets in seconds. I walked into this assuming it would be easy. I pictured pentesting as a cinematic lifestyle: a blacked-out room, hoodie up, five monitors, hacking the mainframe. The reality was one laptop, a questionable amount of caffeine, and me trying to stay awake for a 9:00 AM call.

Here is the part the flashy posts skip: without strict guardrails, an LLM will not find zero-days while you sleep. It may confidently hallucinate two dozen fake Semgrep hits and recommend an out-of-scope bug that gets your account a stern talking-to. The model is a fantastic assistant and a terrible oracle. The sprint came down to knowing which one it was at any given moment.

Over two weeks, I used Olympus, GreenHat Security’s AI-assisted research loop, to break open-source enterprise codebases into pieces I could read and to help draft strict dynamic test harnesses. The system did not find the bugs. I found them by reading. The AI made that reading faster; the harnesses are where the real story lives.

A tool that confidently says “I don’t know” is infinitely more valuable than one that hands you a green checkmark based on a lie.

Vincent Suen

02The pivot

The rejection that taught me the most.

My entire sprint actually started with a rejection. I submitted what I thought was a solid session-invalidation bug, only to get hit with the dreaded “Informative” status.

The triage response

“Organizations requiring immediate revocation typically implement stateful session management or token blacklists, which are architectural design choices.”

My first instinct was denial. My second, better one, was to read the reasoning again and understand where the triager was coming from. It was, annoyingly, pretty good. Stateless JWTs with a refresh interval behave like that by design: the user had legitimate access a moment earlier, and no privilege boundary had been crossed.

My boss’s advice was simple: never walk away from a rejection. Use the feedback as a roadmap, dig deeper, and pivot. “Not Applicable” rarely means “you are wrong.” Often it means “you described a behaviour, not an impact.” I had described the behaviour beautifully, with line numbers. So I went back to the source and looked at neighboring preview and metadata routes where a convenience shortcut might skip the guard enforced on the primary route.

03The method

I built tools that were allowed to say “I don’t know.”

Before touching either target, I decided every proof of concept needed three possible verdicts. That third one is the point of this story.

A proof of concept that can only shout “VULNERABLE” is not proof. It is a slot machine with one symbol. If a test cannot distinguish “the control was bypassed” from “the control was never switched on in this environment,” its green light means nothing.

So every harness got negative controls: run the same attack with the malicious flag off and confirm nothing changes; supply a deliberately invalid identifier and confirm it errors; after the attack, check whether the effect reverts. That last check helps distinguish transient state from something permanent, the difference between a scary demo and a scary incident.

I also made one INCONCLUSIVE result poison the whole run. No partial credit. That decision saved me twice in one afternoon.

04Review setup

Two targets. One hypothesis.

To avoid LLM context rot, we don’t feed Olympus entire repositories. I constrained the AI to specific, high-risk authentication handlers and controllers. For this sprint, we targeted two distinct threat models:

The core hypothesis was identical for both: developers apply robust authorization checks to primary state-changing routes, then forget those same checks on adjacent metadata or preview routes. The front door has three locks. The side window is held shut with hope.

05The hunt / Target A

The clean win.

The enterprise ERP hunt was a textbook win. Reading the SAML controller, one asymmetry lit up like a warning label: the primary sign-in route enforced a strict tenant filter, while the adjacent metadata route accepted an attacker-controlled query parameter with zero filtering. The authorization on the main route simply did not extend to its next-door neighbor.

To prove this was not a static-analysis artifact, I spun up a purpose-built two-tenant Docker lab and bracketed it with positive and negative controls. An unauthenticated actor could cross the isolation boundary, read another tenant’s SAML configuration, and enumerate valid internal database names. I routed a private, responsible disclosure to the vendor’s security team.

It is worth saying out loud: most of this codebase was good. Permission checks were in the right places, and data access was gated correctly. Bug-bounty writing has a bad habit of making every target sound like a dumpster fire on wheels. You cannot spot the outlier if outliers are the only thing you have ever looked at.

06The hunt / Target B

The twist was in the timing.

Then came the government forms platform. I found a form-submission action with a client-controlled preview flag. When set to true, it bypassed the captcha and cached a draft form under a shared key that was not scoped by publish state. I had already written up the finding as “an attacker can do this at will.” I built the harness, hit Enter with the quiet smugness of a man about to be correct, and got this instead:

My first reaction was to assume the test was broken. That is the dangerous instinct: when a tool disagrees with your belief, the cheapest move is to “fix” the tool until it agrees. Instead, I looked at the state the code was actually reading. The code was right; I was wrong. A feeling I knew well.

This was not an arbitrary write. The target short-circuited on a warm cache hit. What I actually had was a repopulation race: you cannot write the cache whenever you like, but you can win the window just after the 30–60 second TTL expires and repopulate the shared key with the draft before legitimate traffic does. Same bug. Real bug. Completely different claim.

If I had shipped my original write-up, a competent triager would have tried it once, seen nothing happen, and closed it as Not Applicable. Twice burned by the same word. The same heartbreak.

Inside the race

A cache miss changes the outcome.

The cache race, in motion. Timing is compressed.

01Warm cache

The existing value is returned. The first harness cannot overwrite it.

02TTL expires

After 30–60 seconds, the shared cache needs to be repopulated.

03Preview request wins

The draft fills the shared key before legitimate traffic does.

↳

Public impact The live page serves the unapproved draft template.

The exploit logic flow

1. Request. Anonymous POST to submitForm with client-controlled isPreview: true.
2. Missing guard. The application skips the captcha check, bypassing anti-automation.
3. The race. The harness loops 76 requests to beat legitimate traffic during the 30–60 second cache TTL window.
4. Impact. The Redis key is populated with the draft template, serving unapproved content to the public.

I rewrote the harness to model the race, with the flag off for the control, on for the exploit, and a post-run revert check to confirm the result was transient cache state rather than a persisted write. The live view flipped to the draft in the successful run.

Elapsed time

38.0seconds

Unauthenticated requests

76requests

Successful runs measured

1sample

I submitted the report to HackerOne.

07What stays with me

The process was worth more.

Finding a vulnerability, understanding it, and successfully exploiting it with AI brings an incredible rush of dopamine. Hitting an “out of scope” wall or getting a “Duplicate” status is definitively the complete opposite. It’s gut-wrenching.

But it was not a total loss. The triager noted the depth of the proof of concept, controls, and harness, and that made it sting a little less.

It was like putting a bandaid on a hole in my chest.

Here is what I actually walked away with. “Not Applicable” is free tuition: the triager showed me exactly what my report was missing, and pride is not a scarce resource. Describe impact, not behaviour. “The token stays valid for 30 minutes” is a behaviour; “a user explicitly denied access can get it anyway” is an impact. Same codebase, wildly different reception.

Build the thing that can tell you no. Tooling that can only confirm you is a mirror. Measure the scary number before you print it. Read the good code too; knowing what correct looks like is how the broken part stands out. And be generous about the rejection: the triager was neither lazy nor wrong. Holding that view made me a better researcher and the experience considerably less miserable.

Where this goes next

We are feeding these false starts, dead ends, and authorization-boundary failures back into Argus, our broken-access-control engine, through Olympus, the harness it runs in. Inspired by offline replay methods like the Dream-RSI framework, we are building simulation loops that score and refine exploration paths against historical traces, to measure whether the automation improves without spending extra compute on live targets.

Two targets, one self-correction, one duplicate, and a triager who taught me the value of a tool that can say, “I don’t know.” The process was worth more than the CVE. This will go down in my books as the best rejection I’ve ever faced.

Written by Vincent Suen

Field notes on AI-assisted pentesting · Entry 01