Utkarsh

+Profile

I'm a second-year CS undergrad at BITS Pilani working on AI security and red-teaming. I build adversarial tooling, ship open-source fixes to the systems people actually run, and write pre-registered research on how models reason under pressure.

11PRs Merged

07OSS Orgs

04Projects

01Pre-Reg Study

03

Selected Work

FOUR PROJECTS // SYSTEMS x SECURITY // 2025–2026

01ENGINE

Shadowbook

LIMIT ORDER BOOK MATCHING ENGINE

ROLECreator

A single-instrument limit order book matching engine in Rust, verified by differential fuzzing. The engine, a reference oracle, and a fuzz harness run side by side across ~100M operations with zero divergences. Once the engine stabilised, the real bugs surfaced in the verification machinery.

386nsp50 Latency

667nsp99 Latency

0Hot-Path Allocs

RustCriterionDiff Fuzzing

GitHub

01SB

DIFF FUZZ // 100M OPS // ZERO DIVERGENCES

02LIVE

Cairn

SELF-HOSTED GIT HOST // CONTENT-ADDRESSABLE VCS

ROLECreator

A self-hosted Git host built around a from-scratch content-addressable version control engine: object store, commit DAG with generation numbers, packfiles with delta compression, Myers diff, three-way merge, and trigram code search. Cross-verified against the real git binary. Spring Boot platform with orgs, teams, issues, PRs, and line-anchored review comments.

198JVM Tests

LiveGit Clone

FullVCS Engine

JavaSpring BootNext.jsJVM

GitHub

02CR

LIVE CLONE // 198 JVM TESTS // FROM-SCRATCH VCS

03BUILDING

Starling

REAL-TIME CRDT COLLABORATIVE EDITOR

ROLECreator

A real-time collaborative text editor built on CRDTs, for conflict-free concurrent editing across peers with local-first sync. Chosen to show distributed-systems depth alongside frontend and design work.

TypeScriptCRDTReal-Time

GitHub

03ST

REAL-TIME // CRDT // LOCAL-FIRST

04PROTOCOL

Tessera

POISONING-RESISTANT ML DISTRIBUTION

ROLECreator

Poisoning-resistant distribution for ML models and datasets. Publisher signs once; untrusted mirrors distribute; any consumer verifies the exact signed bytes with full provenance. Designed against a concrete threat model: malicious mirrors, in-transit tampering, replay, key compromise with rotation and revocation, and Sybil attacks. Each guarantee is backed by adversarial and property-based tests.

SecurityDistributionProvenanceAdversarial Tests

GitHub

04TR

SIGN ONCE // VERIFY ANYWHERE // PROVENANCE

04

Open Source

MERGED INTO THE SYSTEMS PEOPLE ACTUALLY RUN // 7 PROJECTS // 11 PRS

Not toy PRs. Real fixes merged into the AI-security and ML stack: NVIDIA garak, HuggingFace OpenEnv, vLLM llm-compressor, Semgrep rules, DEF CON AI Village, llm-guard, and dottxt-ai Outlines.

NVIDIAMERGED

Bedrock generator param suppression

NVIDIA/garak

Added a suppressed_params hook to BedrockGenerator so inference parameters rejected by Claude on Bedrock are stripped before the request goes out.

+ Interactive // NVIDIA / garak Merged PR #1842 Filed & Fixed

Bedrock generator param suppression

garak is the leading LLM vulnerability scanner. Its AWS Bedrock generator sent temperature and topP in the same call, a pairing Claude on Bedrock rejects. This PR adds a suppressed_params hook that strips rejected parameters before the request goes out.

Request to Bedrock

{
  "modelId": "anthropic.claude-...",
  "inferenceConfig": {
    "temperature": 0.7,
    "topP": 0.9
    # suppressed_params: [topP]
  }
}

Response

400 ValidationException

temperature and topP may not both be set for this model. Remove one and retry.

200 OK

{
  "output": {
    "role": "assistant",
    "content": "..."
  }
}

strips topP before the call

PR #1842View on GitHub ↗PYTHON

HUGGING FACEMERGED

SSRF loopback-bypass fix

huggingface/OpenEnv

Fixed a bug where localhost.evil.com bypassed proxy controls via substring matching. Replaced with urlparse + ipaddress.is_loopback, covering 127.0.0.0/8 and IPv6 loopback.

PR #742View on GitHub ↗PYTHON

vLLMMERGED

Type-Hint Modernization

vllm-project/llm-compressor

Two PRs modernizing type annotations across the llm-compressor codebase for Python 3.10+ compatibility.

PR #2777, #2779View on GitHub ↗PYTHON

SEMGREPMERGED

Shadow-AI Detection Rules

semgrep/semgrep-rules

Two static-analysis rules detecting unauthorized use of Ollama and Groq APIs, added to the Semgrep community registry.

PR #3858, #3859View on GitHub ↗YAML

AI VILLAGEMERGED

AI Village Site

aivillage/aiv_website

Community and sponsors pages contributed to the DEF CON AI Village website.

PR #53, #54View on GitHub ↗HTML / JS

dottxt-aiMERGED

RFC 4291 IPv6 Type

dottxt-ai/outlines

Strict RFC 4291 IPv6 address type in outlines.types, filed as issue #1866 and merged as PR #1867.

PR #1867View on GitHub ↗PYTHON

PROTECT AIMERGED

Contributions

protectai/llm-guard

Two pull requests merged into protectai/llm-guard.

PR #344, #345View on GitHub ↗PYTHON

+Full contribution index

#	Contribution	Org / Repo	Ref
Pull Requests11 merged · 2026
01	Bedrock generator param suppressionfiled & fixed	NVIDIA / garak	PR #1842
02	MaliciousURLs input scanner	Protect AI / llm-guard	PR #344, #345
03	SSRF loopback-bypass fix	Hugging Face / OpenEnv	PR #742
04	RFC 4291 IPv6 regex typefiled & fixed	dottxt-ai / outlines	PR #1867
05	Shadow-AI detection rules (Ollama, Groq)	Semgrep / semgrep-rules	PR #3858, #3859
06	Type-hint modernization	vLLM / llm-compressor	PR #2777, #2779
07	Community & sponsors pages	AI Village / aiv_website	PR #53, #54

05

Research

2 PAPERS // PRE-REGISTERED // SOLE AUTHOR // 2026

01

When Self-Consistency Backfires

2026

Can making a model think more actually make it worse?

Pre-registered sole-author study on majority-vote self-consistency in large language models. Tested on GPQA Diamond using Qwen2.5-7B-Instruct and Llama-3-8B-Instruct across sampling temperatures and vote counts. Confirmatory design with pre-specified hypotheses. Result: self-consistency reliably hurts performance on expert-level reasoning problems, counter to its established gains on standard benchmarks.

+ Interactive // Paper 01

Majority Vote Backfires

Self-consistency samples a model many times and takes the majority answer. It lifts accuracy on standard benchmarks. On expert problems it does the opposite. Drag the vote count and watch the two regimes split apart. Accuracy is computed live by Monte Carlo majority vote over 4,000 trials.

Votes per question k = 1

Standard Benchmark

errors are random noise · most likely answer is correct

52%

majority-vote
accuracy

GPQA Diamond · Expert

errors are correlated · model is drawn to a plausible trap

28%

majority-vote
accuracy

Accuracy vs vote count

Standard Expert

▸

Illustrative per-sample answer distributions chosen to reflect each regime (standard: correct is the model's plurality answer; expert: a wrong "trap" answer is). The accuracy figure is a live Monte Carlo simulation of majority voting, not a fixed number. Mechanism mirrors "When Self-Consistency Backfires" (GPQA Diamond, sole author).

Pre-registeredGPQA DiamondSole AuthorQwen2.5-7BLlama-3-8B

02

Reading the Judges

2026

Do AI judges score answers the way humans do?

An empirical study into the failure modes of LLM-as-judge evaluation pipelines. Examines how judge models form verdicts, where they systematically disagree with human raters, and what structural properties of a response reliably shift judge scores independent of ground-truth quality.

+ Interactive // Paper 02

Game the Judge

LLM-as-judge pipelines are supposed to score answers on correctness. They also reward surface features that have nothing to do with being right. Dress up the wrong answer and watch the judge rank it above the correct one.

QuestionWhat is the time complexity of binary search?

Judge's pick

Answer A ✓ Correct

Binary search is O(log n). It halves the search space each step, so n elements take about log₂(n) comparisons.

Judge score0/10

Judge's pick

Answer B ✗ Wrong

Judge score0/10

Manipulate Answer B (still wrong, just dressed up)

▸

The judge here is a transparent scorer that weights surface features the way real judge models do in my study. Scores update live from your toggles. Ground-truth correctness is fixed: A is right, B is wrong, no matter what the judge says. From "Reading the Judges" (sole author). Repo ↗

LLM EvaluationJudge ModelsSole Author

GitHub

I work on how intelligent systems fail.

06 Who I Am

You've watched me
break things. Here's what
built me.

There's a person behind all this. Here's who.

Read my story

Utkarsh Bahuguna, national-level athlete, four-instrument musician, and builder community founder — FIG. 06 Off the Clock

Let's work together.

Available for 2026. I build adversarial tooling and observability for the AI systems people run in production.

Email Me GitHub LinkedIn

Selected Work

Shadowbook

Cairn

Starling

Tessera

Open Source

Bedrock generator param suppression

Bedrock generator param suppression

SSRF loopback-bypass fix

Type-Hint Modernization

Shadow-AI Detection Rules

AI Village Site

RFC 4291 IPv6 Type

Contributions

Research

When Self-Consistency Backfires

Majority Vote Backfires

Reading the Judges

Game the Judge

You've watched mebreak things. Here's whatbuilt me.

Let's work together.

You've watched me
break things. Here's what
built me.