Portfolio

Selected engineering proof: what I built, the systems underneath, and the tradeoffs that mattered. Raintree product pages live at raintree.technology.

Agent Safety & Evals

Context & Developer Tools

Data & Intelligence Systems

Product SaaS

Systems & Low-Level

Music & Creative Systems

Web context / personal case study

docpull

source-linked web capture

What I built

Browser-free crawler that turns web pages into source-linked Markdown for coding agents.

Stack

Python 3.10+ / aiohttp / BeautifulSoup / Pydantic / Rich / SQLite FTS5 / MCP

Runtime

CLI, Python SDK, and stdio MCP server call the same fetch/discover/convert/pipeline core.
Async HTTP crawler handles static HTML, SSR docs, OpenAPI, Docusaurus, Sphinx, MkDocs, Mintlify, GitBook, Redoc, Scalar, and embedded data.
Pipeline steps fetch, validate, extract metadata, convert, dedupe, chunk, and save to the selected sink.

Data model

Markdown, NDJSON chunks, SQLite, OKF bundles, archives, mirrors, and `corpus.manifest.json` share stable IDs.
Documents carry source URL, source type, title/headings, hashes, chunk IDs, token counts, and output paths.
Profiles tune behavior for RAG, LLM chunks, OKF bundles, mirrors, quick crawls, and SEC filing evidence packs.

Techniques

No-browser default keeps installs small and fetch behavior auditable; JS-only pages skip with explicit reasons.
SSRF defenses block private IPs, CGNAT, IPv4-mapped IPv6, DNS-rebinding paths, and unsafe redirects.
Magic-byte and download-policy checks reject binary formats before parsing as text.
Optional tiktoken chunking, trafilatura extraction, parallel context packs, and Raindrop benchmark tracing plug in by profile.

Verification

URL validation blocks private IPs, CGNAT, IPv4-mapped IPv6, and DNS-rebinding paths before fetch or write.
Magic-byte checks reject common binary formats before parsing content as text.
Robots and HTTPS defaults make the crawler conservative unless the caller explicitly chooses otherwise.
Source-linked Markdown preserves enough provenance for agents and humans to trace each extracted section back to the originating page.

Portfolio

Selected engineering proof: what I built, the systems underneath, and the tradeoffs that mattered. Raintree product pages live at raintree.technology.

Agent Safety & Evals

Context & Developer Tools

Data & Intelligence Systems

Product SaaS

Systems & Low-Level

Music & Creative Systems

Web context / personal case study

docpull

source-linked web capture

What I built

Browser-free crawler that turns web pages into source-linked Markdown for coding agents.

Stack

Python 3.10+ / aiohttp / BeautifulSoup / Pydantic / Rich / SQLite FTS5 / MCP

Runtime

CLI, Python SDK, and stdio MCP server call the same fetch/discover/convert/pipeline core.
Async HTTP crawler handles static HTML, SSR docs, OpenAPI, Docusaurus, Sphinx, MkDocs, Mintlify, GitBook, Redoc, Scalar, and embedded data.
Pipeline steps fetch, validate, extract metadata, convert, dedupe, chunk, and save to the selected sink.

Data model

Markdown, NDJSON chunks, SQLite, OKF bundles, archives, mirrors, and `corpus.manifest.json` share stable IDs.
Documents carry source URL, source type, title/headings, hashes, chunk IDs, token counts, and output paths.
Profiles tune behavior for RAG, LLM chunks, OKF bundles, mirrors, quick crawls, and SEC filing evidence packs.

Techniques

No-browser default keeps installs small and fetch behavior auditable; JS-only pages skip with explicit reasons.
SSRF defenses block private IPs, CGNAT, IPv4-mapped IPv6, DNS-rebinding paths, and unsafe redirects.
Magic-byte and download-policy checks reject binary formats before parsing as text.
Optional tiktoken chunking, trafilatura extraction, parallel context packs, and Raindrop benchmark tracing plug in by profile.

Verification

URL validation blocks private IPs, CGNAT, IPv4-mapped IPv6, and DNS-rebinding paths before fetch or write.
Magic-byte checks reject common binary formats before parsing content as text.
Robots and HTTPS defaults make the crawler conservative unless the caller explicitly chooses otherwise.
Source-linked Markdown preserves enough provenance for agents and humans to trace each extracted section back to the originating page.

Selected projects

What I built

Stack

Selected projects

What I built

Stack