You'll own the search and information retrieval systems at the core of Firecrawl , the infrastructure that determines how we find, rank, index, and serve web content at scale. Retrieval quality is Firecrawl's deepest moat. As AI agents increasingly depend on multi-step search and enrichment, the gap between good retrieval and great retrieval compounds. You're the person who closes that gap , and widens it against every competitor. This is a full-stack search role where you'll build and operate everything from ingestion pipelines to serving layers. If you've built search indexes at massive scale and care deeply about ranking quality, freshness, and retrieval speed, this is the role.
You'll design, build, and maintain the indexing infrastructure that powers Firecrawl's core product. You'll handle billions of documents and care about every millisecond of latency and every byte of storage. You'll own the full stack from ingestion to serving. You don't just build one piece , you own the entire pipeline. Ingestion, processing, indexing, ranking, query understanding, and serving. When something breaks at 3am, you know where to look because you built it.
You'll make sure the right content surfaces for the right queries. You'll build and iterate on ranking models, relevance scoring, and query parsing systems that directly impact product quality. You'll tackle freshness, dedup, and incremental indexing. The web changes constantly. You'll build systems that keep our index fresh without re-crawling everything, deduplicate content intelligently, and handle incremental updates at scale without rebuilding from scratch.
You'll run experiments and ship results to production. You design experiments, measure results rigorously, and ship winners to production fast. You don't need someone to tell you what to try next , you have a backlog of ideas and the judgment to prioritize them.
You'll collaborate closely with the team. Work directly with the RL-focused Research Engineer and the engineering team to connect search/IR improvements with model training and the broader product roadmap.
XML job scraping automation by YubHub
