Member of Technical Staff (Web Access Engineer, Crawler)

Perplexity · London, Belgrade, Ireland · Posted Nov 13, 2026


Transparency

Fresh

Online for 7 days and never reposted. Early applicants get the most attention, so now is the time to apply.

First seen
Nov 13, 2026
Offered for
7 days

Overview

Category
Software engineering
Employment type
Permanent
Workplace type
Hybrid
Country
United Kingdom, Serbia, Ireland
Seniority
Staff
Schedule
Full time
Salary
Not listed

Key skills

Fetching and Rendering Engineering
Own end‑to‑end improvements to crawler fetching and page rendering pipelines.
Network Protocol Expertise
Diagnose and fix failures across DNS, TLS, HTTP/1.1, HTTP/2, HTTP/3, caching and redirects.
Client‑Side Rendering Debugging
Use Chrome DevTools, CDP, HAR files and packet captures to troubleshoot JavaScript rendering.
Programming in Go/Python/Rust/C++
Write production code for crawling infrastructure using at least one of these languages.
Reliability Metrics and Regression Testing
Define and track metrics for fetching reliability and build regression tests for failure cases.

Key phrases

  • fetching and rendering improvements
  • diagnosing network errors
  • page-loading issues
  • DNS, TLS, HTTP, caching, redirects
  • client-side rendering
  • rate limits
  • regression tests
  • metrics for fetching reliability
  • HTTP/3
  • Chrome DevTools
  • CDP
  • HAR files
  • packet captures
  • headless browsers
  • distributed content-acquisition systems

From the posting

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.

Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.

Responsibilities

  • Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.

  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.

  • Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.

  • Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.

  • Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.

  • Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.

Qualifications

  • 5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.

  • Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.

  • Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.

  • Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.

  • Comfort working independently when the path forward is not fully defined.

Nice to have

  • Experience developing CDN, caching, or web performance systems.

  • Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.

  • Contributions to a browser engine or JavaScript runtime.

More jobs at Perplexity

Member of Technical Staff (Index Engineer, Search Crawler)

Perplexity · 2 days ago | London, Belgrade, Ireland | Remote · Staff · Full time

Create CV

Enterprise Account Executive, Financial Services

Perplexity · 2 days ago | San Francisco | $240K–$400K / year | On-site · Senior · Full time

Create CV

Enterprise Account Executive, Legal

Perplexity · 2 days ago | San Francisco, Austin, New York City | $240K–$400K / year | Hybrid · Senior · Full time

Create CV

Enterprise Account Manager

Perplexity · 2 days ago | San Francisco, Austin, New York City | $240K–$400K / year | Hybrid · Senior · Full time

Create CV

Strategic Sourcing Lead

Perplexity · 3 days ago | San Francisco | 190000–240000 USD / 1 YEAR; EquityPercentage | Hybrid · Lead · Full time

Create CV
See all 145 jobs at Perplexity