We are looking for a skilled and motivated Web Crawling Engineer to join our dynamic engineering team. The ideal candidate should have a solid background in distributed web crawling, scraping and data extraction, with experience using advanced tools and technologies to collect and process large-scale data from diverse web sources at large scale.
What you'll do
As a Web crawling engineer, you will be responsible for:
- Developing and maintaining web crawlers using Go to extract data from target websites.
- Utilize headless browsing techniques, such as Chrome DevTools, to automate and optimize data collection processes.
What you need
- Proficiency in Go (Golang)/Rust/Zig for building scalable and efficient web crawlers.
- Deep understanding of TCP, UDP, TLS and HTTP/1.1,2,3 protocols and web communication.
