Software Engineer, Web Crawling
Skills
- Large-scale web crawling experience
- Distributed systems knowledge
- System-level coding in Python, Go, or C++
- Integration with AI/NLP pipelines
- Understanding of web technologies and protocols
Responsibilities
- Design, develop, and deploy a real-time, adaptive web crawling infrastructure
- Architect dynamic crawling strategies based on real-time demand signals
- Implement scalable crawling systems for high URL volumes
- Collaborate closely with AI and search teams
- Own the full lifecycle of the crawler infrastructure
- Optimize crawler performance and reliability
- Mentor junior engineers
Education
- Bachelor's degree or higher in Computer Science, Engineering, or a related field
Benefits
- Health, dental, and vision care (100% coverage for employee)
- Top-tier 401(K) plan with company matching
- Paid time off and holidays
- FSA, HSA, and commuter benefits
- Team activity budget
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
Join Mistral AI as a Software Engineer on their Research Platform team with a focus on driving innovation in AI technology.
Seeking an experienced Senior Python Developer to lead sophisticated software projects with Truelogic, delivering innovative solutions for top American companies.
Grab is looking for a passionate Senior Software Engineer to contribute to their backend applications and enhance features for millions of users in Southeast Asia.
Contribute to cutting-edge diagnostic solutions as a Delphi Software Developer at Eurofins Scientific in Lisbon.
Be part of a high-impact team at Coinbase as a Senior Staff Software Engineer, driving the technical vision for core trading systems.
ATOS is looking for a talented Application Developer with Java expertise to enhance their software solutions remotely.
Join Ripple as a Senior Software Engineer and help shape the future of the global financial system with innovative crypto solutions.
Join Bilue as a Lead Backend Engineer and leverage your technical expertise in Kotlin and Java to mentor teams and deliver cutting-edge digital solutions.
Join Microsoft's PAC team as a Software Engineer II and innovate AI-driven solutions for internal systems architecture.
Step into the role of an iOS Developer at Accenture, where creativity and technical expertise meet to drive client success in a hybrid working model.
Join our team as an AI Developer Engineer to craft innovative AI solutions leveraging the latest in Generative AI and cloud technologies.
Join Palo Alto Networks as a Senior Engineer Software to drive cybersecurity innovations while working alongside industry leaders.
As a Senior Solution Architect at Rossum, you will passionately bridge technical solutions and customer needs while utilizing advanced AI technologies for paperless solutions.
NewsBreak connects and empowers local users, local content creators, and local businesses at scale, with the goal of helping people everywhere live safer, more vibrant, more truly connected lives. By forging close partnerships with thousands of lo...
4 jobsSubscribe to Rise newsletter
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
