Big Data Engineer
Company Description
Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money.
We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments.
We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more at experianplc.com.
Job Description
Requirements
- BS degree in computer science, computer engineering or equivalent
- 5–6 years of experience delivering enterprise software solutions
- Proficient in Spark, Scala, Python, AWS Cloud technologies
- 3+ years of experience across multiple Hadoop / Spark technologies such as Hadoop, MapReduce, HDFS, HBase, Hive, Flume, Sqoop, Kafka, Scala
- Flair for data, schema, data model, how to bring efficiency in big data related life cycle
- Technical requirements and can translate them into technical implementations
- Experience with Agile Development methodologies
- Experience with data ingestion and transformation
- Solid understanding of secure application development methodologies
- Experience with Airflow and Python
- Understanding of automated QA needs related to Big data
- Object-oriented design and experience approaching a problem from different angles, analyzing the pros and cons of different solutions skills
- Experience communicating updates and resolutions to customers and other partners
Responsibilities
- Use your software engineering skills, including Java, Spark, Python, Scala, to Analyze different, complex systems and collaboratively design new products and services
- Integrate new data sources and tools
- Implement scalable and reliable distributed data replication strategies
- Mentor and provide direction in architecture and design to onsite/offshore developers
- Collaborate with other teams to design and deploy data tools that support both operations and product use cases
- Perform analysis of large data sets using components from the Hadoop ecosystem
- Manage product features from the development, testing through to production deployment
- Evaluate big data technologies and prototype solutions to improve our data processing architecture
- As a Big Data Engineer, you will be reporting to the Director, Software Development
Qualifications
- Bachelor's degree required
- Proficiency in CI/CD tools (Jenkins, GitLab, etc.)
Benefits:
- Health, life and disability insurance,
- Paid parental and family care leave
- 20 days of vacation accrued annually, five sick days, and two volunteer days (plus twelve paid holidays)
- This role can be 100% remote long-term, or you can work out of one of our offices #Li-Remote
- Employee stock purchase plan
Additional Information
Our uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplaces™ 2024 (Fortune Top 25), Great Place To Work™ in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site to understand why.
Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is an important part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, colour, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.
Average salary estimate
If an employer mentions a salary or salary range on their job, we display it as an "Employer Estimate". If a job has no salary data, Rise displays an estimate if available.
At Experian, we’re seeking a passionate Big Data Engineer to join our dynamic team located at One Beacon Street, Boston. As a key player in our software development squad, you’ll leverage your proficiency in Spark, Scala, Python, and AWS to dissect complex systems and innovate new products and services. You’ll be responsible for integrating data sources and devising scalable, reliable data replication strategies that enhance our architecture. With 5-6 years of experience in enterprise software solutions, you will collaborate extensively with cross-functional teams to develop data tools that empower both product use and operational capabilities. The ideal candidate has a flair for data modeling, is well-versed in agile methodologies, and has experience in data ingestion and transformation practices. You’ll also mentor both onsite and offshore developers, providing them with guidance in architecture and design. We encourage our engineers to experiment, prototype, and evaluate big data technologies, allowing us to continually improve our data processing architecture. Plus, with opportunities to work remotely or in-office, we promote a culture that values work-life balance and diversity. If you’re excited to make a significant impact with data at a recognized workplace like Experian, we’d love to hear from you!
Join Experian as a Controls Implementation Manager to lead vital risk and controls transformation initiatives within a global data and technology company.
Join Experian as a Security Engineer to enhance their cybersecurity measures and incident response efforts in a remote role.
As the Director of Engineering at Quantum Metric, you'll lead teams in building innovative SDKs for native apps while fostering a diverse and inclusive remote culture.
Join Kimley-Horn as a Civil Engineering Intern and contribute to innovative water/wastewater projects in West Palm Beach.
Lead a professional weld team at Component Repair Technologies to drive quality and safety in aerospace component repairs.
Take the lead in maintaining and optimizing production processes at Zoetis as a Team Leader in Atlanta.
Horton, Inc. is looking for a Truck Application Engineer to enhance product offerings and support customer needs in a collaborative engineering environment.
Join DeepSky as a Development Engineer and lead innovative carbon capture projects while collaborating with experts in the field.
The Boeing Company is looking for an experienced Avionics Manager to oversee the Cabin Modifications Maintenance team in Long Beach, CA.
We pride ourselves on being certified as a Great Place To Work and firmly believe that creating a positive company culture is less about ping pong tables and more about transparency, connection, and "work with purpose." The unique perspective of e...
741 jobs
Subscribe to Rise newsletter
🔮
Hi, I'm Risa! Your AI Career Copilot
you, just ask me below!
Sign up for our weekly
newsletter of fresh jobs
