Big Data Engineer

As a Big Data Engineer at GiG, your main responsibilities include working with real-time data sources, supporting the event-driven microservices architecture, and collaborating with agile teams to maintain and improve our data platform. You'll be tasked with ensuring data integrity, enhancing data ingestion capabilities, and enabling other stakeholders such as Business Intelligence and Data Science teams to effectively use data.

To be considered for the Big Data Engineer position at GiG, you should hold a Bachelor’s Degree in Computer Science or a related field. Additionally, your expertise should include object-oriented programming (Java), experience with ETL processes, knowledge of Apache Spark or Apache NiFi, and familiarity with both Linux and Windows environments.

As a Big Data Engineer at GiG, you will engage with a wide array of technologies including Java for object-oriented programming, Apache Spark or Apache NiFi for data processing, and low-latency databases like ClickHouse. You'll also utilize Docker for containerization and engage in agile methodologies for team collaboration.

At GiG, you will have the flexibility to work in a 100% remote or hybrid setup depending on your preferences. The company thrives on collaboration and creativity, and you’ll find yourself in a nurturing environment designed to support your growth and productivity.

GiG is committed to the professional growth of its employees. As a Big Data Engineer, you will have access to great career development opportunities such as mentorship programs, workshops, and access to leading industry talent, enabling you to enhance your skillsets and advance your career.

When answering this question, focus on specific projects where you utilized Apache Spark or Apache NiFi, emphasizing challenges you faced and how you overcame them. Discuss the data processing pipelines you created and the impact they had on the business outcomes.

Discuss various validation techniques you have applied in your ETL processes, including data profiling, automated tests, and error-handling mechanisms. Show your understanding of the importance of data integrity in driving business decisions.

Emphasize your experience with agile methodologies, discussing how you adapt to changes, communicate effectively, and maintain a focus on team objectives. Highlight examples where you contributed to sprint planning and retrospectives.

Share specific projects where you applied object-oriented principles using Java. Talk about design patterns you have implemented and how they improved code maintainability.

Explain your understanding of Continuous Integration and Continuous Deployment practices, detailing the tools you've used, how you've streamlined the development workflow, and any challenges you navigated regarding change management.

Discuss the methodologies you favor for data modeling, such as entity-relationship modeling or dimensional modeling. Include examples from your past experience and the benefits each method brought to your projects.

Highlight your experience with event-driven architectures and any tools or frameworks you have used to handle real-time data streams. Discuss how you have ensured low latency and high throughput.

Narrate a specific challenge where you successfully solved a complex data issue. Detail the steps you took, how you collaborated with other teams, and what the outcome was, underscoring your analytical mindset.

Discuss your approach to mentoring, focusing on the importance of sharing knowledge, providing constructive feedback, and creating an environment where junior engineers feel comfortable asking questions and taking risks.

Share your strategies for staying current with industry trends, such as attending conferences, participating in online courses, or following relevant publications and thought leaders in the big data technology space.