Senior Data Engineer - AWS

As a Senior Data Engineer at 3Pillar, you'll develop, maintain, and enhance ETL pipelines, craft performant and secure client code, and ensure that your projects meet the highest quality standards. You'll work on gathering requirements, collaborating with team members, and providing accurate status tracking and documentation.

To excel as a Senior Data Engineer at 3Pillar, you'll need at least 5 years of experience in data engineering or a related field, with a strong proficiency in Python and a deep understanding of AWS services such as Kinesis, S3, and Redshift. Experience with SQL and NoSQL databases, as well as data modeling and security best practices, is also essential.

At 3Pillar, we prioritize your professional development. Our services model allows you to work across various projects and industries, providing diverse experiences that contribute to your career growth. We also have a global workforce that facilitates learning from top talent around the world.

3Pillar embraces a remote-first approach, allowing you flexibility in your work environment. Whether you prefer to work from home, the office, or a mix of both, you’ll find that our culture supports a balanced work-life integration while collaborating with a global team.

As a Senior Data Engineer at 3Pillar, you can look forward to a culture that emphasizes well-being through fitness offerings, mental health plans, and generous time off. We are committed to your personal and professional support, ensuring that you operate at your best.

During the interview, articulate specific AWS services you've used, such as S3 for data storage or Lambda for serverless processing. Explain the context in which you've used these services and how they contributed to the success of your projects.

Highlight your approach to data validation, monitoring, and error handling within ETL pipelines. Discuss tools or frameworks you've employed to maintain data integrity and accuracy, as well as any specific strategies you find effective.

Share practical examples of Python scripts or applications you've developed for data processing. Discuss libraries you have used, such as Pandas for data manipulation or SQLAlchemy for database connectivity, to illustrate your programming skills.

Choose a specific project and explain the challenges faced, such as handling large data volumes or integrating disparate data sources. Discuss the steps you took to troubleshoot and resolve these issues, showcasing your problem-solving abilities.

Explain your communication style and how you work with designers, product managers, and other engineers. Give examples of successful collaborations that resulted in improved understanding of user needs and subsequent project success.

Discuss your experience designing schemas tailored to specific application needs. Emphasize your understanding of normalization versus denormalization and considerations for performance and scalability in your designs.

Mention any courses, certifications, or communities you engage with to keep your skills sharp. Discuss any recent technologies or methodologies you've adopted and how they’ve impacted your work.

Outline your experience with implementing data security measures, such as encryption or access controls. Describe how you’ve incorporated data privacy best practices into your data engineering workflow.

Discuss specific SQL databases you've worked with, such as MySQL or PostgreSQL, as well as any NoSQL databases like MongoDB or DynamoDB. Share examples of projects where you utilized these databases effectively.

Express your enthusiasm for the company’s mission and values. Highlight how your skills not only align with the role but also how you're excited about contributing to projects that have a real-world impact across industries.