Big Data Platform Engineer

Responsibilities:

• Design, develop and maintain reliable and scalable big data platform.
• Design and develop micro-service based distributed system that involves lots of components.
• Design and develop high throughput data processing pipeline.
• Work with data scientists and UI team to deliver high quality data for machine learning and security analysis.
• Undertake performance bench-marking and system capacity scheduling.
• Work with support team to troubleshoot system failure accidents.

Qualifications:

• 3+ years’ experience in Java programming with solid design pattern knowledge.
• 3+ years’ experience in Python programming.
• Solid knowledge in following areas: network, operating system, distributed system, database.
• Hands on experience with multi-threading, multi-processing and distributed system programming.
• Hands on experience with micro-service architecture.
• Capability of solving challenging problems that requires complex algorithms.
• Self-motivated with great passion in tackling technical challenges.
• Good communication skills and being a team player.
• Bachelor and above degree in computer science or computer engineering.
• Hands on experience with Spark is a plus.
• Hands on experience with Elastic search is a plus.
• Trackable record of delivering big data platform or distributed system based product is a plus.
• Solid cyber/system security background is a plus.