Position: Senior Data Engineer – Databricks Specialist
Overview
Join one of the world’s leading integrated energy and commodity trading firms in a pivotal role that drives data innovation. We are looking for a seasoned Data Engineer with deep expertise in Databricks and modern data architecture. This role is ideal for someone passionate about transforming legacy systems and delivering scalable, high-impact data solutions.
Key Responsibilities
- Design and implement scalable, secure, and high-performance data solutions using Databricks and other modern data platforms.
- Lead the migration of data from traditional RDBMS systems to Databricks, ensuring minimal disruption and maximum efficiency.
- Build and optimize ETL pipelines to support data ingestion, transformation, and delivery across the organization.
- Collaborate with stakeholders to gather requirements, analyze business processes, and deliver data solutions that drive measurable value.
- Monitor and enhance the performance of data systems to ensure reliability, scalability, and cost-effectiveness.
- Define and enforce best practices for data engineering, including data quality, governance, and security.
- Partner with cross-functional teams—data scientists, analysts, DevOps, and product managers—to align data solutions with business goals.
- Provide technical leadership and mentorship to junior engineers.
- Stay current with emerging technologies and recommend tools to enhance data infrastructure and workflows.
- Ensure compliance with organizational policies, industry standards, and regulatory requirements.
Qualifications
- Bachelor’s or Master’s degree in Computer Science or a related field, or equivalent experience.
- Proven experience with Databricks, including ETL development and data migration.
- Databricks certification(s) strongly preferred.
- Proficiency in cloud platforms such as AWS, Azure, or GCP.
- Strong understanding of big data technologies, data warehousing, and data modeling.
- Expertise in SQL, Python, and scripting languages.
- Experience with RDBMS systems and transitioning to cloud-native architectures.
- Familiarity with containerization (e.g., Docker) and orchestration tools (e.g., Kubernetes).
- Solid grasp of data governance, data quality, and security principles.
- Excellent problem-solving, communication, and collaboration skills.
