Lucas Group is looking for a, Senior Data Engineer, responsible for the design, development, testing, and deployment of solutions using the latest tools and techniques including Python, Scala, Spark, Postgres and Redshift for a web-based solution company in Atlanta.
The Senior Data Engineer will be involved in all aspects of the development lifecycle and will have a key role in driving the direction of the data platform's architecture as it continues to evolve.
Responsibilities:
- Designing, implementing, and maintaining ETL solutions using the latest in big data technologies
- Collaborating with product teams to turn business requirements into technical specifications
- Reviewing other team member's code for correctness and quality
- Writing automated test scripts that power a continuous delivery pipeline
- Refactoring and improving the existing code base for simplicity and clarity
- Defining and managing SLAs for new data solutions as they are brought online
Qualifications:
- 5+ years of professional experience in a development role
- 2+ years of experience in Scala, Java or another JVM language
- A strong understanding of SQL, relational databases, and data modeling
- Ability to analyze data for gaps, inconsistencies, and performance optimizations
- Knowledge of TDD, automated testing principles, and testing best practices
- Solid communication skills and the ability to manage multiple tasks efficiently
- BSCS/BSEE or comparable equivalent
Desired Experience:
- Hands-on experience with custom ETL processes in an enterprise setting
- Working knowledge of modern data management solutions and their trade-offs
- An understanding of functional programming techniques
- Experience using Amazon Web Services for cloud-native and serverless development
- Hands-on experience with Python/Pandas, Scala/Spark, and Elasticsearch
