Position Overview:
We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate will have a deep understanding of Azure Data Lake and Databricks, along with a strong background in data engineering principles. You will play a crucial role in designing, building, and managing our data infrastructure to support data transformation, storage, and analysis needs.
Key Responsibilities:
- Design and implement scalable and reliable data pipelines using Azure Data Lake and Databricks.
- Collaborate with cross-functional teams to understand data needs and deliver comprehensive data solutions.
- Ensure data quality and consistency across all data stores and processes.
- Optimize data processing performance and costs in Azure.
- Develop and maintain data models and architectures for data ingestion, processing, and delivery.
- Implement data security and compliance measures.
- Provide technical leadership and mentorship to junior data engineers.
- Stay abreast of industry trends and advancements in data engineering and cloud technologies.
Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering, with a focus on Azure Data Lake and Databricks.
- Strong proficiency in SQL and experience with programming languages such as Python or Scala.
- Experience designing and implementing large-scale data pipelines and architectures.
- Knowledge of data modeling, data warehousing, and building ETL pipelines.
- Familiarity with data governance, security, and compliance best practices.
- Excellent problem-solving, analytical, and communication skills.
- Ability to work independently and as part of a team in a fast-paced environment.
