Software Engineer

Job Summary:
We are seeking a talented and motivated Data Engineer with experience in Databricks, Azure, and big data technologies. The ideal candidate should have strong expertise in building scalable ETL pipelines using Python and Scala and be familiar with Azure's data services ecosystem. You will play a key role in managing, transforming, and analyzing large datasets to drive business insights.

Key Responsibilities:

• Design, build, and maintain ETL pipelines using Databricks in Azure.
• Work with large-scale data in distributed systems to ensure data quality, integrity, and reliability.
• Develop and implement scalable data models and architectures.
• Optimize existing data workflows for performance and scalability.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements.
• Monitor, troubleshoot, and improve the performance of data pipelines.
• Integrate data from various sources and ensure proper data governance and security.
• Utilize Azure services such as Data Lake, Azure SQL Database, Azure Data Factory, and more for end-to-end data solutions.

Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• 3-5 years of experience in data engineering roles, preferably with Databricks and Azure.
• Proficiency in Python and Scala for building ETL processes.
• Strong knowledge of big data technologies like Hadoop, Spark, and distributed systems.
• Hands-on experience with Azure data services (e.g., Data Lake, Data Factory, Databricks).
• Experience with SQL and NoSQL databases.