Sr. Azure Data Engineer with Databricks

Job Description Data Engineer (Onshore) **Location:** Philadelphia, PA (Hybrid: 3 Days Per Week Onsite) **Experience Required:** 7+ Years Position Overview We are seeking an experienced Data Engineer to support enterprise data modernization and analytics initiatives. This individual will be responsible for designing, developing, and optimizing scalable cloud-based data solutions leveraging Azure, Databricks, and modern Data Lake architectures. The ideal candidate will have strong hands-on experience building enterprise-grade data pipelines, implementing data transformation frameworks, and supporting large-scale analytics platforms. This role requires close collaboration with business stakeholders, data architects, analysts, and product teams to enable data-driven decision-making across the organization. This is a hybrid role requiring onsite presence in the Philadelphia office at least three days per week. Key Responsibilities Data Engineering & Development Design, develop, and maintain scalable ETL/ELT pipelines supporting enterprise data and analytics initiatives. Build data ingestion, transformation, and orchestration frameworks using Azure-native technologies. Develop and optimize data processing solutions using Databricks and PySpark. Implement and maintain Delta Lake and Lakehouse architectures. Ensure data quality, accuracy, lineage, and reliability across all data assets. Perform root cause analysis and troubleshoot data-related issues. Azure Cloud Data Platform **Develop solutions leveraging:** Azure Databricks Azure Data Lake Storage (ADLS Gen2) Azure Data Factory (ADF) Azure Synapse Analytics Azure SQL Database Support data migration, modernization, and cloud transformation efforts. Optimize performance, scalability, and cost efficiency of cloud-based data platforms. Data Modeling & Analytics Enablement Build and maintain logical and physical data models. Create curated data layers to support business intelligence, reporting, and advanced analytics workloads. Partner with business users and data consumers to understand requirements and translate them into scalable technical solutions. DevOps & Operational Excellence Implement CI/CD processes for data engineering solutions. Support automated deployment and release management processes. Develop monitoring, ing, and observability capabilities for data pipelines. Follow enterprise data governance and security standards. Agile Delivery Participate in Agile/Scrum ceremonies including sprint planning, daily stand-ups, retrospectives, and backlog refinement. Collaborate effectively with Product Owners, Architects, QA teams, and business stakeholders. Contribute to continuous improvement initiatives and engineering best practices. Required Qualifications Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field. 7+ years of experience in Data Engineering. 5+ years of hands-on experience with Azure data technologies. 2+ years of experience in Healthcare, Managed Care, Health Plans, or Payer organizations. **Strong expertise in:** Azure Databricks Azure Data Lake Storage (ADLS) Azure Data Factory (ADF) PySpark Python SQL Extensive experience building enterprise-scale data pipelines and data integration solutions. Strong understanding of Data Warehousing, Data Lake, and Lakehouse concepts Experience working with large volumes of structured and unstructured data. Experience with Git, CI/CD, and DevOps practices. Strong analytical, troubleshooting, and problem-solving skills. Preferred Qualifications **Experience working with:** Claims Data Membership Data Provider Data Care Management Data Experience with Power BI or other reporting tools. Familiarity with FHIR, HL7, or healthcare interoperability standards. Experience with Azure DevOps and Infrastructure as Code (IaC). Exposure to AI/ML data pipelines and advanced analytics environments. Required Technical Skills Must Have Azure Databricks Azure Data Lake (ADLS Gen2) Azure Data Factory (ADF) PySpark Python SQL Delta Lake Data Modeling ETL/ELT Development Azure Cloud Platform Git CI/CD Agile/Scrum Skills Azure Databricks Azure Data Lake Storage (ADLS) Azure Data Factory (ADF) PySpark Python SQL