Data Engineer
General Job Summary
The Data Engineer will play a pivotal role in building a modern, scalable data platform to support the company's transformation into a cloud-first, API-first enterprise. This role is responsible for developing secure and reliable data ingestion pipelines, curating data for analytics, and enabling real-time data integration. As the enterprise moves from legacy integrations to event-driven and API-based models, the Data Engineer will actively support modernization of the data ecosystem across the bronze, silver, and gold layers.
Summary of Essential Job Functions
Core Development Stack
Education, Training and Work Experience
Knowledge, Skills, and Abilities
The Data Engineer will play a pivotal role in building a modern, scalable data platform to support the company's transformation into a cloud-first, API-first enterprise. This role is responsible for developing secure and reliable data ingestion pipelines, curating data for analytics, and enabling real-time data integration. As the enterprise moves from legacy integrations to event-driven and API-based models, the Data Engineer will actively support modernization of the data ecosystem across the bronze, silver, and gold layers.
Summary of Essential Job Functions
- Design and implement scalable data pipelines in Azure using Databricks, Spark, Delta Lake, dbt, Dagster , Airflow, and Parquet.
- Develop data ingestion workflows from multiple sources (e.g. SFTP, vendor APIs) into Azure Data Lake.
- Build and maintain Bronze/Silver/Gold layer transformations within a medallion architecture.
- Implement data quality, deduplication, and validation logic across ingestion layers.
- Develop parameterized and reusable notebooks for batch and streaming jobs.
- Create robust merge/update logic into Delta Lake using optimized key-based partitioning.
- Collaborate with business and application teams to identify data integration requirements.
- Support integration with downstream APIs, Power BI models, and SQL-based reporting.
- Implement monitoring, logging, and lineage for data pipelines using tools like Unity Catalog and Azure Monitor.
- Participate in code reviews, technical design discussions, and backlog grooming
- SQL Server Development & Optimization – Develop, optimize, and maintain SQL Server stored procedures, functions, views, and indexing strategies to ensure high-performance data processing.
- ETL & Data Integration – Design and manage ETL/ELT processes using SQL Server Integration Services (SSIS) and SQL batch jobs to extract, transform, and load data efficiently.
- Automate deployment of Azure App Services, Azure SQL Server, Azure Storage Accounts, Azure Functions, Logic Apps, Databricks resources, and other supporting services
- Monitor and troubleshoot pipeline failures, environmental drift, and deployment errors
- Support cloud-native observability stack (e.g., Azure Monitor, Application Insights, Log Analytics)
- Contribute to containerization and orchestration
- Integrate with enterprise API gateways and platform services
- Extend support for AI workloads, data lake deployments, and ML pipelines
Core Development Stack
- Languages & Frameworks: Python, C#, .NET Core, SQL, T-SQL
- Database & ETL: SQL Server, SSIS, SSRS, Power BI
- API Development: ASP.NET Core Web API, RESTful APIs
- Cloud & Data Services (Future Roadmap): Azure Data Factory, Azure Functions, Azure Databricks, Azure SQL Database, Azure Data Lake, Azure Storage, and other supporting services.
- Streaming & Big Data (Future Roadmap): Delta Lake, Databricks, Kafka (preferred but not required)
- Data Governance & Security: Data integrity, performance optimization, access control, compliance, best practices
- Project & Collaboration Tools: Jira, Confluence, Visio, Smartsheet
Education, Training and Work Experience
- Bachelor’s degree in computer science, Engineering, or a related field (or equivalent experience)
- 5+ years of hands-on experience building data pipelines and distributed data systems
- Strong experience in Databricks, Delta Lake, and Azure-based big data tools
- Experience with financial or regulated data environments preferred
- Familiarity with version control (Git), CI/CD practices, and agile delivery
- Experience in the mortgage servicing or mortgage lending industry is a plus.
Knowledge, Skills, and Abilities
- SQL Server & Database Management – Expertise in T-SQL, stored procedures, indexing, performance tuning, and query optimization.
- Strong understanding of data ingestion patterns and partitioning strategies
- Proficient in PySpark/SQL with a focus on performance tuning
- Solid grasp of modern data lake architecture and structured streaming
- Excellent problem-solving and debugging skills
- Ability to collaborate across business, product, and technology teams
- Strong communication skills and documentation discipline.
- Good understanding of Azure DevOps practices and cloud automation patterns
- Good analytical and troubleshooting skills for CI/CD and deployment workflows
Job Type
Full-Time Regular
Full-Time Regular
Location
Remote
Salary Range$99,000.00 — $165,000.00Remote