DataOps/Cloud Data Engineer
Overview
The Health Services Cluster, Ministry of Public and Business Service Delivery and Procurement, is seeking a Senior DataOps/Cloud Data Engineer to join their team in Toronto. The successful candidate will be responsible for designing, developing, and maintaining robust data pipelines, data models, and cloud-based data infrastructure. This is a hands-on technical role requiring deep expertise in data engineering, cloud platforms, and modern lakehouse architectures.
Key Responsibilities
- Design, develop, test, implement, and troubleshoot data pipelines using tools such as Databricks, Azure Data Factory, and AWS Glue
- Build and maintain complex data transformation procedures to support business and analytical needs
- Develop data models for data warehouses and lakehouses including relational and dimensional modeling
- Perform data manipulation and analysis using pandas and PySpark
- Work with diverse data formats including XLSX, CSV, and JSON files, relational databases, cloud storage, and structured/unstructured data
- Implement and manage ETL/ELT processes using cloud-native and cloud-agnostic tools such as Informatica IDMC
- Leverage AWS services including Glue, Step Functions, Lambda, and S3 for data processing and orchestration
- Support data reporting and visualization initiatives
- Participate in the full SDLC from requirements gathering and design through implementation, testing, deployment, and production support
- Apply project management methodologies including Agile/Scrum and Waterfall
- Manage change and incident processes related to data platform operations
- Collaborate with stakeholders through strong communication, presentation, and consulting skills
Must-Have Requirements
- Demonstrated experience in data engineering with proficiency in Python, SQL, Linux shell, and PowerShell
- Hands-on experience with data manipulation and analysis using pandas and PySpark
- Experience with data lakehouse solutions, specifically Databricks (Delta Lake)
- Proficiency working with XLSX, CSV, and JSON files, relational databases, cloud storage, and both structured and unstructured data
- Experience using AWS services including Glue, Step Functions, Lambda, and S3
- Experience performing Extract/Transform/Load (ETL) operations using tools such as Informatica IDMC
Nice-to-Have Skills
- Experience with Azure services and cloud data warehouse solutions such as AWS Redshift
- Experience with additional ETL/ELT tools such as Azure Data Factory
- Data reporting and visualization experience
- Experience working in the Ontario Public Service (OPS) or public sector environment
Work Environment
This is a fully onsite role based at 5700 Yonge St., Toronto. The resource is expected to work 7.25 hours per calendar day, Monday to Friday, between 8:00 AM and 5:00 PM (excluding lunch breaks). A Criminal Record and Judicial Matters Check (CRJMC) security clearance is required. The team operates using both Agile/Scrum and Waterfall methodologies.