Azure Data Engineer
- Lisboa
- Permanente
- Horário completo
- Design, implement, and optimize Azure-based architectures for data platforms.
- Configure and manage Azure Databricks environments including development, monitoring, and performance tuning.
- Build and orchestrate data pipelines using tools such as Azure Data Factory, Delta Lake, and Databricks workflows.
- Apply strong security configuration practices, including the use of Azure Key Vault, RBAC, private endpoints, and network security settings.
- Ensure compliance with data governance standards and internal security policies.
- Analyze and troubleshoot cloud-based data workflows for performance and reliability.
- Define and implement Infrastructure as Code and CI/CD pipelines for deployment and lifecycle management of data components.
- Expertise in building and orchestrating data pipelines (ADF, Azure Databricks), conducting data processing and transformation (batch and streaming), scripting in Python or Scala, configuring clusters, and optimizing workloads for performance and cost.
- Proficient in creating and managing complex ETL/ELT workflows, with secure integration across hybrid and multi-cloud environments. Strong understanding of firewall rules, linked services, and data movement strategies.
- Understanding of Azure Storage Accounts, including Blob Storage and Data Lake Storage, with hands-on experience managing metadata, access tiers, lifecycle policies, and securing storage via SAS tokens and RBAC.
- Knowledge of the Azure ecosystem with hands-on experience in designing, deploying, and maintaining cloud-native data solutions. Skilled in cost optimization, automation, and secure resource management.
- Managing secrets, certificates, and encryption keys, with automated key rotation and seamless integration with other Azure services.
- Skilled in designing and enforcing access controls through custom roles, resource scopes, and principle of least privilege strategies.