Data Engineer - Senior

The Data Engineer creates the data infrastructure in a simple and efficient matter by design, build and integrate data from various sources. The Data Engineer builds data pipelines from existing data sources and other (on-prem) systems into the Big Data platform; Creates consistent e2e BI solutions with a data lifecycle starting with collecting the needed data until visualizing them; Creates AI/ML models in BigData use cases and helping business to leverage the results; User story creation with good understanding of business processes, product and customer journey; Rates user stories for customer value and to challenge the product owner on prioritization of user stories (e.g., stability versus functionality); User story refinement: Is able to translate user stories into technical stories; Development: Able to design, develop and optimize data storage and data processing solutions and proficient in in one or more program languages like python and SQL to ensure sufficient quality level ,including test criteria; Deployment: Is able to create and execute script based deployments, including rollbacks. Is able to maintain/setup automated deployment using the VF standard toolset; Operations: Is able to setup automated monitoring and proactive alerting, is able to provide parameters to size the underlying application infrastructure.

The Data Kong Team is responsible for building and maintaining the Asset Analytics Wind data pipelines on Databricks, delivering trusted and scalable data products up to the Silver layer and enabling Value Delivery Teams through platform support, infrastructure management, governance, and technical expertise.

We are looking for a highly experienced, structured, and self-driven Senior Data Engineer / Databricks Platform Engineer who can independently own complex technical challenges, support the wider team, and contribute across both data engineering and platform engineering domains.

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Delta Lake.
• Build and support Bronze and Silver layer data products following Medallion architecture principles.
• Develop and optimize batch and streaming ingestion pipelines.
• Monitor, troubleshoot, and improve Databricks performance, Spark jobs.
• Support Value Delivery Teams with platform onboarding, troubleshooting, best practices, and operational support.
• Implement and maintain Databricks and supporting Azure infrastructure using Terraform.
• Manage security, access control, and governance using RBAC, Unity Catalog, Service Principals, and Managed Identities.
• Develop and maintain CI/CD pipelines for automated deployment of data and infrastructure components.
• Ensure platform reliability, scalability, cost optimization, and operational excellence.

The team is present in the Amsterdam office on Tuesdays and Thursdays and we expect the candidate to also be avaiable in Amsterdam on those days.