Job Title: Senior Data EngineerAbout the role:The Senior Data Engineer role focuses on building and maintaining data pipelines, ensuring data quality, and collaborating with various teams within Springer Nature.About us:We’re looking for a Senior Data Engineer to join Springer Nature Global Business Systems (GBS) within Springer Nature Technology. Springer Nature is a leading publisher of scientific books, journals and magazines with over 3000 journal titles and one of the world’s largest corpora of peer-reviewed scientific text data. You would be joining a new programme of work to transform how Springer Nature uses its data: building up data capabilities, creating a data platform and engineering capability (technology, people and process) to create a foundation for the future, adding value to cross-organisation initiatives and kick-starting data-driven innovation.Across the programme, our teams are cross-functional, diverse and made up of different experience levels. All team members collaborate to deliver the best solutions that satisfy our customers’ needs.We are committed to growing and nurturing our people for the long-term. We attend conferences and make time for people to explore new technology that interests them and is relevant to our work as well as host data engineering community-of-practice sessions to share knowledge.We work in a supportive environment. We value face-to-face co-working and require two days per week in the office on average in a month.The position is based in either Pune (India) or Lisbon (Portugal), with some travel required, typically 1-2 times a year. This role is on a small, autonomous team and you will be expected to impact what we do and how we work. We like to keep our processes light, and bureaucracy slim.About youRole responsibilities:Design, implement, test and maintain scalable and efficient ETL/ELT pipelines to ingest and transform large datasets from diverse sources.Architect, build, test and optimize data warehouses and data lakes, focusing on data consistency, performance, and best practices for analytics.Implement data quality checks, validation rules, and monitoring to ensure data accuracy, integrity, and security. Contribute to data governance initiatives.Continuously monitor and enhance database and query performance, identifying opportunities to streamline and improve data retrieval times.Define and design frameworks for monitoring data quality and data pipelinesMonitor Google BigQuery consumption.Evaluate, select, and integrate new tools and technologies to enhance data capabilities. Automate manual processes to improve efficiency and accuracy.Mentor junior engineers, sharing best practices in data engineering and analysis, and helping to build a culture of technical excellence within the team.Proactively lead streams of data engineering work, collaborating with stakeholders and other non-engineering roles to come up with creative solutions to the problems your team is presented with.Work with both data producers and consumers to optimise existing data products and the data within them to meet evolving business needs.Work collaboratively with other engineers, using techniques like pair and ensemble programming, to foster collective ownership and help upskill, reskill and learn from other team members.Help guide technology choices, adopting company-standard tech stacks by default while responsibly investigating alternative tools and services and looking for opportunities to innovate.Skills & experience:Technologies you will be working with:SQL, Python, Google Cloud Platform (GCP), and Google BigQueryData pipeline tools such as Apache Airflow and DBTData modeling concepts, including dimensional and star schema designData visualization tools like LookerEssential:Several years of experience in Data / Software engineering on a cloud platform.An understanding of data and distributed systems concepts.Several years of experience working with data from large ERP systems.Knowledge of data quality concepts and experience using tools to identify data quality problems and come up with strategies to tackle them.Experience working with iterative software development principles:Continuous Integration & Deployment (CI/CD)Automated testing at different levelsCollaborative development techniques like pair and ensemble programmingExperience owning software systems and/or data pipelines end-to-end, having full responsibility for operating them in production and responding to problems that arise.The ability to work with stakeholders and other non-technical roles to translate business requirements into technical work, and conversely to articulate necessary technical work to the same people for prioritisation.High competence in SQL and deep experience of at least one programming language e.g. Python, JavaDesirable:Experience with decentralised Data Mesh and Data Product architecture principles.Experience building and testing data pipelines with DBTUnderstanding and experience of User-Centric Design principlesWhat you will be doing:1 month:Actively contributing to the codebase in collaboration with other engineers and deploying changes into productionFamiliarising yourself with our tech stack and processesStarting to understand the data landscape in and around your teamGetting to know the various stakeholders and their general requirementsCollaborating effectively with each discipline on the teamParticipating in technical discussions and sharing ideas3 months:Gaining an understanding of the team’s context within the wider organisationLeading a stream of work, working with both engineers and stakeholdersTriaging support queries and diagnosing issues in live data pipelinesSetting the technical direction of the work done by the teamHolding discussions within the engineering team in order to improve product architecture and code qualityEnsuring that data is stored securely and in compliance with GDPR6 months:Take ownership of key projects and drive initiatives to enhance data capabilities.Switching context between multiple streams of work, providing guidance to both developers and stakeholdersFostering a culture of continuous feedback, giving and receiving constructive feedback within your team, and proactively improving ways of workingMentoring others in the principles of data engineering and best practices and looking for opportunities to help other engineers on the team growArbitrating disagreements within the team and not avoiding difficult conversationsGauging the complexity and scope of a piece of work, breaking it into smaller pieces when appropriate with a focus on iteratively delivering value to end usersAt Springer Nature, we value the diversity of our teams and work to build an inclusive culture, where people are treated fairly and can bring their differences to work and thrive. We empower our colleagues and value their diverse perspectives as we strive to attract, nurture and develop the very best talent.Springer Nature was awarded Diversity Team of the Year at the 2022 British Diversity Awards. Find out more about our DEI work here#LI-SS2
