Senior Data Engineer

Location London, Greater London, England
Discipline: Technology
Job type: Permanent
Salary: £Salary: £75,000 - £85,000 Per annum Plus benefits
Contact name: Rob Smith

Contact email:
Contact phone: 01293584300
Job ref: 21317
Published: about 1 month ago
Expiry date: 30 July 2021

Location: London/Home working

My client is looking to recruit an experienced data engineer to help rearchitect one of their leading platforms. You will be joining a state-of-the-art R&D team that utilises AI, ML, NLP and modern development to build innovative solutions and best in industry products.

Responsibilities and Performance Objectives:

  • Architect and build distributed, scalable, and reliable data pipelines to ingest and process data.

  • Collaborate closely with stakeholders and researchers to support machine learning, analytical and product use cases.

  • On-board, document and curate external datasets for internal usage.

  • Engage in data interpretation and forensic data analysis and troubleshooting.

Required skills:

  • We’re looking for an experienced engineer who can hit the ground running.

  • Self-motivated and creative. We’re building from ground up, not tweaking legacy.

  • Proficiency in scripting languages (especially Python).

  • Good communicator and team player.

  • Solid understanding of algorithms and data structures.

  • Experience of cloud platforms (e.g., Amazon Web Services).

  • Commercial experience of writing and optimizing SQL queries.

  • Experience of data orchestration platforms (e.g., Airflow).

Nice to have skills:

  • Experience in feature engineering for Machine Learning applications

  • Experience working within an agile team

  • Knowledge of data versioning libraries/ methods (e.g., DVC)

  • Familiarity with data lakes

  • Experience of cloud data warehouses (e.g., Snowflake)

  • Familiarity with migrations frameworks for change management (e.g., Flyway, Alembic)

  • Experience of dynamic schemas/schemas evolution

  • Experience building streaming pipelines (e.g., Kafka, Kinesis)

  • Familiarity with continuous deployment

  • Experience ingesting data from web (e.g., HTTP requests, HTML/ XML parsing)

  • Familiarity with Big Data ecosystem (e.g., Spark/ MapReduce)

  • Experience of load testing data pipelines

  • Experience of geo-indexing technologies (e.g., PostGIS)

  • Knowledge of columnar data storage formats (e.g., Parquet, ORC)

Salary: £75,000 - £85,000 Per annum Plus benefits