51 to 75 of 116 Data Lake Jobs in London

Python Team Lead (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS GCP Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Apache Iceburg Iceberg PySpark MWAA Arrow DBT gRPC protobuf Snowflake TypeScript Manager Finance Trading Front Office Investment Banking Asset … external sources such as Bloomberg and Reuters and it's distribution across investment technology. The data enters an AWS data lake which, over the next 12 months, will be migrated to a data mesh architecture. You will be leading one of the teams … in the area that is responsible for this core data lake and its migration to a data mesh. You will be working with Java, Python, AWS (EC2, ECS, EKS, Glue), SQL, Spark, MWAA/Airflow, Iceberg and DBT. You will have line-management responsibility for more »

Senior Data Engineer 💰 £80,000-£100,000 🖥️ AWS, Python, Docker 🌍 London, 3 days a week in office We are exclusively partnered with a high growth, early stage startup operating in the beauty and wellness industry (in the UK and now US!) and supporting them on their search for … a Senior Data Engineer. The role... As the company's 1st Data Engineer, you’ll be the central player in building the integrated data architecture from the ground up. You'll gather data from diverse sources, build and maintain data … What you'll be doing... You will gather data from various departments and tools, integrating it into a central data lake or warehouse. You will design and construct data pipelines to automate data flow, involving ETL processes as needed. You will more »

We are looking for a Senior Data & Analytics Developer to work within the transport sector. Location: Hybrid working - London About the role : You will be responsible for working with others as necessary within the project including the Development Stream Lead, Solution/Data Architects, Product Owners … an optimal solution which meets requirements, adheres to technical standards and strategy constraints. Essential Skills: The ideal candidates will have a proven Senior Data & Analytics Development background, with the following skills/experience: Knowledge of ETL/ELT, data warehousing/business intelligence methodologies and best … and hybrid data warehouse design. Knowledge using cloud data products like Data Factory, Event Hubs, Data Lake, Synapse, Azure SQL server. Knowledge in developing in Databricks and experience in coding with PySpark, Spark SQL. Experience in design and development of complex more »

Our public sector client requires a Senior Data & Analytics Developer to join their team as part of an on-going project on a 12-month contract. You will have the following skills and experience: Knowledge of ETL& ELT, data warehousing/business intelligence methodologies. Experience handling … big data, cloud technology and unstructured data. Star Schema structure & design, Kimball … inmon, and hybrid data warehouse design. Cloud data products such as: Data factory Events Hubs Data Lake Synapse Azure SQL Server Experience developing Databricks and coding with PySpark and Spark SQL. Proficient in ETL coding standards Data encryption techniques more »

Write high-quality, maintainable code using languages such as Python , SQL . Establish data tools like Snowflake and Azure Data Lake Services (ADLS) Gen 2 Utilize PowerBI, Tableau, or similar tools to design and create interactive and visually appealing dashboards and reports. Work closely with … PoCs to validate new technologies, frameworks, or solutions. Drive the development of MVPs, ensuring timely delivery and alignment with business goals. Azure and Data Engineering: Leverage your Azure expertise to design and implement cloud-based solutions. Collaborate with data engineers to build robust data pipelines and ensure data quality. SQL and Data Interface: Develop and optimize SQL queries for data retrieval and manipulation. Work on data interfaces to connect various systems within the bank. Qualifications: Bachelor’s degree in Computer Science, Information Technology, or more »

Senior Data Engineer - Rapidly growing highstreet brand - Hybrid - up to £100,000 Burns Sheehan is partnered with a rapidly growing high-street chain who have seen great success recently and are in the process of expanding further in the UK and abroad. This will be s a great … opportunity to have a real impact on the strategy and roadmap for data across the business. Working closely with the wider tech teams on a super greenfield project, this is the perfect opportunity to have a real impact on a scaling business. You will be working very closely … be up to?: Data Integration: Gather data from various departments and tools, integrating it into a central data lake or warehouse. Data Pipeline Development: Design and construct data pipelines to automate data flow, involving ETL processes as more »

Data Engineer London - Hybrid working £55,000 - £60,000 A global leading HR staffing organisation based in London is currently looking to recruit a Data Engineer who will play a crucial role in designing, developing, and maintaining the data infrastructure that powers innovative solutions … within the client. We require an engineer who has a passion for data and has a good background in Azure, synapse, Databricks, SQL, Python and DataOps. This is a fantastic opportunity for a Data Engineer to gain exposure to different business areas from Visualisation, Machine Learning … Alongside this my client is going to be going through a cloud migration to Azure. Responsibilities : Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines. Develop and maintain scalable data models and databases. Design and implement data more »

they are always looking to adapt to technological changes to stay at the forefront of the industry About the Role: As a Principal Data Engineer, you’ll play a pivotal role to help drive the adoption of Microsoft Power platform and streamline reporting and analytics processes across the … organisation, we are looking for a Data Engineer with extensive multi-cloud data engineering experience and in-depth understanding of end-to-end development and deployment processes. Whilst this is a data engineer role, the focus is on data analytics and migrating … from Tableau to Power BI. Experience/skills required: Proficient in data analysis, data modelling and designing ETL processes. Familiar with Data Mesh architecture and Data Products Highly skilled in T-SQL with experience in performance tuning and dealing with large data more »

Scalexperts are partnered with a VC backed education group who after making a few recent senior leadership hires in Data & Technology are now scaling their Data Engineering function. They are looking for a Senior Data Engineer to join them to work on building a … brand new data platform which is still in early stage development. They are looking to combine all the data into one common data platform heavily on Azure so that reporting & insight/analytics becomes much clearer and more consistent. This role will require someone … years experience in Data Engineering Azure experience ideally worked with tools such as Synapse, Azure Data Factory, Data Lake etc Experience with SQL is a must, some Python is a bonus Confident in providing technical leadership Able to gather requirements for a variety more »

Senior Data Engineer (No Sponsorship Available) I am seeking an experienced Data Engineer who has experience managing green field projects and implementing modern data platforms that connect business systems and data sources to drive insights through data analytics. The ideal … candidate will have experience using Azure Platforms for consolidating data sources and making them accessible to the wider business to support growth and business strategy. Must have strong communication skills and be comfortable liaising with key stakeholders and other business units to understand business needs and gather requirements. … designing and building data infrastructure and ETL pipelines Experience with Azure Platform including Data Factory, Synapse, and Data Lake DB development experience with SQL Software development experience in relevant languages like Python or C# Experienced with T-SQL and RESTful APIs Experience with more »

working with companies in the finance sector highly desirable • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment. • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

will develop superior digital and technology concepts and architecture solutions as well as support technical implementations actively and on site, applying your sound Data/Cloud technical know-how, your understanding of business contexts, along with your analytical and conceptual skills. As you acquire experience, you will be … technical project management, digital delivery, or technology consulting environment • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

Lead Data Project Manager | Data | Data Lakes | Data Warehouse | Azure Lead Data Project Manager needed on a Permanent basis. Gravitas continue their support for a leading non for profit organisation on their search for a Lead Data Project … company is rolling out a new delivery focused function, the successful candidate will need to take ownership for all things Data. Role: Lead Data Project Manager Location: London - Minimum of once a week in the office. Salary: £58,000 - £63,000 Type: Permanent. The essential skills needed for … a project manager. Ability to provide programme/project management expertise to help support the workstream delivery. Previous experience with end to end Data project. Good communication and stakeholder management. An ability to be a self-starter/autonomous worker who deep dives into detail. MSP/Prince2 more »

Synapri collaborates with a large transport company, who is seeking a Senior Data & Analytics Developer for a 12 months contract. You will be involved in the design, development and configuration of applications, components and tools according to the technical plans set by the Development Technical Lead. Knowledge and … experience required: In-depth working knowledge of ETL/ELT, data warehousing/business intelligence methodologies including dealing with big data, cloud technology and unstructured data Knowledge of star schema structure & design, detailed understanding of Kimball & Inmon and … hybrid data warehouse design Detailed knowledge using cloud data products like Data Factory, Event Hubs, Data Lake, Synapse, Azure SQL server. Detailed knowledge in developing in Databricks and experience in coding with PySpark. Spark SQL ETL coding standards: ensuring that code more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … tools into existing structures. Create data tools to support sport data modelling, prediction and analytics. Work with data and analytics experts to strive for greater functionality in our data systems. Essential Skills Proven experience as a Data Engineer, with expertise more »

Data Architect Salary: £90,000 - £110,000 Location: Remote/Hybrid Data Idols are working with a fast-growing technology company who are disrupting the software engineering industry. They are looking for a Data Architect with strong experience in data engineering and … experience in delivering innovative solutions. The Opportunity In this role as a Data Architect, you will play a pivotal role in ensuring the efficiency of their data architecture while heading the migration from legacy systems to modern cloud solutions. You will design and build scalable database and data lake solutions, implement CICD pipelines and automate the deployment of data products into production environments. Skills and Experience Proficiency in AWS Great communication more »

Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial … Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

the future of Azure solutions with Ascent? If so, we are looking for you. We are looking for Cloud Solution Architects with a Data and AI specialism who are eager to be a part of the team—you'll be instrumental in our mission to leverage technology accelerators … and we help our customers solve problems, elevate, and do existing things better. We are on a mission to help our customers connect data, software, and purpose to create extraordinary outcomes. You could say we are a digital transformation business. We specialize in software product development, analytics, data … experience in Azure data and AI technologies, like Fabric, Synapse, Databricks, Spark, Python, GitHub, Data Factory, Azure Data Lake, Power BI, Cognitive Services, Purview, etc. Strong data analysis and modeling skills. High-level understanding of Azure infrastructure. Experience with CI/ more »

Database, Azure App Services, Azure virtual networks, Exchange Online, and Intune Azure security best practices, and Microsoft Defender products and services Protect sensitive data and ensure compliance with data governance standards Work closely with … Data and AI teams to ensure their Azure-based solutions are designed, implemented, and maintained correctly and securely Azure Data Lake, Azure Data Factory and Azure Open AI Participate in joint projects to integrate Data and AI solutions On Prem Infrastructure more »

Permanent - Hybrid - Azure Data Engineer Azure Data Engineer required for an exciting perm opportunity. Our client, a leading property investment business are looking to hire an Azure Data Engineer to join their growing tech function. The key responsibilities are the following: Data processing, consolidation, clean-up and migration Data Transformation Azure roadmap discussions Maintain Azure Automation Runbooks. Configure virtual cloud instances and monitor backups. The key skills they are looking for are: 3+ years experience within an Azure Data Specialist position Microsoft T-SQL script skills Experience with Azure Data storage such as Azure blob storage/data lake Azure Key vault Active Directory - Azure AD SAML, Enterprise Apps A strong knowledge of databases (SQL/Oracle) Server Administration If you are looking more »

s biggest brands, including Uber, Major League Baseball, NHS, Disney, Standard Chartered, Mediacom and British Airways. Our services support the whole organisation with data-driven performance insights for leaders, training for managers, wellbeing tools for all employees, as well as access to a global network of accredited mental … hiring process more accessible, just let us know when you apply, or email talent@unmind.com. The Role – Analytics Engineer The core mission of data analytics is to provide actionable insights to all Unmind Teams that drive product and business decisions that build the best version of Unmind. As … within reports that surface relevant trends. In this role you'll get to: Own - You will take the architectural lead for the key data models in the company, and guide your team in finding the right balance between technical excellence and delivery of value that brings you closer more »

About Tredence: - Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision … methodology; think strategically and execute methodically. Develop and manage capacity and growth projection forecasts of the environment within budgets. Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional/non-functional business requirements. Drive optimization, testing and tooling to improve … in Data bricks Experience managing distributed teams preferred. Strong technical experience in large distributed systems, Data Warehousing, Data Lake at scale Project management skills: financial/budget management, scheduling and resource management experience with medium and large-scale projects Comfortable working with ambiguity more »