technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and unstructured data on AWS. • Familiarity with AWS best practices in data engineering, data science, and product development. • Knowledgeable in … both stream and batch processing. • Comfortable designing and building for AWS cloud, including Platform-as-a-Service, server-less, and container technologies. • Diverse skills in ETL, data warehouse, and Data Lake design. • Familiarity with various tools, cloud technologies, and approaches. • Expertise in AWS or equivalent open-source tools. • Proficient more »
City Of Bristol, England, United Kingdom Hybrid / WFH Options
Anson McCade
Azure Stream Analytics, and Azure Synapse Analytics. A solid foundation in data engineering, data science, and product development, encompassing experience in both stream and batch processing. Designing and deploying production data pipelines, utilizing languages such as Java, Python, Scala, Spark, and SQL. In addition, you should have proficiency or … and unstructured data, including integration of data through Azure cloud-based ingestion and curation processes. Knowledge of data mining, machine learning, and natural language processing is considered advantageous. Familiarity with Agile methodologies and participation in Scrum ceremonies. Competence in designing and constructing solutions tailored for the Azure cloud environment more »
Knowledge of developing real-time data stream systems (ideally Kafka). Proven track record in developing data systems using PySpark and Apache Spark for batch processing. Capable of managing data intake from various sources, including data streams, unstructured data, relational databases, and NoSQL databases. Extensive knowledge in distributed systems … visibility. Play a major role in the development and optimisation of data systems to support our Data Lake/Data Mesh architecture and data processing needs, ensuring the provision of scalable, high-performance data products for the entire organisation and proper data governance. Utilise and improve our current AWS … possible solutions. This position offers the opportunity to work with a bright and varied team to develop and improve our data platform for data processing, access, gathering, and monitoring, building the future of our data infrastructure. more »
be able to provide an accurate assessment of the current capabilities and limitations of the systems. They will understand the critical path of their batch and the associated downstream, upstream and business impacts. They will work closely with Level 3 team and other cross functional teams and check the … expectation setting. * You will actively work on and resolve any IT risks and vulnerabilities across the application stack. * You will contribute towards ensuring all batch SLA's/OLA's are appropriate, understood and agreed for the critical path of your batch processes, ensuring that you understand the more »
be able to provide an accurate assessment of the current capabilities and limitations of the systems. They will understand the critical path of their batch and the associated downstream, upstream and business impacts. They will work closely with Level 3 team and other cross functional teams and check the … expectation setting. * You will actively work on and resolve any IT risks and vulnerabilities across the application stack. * You will contribute towards ensuring all batch SLA's/OLA's are appropriate, understood and agreed for the critical path of your batch processes, ensuring that you understand the more »
