Data Engineer
since 10.2024 - Till the present day |Rapid Innovation GmbH
Google Cloud, GKE, SQL, Dbt, Kotlin, Kafka, Pekko, GitHub
Supply chain management consulting and implementation services.
● Support in creation of master data set for a supply chain management application from multiple heterogeneous sources.
● Design and implementation of ETL and data pipelines
● Implementation data quality dashboard for business critical KPIs
● Deployment and monitoring of various data pipelines
Consultant / Data Engineer
09.2023 - 10.2024 |Data warehouse consulting services
AWS Cloud, Spring Boot, Java, Docker, GitLab
Support and implementation services.
- Consulting services data model, deployment strategies, process design
- Design and implementation of various ETL processes
- Workshops and knowledge transfer on cloud and data architecture, process design
Solution Architect / Cloud Architect
07.2021 - 10.2023 |Data warehouse migration
AWS Cloud, LakeFormation, Glue, EMR, Athena, Lambda, SQS, S3, Docker, Spark, Hadoop, Airflow, Terraform, Python, GitLab
Concept and support of an on-premise data warehouse migration towards the cloud.
Design and implementation of a cloud-based data warehouse infrastructure based on AWS Managed Services
Help in migrating various data processing jobs from Cloudera to AWS Managed Services
Design and implementation of deployment pipelines for testing and rollout
Workshops and knowledge transfer on cloud and data architecture, process design
Solution Architect / Data Engineer
06.2020 - 12.2022 |Cloud based data warehouse
AWS Cloud, Kubernetes, Docker, Spark, Airflow, Kafka, Terraform, Python, Kustomize, GitLab
Design and implementation of a cloud-based data ware house for processing user related data.
Implemented workflows to fetch data from various third-party providers
Building and enabling a team to create new ETL processes
Realtime integration of a market automation software suite
Implemented modern ETL processing environment using Airflow, Spark and Kubernetes
General advice on data architecture and data management
Solution Architect
08.2019 - 12.2019 |Sensor data processing
AWS Cloud, Kubernetes, Kafka, Spark, Bamboo, Java, Docker, Terraform
Design of AWS managed infrastructure platform for sensor data processing, extension of an existing data science environment.
Advice on design and tools for building a Kubernetes based infrastructure platform for sensor data processing
Design and implementation of infrastructure components on Kubernetes
Design and implementation of the ETL pipeline for data collection
Design and implementation of CI/CD pipeline with Bamboo & Kubernetes
Solution Architect / Data Engineer
01.2019 - 03.2020 |Data analytics on car measurement data
AWS Cloud, Lambda, IAM, Airflow, Kubernetes, Terraform, Python, Jenkins
Design and implementation of a cloud-based data warehouse for evaluation of vehicle data. Design and implementation of a data science environment.
Extented a prototype and put into operational readiness for a production environment
Design and implementation of a CI / CD pipeline
Setup of project structure and release management
Implemented various ETL pipelines for car measurement data collection, validation and transformation
Solution Architect / Data Engineer
10.2017 - 11.2018 |Data science environment
AWS Cloud, Kubernetes, Spark, R, NiFi, Terraform, Docker, Jupyter NB
Design and implementation of a cloud-based data warehouse & data science environment.
Designing data warehouse architecture & data storage strategies
Architecture proposal of a dynamically scalable data warehouse
Implementation of infrastructure components in Terraform & Kubernetes
Implementation of infrastructure components in Kubernetes
Development of ETL pipelines for data collection
System Architect / Consultant Data Strategy
05.2017 - 12.2018 |Micro service architecture
Micro Services, Java, Docker, Kafka, Liquibase, Jenkins
Support conception and implementation/migration of a monolith into a micro service architecture.
Alignment and coordination of different teams regarding technology usage
Introduction of Kafka as the central message bus for micro service communications
Introduction of LiquiBase for database schema management
Professional / technical support for a specific micro service