Achievements and responsibilities: — Reduced Spark EMR costs by 40% by migrating ETLs to more efficient EC2 instances (c6g over m5). — Designed ranking API for KaiOS Apps Store. — Migrated business-critical legacy ETLs to Airflow, EMR Spark & Athena from AWS Batch, Lambda, Redshift. — Adopted code quality improvement process and tools: linters, CI, code reviewing and refactoring. — Designed an Airflow alerting framework. — Adopted several remote-first development practices for a distributed team of 15 data engineers and analysts.

Achievements and responsibilities: — Founded a data pipelines infrastructure using Airflow (Google Cloud Composer) and Apache Beam (Google Cloud Dataflow) maintaining multiple ML-related data pipelines. — Designed and implemented Data Access Layer from the ground up. API is similar to Apache Superset. — Automation of ML: adopting data scientists’ notebooks to ETL jobs. Includes active work with ML and data wrangling libraries such as pandas, PyTorch, FB Prophet. — Major ClickHouse SQL experience. — Minor analytical tasks. — Major requirements negotiation. Project description: — Whisk is a part of Samsung NEXT and is a highly AI-oriented food-tech company. — Data repositories contain various recipes-related datasets including webpages impressions, managed ingredients ontology, customers carts checkouts, etc. — Data infrastructure is built using Google Cloud PubSub for data ingestion, ClickHouse for data analytics, Google Cloud Composer (Apache Airflow) for ETL, Data Studio and Redash for BI reports. — Data Access Layer is implemented in form of asynchronous web-application on Fast API and SQL Alchemy.

Achievements and responsibilities: — Designed a Serverless AWS-based Data Platform (Data Lake + Data Marts) from the ground up. — Set up development and release processes as a Data Engineers Team Lead. The team is distributed: contains 5 full-time employees + 3 outsourcers. — Reduced cloud infrastructure costs by 30% by choosing AWS over Azure on the evaluation stage. — Successfully presented the project to the headquarters and promoted Data Platform to OMD departments. — Collected, negotiated and prioritized requirements from 5 major departments. Project description: — OMD is an advertising placements buyer. — Data platform is developed for business automation: range of applications varies from reporting to automated advertisement campaigns planning and data science purposes. — Huge number of various sources: analytical systems (e.g. Google Analytics, Yandex.Metrika), media advertisement accounts (e.g. FB, Instagram, RB Mail.Ru, VK), digital advertisement accounts (e.g. Google Adwords, Yandex.Direct), clients' CRM systems, and even offline placements statistical data (outdoor, TV and radio) — about 60 sources total. Consider the need to support different authentication protocols and rate limits. DWH Stack: — Amazon Web Services: S3, Lambda, Batch, Glue, Step Functions, DynamoDB, EC2, Redshift (and other secondary services, e.g. IAM, Secrets Manager). — Async Python 3.7 (aiohttp, aiobotocore). — ClickHouse.

Development of the DWH and Big Data Analytics Infrastructure for Yandex.Taxi. Achievements and responsibilities: — Designed, implemented, maintained critical Data Warehouse entities like rides and customers aggregates; — Developed full-cycle ETL-process from the rides logs, ads platforms, etc. to Data Marts for over 10 entities; — Developed (using JS, StatFace, Tableau) and maintained the main BI dashboard of the whole business; — Performed basic analytical tasks (using MapReduce frameworks and/or YQL — analogue of Hive for YT); — Handled DWH users technical support requests. DWH stack: — YT (Yandex in-house MapReduce system); — Python as the general-purpose language for describing ETL-processes and MapReduce procedures; — Greenplum; — ClickHouse; — Tableau, StatFace (Yandex in-house BI tool with JavaScript support); — Microsoft Analysis Services, SSAS (minor experience).

DWH and ETL-process development. Technology stack: — Teradata; — Informatica; — Automation via Python.

Monthly audience: ≥2.5 millions. Formerly, as a chief editor I led a distributed team of ≥40 authors and editors. Achievements and responsibilities as a Lead Data Engineer: — Founded a serverless analytical infrastructure using ClickHouse-based events aggregating system. The tool collects web-clients activity and provides a reporting API. — Adopted Lighthouse CI which encouraged web developers to increase site performance by 150%! — Implemented data processing tools for content analysis: extracting data from Facebook, on-site analytics, OneSignal, Telegram, VK, etc. and storing to relational database with data transformations. — Maintain a chatbot for business automation purposes based on Django+SQLite3, Nginx, Redis and Celery. — Designed and implemented reporting system for commercial placements based on Django, Postgres and Bootstrap. Used for live monitoring and reporting on marketing campaigns. — Perform analytical tasks for advertising purposes and data visualization with Tableau, Redash, DataLens.