Data Engineer for AI-Powered Social Media Platform | Remote

We're seeking an experienced Lead Data Engineer to join our cutting-edge artificial intelligence platform that revolutionizes social media content creation and publishing. In this pivotal role, you'll architect robust data pipelines that power our AI models, ensuring seamless data flow from diverse social media sources. You'll work at the intersection of big data and machine learning, implementing sophisticated ETL processes while maintaining the highest standards of data quality and governance. Key Responsibilities - Design, develop, and maintain scalable data pipelines collecting and processing information from multiple social media platforms and user interactions. - Architect comprehensive data warehouse solutions optimized for AI model training and analytics. - Implement real-time data processing systems to support dynamic content generation. - Establish rigorous validation frameworks ensuring data integrity, accuracy, and reliability. - Enforce enterprise-grade data governance practices guaranteeing compliance with GDPR and other relevant regulations. - Develop and maintain data documentation and lineage tracking systems. - Automate Extract, Transform, Load workflows using modern data engineering tools and frameworks. - Continuously monitor and optimize data pipelines for enhanced performance, reliability, and scalability. - Implement data partitioning and indexing strategies for improved query performance. - Partner with Data Scientists and ML Engineers to develop data infrastructure supporting model development. - Work alongside analysts to create interactive dashboards enabling data-driven decision making. - Communicate complex technical concepts to non-technical stakeholders effectively. - Evaluate emerging data technologies and frameworks for potential implementation. - Establish performance benchmarks and monitoring solutions to identify pipeline bottlenecks. - Develop data marts and dashboards providing real-time insights into social media metrics. Nice to Have - Experience with streaming data technologies (Kafka, Kinesis, Spark Streaming). - Knowledge of containerization and orchestration (Docker, Kubernetes). - Familiarity with NoSQL databases (MongoDB, Cassandra, DynamoDB). - Experience with data visualization tools (Tableau, Power BI, Looker). - Background in social media analytics or working with social media APIs. - DataOps certification or formal training. - Knowledge of data privacy regulations beyond GDPR (CCPA, HIPAA). Why Join Our Team Join us to tackle fascinating data engineering challenges at the forefront of artificial intelligence and social media. You'll work with cutting-edge technologies in a collaborative environment that values innovation and technical excellence. Our remote-first culture offers the flexibility to work from anywhere while contributing to a product that's transforming how businesses create and publish content online. We offer competitive compensation, professional development opportunities, and the chance to shape the architecture of a rapidly growing AI platform.