Data Engineer LATAM at onebeat

Brazil

<p>We are seeking an experienced Data Engineer. The ideal candidate is self-motivated, can multitask, and is a proven team player. You will design, develop, manage, and maintain our open-source data platform, including our Data Lakehouse (S3, Apache Iceberg, and ClickHouse), ETL processes, and orchestration.</p> <br><h3>What You Will Do</h3> <p>●&nbsp;&nbsp;&nbsp;&nbsp;Develop a scalable data platform integrating multiple sources for easy access.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Design and enhance data tools (orchestration, governance, data lakehouse, BI, etc.).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Ensure smooth operation of data systems for analysts, scientists, and engineers.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Optimize data pipelines (ingestion, processing, and output) in a microservices environment.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Build, maintain, and monitor ETL/ELT processes .</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Troubleshoot and improve the performance, scalability, and reliability of the data infrastructure (S3, Apache Iceberg, ClickHouse).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Collaborate cross-functionally with data scientists, analysts, and backend engineers to understand data needs and deliver solutions.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Implement and champion data quality, governance, and security best practices across the platform.</p> <br><h3>Requirements</h3> <p>●&nbsp;&nbsp;&nbsp;&nbsp;3+ years of experience as a Data Engineer or in a similar data infrastructure role.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Strong proficiency in SQL and hands-on experience with data modeling.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience with data lake/lakehouse architectures (e.g., Apache Iceberg, S3, or similar).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience with analytical / columnar databases (e.g., ClickHouse or similar).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience building and orchestrating ETL/ELT pipelines .</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Strong programming skills in Python and/or Scala/Java.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience working within a microservices architecture and cloud environments (AWS preferred).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Self-motivated, strong multitasking skills, and a demonstrated team player.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Excellent communication skills and the ability to work both independently and collaboratively.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Hands-on experience with Apache Spark (or similar technologies) for large-scale data processing.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Professional proficiency in written and spoken English.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Note: this role is focused on batch data processing (not real-time streaming).</p><p><strong>Nice to Have</strong></p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience working with and contributing to open-source data platforms and tools.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Familiarity with BI and visualization tools (e.g., Superset, Looker, Tableau, Metabase, or similar).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience with containerization and orchestration (Docker, Kubernetes).</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience with infrastructure-as-code and CI/CD practices.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience with AWS EMR and running Apache Spark workloads in a cloud environment.</p><p>●&nbsp;&nbsp;&nbsp;&nbsp;Experience leveraging AI-assisted development tools (e.g., GitHub Copilot, Cursor, or similar) to boost engineering productivity.</p>