As a Senior Data Engineer Consultant, you'll be leading efforts in building scalable backend services and data pipelines that support hundred of millions of users and thousands of events per second. You will get to work across multiple microservices written in Python, Scala, and Ruby, using technologies such as AWS Lambda, Kinesis, SQS, RDS, Dynamo, Snowflake, Spark, HDFS, S3, ElasticSearch, ECS/Fargate, Athena, Presto, Cadence, Serverless Framework, Docker, Terraform, and more.

- Development efforts of new services that deal with large scale data or high throughput data pipelines/streams

- Architect and scale out our ETL framework to support back data processing in technologies such as Spark

- Manage and implement scalable monitoring and escalation strategies across our systems

- Act as a technical architect, elegantly separating domain models to ensure partner complexity doesn't leak into our app

- Invest in infrastructure that ensures our small team can efficiently manage hundreds of integrations

 

Qualifications:

- 8+ years software engineering experience

- Python and SQL
- Experience in a few of those is ideal

- Experience in designing and implementing scalable applications/microservices

- Experience creating robust RESTful APIs

- Emphasis on clean, well-designed code

- Deep understanding of Postgres, MySQL, and other relational databases

- Experience working with large datasets/databases and scaling memory-intensive applications

- Gritty mentality with a focus on shipping

- Process-oriented executor; you can manage multiple projects concurrently and prioritize effectively