JD – Data Engineer role:
Experience : 3+ Years
· 1. Passion for building quality production systems software and tools. Research is an important component of what we do.
2. Team players who enjoy working alongside other brilliant people to come up with the best solutions together. Should be comfortable with accepting, giving and applying constructive criticism.
3. Curiosity and open mindedness are critical. We are building critical infrastructure for institutions to adopt crypto! The problems are challenging, and we need innovators to help solve them!
Responsibilities
- Ownership of the end-to-end data engineering component of the solution. - Take decisions on architecture for a scalable data platform
- Design, build and launch efficient & reliable data pipelines to move and transform data (both large and small amounts).
- Intelligently design data models for optimal storage and retrieval.
- Deploy inclusive data quality checks to ensure high quality of data
- Maintain and scale our data platform to meet our needs, including monitoring, logging, and alerting
- Improve the platform’s availability, scalability, performance, and security
- Participate in implementing features, addressing issues, and improving performance - Be the responsible person for data systems
Must have Requirements/Experience
- At least 3 years of experience in data engineering or similar role
- Experience with spark batch-job scripting
- Experience with spark job tuning (esp. spark-sql optimisations)
- Have dbtframework experience
- Knowledge on any job orchestration component (eg: Airflow/StepFunction) - Experience with AWS components (eg: Glue/EMR/StepFunctions/Athena) - Worked with transactional data lake table formats (eg: HUDI/DeltaLake/Iceberg)
Nice to have Requirements/Experience
- A genuine interest/exposure in Web3 and/or decentralized technologies. - Experience in data landscape or self-service analytics space (Pipelining/Warehousing/Visualisations).
- Participation in open source projects (even if it is not to the level of substantial contribution) - Experience on high frequency Time series databases
- Exposure to making decisions on multi cloud architecture & integration
- Exposure to Identity & Access Management.