The Data Engineer will be responsible for all the data processing and Data Infrastructure of the company. Working in a diverse and challenging environment providing data solutions. He will be a lead partner in implementing and maintaining new data tools.
Reporting to Head of Data, the duties of the Data Engineer include:
- Oversee the data engineering efforts in the company
- Design and maintain schemas and data structures
- Develop data integration processes, pipelines and ETLs
- Tune and Optimize data pipeline, using native monitoring and troubleshooting tools
- Troubleshoot data issues, validate result sets, and provide ad-hoc reports and analysis on time and with high quality.
- Build infrastructure for automatic reporting tools for relevant teams and stakeholders
- Effectively collaborate with other teams across the organization
- Manage multiple projects at a time in a fast-paced, results-oriented environment.
- Passionate about data with strong analytical as well as technical skills!
- 2+ years’ experience as a Data Engineer (preferably in a cross discipline background, as a single Data Engineer in a startup company)
- Experienced in working with large DWH or Data lake solutions and developing ETL processes – Hands on experience – a must
- Advanced SQL knowledge and experience working with relational and non-relational data sources, schema design, data-modeling and excellent SQL troubleshooting skills working with large datasets (Postgresql, Snowflake, SQL Server, S3, Redshift and others)
- Strong background in ETL development, data-modeling, metadata management, and data quality
- Experience working with Linux
- Experienced in message queue (e.g. Kinesis/ Kafka/ SQS/ RabbitMQ)
- Fluent in at least one scripting language (e.g. bash, Python)
- Familiar with the AWS cloud platform
- Familiar with Snowflake DWH – an advantage
- Familiar with Elasticsearch (not only in the ELK framework) – an advantage
- Working with NoSQL databases, (e.g. key-value stores- redis) – an advantage
- Internally motivated, able to work proficiently both independently and in a team environment
- College or University degree in Engineering/ Computer Science or a related discipline, or equivalent experience in other frameworks (e.g. same position in 8200)