Senior Data Engineer
Location:
Hong Kong or adjacent timezone (+/- 2 hour difference)
We are looking for a Senior Data Engineer to join Bilby, a company that applies machine learning and AI techniques to scrape, parse and analyse official government documents, in order to extract alpha-generating features for financial market analysis. You will play a central role in designing, building, and maintaining data pipelines and downstream applications, including an API for delivering data to end users.
Key responsibilities
You would report to the CTO, and work closely with the existing Head of Data (in charge of designing and maintaining robust data pipelines for scraping and processing large volumes of text data from a variety of government sources) and our API product developers. The successful candidate will be responsible for:
- Downstream application pipelines: Build and manage pipelines for downstream applications to facilitate efficient data delivery and integration with end-user systems.
- ML model deployment: Deploy machine learning models and construct ML pipelines to extract relevant features from texts, supporting market analysis and decision-making processes.
- Architecture and technology selection: Choose appropriate technologies and architectures in a pragmatic manner, balancing innovation with pragmatism.
- Team collaboration: Work with and mentor more junior team members.
Required attributes
The successful candidate will possess:
- Experience: Proven track record as a Senior Data Engineer or similar role, with significant experience in architecting, developing and maintaining data pipeline and backend systems. The successful candidate will be a fully-fledged engineer, not simply a developer. As a result, they will pay close attention to lifecycle questions, such as making systems scalable, maintainable and deployable.
- Technical skills:
- Strong verbal and written communicator, and a propensity to clearly document both thought processes and deliverable tasks.
- Proficiency in data workflow management platforms, such as AirFlow.
- Familiarity with data warehouses such as Google BigQuery.
- Expertise in cloud platforms, particularly Google Cloud Platform (GCP).
- Strong capability in deploying models and building ML pipelines, with familiarity in AI and ML frameworks (e.g., TensorFlow, PyTorch).
- Solid backend development skills, including API design and implementation. The successful candidate must be proficient in Python. Experience in other languages like Javascript, and Go will be valued.
- Knowledge of how to deploy software in a sustainable manner, such as using CI/CD and writing tests, to ensure quality and reliability.
- Design taste: Ability to make pragmatic decisions on technology and architecture, balancing short-term needs with long-term scalability.
- Industry awareness: Openness to new technologies and a keen awareness of current events and developments in data engineering and AI/ML.
Preferred attributes
Ideally, the successful candidate would also have the following qualities:
- Expertise in developing end-user APIs and SDKs.
- Experience in the data operations behind financial markets is plus.
- Leadership experience: Capable of mentoring mid-level developers and some project management.
What's next
To apply for this position, please complete the application form here in Dover
