Data Engineer II
Reports to: Team LeadOverview:The Data Engineer II will architect and implement data systems at TruBase. This role focuses on designing distributed data processing systems, optimizing data pipelines, and ensuring system reliability at scale.Essential Behaviors & Activities:
Build distributed data systems
Design for fault tolerance Balance consistency and availability requirementsScale processing across multiple instancesImplement database solutions
Recommend appropriate database technologies based on workloadsDesign efficient schemas and indicesOptimize query patterns and data accessBuild efficient and correct ETL pipelines
Design high-throughput data flowsImplement data transformation logicMonitor pipeline performance and correctnessEnsure data integrity across systemsTroubleshoot performance issues
Profile system bottlenecksAnalyze memory, CPU, and network usageImplement distributed tracingAdvance technical capabilities
Be familiar with emerging technologiesImplement proof-of-conceptsDocument architectural decisionsMentor junior engineersCompetencies and Experience:Core Competencies
- Distributed systems knowledge (idempotence, consistency, consensus)
- Understand how databases work (B+-trees vs LSMs, columnar vs. row storage, storage tiering)
- Performance debugging and profiling practices and tools (strace, py-spy, flamegraphs)
- Memory access/usage, IO, and CPU usage optimization
- Data transformation and aggregation at scale
- Experience with log-based distributed streaming systems (Kafka, Pulsar, Kinesis)
- Experience with relational and columnar databases (Postgres, Redshift, Clickhouse, Dynamo)
- Advanced language skills (Python, TypeScript)
- Container orchestration (Kubernetes)
- CI/CD for data pipelines
Desirable Technical Experience
- Experience with lower level lang
- Understanding of codecs and compression (JSON, Protobuf, FlatBuffers)
- AWS cloud architecture
- Data governance implementation
Measurables
- System reliability and scalability
- Query and pipeline performance improvements
- Architecture quality
- Technical documentation clarity
- Knowledge sharing effectiveness
Other Information:Supervisory Responsibility: This position has no direct reports.Position Type/Expected Hours of Work
- This is a full-time position; days and hours of work are Monday through Friday, 40 hours a week.
- (Hybrid, Atlanta Area — 2 Days/Week in Office)
- Work Authorization:Applicants must be legally authorized to work in the United States for any employer. We are unable to provide visa sponsorship (e.g., H-1B, OPT extensions) at this time.
This job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities and activities may change at any time with or without notice.
