• Our focus is on enhancing Spark stability/usability/performance and extending Iceberg's capabilities
• Additionally, we are building internal storage and metadata services to power our lakehouse architecture
• Troubleshoot and resolve production issues, ensuring minimal downtime

Maintain and improve system monitoring, logging, and alerting

• A validated, successful track record of 5+ years with strong programming skills in either Java or Scala
• Familiarity with Linux/Unix systems, including shell scripting and system commands
• Understanding of networking principles, as Hadoop often operates in distributed environments
• Overall, the candidate is expected to have analytical skills to tackle complex distributed system challenges and optimize solutions
• Excellent problem solving and analytical skill on both defect and performance issue resolution
• Great teammate and able to work with a global team
• Experience in designing and implementing distributed systems

Good understanding of other Hadoop related projects including HDFS/HBase/Yarn

• Comprehensive knowledge of Apache Spark’s internals, including Catalyst optimizer, Tungsten execution engine, shuffle mechanics, AQE, and memory management

Solid understanding of file formats like Apache Parquet and table formats such as Apache Iceberg, Delta or Apache Hudi internals, including snapshots, manifests, partition specs, and table metadata

• Plus Points

Contribution to related open source projects like Apache Spark/Iceberg $132,000 - $222,100

• Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience

The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including 401(k) eligibility and various paid time off benefits, such as PTO and parental leave) If hired, employees will be in an “at-will position” and the Company reserves the right to modify base salary (as well as any other discretionary payment or compensation program) at any time, including for reasons related to individual performance, Company or individual department/team performance, and market factors