Senior Staff Software Engineer, Data Infrastructure - Slack
To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.
Job CategorySoftware Engineering
We’re Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too — driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good – you’ve come to the right place.
Slack is looking for an expert Engineer to join us in Data Engineering. You will conceptualize, design, build and integrate scalable systems, platforms and tools that power the Data Warehouse. These systems help our partners implement, deploy and analyze data assets with a high level of autonomy and limited friction. You will play a meaningful role in scaling our data infrastructure as well as making our user experience interacting with data more efficient, cheerful and productive.
As Slack’s data grows, the goal of the Data Infrastructure team is to provide the best blend of the tools and technologies that will strengthen the efficiency and dependability of the way we make decisions. Our Data Warehouse powers processing of our large and diverse datasets to allow our partners to focus on their strengths: data applications, data modeling, data analysis, search or machine learning.
Data Infrastructure at Slack
Slack's data infrastructure operates at tremendous scale with systems that process billions of records daily and millions of events per second. Our teams build and maintain the distributed systems including but not limited to Query and Compute engines, Scheduler, Authorization framework for data, Core infrastructure and Orchestration services for DW.
This data infrastructure is 65PB+ of storage, processes ~900B records a day, 400+ ETL pipelines, 900+ Active DAGs, a growing community of 100+ developers, one of the largest Presto deployments in the industry. We recommend reading some of our blog posts that shed some light into what we do here at Slack.
Slack's data warehouse powers product features like Enterprise Analytics and Search. We support the ML, Reporting & Analytics for Marketing, Sales and Finance aiding them in making data-informed decisions.
Our infrastructure is built on EMR, Hadoop, Hive, Spark, Flink, Hudi, Apache Ranger, Sqoop, Airflow, Presto, Kafka, Secor and Aurora/ MySQL/ Vitess. Our services are developed using Python, Java and Golang and deployed on AWS.
What you will be doing:
- Help envision, guide and implement the generation Data Warehouse at Slack solving for 10x scale.
- Comfortably communicate sophisticated issues to a diverse audience. Inspire and lead technical discussions that uncover new problems or novel solutions. These discussions further help drive decision making within your organization.
- Collaborate with peers in Engineering to identify and surface fundamental issues in the system, providing strategy and direction for major functional areas.
- Advise engineers to triage bugs, troubleshoot sophisticated production issues and provide a fix across the stack related to our services.
- Be a role model and a multiplier, coaching and mentoring engineers.
- Define and articulate the guiding principles for strategic development of methods, techniques, and evaluation criteria for projects and programs. This will include assessment of build vs buy decisions at every stage, backed by proof of concepts, benchmarking, etc.
- Own the software lifecycle all the way from conceptualizing high level and complex requirements to design, build and ship with informative and forward-looking approaches.
Provide visibility into the health of our systems and infrastructure with a comprehensive view of data flow, resources usage, data lineage, etc.
- Evolve maturity of our monitoring systems and processes to improve visibility and failures detection in our infrastructure.
- Write, review, or provide feedback on technical design proposals.
What you should have:
- 10+ years of relevant technical experience, including significant experience with Data Warehouse technologies building pipelines, working with ingestion frameworks and building & maintaining large scale data warehouse systems.
- You have operated large distributed systems at low cost while at the same time worked on solving Scalability, Performance, Reliability challenges.
- You have a strong dedication to code quality, automation and operational excellence: unit/integration tests, scripts, workflows.
- You are a strong communicator. Explaining sophisticated technical concepts to designers, support, and other engineers is no problem for you.
In-depth knowledge of not only the engineering functions but in general data technologies that power analytics but also the business strategies and company goals.
- Good knowledge of the production technologies we use including AWS (CloudFormation, EC2), Docker, Terraform, Kubernetes, Chef.
- A Bachelor's degree in Computer Science, Engineering or a related field, or equivalent training, fellowship, or work experience.
Slack has a positive, diverse, and supportive culture—we look for people who are curious, inventive, and work to be a little better every single day. In our work together we seek to be smart, humble, hardworking and, above all, collaborative.
If you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form.
At Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com.
Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce.
Salesforce welcomes all.For Colorado-based roles, the base salary hiring range for this position is $194,400 to $296,600.Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for incentive compensation, equity, benefits. More details about our company benefits can be found at the following link: https://www.salesforcebenefits.com.