Senior Data Engineer
Description
Work on something you believe in. At Discogs, we believe in sharing music with the world. Our technical teams develop a product that opens the doors to music discovery. The Senior Data Engineer will work to drive adoption of data management standards throughout the organization while building and deploying ETL pipelines and data modeling, and databases.
Location
This is a remote position. Open to candidates located in OR, WA, CA, CO, ID, AZ, TX, IL
Who We Are
Discogs is the largest crowd-sourced, community-driven database of recorded music information in the world. Every day, millions of people use the Discogs Marketplace to connect, learn about music, and buy and sell vinyl records, CDs, and cassettes. As Discogs continues to grow, we are looking for bright, dedicated, creative, and highly motivated people to help us realize our mission to serve the music fan in everyone. We are relatively small, so individual contributions can have a large impact. High value is placed on quality, critical thinking, and continuous improvement. Our teams work collaboratively but are distributed geographically and open-source tools are important to who we are and how we work. We value the experiences and skills each team member contributes to helping us serve our music community.
Who We’re Looking For
With a background in data modeling and building robust data pipelines, the Senior Data Engineer will act as the team’s voice of data integrity and reliability and will work to drive adoption of data management standards throughout the organization. This position reports to the Data Engineering Manager.
What You’ll Accomplish
- Makes key contributions to defining the Discogs’ data strategy. Along with planning, writing, and deploying data pipelines, is responsible for unit testing, and championing reliability and security while quickly delivering value to the business.
- Works with other agile delivery team members to design, implement, and optimize data pipelines, data processing, and other data needs.
- As a data team member, leans on using functional programming practices and performs end-to-end data testing.
- Applies understanding of data engineering methodologies, processes, and tools to create a unified Discogs’ experience across a variety of platforms and devices for a global audience.
- Communicates effectively with tech and non-tech members and is capable of delivering feedback to peers and their lead.
- Assists squad with estimation and prioritization of data efforts.
- Actively engages with members across the organization to design and deploy useful data services.
- Coaches and mentors other data engineers by providing recommendations, technical oversight, reviews, and feedback.
- Capable of delivering data pipelines and architecture from technical design to launch with little oversight.
- Collaborates closely with the Business and with other Engineering Teams to quickly deliver high quality products that maximize value to the Discogs ecosystem.
Requirements
Must Have:
- 10+ years of data engineering experience
- Experience building and deploying ETL pipelines and data modeling, and databases
- Experience with orchestration tools (Airflow, Luigi, etc)
- Experience with languages such as SQL and Python
- Thorough understanding of data technologies such as Spark, Kafka, and Kafka Connect
- Familiarity with AWS services such as S3, Redshift, EKS / Kubernetes, and Personalize
- Familiarity with data analysis, warehousing, and visualization
- Thoroughly understands the business vision, mission, and strategy and how their code supports and affects business outcomes.
- Shows initiative and seeks evidence in data to support ideas.
Great to have:
- Experience with handling data lineage, data quality, and data discoverability
- Experience with change data capture (CDC) tools such as Debezium, Hudi, etc
- Strong understanding of data security and privacy
- 2+ years of experience working in Agile/Scrum/Kanban teams
- Experience with business intelligence tools (Tableau, Superset, etc)
Education:
Bachelor's degree from four-year college or university, or equivalent technical work experience
Benefits & Perks
What We Provide
- Competitive compensation: salary, plus performance-related bonus program
- 401(k) with employer match
- 100% company-paid medical and dental insurance benefits for you and your dependents
- 4 weeks paid vacation, increasing based on tenure
- 18 weeks paid leave for birth moms
- 8 weeks paid parental leave, including for adoption
- Monthly wellness allowance
- Annual professional and personal development allowance
- Work from home office set-up and expense allowances
- Flexible work location opportunities
- Employer matching toward charitable contributions
What We Believe In
Discogs’ mission is to serve the music fan in everyone. We represent a diverse and inclusive community, and we are committed to serving our community with innovative and creative solutions. We know that innovation happens best when varying perspectives are embraced and integrated. Our global team reflects our global community.
Discogs is an Equal Opportunity Employer.
Applicants needing accommodation to apply should contact us at 503-597-6340
How to apply
Apply via link
Discogs
Discover more jobs
Discogs is the biggest worldwide music database. Discogs also offers a Marketplace where Buyers and Sellers from all over the world can buy and sell music in all physical formats. Thanks to the accuracy of the database, millions of Internet users use our website to research, buy and sell music, and also to track their music Collection and Wantlist. Discogs is directly supported by a large community of music enthusiasts and collectors.