Databricks Specialist

Job overview

The Databricks Specialist role is onsite in Washington, DC. Annual compensation is $110,000–$115,000. The role supports a federal project by developing and optimizing data infrastructure for mission-critical environments. The specialist builds scalable data pipelines and Spark applications, administers Databricks and AWS resources, and integrates security and data-quality controls. This work supports reliable, secure data processing and delivery in federal environments.

What you'll do

• Develop and optimize Spark applications in AWS Databricks using Python, PySpark, and SQL
• build and maintain ETL/ELT pipelines using Medallion Architecture
• administer Databricks workspaces, clusters, jobs, access controls, Unity Catalog, and notebooks
• manage AWS services and provision infrastructure with Terraform
• optimize and troubleshoot data processing
• integrate data from APIs and databases
• support security and compliance controls in CI/CD pipelines
• implement data quality checks and lineage tracking
• and collaborate with Agile/Scrum teams. Requires at least three years of hands-on Databricks development, relevant production experience, a bachelor's degree, and active or previous High-Risk Public Trust (Tier 4) clearance.

What we're looking for

Skills & competencies

databricks specialist

washington dc

red cedar consultancy

aws databricks

pyspark

apache spark

python

sql

delta lake

etl/elt

terraform

high-risk public trust

Work arrangement

• Weekend coverage required

Why this role

Federal project work includes FISMA/FedRAMP requirements, ATO reassessments, and POA&M resolution; production on-call support is part of the role.

About the employer

Red Cedar Consultancy is hiring for this role. Industry: Administrative Management and General Management Consulting Services. Sector: 54.