Databricks Engineer

Databricks Engineer

woodcons.com

Location: Dulles VA, USA

Job Type: Full-Time

Shift: Days

Telework: None

Salary Range: $135,000 to $150,000

** Starting salary is based on minimum education and years of experience and increases based on education and/or experience.

Overview: Seeking a Databricks Engineer to support a mission-critical U.S. Government initiative focused on developing and sustaining a secure network operations environment. This role will lead the migration of legacy applications, services, and platforms to a modern Medallion Model architecture using Databricks, helping shape the future of the organization's data infrastructure.

Application Process: Interested candidates should submit their resume detailing their qualifications and experience.

Security Clearance Requirements:

This position requires all candidates to be U.S. Citizens and possess an active TS/SCI Security Clearance with a Polygraph.

Migrate services, applications, and platforms from legacy systems to Databricks.
• Design and execute migration plans, including validation and optimization.
• Transition legacy data pipelines from NiFi to Databricks.
• Implement the Medallion Model (Bronze Silver Gold) for data assets.
• Develop SOPs for integrating data assets into Databricks with a focus on performance and instrumentation.
• Optimize development, testing, monitoring, and security for data assets.
• Automate and enhance data ingestion and pipeline orchestration using scripting and programming languages.
• Leverage AWS CloudWatch and similar tools for pipeline performance monitoring.
• Ensure team SOPs are documented and communicated.
• Maintain high standards of technical accuracy, timeliness, and quality.
• Communicate effectively with all levels of management and government stakeholders.

Required Education & Experience

• Bachelor's degree in Software Engineering, Computer Science, or a related discipline.
  [Ten (10) years of experience (for a total of eighteen (18) or more years) may be substituted for a degree.]
• 8+ years of software engineering experience.

Required Skills

• 5+ years of data engineering experience in a cloud environment.
• Experience with:
  • Relational, NoSQL, and file-based storage (e.g., Databricks, ElasticSearch, PostgreSQL, S3, Athena)
  • CI/CD pipeline environments
  • Databricks, Cloud RDS, NiFi, AWS Redshift, and Elasticsearch

Desired Skills

• Experience with:
  • Databricks workflows, Unity Catalog, Autoloader, Delta Live Tables/Delta Lake
  • Databricks Workspace/Notebooks, MLflow, Apache Spark
  • Collaboration tools: MS Teams, Outlook, SharePoint, Confluence
• AWS Professional Certification or equivalent
• Strong problem-solving and communication skills
• Familiarity with:
  • CISA: Securing the Software Supply Chain
  • CISA: Cybersecurity Best Practices
  • CISA: Open-Source Software Security
  • NIST SP 800-218: Secure Software Development Framework V1.1

Fringe Benefits:

• Health Insurance: Comprehensive medical, dental, and vision plans.
• Retirement Plan: 401(k) with company match.
• Paid Time Off: Generous PTO policy including vacation, sick leave, and holidays.
• Professional Development: Opportunities for training, certifications, and career advancement.
• Work-Life Balance: Flexible work schedules and remote work options.
• Wellness Programs: Employee assistance programs, wellness initiatives, and gym membership discounts.

Why Join Us?

• Impactful Work: Contribute to critical government projects that make a difference.
• Career Growth: Take advantage of professional development opportunities and career advancement.
• Supportive Environment: Work in a collaborative and flexible environment that values work-life balance.
• Competitive Compensation: Enjoy a competitive salary and comprehensive benefits package.