New

Manager Data Engineering (Big Data & Databricks)

Norfolk Southern
United States, Georgia, Atlanta
1200 Peachtree Street Northeast (Show on map)
Aug 19, 2025
Requisition 39058: B5 Manager Data Engineering A resume helps you stand out to hiring managers and recruiters; your resume communicates your experience and your brand. While it is not required, we encourage you to include an up-to-date resume along with a completed job application to give you the best opportunity to be considered. A complete resume helps us to better understand your unique background, relevant experiences, and passions. We look forward to learning about you. Norfolk Southern offers a unique opportunity to be part of our proud legacy that spans nearly 200 years. We are a customer-centric, operations-driven team dedicated to advancing safety, serving communities, and driving innovation for tomorrow's rail.As part of Norfolk Southern, you'll join a collaborative team where there are opportunities for growth across the organization. We are building a culture where everyone can thrive by owning and driving exceptional results, being humble and leading with trust, serving our customers with excellence, and collaborating and coaching to win. Job Description The Manager Data Engineering will lead a team responsible for building various cutting-edge Big Data solutions and Big Data pipelines in collaboration with the Data Science, Business Intelligence (BI), software development, engineering, and business teams. They will be focused on driving part of the strategic horizontal Big Data initiatives that advance the skills and capabilities of the organization, while improving the level of enablement for partners and stakeholders. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions, perform data ingestion development, support duties, coach and mentor Data Engineers, and set the data vision for the organization. Responsibilities: * Develop the vision and manage the execution of the Data Engineering roadmap by collaborating with cross-functional teams, products, analytics, engineering, and machine learning experts. * Oversee various parts of the data lifecycle from acquirement to insights, in particular: data ingestion, data warehouse, Data Lakehouse, and data engineering. * Hire, coach, and mentor talented Data Engineers. * Lead Data Engineering team to use state-of-the-art Big Data tools and technologies to build scalable data architecture and performant data pipelines to acquire high-velocity real-time and batch data of different sizes and scales. * Design, implement, and support scalable data infrastructure solutions to integrate with multi-heterogeneous data sources, aggregate and retrieve Big Data in a fast and safe mode, curate data that can be used in BI reporting, analysis, machine learning models and ad-hoc data requests. * Ensure appropriate levels of governance and testing procedures for developed solutions. * Stay up to date with technology, trends, and tools in Big Data. * Take part in development of data pipelines and solutions. * Optimize and tune data ingestion and data retrieval pipelines. * Implement the solutions in production environment and implement efficient and timely support models. * Manage multiple projects concurrently. Education Required Bachelor's Degree, preferably in Information Systems, Computer Science, Computer Information Systems or related field. Skills Required * Experience as a People Leader, directly managing technical teams, including hiring, developing, motivating, and directing people as they work. * 5 years of experience in Big Data, BI, and Advance Analytics. * Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python. * Advanced knowledge of Big Data, Data Engineering, and BI with hands-on and technical experience in tools/technologies like Apache Kafka, Spark, Hadoop, Spotfire, Tableau, PowerBI, etc. * 5 years of experience in building data ingestion pipelines using: python/PySpark and/or Spark-Streaming, etc. * Experience with relational databases e.g. Teradata, Oracle, SQL Server, as well as NoSQL databases including HBase and Cassandra. Skills Desired * Experience with data engineering productivity toolchains and concepts (CI/CD, DevOps, GitHub, Jira, Confluence, Agile Scrum / Kanban, etc.) * Hands-on coding experience in at least one modern programming language: Python, Scala, etc. Strong SQL and Python skills, including knowledge of Python libraries/frameworks. * Knowledge of Unix/Linux platform and shell scripting. * Strong project management skills, including work prioritization, planning, and task delegation. * Ability to effectively communicate (both written and verbally) technical information at all levels of the organization. * Experience in successfully building and deploying a new data platform on Azure/ AWS. * Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue. * Strong knowledge of Messaging Platforms like Confluent Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series. * Lake with Spark SQL, Delta Live Tables, Unity Catalog. * Experience with Data governance tools like Collibra, etc. Work Conditions Location: Atlanta, GA Environment: Hybrid (three days in office; two days remote) Shift Work: No On-Call: Yes Weekend Work: No Company Overview Since 1827, Norfolk Southern Corporation (NYSE: NSC) and its predecessor companies have safely moved the goods and materials that drive the U.S. economy. Today, it operates a customer-centric and operations-driven freight transportation network. Committed to furthering sustainability, Norfolk Southern helps its customers avoid 15 million tons of yearly carbon emissions by shipping via rail. Its dedicated team members deliver more than 7 million carloads annually, from agriculture to consumer goods, and is the largest rail shipper of auto products and metals in North America. Norfolk Southern also has the most extensive intermodal network in the eastern U.S., serving a majority of the country's population and manufacturing base, with connections to every major container port on the Atlantic coast as well as the Gulf of Mexico and Great Lakes. Learn more by visiting www.NorfolkSouthern.com. At Norfolk Southern, we believe in celebrating our individuality. By leveraging the unique backgrounds and viewpoints of our employees, we can create a culture of innovation, respect, and inclusion. We know that employees thrive in a workplace where differing viewpoints, ideas, and experiences are freely shared and valued. As such, we encourage all employees to contribute their distinctive skills and capabilities to our organization. Equal employment opportunities are available to all applicants regardless of race, color, religion, age, sex, national origin, disability status, genetic information, veteran status, sexual orientation, and gender identity. Together, we power progress.