Senior Data Engineer

Description

Job Description:

The Leidos Innovations Center has an exciting opening for you, our next Senior Data Engineer, to play key role with the design, engineering, development and deployment of the geospatial data layer for the DOMEX Data Discovery Platform (D3P). The DOMEX Data Discovery Platform (D3P) program is a next generation machine learning pipeline platform that provides cutting edge data enrichment, triage, and analytics capabilities to Defense and Intelligence Community members. Our Senior Data Engineer will lead the requirements, design and development of a DOMEX geospatial data layer as well as collaborate as part of a cross-functional Agile team to create and enhance data ingestion pipelines and addressing Big Data challenges. You can work in our Bethesda, MD or Reston, VA office.

Exiting things you will be doing on the job:
• Provide Extraction, Transformation, and Load (ETL) experience coupled with enterprise search capabilities to solve Big Data challenges
• Perform data profiling of new data sources to discover metadata and identify anomalies
• Design and implement high-volume data ingestion and streaming pipelines using Open-Source frameworks like Apache Spark, Flink, NiFi, and Kafka on AWS Cloud
• Collaborate and work with a diverse group of data scientists and engineers in developing solutions for ingesting heterogeneous datasets with geospatial enrichment
• Design, develop, test, maintain, and support data pipelines for new geospatial data layer
• Help drive geospatial data pipeline technology evaluation and proof of concept
• Collaborate with Data Science team to analyze the output of geospatial data layer
• Leverage strategic and analytical skills to understand and solve customer and business centric questions
• Assist with definition and implementation of standards / best practices for database & pipeline development
• Monitor and troubleshoot performance issues on the enterprise data pipelines and the data lake

This is you:
• Bachelor’s Degree with 8 years of relevant experience or Master’s Degree with 6 years of relevant experience or 4 additional years of experience in lieu of degree
• Possess a DoD TS security clearance and be able to obtain a DoD TS/SCI with CI Poly
• Fluent with Python and preferably one other development language
• Experience performing ETL processes, designing data infrastructure and optimizing performance
• Experience with database administration such as PostgreSQL and SQL
• Experience with geospatial data processing and enrichment
• Significant experience working in a Linux environment
• Experience with building microservices in Python
• Experience with big-data tools, e.g., Hadoop, Spark, Kafka and NiFi
• Experience transforming data in various formats, including JSON, XML, CSV, and zipped files
• Good interpersonal and communication skills necessary to work effectively with customers and other team members.

You will wow us even more if you have these skills:
• Expertise in data profiling techniques and understanding the content from both a data quality and business perspective
• Working knowledge on containers and Kubernetes (K8S)
• Familiarity with Geospatial analysis tools including but not limited to QGIS and/or ArcGIS or R
• Familiarity with Arcpy and Geopandas
• Cloud geospatial data processing / pipelines
• Experience with Elasticsearch
• Experience with Agile software development (Scrum, SAFe)

LInC
D3P

Pay Range: