Data Engineer Job at Bespoke Technologies, Chantilly, Loudoun County, VA

WEY5R01uMFc2SHllOFhPRGZ3OXBQbmJPSUE9PQ==
  • Bespoke Technologies
  • Chantilly, Loudoun County, VA

Job Description


BT-66 – Data Engineer
Skill Level: Senior
Location: Herndon/Chantilly (fully on-site, no remote option)

** MUST HAVE A POLY CLEARANCE TO APPLY. Those who do not have a Poly clearance will not be considered.**
Develop new tools, code, and services to execute data engineering activities involving data of varying types and in varying conditions. Activities include the following tasks: Movement of structure and unstructured data using approved methods. Execute data ingestion activities for storing data in a local or enterprise level location. Develop code to format data that supports exploration. Analyze source data formats and work with Data Scientists and partners to determine the formats and transforms that best meet objectives. Develop code and tools to provide one-time and on-going data extraction from various repositories, formatting and transformations into enterprise or standalone data models. Develop new ETL and perform O&M and enhancements on existing ETL code using best practices/standards. Develop and deliver documentation for each project including ETL mappings, code use guide, code location and access instructions.
  • Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers
  • Ensure the pedigree and provenance of the data is maintained such that the access to data is protected
  • Clean and preprocess data to enable access for advanced analytics
  • Collaborate with enterprise working groups to advance the state of data standards
  • Collaborate with the engineering team, data stewards, and partners to aid in getting actionable value out of the data holdings
  • Collaborate with software engineers to update, configure, and maintain data services based on the requirements
  • Ensure data quality by working with the testing and data quality team to enhance standardization of data conditioning pipelines
  • Experience adapting to various types and formats of data, and working with development teams to integrate new data processing platforms
Required Skills:
10+ years' experience with:
  • Data lifecycle engineering
  • Development and maintenance of extract, transform and load (ETL) tools and services
  • Cloud and on-prem data storage and processing solutions
  • Python, SQL, Spark and other data engineering programming
  • COTS and open source data engineering tools such as ElasticSearch and NiFi
  • Processing data within the Agile Lifecycle

Job Tags

Full time, Local area,

Similar Jobs

Medisys Health Network, Inc.

Epic Willow Analyst Job at Medisys Health Network, Inc.

 ...Job Description EPIC WILLOW ANALYST (Hybrid with 2 remote days after 90 days from hire date) JOB SUMMARY: The candidate will hold a dual role within Medisys. They will equally split their support/build assignments between the Willow application, and another EPIC... 

KLAMATH BASIN BEHAVIORAL HEALTH

Peer Support Specialist Job at KLAMATH BASIN BEHAVIORAL HEALTH

Job Description Job Description Job Title: Peer Support Specialist Department: Corrections Shift: Day shift M-F; One monthly Saturday Full time Intro and job overview: Join our KBBH team and find meaningful work, inspiring colleagues, career growth...

Manhattan Psychiatric Center

Training Specialist (New York) Job at Manhattan Psychiatric Center

 ...Recruiting Training Specialist 1 for Manhattan Psychiatric Center Manhattan Psychiatric Center (MPC), an Office of Mental Health (OMH) Joint Commission-accredited facility, is recruiting a Training Specialist 1 (Mental Hygiene) to join its Training and Education Department... 

Lockheed Martin

Sr. Linux/Windows Systems Administrator - Top Secret Clearance Job at Lockheed Martin

 ...configuration Effective oral and written communication skills Ability to work effectively in a rapid paced, team environment CompTIA Security+ Certification (current - will be required within 6 months of start) CompTIA Linux+ or Red Hat Certified Systems... 

DK Security

Armed Security Officer Job at DK Security

 ...Overview DK Security is Michigans fastest growing security company. Established in 1995, we began as an investigations firm and quickly...  ...*Military veterans are encouraged to apply* SCOPE: Armed Security Officers are responsible for protecting the sites building...