NIRA, Inc. logo

Senior Data Engineer

NIRA, Inc.
1 day ago
Full-time
On-site
Washington, District of Columbia, United States

NIRA is seeking a highly experienced Senior Data Engineer to support the U.S. Department of Justice Digital Evidence Review Platform program. The Data Engineer will design, develop, test, operate, and optimize high-volume data ingestion and processing pipelines for digital evidence and associated metadata. The platform must process large and diverse datasets while preserving integrity, provenance, auditability, access controls, and chain-of-custody information throughout the evidence lifecycle.

Key Responsibilities

  • Design, develop, test, operate, and optimize high-volume data ingestion, transformation, integration, and processing pipelines.
  • Ingest, normalize, enrich, validate, store, search, export, archive, and restore structured and unstructured data.
  • Process sources such as forensic extraction packages, documents, messages, multimedia, call records, geolocation data, cloud-provider returns, metadata, and outputs from Government-furnished parsers and external systems.
  • Implement validation, hashing, error handling, retry, reconciliation, reprocessing, and data-quality controls.
  • Preserve source identifiers, processing history, lineage, provenance, and chain-of-custody events.
  • Support OCR, transcription, translation, deduplication, metadata extraction, entity extraction, and enrichment workflows.
  • Develop schemas, canonical data models, metadata structures, and open, portable export formats.
  • Support search, analytics, and AI/ML-enabled processing while maintaining human oversight and auditability.
  • Automate deployment and testing through CI/CD and configuration-management practices.
  • Tune pipeline performance and troubleshoot failures across development, test, UAT, production, and disaster-recovery environments.
  • Maintain technical documentation and collaborate with architects, software engineers, cloud integrators, cybersecurity personnel, and product specialists.

Minimum Qualifications

  • Bachelor's degree in engineering, mathematics, or science.
  • At least 15 years of relevant data engineering, data integration, data architecture, software engineering, or large-scale data-processing experience.
  • A master's degree may substitute for two years of required experience; a PhD may substitute for four years.
  • Extensive experience designing, developing, and maintaining enterprise data ingestion, transformation, integration, and processing pipelines.
  • Experience processing structured and unstructured data from multiple sources and formats.
  • Experience with data modeling, metadata, schema design, data lineage, data quality, error handling, reconciliation, and audit logging.
  • Experience developing in Python, Java, Scala, SQL, or comparable data-engineering languages.
  • Experience with cloud-based data platforms, distributed processing, object storage, databases, search platforms, or analytics environments.
  • Experience supporting CI/CD, automated testing, configuration management, performance tuning, and production troubleshooting.
  • S. citizenship and ability to meet DOJ residency and personnel-security requirements.
  • Ability to obtain and maintain a High-Risk Public Trust investigation and PIV credential.

Preferred Qualifications

  • Experience with digital evidence, forensic data, eDiscovery, law-enforcement data, case-management data, or other high-integrity federal datasets.
  • Experience with AWS GovCloud, Azure Government, FedRAMP-authorized SaaS, or hybrid federal cloud environments.
  • Experience processing PDFs, text, messages, multimedia, call records, geolocation data, forensic extraction files, or large document collections.
  • Experience with OCR, natural-language processing, entity extraction, transcription, translation, deduplication, AI/ML enrichment, or LLM/RAG pipelines.
  • Experience with Python-based data frameworks, Spark, Databricks, Kafka, Airflow, cloud-native data services, or comparable technologies.
  • Experience implementing hashing, immutable audit records, provenance, lineage, retention, archival, and data-restoration controls.
  • Experience designing open-format exports, APIs, canonical schemas, or data-exchange interfaces that reduce vendor lock-in.