Apache NiFi Engineer / Data Integration Engineer
Position Overview
Red Arch Solutions is seeking a highly skilled Apache NiFi Engineer to support enterprise data onboarding, integration, and transformation initiatives within a Zero Trust environment. The successful candidate will design, develop, deploy, and maintain complex Apache NiFi data flows that enable secure, scalable, and reliable movement of mission and enterprise data across disparate systems.
This role requires deep expertise in Apache NiFi architecture, custom processor development, enterprise integration patterns, metadata management, data transformation, and workflow automation. The engineer will work closely with Data Stewards, Enterprise Architects, Zero Trust Core Services teams, Data Catalog personnel, system owners, and mission stakeholders to onboard new data sources and ensure compliance with enterprise data standards and security requirements.
KEY RESPONSIBILITIES
Apache NiFi Development & Engineering Design, develop, test, deploy, and maintain complex Apache NiFi data flows supporting enterprise data onboarding and integration requirements.
Engineer scalable and fault-tolerant ingestion pipelines capable of processing structured, semi-structured, and unstructured data.
Develop reusable flow frameworks and integration patterns to accelerate onboarding activities.
Optimize flow performance, throughput, latency, and resource utilization across clustered environments.
Troubleshoot and resolve complex data movement, transformation, and processing issues.
Advanced Processor Development Develop custom Apache NiFi processors, controller services, reporting tasks, and extensions using Java.
Create specialized processors supporting: Metadata extraction
Schema validation
Data tagging
Attribute enrichment
Security classification enforcement
Data quality validation
API integrations
Workflow automation
Extend NiFi capabilities to support mission-specific onboarding requirements.
Develop shared processor libraries and reusable components.
Enterprise Data Integration Integrate enterprise systems using: REST APIs
SOAP Services
JDBC/ODBC Connections
Message Queues
Kafka
File-Based Transfers
Cloud Storage Services
Design data ingestion and routing strategies across multiple environments.
Support onboarding of mission and enterprise data assets into approved repositories and data services.
Metadata & Data Governance Implement metadata collection, tagging, and enrichment capabilities within data onboarding workflows.
Support enterprise data catalog and governance initiatives.
Ensure onboarding workflows comply with metadata standards, taxonomy requirements, and stewardship processes.
Collaborate with Data Stewards and Data Owners to improve data quality and discoverability.
Zero Trust & Security Integration Implement secure data movement patterns aligned with Zero Trust principles.
Integrate authentication, authorization, encryption, and auditing capabilities into data pipelines.
Coordinate with cybersecurity and Zero Trust teams to ensure compliance with enterprise security requirements.
Support Attribute-Based Access Control (ABAC) and policy-driven data access models.
REQUIRED QUALIFICATIONS
Experience 12+ years experience. Experience working with users to help define their database use, data analytics, and the data user experience.
12+ years of data integration, ETL, or enterprise middleware experience.
8+ years of hands-on Apache NiFi development experience.
Experience designing and supporting large-scale enterprise data ingestion architectures.
Experience onboarding new data sources and integrating complex systems.
Advanced Apache NiFi Skills
Demonstrated expertise with: NiFi Flow Design
Process Groups
Remote Process Groups
Site-to-Site Communications
Controller Services
Parameter Contexts
Flow Versioning
NiFi Registry
Clustered Deployments
Provenance Tracking
Back-Pressure Management
Flow Performance Tuning
Secure Flow Design
Exceptional Processor Development…