Job Description
Primary Responsibilities:
- Integration Solutions: Develop and implement integration solutions for the GAINS project using Kafka and Elastic as the primary data architecture platforms, with expanded integration to other technologies, including but not limited to Databricks.
- Data Integration: Integrate data sources into Confluent (Kafka), and Elastic and Databricks platforms. Develop Kafka system integrations between Elasticsearch/Logstash and other systems.
- Kafka Integration & Development: Develop Kafka system integrations, custom connectors, and work with ksqlDB and Kafka Streams for data processing based on the design solution.
Kafka Cluster Management: Deploy and manage Kafka clusters on Kubernetes in multi-site environments (both on-premise and cloud).
- Software Lifecycle Automation: Automate the full software lifecycle, from design and development to testing and deploym...