Job Type
Contract Full-Time

Job Description

BCTG's direct client is looking for a Kafka & Migration Developer to help with an ongoing migration from Splunk to Grafana.

Core Skillets Required include:

  • Kafka: Expertise in Kafka architecture, producers, consumers, topics, partitions, and Kafka Streams.
  • Grafana: Experience building and managing dashboards, panel customization, and alerting in Grafana.
  • Splunk: Strong knowledge of Splunk for log ingestion, querying (SPL), and dashboard creation.
  • Terraform: Infrastructure as Code (IaC) for provisioning and managing Kafka clusters, Grafana, and other resources.
  • Python: Proficient in Python scripting for automation, data processing, and integration tasks.

Recommended Additional Skills & Tools:

  • Kafka Ecosystem: Kafka Connect, Kafka Streams or ksqlDB, Schema Registry, Confluent Platform
  • Observability & Monitoring: Prometheus (metrics collection), OpenTelemetry (distributed tracing), ElasticSearch/Logstash/Beats (log processing), Alertmanager, Grafana Alerting
  • Cloud & Infrastructure: Kubernetes (deployment & orchestration), Helm (package management), AWS/Azure/GCP cloud services related to Kafka and monitoring
  • Dashboard & Visualization: Grafana Loki (log aggregation), JSON/YAML for dashboard templating, API usage for automation
  • Data Processing & Scripting: Advanced Python libraries (e.g., Pandas), Bash/PowerShell scripting for automation
  • Security & Compliance: Kafka security (SSL, SASL, ACLs), securing data pipelines and observability infrastructure
  • Version Control & CI/CD: Git workflows, Jenkins/GitLab CI/GitHub Actions for automated deployment pipelines
  • Performance & Troubleshooting: Kafka performance tuning, troubleshooting, Grafana dashboard optimization

Additional Notes:

  • Experience with containerization and orchestration (Docker, Kubernetes) is highly valuable for deploying scalable Kafka and Grafana environments.
  • Familiarity with monitoring frameworks and distributed tracing will improve observability during migration.
  • Strong automation skills help reduce manual effort in dashboard migration and infrastructure provisioning.
  • Understanding security best practices ensures data integrity and access control across Kafka and observability stacks.
Apply With