Apache Kafka 4.3, a release powered by 25 new KIPs. Apache Kafka 4.3 includes first-class support for preserving record headers in Kafka Streams
#apache kafka
24 posts
1 Jun
19 May
Enforce encryption & governance centrally with Confluent Private Cloud. Reduce brokers by 73% and secure legacy Kafka clients without code changes or sprawl.
15 Aug 2024
Modern applications have an unceasing buzz of user activity and data flows. Users send a flurry of one-click reactions to social media posts. Wearable tech and other IoT sensors work nonstop to transmit event data from their environments. Meanwhile, customers on e-commerce sites perform shopping cart actions or product searches which can bring immediate impact […] The post Building an…
9 May 2024
One of our most important goals at Heroku is to be boring. Don’t get us wrong, we certainly hope that you’re excited about the Heroku developer experience — as heavy users of Heroku ourselves, we certainly are! But, even more so, we hope that you don’t have to spend all that much time thinking about […] The post Evolving the…
8 Oct 2020
This summer, we announced the beta release of our new streaming data connectors between Heroku Postgres and Apache Kafka on Heroku. These connectors make Change Data Capture (CDC) possible on Heroku with minimal effort. Anyone with a Private or Shield Space, as well as a Postgres and an Apache Kafka add-on in that space, can […] The post Heroku Streaming…
10 Jul 2020
Today we are announcing a beta release of our new streaming data connector between Heroku Postgres and Apache Kafka on Heroku. Heroku runs millions of Postgres services and tens of thousands of Apache Kafka services, and we increasingly see developers choosing to start with Apache Kafka as the foundation of their data architecture. But for […] The post Streaming Data…
11 Jun 2020
We are thrilled to announce that Heroku Shield for Redis is now generally available and certified for handling PHI, PII, and HIPAA-compliant data. Heroku Shield for Redis is the final missing data service for Heroku Shield, which is an integrated set of Heroku services with additional security features needed for building high compliance applications. All […] The post Heroku Shield…
6 May 2020
Security is always top of mind for Heroku customers; COVID-19 has further increased the urgency for enterprises and developers to deliver more mission-critical applications with sensitive and regulated data. Given the needs of our customers, including those in regulated industries like Health & Life Sciences and Financial Services, we are thrilled to announce that Heroku […] The post Bring Your…
7 Nov 2019
Today, we’re thrilled to announce four new trusted data integrations that allow data to flow seamlessly and securely between Heroku and external resources in public clouds and private data centers: Heroku Postgres via mutual TLS Heroku Postgres via PrivateLink Apache Kafka on Heroku via PrivateLink Heroku Redis via PrivateLink These integrations expand Heroku’s security and […] The post Announcing Heroku…
1 Oct 2019
We are thrilled to announce that Apache Kafka on Heroku Shield is now generally available and certified for handling PHI, PII, and HIPAA-compliant data. Our newest managed data service unifies Heroku Shield, a set of Heroku platform services that offer additional security features needed for building high compliance applications, with Apache Kafka on Heroku, our […] The post Apache Kafka…
23 Jul 2019
There are many reasons to choose Heroku Data services, but keeping the services you use secure and up-to-date rank near the top. This foundation of trust is the most important commitment we make to our customers, and frequent and timely maintenances are one way we deliver on this promise. We do everything we can to […] The post Why Frequent…
11 Jul 2019
This blog post is adapted from a talk given by Ali Hamidi at Data Council SF ’19 titled “Operating Multi-Tenant Kafka Services for Developers on Heroku.” Thousands of developers use Heroku’s Apache Kafka service to process millions of transactions on our platform—and many of them do so through our multi-tenant Kafka service. Operating Kafka clusters at this scale requires […]…
14 Jan 2019
This blog post is adapted from a talk given by Stella Cotton at RailsConf 2018 titled “So You’ve Got Yourself a Kafka.” In recent years, designing software as a collection of services, rather than a single, monolithic codebase, has become a popular way to build applications. In this post, we’ll learn the basics of Kafka […] The post Building a…
6 Dec 2018
Building a SaaS product, a system to handle sensor data from an internet-connected thermostat or car, or an e-commerce store often requires handling a large stream of product usage data, or events. Managing event streams lets you view, in near real-time, how users are interacting with your SaaS app or the products on your e-commerce […] The post Manage Real-time…
13 Mar 2018
Editor’s Note: One of the joys of building Heroku is hearing about the exciting applications our customers are crafting. SHIFT Commerce – a platform helping retailers optimize their e-commerce strategy – is a proud and active user of Heroku in building its technology stack. Today, we’re clearing the stage for Ryan Townsend, CTO of SHIFT, […] The post Deconstructing Monolithic…
19 Dec 2017
Designing scalable, fault tolerant, and maintainable stream processing systems is not trivial. The Kafka Streams Java library paired with an Apache Kafka cluster simplifies the amount and complexity of the code you have to write for your stream processing system. Unlike other stream processing systems, Kafka Streams frees you from having to worry about building […] The post Kafka Streams…
14 Sept 2017
Event-driven architectures are on the rise, in response to fast-moving data and constellations of inter-connected systems. In order to support this trend, last year we released Apache Kafka on Heroku – a gracefully integrated, fully managed, and carefully optimized element of Heroku’s platform that is the culmination of years of experience of running many hundreds […] The post Kafka Everywhere:…
11 Jan 2017
At Heroku, we’re always working towards improving operational stability with the services we offer. As we recently launched Apache Kafka on Heroku, we’ve been increasingly focused on hardening Apache Kafka, as well as our automation around it. This particular improvement in stability concerns Kafka’s compacted topics, which we haven’t talked about before. Compacted topics are […] The post Pulling the…
29 Nov 2016
Heroku recently released [a managed Apache Kafka][1] offering. As a Node.js developer, I wanted to demystify Kafka by sharing a simple yet practical use case with the many Node.js developers who are curious how this technology might be useful. At Heroku we use Kafka internally for a number of uses including data pipelines. I thought […] The post Data Pipelines…
28 Sept 2016
Many of the compelling and engaging application experiences we enjoy every day are powered by event-based systems; requesting a ride and watching its progress, communicating with a friend or large group in real time, or connecting our increasingly intelligent devices to our phones and each other. Behind the scenes, similar architectures let developers connect separate […] The post Apache Kafka…
23 Aug 2016
At Heroku, we’re always working towards increased operational stability with the services we offer. As we recently launched the beta of Apache Kafka on Heroku, we’ve been running a number of clusters on behalf of our beta customers. Over the course of the beta, we have thoroughly exercised Kafka through a wide range of cases, […] The post Dawn of…
19 Jul 2016
We recently launched Apache Kafka on Heroku into beta. Just like we do with Heroku Postgres, our internal engineering teams have been using our Kafka service to power a number of our internal systems. The Big Idea The Heroku platform comprises a large number of independent services. Traditionally we’ve used HTTP calls to communicate between […] The post Powering the…
27 May 2016
At Heroku, we’re always striving to provide the best operational experience with the services we offer. As we’ve recently launched Heroku Kafka, we were excited to help out with testing of the latest release of Apache Kafka, version 0.10, which landed earlier this week. While testing Kafka 0.10, we uncovered what seemed like a 33% […] The post Apache Kafka…
26 Apr 2016
Today we are happy to announce early access to Heroku Kafka. We think Kafka is interesting and exciting because it provides a powerful and scalable set of primitives for reasoning about, building, and scaling systems that can handle high volumes and velocities of data. Heroku Kafka makes Kafka more accessible, reliable, and easy to integrate […] The post Announcing Heroku…