Search overlay panel for performing site-wide searches

Boost Performance & Scale with Postgres Advanced. Join Pilot Now!

Apache Kafka

Building an Event-Driven Architecture with Managed Data Services

Modern applications have an unceasing buzz of user activity and data flows. Users send a flurry of one-click reactions to social media posts. Wearable tech and other IoT sensors work nonstop to transmit event data from their environments. Meanwhile, customers on e-commerce sites perform shopping cart actions or product searches which can bring immediate impact to operations. Today’s software organizations need the ability to process and respond to this rich stream of real-time data.

That’s why they adopt an event-driven architecture (EDA) for their applications.

Evolving the Backend Storage for Platform Metrics

One of our most important goals at Heroku is to be boring. Don’t get us wrong, we certainly hope that you’re excited about the Heroku developer experience — as heavy users of Heroku ourselves, we certainly are! But, even more so, we hope that you don’t have to spend all that much time thinking about Heroku. We want you to be able to spend your time thinking about the awesome, mission-critical things you’re building with Heroku, rather than worrying about the security, reliability, or performance of the underlying infrastructure they run on.

Heroku Streaming Data Connectors Are Now Generally Available

This summer, we announced the beta release of our new streaming data connectors between Heroku Postgres and Apache Kafka on Heroku. These connectors make Change Data Capture (CDC) possible on Heroku with minimal effort. Anyone with a Private or Shield Space, as well as a Postgres and an Apache Kafka add-on in that space, can use Streaming Data Connectors today at no additional charge.

Customers use connectors to build streaming data pipelines between …

Streaming Data Connectors Beta: Capture Postgres Changes in Apache Kafka on Heroku

Today we are announcing a beta release of our new streaming data connector between Heroku Postgres and Apache Kafka on Heroku. Heroku runs millions of Postgres services and tens of thousands of Apache Kafka services, and we increasingly see developers choosing to start with Apache Kafka as the foundation of their data architecture. But for those who are Postgres-first, it is challenging to adopt without a full app rewrite. Developers want a seamless integration …

Heroku Shield for Redis Is Now Generally Available

We are thrilled to announce that Heroku Shield for Redis is now generally available and certified for handling PHI, PII, and HIPAA-compliant data. Heroku Shield for Redis is the final missing data service for Heroku Shield, which is an integrated set of Heroku services with additional security features needed for building high compliance applications. All Heroku Managed Data Services — Heroku Connect, Heroku Data for Redis, Heroku Postgres, and Apache Kafka on Heroku — are …

Bring Your Own Key for Heroku Managed Data Services Is Now Generally Available

Security is always top of mind for Heroku customers; COVID-19 has further increased the urgency for enterprises and developers to deliver more mission-critical applications with sensitive and regulated data.

Given the needs of our customers, including those in regulated industries like Health & Life Sciences and Financial Services, we are thrilled to announce that Heroku Private Spaces and Shield customers can now deploy a new Postgres, Redis, or Apache Kafka service with a key created …

Announcing Heroku Data Services Integrations Using mutual TLS and PrivateLink

Today, we’re thrilled to announce four new trusted data integrations that allow data to flow seamlessly and securely between Heroku and external resources in public clouds and private data centers:

  • Heroku Postgres via mutual TLS
  • Heroku Postgres via PrivateLink
  • Apache Kafka on Heroku via PrivateLink
  • Heroku Redis via PrivateLink

These integrations expand Heroku's security and trust boundary to cover the connections to external resources and the data that passes through them. They enable true multi-cloud …

Apache Kafka on Heroku Shield is Now Generally Available

We are thrilled to announce that Apache Kafka on Heroku Shield is now generally available and certified for handling PHI, PII, and HIPAA-compliant data. Our newest managed data service unifies Heroku Shield, a set of Heroku platform services that offer additional security features needed for building high compliance applications, with Apache Kafka on Heroku, our fully-managed service based on the leading open-source solution for handling event streams.

Organizations of all sizes face relentless pressure to …

Why Frequent Maintenances Are Essential for Secure Heroku Data Services

There are many reasons to choose Heroku Data services, but keeping the services you use secure and up-to-date rank near the top. This foundation of trust is the most important commitment we make to our customers, and frequent and timely maintenances are one way we deliver on this promise.

We do everything we can to minimize downtime, which is typically between 10 – 60 seconds per maintenance. There are ways for you to minimize disruption too (see the tips and tricks below). The rest of the post explains how we think about Heroku Data maintenances, how we perform them, and when we perform them.

An Ounce of Prevention…

Hackers exploit known but unpatched vulnerabilities or out-of-date software. Minimizing the time between when a patch or update becomes available and when it gets deployed is the most effective means of limiting damage. There’s nothing worse than seeing your company’s high-profile breach at the top of Hacker News and the Wall Street Journal.

This business and reputation risk is real. Like you, we’re faced with the same choice. We believe it’s best to budget some prevention time upfront for patching and updating data services. Otherwise, an incident may cost us (and you) a larger amount of remediation time and effort, to say nothing of the potential damage done to our (and your) brand, business, and customers.

That’s why we invest significant engineering, security, and operations effort into creating a proactive security posture that keeps your stack up-to-date through frequent, scheduled maintenances.

How Heroku Operates its Multi-Tenant Apache Kafka Services

This blog post is adapted from a talk given by Ali Hamidi at Data Council SF ’19 titled “Operating Multi-Tenant Kafka Services for Developers on Heroku.”

Thousands of developers use Heroku’s Apache Kafka service to process millions of transactions on our platform—and many of them do so through our multi-tenant Kafka service. Operating Kafka clusters at this scale requires careful planning to ensure capacity and uptime across a wide range of customer use …

Subscribe to the full-text RSS feed for Apache Kafka.