Announcing the Custom Destination Toolkit: Build Your Own Destination in Minutes

Our goal is simple yet ambitious: give customers the tools they need to build their own integrations without sacrificing reliability, observability, or security. Data teams should not feel limited by their vendors’ roadmaps and priorities. Do you need to sync data into legacy APIs or niche, industry-specific software? Just pick the right tool for the job and get it done.

HTTP requests

Our custom destination toolkit exposes the building blocks of software-to-software communication, so where else could we begin but with the venerable HTTP request?

Most data activation tools offer some basic HTTP functionality, usually as a simple “webhook” destination. We quickly learned with our customers that these webhook connectors—ours included—lacked the flexibility required to interface with many popular APIs. Rate limits, retries, dynamic URLs? Never heard of ‘em. Support for nested objects and arrays was also poor or nonexistent. In fact, the most common use case for our webhook destination involved making requests to internal services that would modify the requests before proxying them to third-party services. 🥴

One customer gave it to me straight: “Right now, I need to redesign my API to accept Hightouch webhooks. I want it to work the other way around. Why can’t I configure my Hightouch webhooks to interface with existing APIs?”

That question presented an irresistible challenge for our engineering team, leading to today’s announcement.

Our all-new HTTP Request destination is—by far—the most powerful and flexible HTTP connector on the market. Using our no-code visual interface, you can exercise granular control over:

• Request triggers, like rows added, changed, and/or deleted
• Methods, such as POST, PUT, PATCH, and DELETE
• Dynamic endpoints, including templated URLs with variable injection
• Dynamic payloads, supporting JSON objects, XML documents, and URL encoded forms
• Rate limits, up to 10,000 requests per second
• Concurrency limits, up to 1,000 requests in parallel
• Timeout thresholds, up to 60 seconds per request
• Custom error handling logic, including retries across multiple sync runs
• And much more…

You can compose nested JSON objects and arrays, or create XML documents using our new Liquid-based templating engine. Our payload builder supports variable injection, control flow, and loops. Gone are the old days of building complex payloads in SQL.

Check out the docs to see it in action!

Simplified mock-up of the HTTP Request destination

Serverless functions

If you’re still itching to write code, I’m delighted to share that Hightouch now integrates directly with AWS Lambda, Google Cloud Functions, and Azure Functions.

These destinations make it possible to build your own custom connector by invoking a serverless function whenever rows are added, changed, or removed in your data model. Here’s why you might want to do that:

• Sometimes, it’s easier to transform and filter data using code instead of SQL. With serverless functions, you can use your favorite language to get the job done fast.
• Client libraries help you interact with complex APIs. (No need to read—and reread—confusing specs to understand HTTP endpoints.)
• Your functions can also enrich warehouse data using external sources like Clearbit and ZoomInfo.

Serverless functions offer the ideal environment to execute custom code during a sync. Hightouch now integrates natively with the three major cloud providers: AWS, Google Cloud, and Azure. All of our serverless function destinations offer first-class support for custom triggers, batching rules, rate limits, concurrency limits, and error handling. What more could you ask for?

Simplified mock-up of a serverless function destination

Message queues

To support high-throughput, distributed, and asynchronous workloads, Hightouch can now publish messages into Apache Kafka, Google Cloud Pub/Sub, RabbitMQ, Amazon SQS, Amazon SNS, Amazon EventBridge, and Amazon Kinesis.

Message queues might be “dumb pipes,” but our integrations are anything but basic. With the Kafka destination, for example, you can connect to multiple brokers, authenticate with SASL, bring your own certificate authority, publish into different topics for each message trigger, and define custom ordering and partition keys. Hightouch supports all managed Kafka services (Amazon MSK, Confluent Cloud, etc.) and can also connect to self-hosted instances. That covers only one of the seven message queue destinations we're releasing today. 🤯

Simplified mock-up of a message queue destination

Transactional databases

Hightouch can also write into transactional databases like PostgreSQL, MySQL, SQL Server, MongoDB, Firestore, Rockset, CockroachDB, and many others. Today, we’re shipping a bunch of optimizations to make these syncs up to 20 times faster, so you can power internal tools and in-app experiences using the freshest data from your warehouse.

Internally at Hightouch, we built our customer-facing billing portal by syncing product usage data from Snowflake into Postgres. If you’re curious about other use cases, we’ve written extensively about combining the analytical power of the data warehouse with the low-latency performance of transactional databases.

Simplified mock-up of a transactional database destination

Customizing the user experience

Our embedded destination framework helps you build a custom connector that looks and feels just like a native destination. It offers programmatic control over the entire sync configuration UI/UX, including questions, answers, and validation rules.

Embedded destinations are meant for data teams tasked with building connectors for less technical users. By customizing the form’s appearance, developers can hide technical complexity and present a point-and-click interface to end users in marketing, sales, and other departments.

Is a custom UI overkill for most use cases? Most definitely. But if you need this level of control, it’s really nice to have these advanced features in your back pocket.

Michelangelo the sculptor used his iron chisel. Picasso the painter used his horsehair brush. And you, the data activator, can use Hightouch’s embedded destination framework to create your visual masterpiece.

Read the docs to learn how it works.

Simplified mock-up of a form created using the embedded destination framework

What’s next?

Starting today, our custom destination toolkit is available for all Hightouch workspaces. I’m excited to see what you build! If you’re new to Hightouch, you can request a demo or create a free account.

Want to chat about your use case? Need help creating your first custom destination? Please don’t hesitate to reach out: omeed@hightouch.com (or ask for me on Intercom!)