How Bracket Works
Introduction
In today's data-driven world, businesses need efficient ways to sync data from different sources and ensure consistency across all platforms. This is where Bracket comes in - a revolutionary two-way ETL (Extract, Transform, Load) platform. In this blog post, we will discuss how Bracket works and the techniques it employs to keep data in sync with its polling method. In a future blog post, we'll discuss how Bracket can also stream data between Salesforce and Postgres.
Field Mapping
One of the key components of Bracket is field mapping. This process involves mapping the fields between the two data sources to effectively sync data. By mapping these fields, Bracket ensures that the data reflects the same information across both sources.
Transformers
Bracket goes beyond simple field mapping by providing transformers, which are bespoke handling mechanisms for each data source. These transformers ensure that fields are correctly mapped to a Plain Old Java Object (POJO) format, making the data integration process seamless and accurate.
Schemify - Introspecting Data Sources
To help with the classification and assignment of field types from each data source, Bracket utilizes our very own open-source project called Schemify. Schemify allows Bracket to introspect the data source and automatically determine the types of fields. This helps in ensuring accurate data mapping and integration.
Connecting to Various Data Sources
Bracket is designed to connect to a wide range of data sources. From databases to APIs, Bracket can seamlessly integrate with different platforms. The list of supported data sources is extensive and can be found on the official Bracket documentation at docs.usebracket.com. This flexibility allows businesses to sync data from multiple sources without any hassle.
Bidirectional ETL is really just a 4 step ETL
In the context of bracket, our bidirectional ETL a.k.a. two-way ETL is a system of 4 ETL processes: It’s a combination of two pipelines, each consisting of one traditional ETL and one rETL (reversal ETL). By leveraging this simple flow of data we are able to address merge conflicts and prevent an infinite loop of changes.
The Infinite Loop Antidote: A Source of Truth
To prevent an infinite loop of changes, Bracket maintains a "source of truth" for data synchronization. This means that when changes occur in one data source, Bracket ensures that they are propagated to the other source. By keeping track of the latest changes and updates, Bracket guarantees that data remains consistent across both platforms. Bracket also allows customers to maintain the source of truth on premises in their own data stack.
Merge Conflicts
In cases where a specific record is updated in both data sources, Bracket handles merge conflicts by giving priority to the primary data source. The primary data source is processed first, ensuring that any conflicting updates from the secondary source are overwritten. This helps maintain data integrity and consistency.
Event Driven Architecture and Polling
Ideally, an event-driven architecture would be the most efficient way to sync data in terms of memory and space usage. However, not all data sources support event-driven notifications. As a result, Bracket defaults to polling the data sources at a set interval. This means that Bracket periodically checks for updates in the data sources, ensuring that any changes are captured and synced accordingly.
However, for data sources that do support event-driven architecture - such as Salesforce and Postgres - Bracket offers event-driven instantaneous syncing.
Conclusion
In conclusion, Bracket is a powerful two-way ETL platform that streamlines the process of syncing data from different sources. By employing techniques that enable robust, reliable two-way syncing, Bracket ensures that data remains consistent and up-to-date across all platforms. With its ability to connect to a variety of data sources, Bracket provides businesses with a reliable solution for data synchronization.
To learn more about Bracket and its features, visit usebracket.com.
