Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

Data is the “new oil” and organizations that are able to quickly analyze both real-time and historical data will have a competitive advantage. But ingesting, managing, storing, and correlating different types of data can become a costly, administrative nightmare.

In this blog, we will walk through details about Macrometa’s Global Data Mesh and its 

, search, and streams components. We will also go over the two other parts of the Macrometa 

 (GDN) - Edge Compute that has features like 

 and Data Protection with features like geo-pinning.

Global Data Mesh: Store and serve any kind of data

, you can use the data model that makes the most sense for your application and make it available locally, globally, or both. The result is a flexible, ultra-low-latency data layer purpose-built for global, real-time, and event-driven use cases.

 is that you don’t have to worry about time-consuming data integrations or storage maintenance. You simply use different databases at the same endpoint - so it is much easier to integrate different data streams into an application. 

In a data mesh model, data can be stored, accessed, and processed where it originates and/or where it is needed. Organizations can use a data mesh to collect and process data where its users are and gather real-time insights.

Data is automatically replicated across the Macrometa GDN, and API requests are routed to the nearest available location. There are over 175 

 around the world. Changes at any location propagate across the network instantly, bringing all nodes to parity within milliseconds. Let’s run through the components of the Global Data Mesh. 

Document Store - NoSQL document collections with flexible indexing and modeling

 makes it easy to read, write, replicate, query, and index JSON data on the Macrometa Global Data Network. Advanced features let you change data feeds for publishing or subscribing to document collections as streams. 

Graphs - Dynamic, high-throughput graph store with ultra low-latency queries

 are a fault tolerant, fully managed graph database service that makes it easy to build and run graph powered apps and 

 on highly connected datasets. You can easily ingest millions of related events across tens of thousands of streams with billions of connected relationships in real-time and query the dynamic graph structure with milliseconds latency.

Streams - Distributed, real-time stream engine with pub/sub and message queues

 integrate seamlessly with your existing workflows, giving you fast, scalable, and reliable access to your data from a wide range of popular frameworks. Build stateful low-latency micro-services and real-time data pipelines. You can create and use a stream on any collection or store. 

Dynamo Store - Feature-complete and fully compatible with AWS DynamoDB

 offers a low latency, globally replicated key-value and document database that is serverless, multi-cloud, multi-region, and multi-master. Macrometa Dynamo mode offers all the features of AWS DynamoDB with a 100% compatible API and wire protocol proxy (you can even use the AWS SDK with Macrometa Dynamo mode). 

Key-Value Store - High-throughput KV collections

 lets you build high-performance web and mobile apps and APIs quickly and easily without the need for managing anything. Use our simple API to quickly add state to your AWS Lambda functions, Cloudflare workers, or container apps. 

Search - Flexible full-text search and similarity ranking across multiple data stores

Search has become such a critical part of application development and is key to improving customer experiences. 

 offers powerful full-text search and real-time ranked search with query parameters like spanning phrases, wild cards, images, proximity, and similarity. Full-text search is available in 20+ languages with dedicated analyzers and advanced features like stop word removal and stemming.

Bringing it together with Edge Compute and Data Protection

What separates Macrometa from other data storage solutions is that it also brings compute closer to people and where data is generated, regardless of where that is. Macrometa’s Edge Compute puts data into action and delivers real-time and event-driven apps and data in <10ms to 80% of the global population. 

 lets developers create simple REST APIs on top of the data mesh. 

 enables Complex Event Processing (CEP) functions and 

 workloads - in minutes instead of days or months. 

Stream Workers play an important role with data collections and stores. With Stream Workers, data could come from different sources (GDN streams, GDN Database, HTTP endpoints, Kafka, and MQTT etc.) and can be published to different sinks or stores. 

 also allows you to transform data from one data type to another, aggregate streams, and clean and filter data. This CEP engine automatically creates objects like streams, collections, and even tables and remote databases. 

Macrometa’s Data Protection supports geo-pinning or geo-fencing to address data sovereignty and privacy use cases. Developers can set up 

 for regional and global data, and then adjust locations as needed. Macrometa offers three different types of 

 (user, token-based, and API keys) and is 

With many tutorials, Quickstarts, and easy-to-follow example 

, you can get started with Macrometa’s Global Data Mesh in minutes. 

Say Yes To The Data Mesh: Explore Multi-Model Data Collections And Stores At The Edge

Cloud is mainstream today, and its evolution ushered in several new businesses, business models & jobs like SRE, DevOps, CloudOps, etc. But conceptually, the model is a client-server model of the 1980s with increasing distance between client & server in each decade, i.e., initially, it was Client → Server (where servers were a few feet from the user), then Client → Data Centers (where servers were 10s and 100s of miles from the user) and finally Client → Cloud (where servers were 1000s of miles from the user).

On the other hand, the demands of enterprise businesses today are very different from what they were in the 80s and 90s. In the 80s, software was a niche area. Today, it is the main driver for companies, and survival/failure depends on how well the company can see the shifts, utilize the right tech and innovate.

In the '90s, 100 MB of data is a big deal. I still recall the excitement of getting 5.25-inch high-density floppy disks that could store 1.2 MB of data. Today, 100 TB of data is frivolity.

Similarly, in the '90s, the concept of digital data privacy or data regulations was insignificant. Today, we have the GDPR law, California Law, and many more regulations to secure data.

Also, can you recall how many enterprises in the 80s and 90s were providing services globally, dealing with millions of users, not many, right? Today, pretty much all companies have their business serving global users.

Companies that could see the shifts early in the 2000s and 2010s adopted the cloud when it was in infancy and benefited handsomely (ex: Netflix). While others found seemingly rational reasons not to adopt and got decimated (ex: Blockbuster). In my opinion, enterprises today have a choice - see the seismic shifts happening now and embrace the next evolution or get left behind due to fear.

This blog post concentrates on how these shifts translate to what is needed beyond the current 

From client→server days in the 80s to current cloud computing, the default approach is to move the data from the source to where the services are running. Whether the distance to move the data is 100 miles or 10,000 miles, it does not really matter.

The challenge is that new-age enterprises generate data in terabytes, petabytes, and zettabytes. It’s not just IoT / 

 / Smart Manufacturing, but it also includes retail, gaming, and other industries where millions of users request or generate data via forms, clickstreams, logs, etc. Moving this data 1000s of miles to a distant cloud location is a recipe for a bad user experience, a slower time to act, and a host of other issues like low reliability, network congestion, high computing costs, 

 issues, etc. It does not matter whether it is event-driven or request-response workload.

A better alternative is moving computing closer to the source of the data, especially for use cases that deal with interaction, personalization, 

, and real-time log processing, etc. This will help process the data closer to the source and turn data into dollars quickly by providing a better user experience, personalization, faster actuation, and reduced egress and other cloud costs. But, there are use cases where the moving data to compute makes more sense, and the current cloud model is a good fit. Example - 

Currently, enormous regulatory activities are happening around the globe to protect their citizens (or to monitor their citizens if it is not a democracy) from cyber crime and attacks. It is safe to assume that we’re just starting on these regulations and compliance path and it’ll only become more taxing in the coming years.

On the other hand, today's data platforms don’t deal with privacy. Similarly, most enterprise services approach privacy as an add-on at the end to comply with regulations. The challenges that come with doing it in this way (as an add-on) are poor architecture, humongous development & operational costs, technical complexities, and many more.

So, even in this scenario, a better alternative for data platforms (databases, search, file systems) is to natively support privacy capabilities so that architects and developers can leverage them right from the beginning. This can enable developers to easily geo-fence the data and compute to designated locations, intelligent routing of requests, real-time tokenization, pseudo-anonymization on demand, etc.

Best way to understand this use case is with an example.

Say we’re at a coffee shop and want to grab some coffee, you go ahead and order like - “One Espresso Macchiato in a tall cup, please”. I order with a straight face and order like this - “1 cup 2% milk, one shot of espresso, one tablespoon of sugar, one dollop of foam mixed with these and heated for 3 min, then pour in a tall cup, please.”

Interestingly the latter approach is what most people do when building their services to achieve business outcomes. The following is a sample reference architecture to build an e-commerce application using AWS services -

e-commerce application architecture on AWS cloud

Building apps using point services like the above is akin to how I ordered coffee when we went to a coffee shop. Unfortunately, the cloud providers have done a great job convincing many folks that this is the right technical approach. The folks who benefit from this model are cloud providers, consultants selling expertise, and point solution vendors. It may not be for someone who’s a CTO/CIO/VP/Architect, who needs to deliver outcomes with the least complexity and higher developer velocity.

Again, a better alternative here is to utilize converged services that provide multiple capabilities (like database, search, graphs, streams, and 

) from one interface on a single copy of data. Cloud providers cannot do this from a single copy of data because they have to re-architect their services.

Today, for a developer to run a service, the first thing they need to do is select a cloud and within that cloud a specific region to deploy and run the service. There are two big reasons for this -

Cloud services are proprietary and hence making it very hard to do multi-cloud.

Data & compute services are centralized architectures making it impossible to do multi-region. The centralized architectures use consensus protocols like Raft, Paxos, etc., which rely on synchronous communication between nodes. So don’t be misled by multi-region claims of centralized architectures. They do not scale well to multi-regions and certainly not when you need them across the globe.

The above characteristics certainly benefit the cloud providers, though. The more lock-in, the better. On the other hand, the problems that architects and developers have are things like how to provide disaster recovery with the least effort, how to route requests to the right location or cloud, how to minimize the egress costs, and finally, how to maximize the developer velocity and spend more time on things that move the needle for business instead of unnecessary ops complexity.

A better alternative is to free the developers from the underlying complexity and the need to choose a cloud or location. This is not radical. 

 were already doing this for the last two decades, albeit for static data. What is needed is CDN-like capabilities but for the more complex services like databases, search, streams, stream processing, computing, etc. We call that GDN, i.e., Global Data Network :-)

I’m, first and foremost, a practitioner. Not a writer/analyst. So as any practitioner does, I spent the last five years building a 

 to address the above challenges. If you want to learn more about Macrometa GDN, then I selfishly and highly recommend learning more about 

Beyond Cloud Computing: The Advent Of GDN

Project Parsec is one of the ways we’re offering unprecedented speed and performance to applications built on the 

. It is an ongoing project still in development and available only in Macrometa.

What’s better than a map? A map that offers turn by turn navigation, and uses real-time traffic information to chart the best route to your destination. A map that’s aware that your favorite coffee shop is closed and routes you to the next nearest outlet. In short, a map that takes the guesswork out of your commute and routes you to whatever you need - quickly and simply.

What’s better than a router? A router that dispatches you to the nearest datacenter with the lowest 

- after confirming that it's active. If it’s not, it dispatches you onto the next nearest datacenter, so that - no matter what - you always get to your data, quickly and simply.

Meet our internal project, Parsec, the geo-aware and latency-aware router that helps you do more with data. Whether you want to super-size to a trillion events per minute scale, run applications at "blink of an eye" speed, or build new "never possible before" features, Parsec helps us help you get there.

 that lets you bind data to specific regions and use any combination of countries and clouds to store, process, and query data. Parsec functions as an intelligence layer that sits atop the Global Data Network spanning across the globe and helps your teams reach your data quickly with ultra-low latency. 

In today’s cloud model, the practice has been to move the data from the source to where the services are running i.e., in a hyperscale cloud data center that is thousands of miles away. Macrometa flips this model and moves the computing closer to the data source.

Modern applications need predictable low latency. So forcing requests to make a trip across the globe to reach data is a bad user experience: it will introduce huge latencies, low reliability, and higher computing costs. Not to mention how applications today have to deal with regulations on data like GDPR etc. It’s critical that appropriate PII data never leaves the applicable regions to avoid violation of regulations. Macrometa’s Project Parsec layer helps you do just that i.e., by automatically routing requests to the nearest location with the right availability and lowest latency.

Parsec is a geo-distributed coordination-free routing & processing engine connected by Anycast to serve the requests originitation from anywhere in the world. When an App or API makes a request to the GDN, the request is routed, processed, and served by our Parsec engine that is closest to request source.

The Parsec engine applies pre-defined policies to quickly determine the right Macrometa 

 (PoP) is a group of nodes within a data center that holds data in a region. By keeping copies of the data in different regions, we are able to reduce the access times since the data is closer to the people requesting it.

Parsec is used to find the optimal pool (based on the configured latency and geo policies) where a user's data is stored. Optimal - in the Parsec world - is defined as first, the user always has access to their data no matter what and second, they are able to access the data in the fastest possible way.

Parsec also provides intelligent request routing and tenant placement among geo-pools. It automatically manages data navigation, request routing, scaling, and load balancing - with the goal of making data always accessible at the lowest possible latency.

Need to handle a trillion events per minute? No problem, Parsec helps your data teams scale easily. Macrometa’s decentralized/distributed data, compute, and governance architecture makes it incredibly easy - also very quick and highly cost efficient.

Traffic isn’t directed to one central location, but geo-distributed. This means, each city’s traffic goes to that closest Parsec engine instance. So, people In India are not contending with traffic from the US.

As regional traffic grows, more pools are added or size of a pool is extended to handle this growth. If a region becomes over provisioned, Parsec can redirect traffic to other regions to help lighten the load. This response to changing demands is managed by Parsec with an eye to reduce latency while maintaining reasonable costs. As demand subsides, Parsec dismantles any additional resources it brought up to handle the increased load. In this way, Parsec helps to maintain the right balance between availability, performance, and cost.

Parsec's number one priority is to always make your data available to you. While it strives to give the fastest possible access to data, it does not sacrifice access. If a pool (data center) isn’t available to deliver data, it redirects the request to other data centers. As resources are added (in step with increasing demand), no downtime is experienced. In other words, when pools or regions are added and removed, users' access to their data is never disrupted.

Parsec offers a smart way to automatically balance workloads without any effort from your teams.

Parsec continuously measures the load and latency of all pools. These measurements are used to determine the most effective pool to service the data request. When it detects increases in either, it schedules more resources to be brought up to that region. This effectively means as the number of events increases we can direct more resources to that region with the intent of handling trillion of events per second.

All data is accessed via unique URLs. Parsec maps the unique URL to the correct pool in the correct region for each customer. If a customer grows and is moved to a new pool with more capacity, Parsec automatically knows how to route the traffic to the correct pool. And it does this transparently for customers. So no interruptions to customer. Internally this also allows us to isolate customers to their own pools thereby providing strong isolation between customers.

How To Scale To “Trillion Events Per Minute” - With Ease And Without Downtime

 - the next generation Kafka Summit organized by Confluent happening in beautiful Austin, Texas October 4-5.

Why are we attending it and you should as well?

There is no better place to learn about data streaming than Current 2022, an event where real-time technology leaders and developers will share best practices, use cases, and explore the vision and future of the technology.

Nothing more exciting than being able to be a part of this! Here’s what we’re looking forward to:

Two days of power-packed keynote sessions highlighting real-time data technology and tons of opportunities to network and collaborate.

Educate what Macrometa’s GDN platform has to offer to build apps in real-time

Network with real-time streaming data technologists and researchers

Come say howdy at the Macrometa booth, see it in action, grab a t-shirt, and learn more about this real-time streaming data stuff the kids love.

 is a Platform as a Service for building and running applications at the edge. Run regionally, globally, or both - the GDN automatically synchronizes data between locations in milliseconds. Build greenfield applications or extend existing applications to improve performance and comply with data locality requirements. We provide data and compute infrastructure from more than 175 locations globally, reaching two billion users and ten billion devices in 50ms.

Create the next generation of high-performance applications

Build APIs on top of your data to create new revenue streams in the API economy

Macrometa At Current 2022 Kafka Summit Organized By Confluent

When you think about data and event-driven analytics, it’s natural to think of stream processing as the most effective way to analyze streams of data in real time. Many organizations have used legacy event stream processing tools like Spark and 

 for the past decade. But what if your organization handles very complex events (like an insurance company processing thousands of claims on a monthly basis)?

Would stream processing still be a good fit if you were trying to track insurance scams across many different geographies? Or would 

 provide more accurate, real-time insights? With Macrometa, you can support CEP use cases like supply chain management and event stream processing use cases like orders and payments. Either way, Macrometa provides a CEP engine “out of the box” that is easy to learn with an SQL-like language and there are many extensions for different sources and sinks (Kafka, MQTT, etc.) and functions (math, pii, statistics, etc).

In this blog post, we’ll take a look at Macrometa’s CEP engine, how to implement it, and its benefits. Read on to learn more!

CEP helps you to automate certain processes, and even predict future outcomes based on your historical and real-time data. Make the most of your data with CEP - like understanding customer behavior patterns - by analyzing the events that trigger it.

In a nutshell, CEP helps you to analyze streams of events from multiple data sources on a very low-latency level to uncover patterns and derive insights. This way, you can take action on the stream of events as soon as they happen. With Macrometa’s CEP engine, you can set up CEP faster than 

traditional event stream processing tools

 offers a simple way of performing CEP. You can easily integrate streaming data and take appropriate actions to analyze and correlate data in different ways to meet your business goals. 

Exploring Macrometa Streams and the CEP architecture

 is designed to work with real-time data. This includes the data flows from any IoT devices like cameras, weather stations, and other sensors. The most important and distinguishable features of Macrometa’s Global Network (GDN) streams are as follows:

 If you create a global stream the data published will be replicated in all the regions. Also, it's possible to simultaneously publish and consume data to/from all the regions. This allows you to deliver low latency across all your locations and anywhere your customers interact with your business. The GDN also provides different ways to publish and consume stream events - REST API interface, drivers for Python, NodeJs, Java, Stream Workers, and Pulsar native clients. 

 You can create and use a stream on a collection. This means all the data inserted, updated or deleted will be published to stream. One of the possible use cases is the change data capture (CDC).

3. Easy set-up from and to different data sources:

 The Stream Worker's data could come from different sources (GDN streams, GDN Database, HTTP endpoints, Kafka, MQTT and etc.) and can be published to different sinks. This is a feature available right “out of the box.” 

 also allows you to transform data from one data type to another, aggregate streams, clean and filter data, and more. 

 Stream Workers automatically create the needed objects like streams, collections, and even tables and remote databases. The CEP engine does this transparently for you behind the scenes. 

Now, let’s look at how you can implement CEP in Macrometa within minutes.

Setting up Macrometa CEP For Simple Producer-Consumer

The best way to get started is to use the samples located beneath Stream Workers listed within the GDN. Every sample has a good description of the use case it covers and how you can get it up and running.

To begin, you can log into the Macrometa platform and then find the Stream Workers menu item on the left. Stream Workers will only appear if it is already enabled within your account. You can enable it by contacting 

 sample below is a great one to get started with since it is the most simple stream worker. It gives you a basic understanding of the Stream Workers structure and language. This stream worker generates random values every second and publishes them as events on SampleStream.

 will start to create and publish events to 

The second stream worker here is a consumer of the events published by events-generator. It reads the events from 

, collects 10 sequential events in a sliding window and processes them by retrieving the biggest one.

Finally the result of the processed data is published to 

Any of the other Stream Workers cover different use cases for reading (source), processing, analyzing, and finally writing/storing/publishing (sink) the processed data. Go ahead and apply the ones that work best for your use cases from your GDN account or the 

Now that you know the steps to implement Macrometa’s CEP engine in your organization. With Stream Workers, you can process events in real-time, trigger automated actions, and more. Overall, CEP can be an extremely powerful tool for event-driven analytics.

To get started with Macrometa and see Stream Workers in action, 

And to know more about how complex use cases require CEP that goes a step beyond just event stream, join us in this webinar 

Complex Event Processing with Macrometa: What’s Next After Streaming Analytics with Apache Spark and Apache Flink

Tutorial: Implement Complex Event Processing In Macrometa Within Minutes

This blog post is written by DevCycle who partnered with Macrometa to launch world's first and fastest Feature-Flag-as-a-Service (all from the Edge). Let's hear from them on why they chose Macrometa and how they built an ultra low-latency global feature flags solution using Macrometa.

The idea of saving user data to the server is not a new concept. It’s a tried and true archetype that is used widely. Our old architecture is built on this premise; a user is introduced into the system through our SDKs, the server receives the request, and the server then saves that user to our database and returns the user their configuration with all of the experiments and feature flags associated with that user.

, we reimagined our systems to separate that logic by creating a service that buckets users into experiments/feature flags. This service would live on 

 hosted by Cloudflare, meaning any request sent to these workers will direct you to the closest node in the Cloudflare network with a lightning-fast response (think ~50ms). This is a step up from our old response times, which averaged around the 150ms mark.

The caveat for this fast response time was that we could not store user data in traditional databases as we valued the response time as paramount. Our old system had the user data fetched from the database to use in bucketing decisions, but it would add significant time to the request.

Imagine having a feature flag gating a feature on your landing page; a fast response time could mean the difference between your user seeing the new feature or seeing a blank page while the request waits to finish.

Instead of having the user data saved, we initially opted for an approach where the developer would pass in the user data in every request, eliminating the need to keep it as it would be available in the request itself. This was a satisfactory solution, but it made using the tool a little restricted. You’d have to pass all the user data in every request to get features instead of piecemeal when the user triggers changes to their attributes. Not having to give in all their details is the explicit goal, but how do we get there?

We explored several different options when considering a solution to storing user data. We had specific criteria when researching the market for solutions, but the most important to us was having low 

A few options we boiled it down to were:


Durable Objects seemed like a good option touting a low latency key-value storage solution that was strongly consistent. The tradeoff, however, was that the Object being updated lives in the data center where it was first created. After creation, the data currently cannot be moved to a new region. This is fine if the user creating the data remains in the exact geographical location and never moves. Still, it breaks down as soon as they attempt to access their data from a location other than where they initially created it. Over time, performance, on average, would decline as users move around the world.



 boasts low latency round trips and average response times of less than 50ms, with a tradeoff of replication lag for regions around the world, typically less than 300ms. We decided this was worth the tradeoff to maintain a faster average read performance. 

 was created for our SDKs to use when a developer has 

 enabled as a feature, taking in user data as the request body and simple authorization handling by reading a client key associated with a DevCycle project from the request header. Macrometa’s library was easy to implement into our Cloudflare Workers, and their 

 were concise and easy to follow.

We’ve also added an option to turn EdgeDB on in the SDKs so that it tells the API to use EdgeDB user data in the request to get user variables and combine any user data sent up in the request. You can find the public API in our 

 to read/write user data into a geographically- replicated data store and a publicly accessible endpoint to save user data!

Okay, but how is a low latency solution useful?

A feature is only outstanding as its use-case, so how would you use EdgeDB?

Let’s imagine you’re creating a product that is a storefront to products regularly sold at a brick-and-mortar store. Maybe it could be artisanal coffee drinks. Let’s call it “Planet Dollars”.

Now “Planet Dollars” will have an accompanying app that you can download on your phone and a website to order ahead and pick up your drink at a store.

You want to give members an incentive to keep spending money, giving them stars on every purchase. Once they accrue a certain number of stars, they’ll achieve a “Platinum” status on their account, unlocking new functionality in the app. The user orders on the website, and the request is sent to your backend:

Reinventing The Wheel: How DevCycle Made Saving User Data Fast With Macrometa

Digital experiences are the new normal, and understanding both customers and their intent is paramount for businesses to stay ahead of the competition. Consumers are increasingly transacting online for everyday experiences, from telehealth appointments to digital banking, online shopping, and even attending virtual events. Every online transaction can be thought of as an event, and organizations are capturing, comparing, and analyzing these events from a variety of sources to predict customer behavior and improve their experiences.

 isn’t trivial and there are a variety of solutions out there that claim to be the best, but how do you choose the right one for your use case? In this blog, we will introduce Macrometa, the new kid on the block in the world of CEP (you can even 

 and try it out - in minutes), and see how it stacks up against two mature offerings for streaming analytics, 

 and Apache Flink. We will provide a high-level overview of CEP in the next section, but check out the chapter 

 to learn more about the two stream analytics platforms. 

Understanding Complex Event Processing vs Event Stream Processing

Before we continue, let’s set the record straight on the differences between CEP and event 

 (ESP), also known as streaming analytics. In short, CEP can be considered as a more complex version of ESP and generally involves detecting patterns or correlations between multiple incoming streams of events that may or may not arrive in the order they were sent, to derive real-time insights.

An example of ESP would be comparing the number of items added to Amazon shopping carts with the number of items that were actually ordered, to ultimately improve the order completion percentage over time. With CEP, you may correlate the zip codes of customers with items in their shopping carts with national weather data to notify customers that deliveries may be delayed. Aggregating a variety of events, both real-time and historical, allows you to more accurately forecast delivery estimates, providing an improved customer experience.

 (GDN) offering a Global Data Mesh, Edge Compute and a CEP engine with native Data Protection. Within the Macrometa GDN is an incredibly powerful component called 

 which help enable CEP use cases. Macrometa’s CEP capabilities are similar to those of Spark and Flink, but we will go into more detail about how Macrometa can help simplify your architecture and expedite your deployments. Take a look at the chart below that provides a high-level overview of the three data processing engines.

Macrometa, Spark, and Flink feature comparison

Stream Workers are only one component of the Macrometa GDN and work seamlessly with the rest of the platform to expedite and simplify the creation of event-driven architectures. To put this into context, imagine how much time and expertise it would take to write stream processing jobs to aggregate a real-time stock quoting stream via a variety of intervals (by second, minute hour, day, etc.) in Spark or Flink. You’ll need to specify and integrate a backend datastore for persistence, then repeat the setup process within multiple regions as close to your end users as possible for low latency. This could take several months! With Macrometa, you could deploy a local or global Stream Worker to process and aggregate the data in as little as five minutes.

Check out the code snippet below to see how easy it is to aggregate, process, and store the data within a Macrometa database.

Today’s challenges increasingly require solutions that can process, analyze, and act on data in real-time. Detecting fraudulent credit card transactions for a global customer base is one such challenge that can be addressed with a 

 engine like Macrometa, Spark or Flink. With low-latency being critical for anomaly detection, a centralized architecture simply won’t work. Instead, you will need to have a processing engine deployed in a variety of regions as close to your customer base as possible.

If operational simplicity is important to you, then you’ll love Macrometa due to its 

. You can choose to deploy Stream Workers into one or more PoPs around the world at the click of a button, ensuring data will be processed as close to where it is created as possible, while at the same time abiding by any 

. On the topic of regulatory requirements, you can geo-fence your data to ensure it stays within a particular region as well as anonymize/tokenize it to ensure access is only granted to those for whom it is intended.

Accelerate Time to Market With Developer Velocity

When processing real-time streams of data, your system will generally contain three components; a source (or sources) where the data originates, a stream processor which will analyze the data (cleanse, transform, enrich, correlate, etc.), and finally a destination where the newly processed data is consumed. Spark and Flink serve as the stream processing engines, thus it will be up to your development teams to specify and integrate the tools for the source and destination.

Macrometa on the other hand is a complete development platform that can dramatically simplify your architecture, unleashing developer velocity to focus on building great products rather than integrating disparate systems. Stream Workers can be thought of as an application blueprint, specifying data ingestion, processing your data, and finally writing to a 

 in whatever data model makes sense for your use case. Oh, and if you need to 

 on your data, Macrometa can do that too.

Macrometa GDN, Spark, and Flink are all fantastic stream processing engines, and which tool is right for you depends on your goals and needs. Spark is more mature, while Flink is likely more suitable for CEP due to improved latency and performance. If a managed service is more your thing, 

 may be able to help reduce complexity of your Spark jobs. 

However, if you need to process globally distributed events with as little latency as possible, or want to reduce your time to market and unleash developer velocity, Macrometa is absolutely the way to go. 

Macrometa offers real-time, stateful CEP close to where the events are occurring - where your customers are creating data - to provide fast, relevant insights. 

Want to see for yourself? Be sure to watch our 

 to quickly begin setting up your first Stream Worker.

Choosing A Complex Event Processing Engine: Macrometa Vs Apache Spark And Apache Flink

The cloud has been changing the way we think about technology for years now and it’s no secret that cloud services are becoming increasingly popular with both users and businesses alike.

Cloud gaming is a growing segment of the video game industry, with some companies reporting more than double-digit growth year-over-year. In its recent 

 on cloud gaming trends, Intricately noted that cloud gaming as a service could reach an estimated value of $5.8 billion by 2022. This high valuation is indicative of the rapid adoption of cloud technology in general and cloud gaming in particular. 

As we enter 2022, the video game market will continue to expand, with emerging concepts such as microservices, 

, and blockchain continuing to drive innovation across hardware, software, and services.

But what does this mean for cloud gaming? How will these trends impact it going forward? Let’s take a look at Intricately’s report in more detail and see if we can predict where cloud gaming is headed in the future.

Cloud Gaming Market Predictions and Trends

The Cloud gaming industry to grow over 300% while adding new ways to engage with gaming

According to the report, in the next two years, the cloud 

As streaming services revolutionize, the way we watch television cloud gaming is changing, and how we play, well, I would say cloud gaming is offering us new ways to engage with our gaming addiction. So we don't think it's changing how we play, but it's adding new ways for us to actually go and play computer games.

Are streaming games going to be a reality?

According to the report, the future of gaming is in the cloud, making streaming games a reality, and you no longer need an expensive gaming console. All you need is a fast internet connection.

However, it is not that you no longer need an expensive gaming console. Cloud gaming does not offer the same experience as gaming on a high-end PC. Gamers typically have both PCS and consoles, and handheld devices, they don't pick between them. We've also seen that all you need is a fast, internet connection isn't strictly true, you can have a great ISP connection put to the internet for sure.

If the cloud is too far away as we often say or wherever the game cloud gaming is being hosted, a fast internet connection will not help you at all. You're still going to have problems and because it's video, it's even more sensitive. So most Consoles and PC gaming is running at anywhere from 1080p, which is a high-definition video, up to 4K which is, you know, obviously 4x of 1080 P. So 4x the resolution is an ultra-high-definition quality that requires a huge amount of bandwidth. But gamers will lower the resolution because they're looking to play a multiplayer game and the 

 between the player and the game is really important.

Archive