Independent Streams

You may need to support multiple parallel measurement sources for the same meter; the flows of data from each of these individual sources is called an independent stream. Octane uses Labels to isolate the measurements from each source, with each set of Primary Label values representing an independent stream.

Independent streams are necessary to ensure that measurements from different sources do not conflict or clobber one another. Octane aggregates each measurement source independently, then combines these aggregated values to arrive at the meter's total usage.

❗️

Primary labels are NOT set by default

If you do not explicitly set primary labels on a meter, all measurements for the meter will be considered a single stream.

Usage

It's not necessary to use independent streams for all configurations, such as counters that only need to track the number of times a feature (API request, file upload, etc.) is used within each billing period.

In contrast, gauges that track measured values over time must be configured to isolate independent streams for each measurement source to avoid the loss of existing values!

Example

At Antler DB, we allow each customer to deploy multiple independent databases from one account, and customers are billed for the number of API requests they make, as well as the total amount of storage they use.

Configuring our Storage Used meter

Our meter for tracking storage usage is a gauge that tracks the total amount of storage used, measured over time. The values for this meter are measured values, and each individual measurement represents the total space in use at the time of measurement.

Because each customer can deploy multiple databases from a single account, we need some way to aggregate the storage usage from different deployments that are measured independently. In this situation, we need to take advantage of independent streams. Therefore, we will configure the Primary Label deployment, which will allow us to isolate the values from each measurement source:

Understanding how independent streams are aggregated

Let's walk through how various measurements would be interpreted by this meter, with and without independent streams configured. In this example, each deployment measures its storage usage once per hour and sends the value to Octane. These tables reflect the updated value of the meter after each measurement is received.

As you can see from the table below, when the 1:00 measurement is received from each measurement source:

• With independent streams configured, each measurement source only updates the running total for the corresponding deployment value.
• Without independent streams, the meter's total reflects the last value received from any measurement source.

For our next set of measurements at 2:00, we can see that Octane received a measurement from the dev deployment before the prod deployment. The measurement from the dev deployment updates the value for that independent stream, and the meter's current total value is updated to reflect the sum of current values from each stream. The second measurement, from the prod deployment, has the same value as the last measurement received from prod, so the meter's value remains the same.

When we have not configured independent streams for our meter, each measurement clobbers the value previously received by Octane.

Now consider what would happen if the storage used by each deployment stayed consistent for a long period of time, and the measurements from each deployment arrived in the same order every time. With a properly-configured Primary Key to isolate independent streams, unchanging values over time will not cause the meter's value to change.

In contrast, without independent streams, the value that is received last will effectively become solely responsible for setting the value of this meter:

Configuring our API Requests meter

Our meter for measuring API requests is a counter. For each request the customer makes, the value of the meter is incremented by 1.

In this case, it's not required to configure individual streams to isolate the requests made to each deployed database; the total number of API requests made by a customer will always equal the sum of the number of API requests made to each deployment.

In this example, each deployment counts the number of API requests that are made to it, and once per hour, it sends the number of new requests made since the last time a measurement was sent to this meter. The table below reflects the updated value of the meter after each measurement is received.

As you can see from this table, the nature of counter meters means that this meter's total value after each update is the same regardless of whether a Primary Key has been specified to enable independent streams:

As you can see from this example, the meter's resulting total will be the same regardless of whether we configure independent streams.