# Bucket export

<FeatureStatus feature="BucketExport" />

A bucket export delivers a workspace's cost rows into a bucket you own, as FOCUS 1.4 files: one
folder per billing day, in gzip-compressed CSV or in Parquet, with a manifest that says what the
folder holds. Costfluent delivers each day once the workspace's data reaches it, and again whenever
the day's rows change. To take one report's numbers into a spreadsheet once, export it instead; see
[Export to CSV or Excel](/cost-reports/explore#export-to-csv-or-excel).

Costfluent only creates files. It never reads, lists or deletes anything in the bucket. Every
request goes to the storage host Costfluent builds from the destination's region, provider or
storage account; no destination takes an address you type. Each destination has a credential of
its own, separate from the connections that read your cloud costs:

| Destination | Costfluent writes with | What it may do |
|---|---|---|
| AWS S3 | a role in your AWS account that trusts Costfluent's principal with your organization's external ID | `s3:PutObject` under the export's folder |
| Google Cloud Storage | the service account Costfluent created for your organization | create objects in the bucket (Storage Object Creator) |
| Azure Blob Storage | a SAS for the container | create and write blobs, nothing else |
| S3-compatible Object Storage of OVHcloud, STACKIT, Scaleway, IONOS or Hetzner | an access key of yours | `s3:PutObject` under the export's folder |

## Before you start

<PlanAvailability feature="BucketExport" plan="Pro" />

- Adding, changing, testing or deleting a destination needs the Owner or Integration Owner
  organization role and **Export costs** on the workspace, so nobody sets up a feed of costs they
  could not export by hand. Anyone who can view the workspace sees its destinations and how far
  delivery reaches.
- A bucket for the feed, in AWS S3, Google Cloud Storage, Azure Blob Storage, or the Object Storage
  of OVHcloud, STACKIT, Scaleway, IONOS or Hetzner.
- A prefix: the folder in the bucket the feed goes under, such as `finops`. Costfluent writes
  everything under `<prefix>/costfluent/`, or under `costfluent/` when you leave the prefix empty,
  and every grant below names that folder. A prefix is up to 200 characters of letters, digits and
  `! - _ . * ' ( )`, in folders split by `/`, without `..`.
- The workspace's ID, which names its folder in the bucket: **Settings**, then **Workspaces**, lists
  it in the **ID** column.

## Grant write access

Give the destination the one permission it needs, scoped to the export's folder. In the policies
below, replace `<bucket>` with the bucket's name and `<prefix>` with your prefix; without a prefix,
the resource is `arn:aws:s3:::<bucket>/costfluent/*`.

### AWS S3

Costfluent writes as a role in your account. Its principal ARN and your organization's external ID
are the same two an AWS connection's role trusts: Costfluent shows them when you add the
destination and under [Connect with the IAM console](/connect/aws#connect-with-the-iam-console),
and the Terraform provider's `costfluent_aws_provider_info` data source reads both. The role's
trust policy:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "AWS": "<Costfluent principal ARN>" },
      "Action": "sts:AssumeRole",
      "Condition": { "StringEquals": { "sts:ExternalId": "<your external ID>" } }
    }
  ]
}
```

Its permissions policy:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket>/<prefix>/costfluent/*"
    }
  ]
}
```

<Stepper>

1. In the IAM console, create a role with a custom trust policy, and paste the trust policy above
   with Costfluent's principal ARN and your external ID filled in.
2. Add the permissions policy above to the role as an inline policy, with your bucket and prefix.
3. Note the role's ARN, such as `arn:aws:iam::123456789012:role/costfluent-export`, and the
   bucket's region, such as `eu-central-1`. The destination asks for both.

</Stepper>

Costfluent also passes that permissions policy as the session policy every time it assumes the
role, so its session can only put objects under the folder, whatever else the role allows.

:::caution
Keep the bucket's default encryption on Amazon S3 managed keys (SSE-S3). The session may call
nothing but `s3:PutObject`, so a bucket whose default encryption uses a KMS key refuses the write.
:::

### Google Cloud Storage

Costfluent writes as the service account it created for your organization, the same one a Google
Cloud connection shares its billing export with. Add the destination first
([Add the destination](#add-the-destination)): it creates the service account if your organization
has none yet, and shows its address. Then grant that address Storage Object Creator
(`roles/storage.objectCreator`) on the bucket, on the bucket's "Permissions" tab in the Google
Cloud console or with:

```bash
gcloud storage buckets add-iam-policy-binding gs://<bucket> \
  --member=serviceAccount:<service account address> \
  --role=roles/storage.objectCreator
```

The role creates objects but cannot read, list, replace or delete them. It covers the whole bucket,
while Costfluent writes only under the export's folder; to keep the grant to the feed as well, give
the feed a bucket of its own. If your organization restricts sharing to known domains, allow
Costfluent's service account first, as a Google Cloud connection needs
([Connect GCP](/connect/gcp#connect-in-the-console)).

### Azure Blob Storage

Costfluent writes with a shared access signature (SAS) for the container. It accepts only a SAS
whose permissions it can read from the token itself:

- a container SAS (`sr=c`), not an account SAS, and not one tied to a stored access policy;
- permissions Create, Write or both (`sp=c`, `sp=w` or `sp=cw`), and nothing else: a SAS that can
  also read, add, list or delete is refused;
- HTTPS only, and an expiry that has not passed.

In the Azure portal, open the container, select "Shared access tokens", tick only "Create" and
"Write", set "Allowed protocols" to HTTPS only, choose the expiry, and generate the token. Copy the
"Blob SAS token", the part after the `?`, not the SAS URL. With the Azure CLI:

```bash
az storage container generate-sas \
  --account-name <storage account> \
  --name <container> \
  --permissions cw \
  --https-only \
  --expiry <YYYY-MM-DDTHH:MMZ> \
  --output tsv
```

:::note
Costfluent shows the SAS's expiry beside the destination. Once it passes, delivery fails until you
paste a new SAS into the destination; a SAS signed with the storage account's key also stops working
when that key is rotated.
:::

### S3-compatible Object Storage

Choose the provider and enter the bucket's region. Together they form the only host Costfluent
writes to:

| Provider | Host | Regions |
|---|---|---|
| OVHcloud | `s3.<region>.io.cloud.ovh.net` | such as `gra`, `sbg`, `de` or `eu-west-par` |
| STACKIT | `object.storage.<region>.onstackit.cloud` | such as `eu01` or `eu02` |
| Scaleway | `s3.<region>.scw.cloud` | `fr-par`, `nl-ams`, `pl-waw`, `it-mil` |
| IONOS | `s3.<region>.ionoscloud.com` | `eu-central-1` (the `de` region), `eu-central-2`, `eu-south-2` |
| Hetzner | `<location>.your-objectstorage.com` | `fsn1`, `nbg1`, `hel1` |

Create an access key for the feed and limit it to putting objects under the export's folder, with
this policy on the key's user, or as a bucket policy statement that names that user as its
principal, whichever the provider offers:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket>/<prefix>/costfluent/*"
    }
  ]
}
```

:::caution
Costfluent cannot tell what a key allows without writing with it, so a key that may also read or
delete is accepted like a write-only one. Give the destination a key of its own that can only put
objects under the folder. Where the provider cannot limit a key to one bucket's folder, give the
feed a bucket, or a project, of its own.
:::

## Add the destination

<Stepper>

1. Open **Settings**, then **Workspaces**, and select the workspace whose costs the bucket receives.
2. Add a bucket export destination and choose where it delivers: AWS S3, Google Cloud Storage, Azure
   Blob Storage or S3-compatible Object Storage.
3. Enter the bucket and the prefix, and what the destination writes with: the role's ARN and the
   bucket's region for AWS; the storage account, the container and the SAS token for Azure; the
   provider, the region and the access key ID and secret access key for S3-compatible storage.
   Cloud Storage needs only the bucket, and shows the service account address to grant.
4. Choose CSV or Parquet.
5. Test the destination. Costfluent writes one small object into the folder, which shows the grant
   works before the first delivery; see [Test objects](#test-objects).

</Stepper>

The key pair and the SAS are stored encrypted and are never shown again; to change one, enter a new
one. An AWS or Cloud Storage destination stores no secret.

### Test objects

Each test writes `<prefix>/costfluent/_costfluent-check/<yyyyMMddTHHmmssZ>.txt`, named by the
instant of the test in UTC, so tests never overwrite each other or a delivery. The object holds one
line saying that Costfluent wrote it to check the folder and that it is safe to delete. Costfluent
never deletes it; delete it whenever you like.

For AWS, the test also asks AWS to let Costfluent assume the role with an external ID that is not
your organization's. The role must refuse: a role that lets Costfluent in without your external ID
would let any Costfluent customer write to your bucket, so the test fails until the trust policy
requires it.

The test runs in the background, and its result appears on the destination after a moment. A failed
test names its reason: the write was refused, the bucket was not found, the credential was refused,
the role does not require your external ID, the SAS has expired, or the storage did not answer.

## What lands in the bucket

### Key layout

```text
<prefix>/costfluent/<workspace ID>/focus-1.4/billing-date=<YYYY-MM-DD>/version=<yyyyMMddTHHmmssZ>/part-00000.csv.gz
<prefix>/costfluent/<workspace ID>/focus-1.4/billing-date=<YYYY-MM-DD>/version=<yyyyMMddTHHmmssZ>/part-00001.csv.gz
<prefix>/costfluent/<workspace ID>/focus-1.4/billing-date=<YYYY-MM-DD>/version=<yyyyMMddTHHmmssZ>/manifest.json
```

- The workspace's folder is its ID (`wsp_…`), never its name, so renaming the workspace does not
  move the feed.
- `billing-date` is the day the rows' charge period starts (`ChargePeriodStart`), in UTC.
- A Parquet destination writes `part-00000.parquet` and so on instead of `.csv.gz`.
- A part holds at most 1,000,000 rows, numbered from `part-00000`. The manifest is written last.

### Versions

Each delivery of a day lands in a new `version=` folder, named by the UTC instant Costfluent
reserved for it before its first upload. To read the feed, take for each billing day the newest
`version=` folder that holds a `manifest.json`, and ignore the others:

- A folder without a manifest is not complete yet. If a run stops partway, the next run finishes the
  same folder, writing the same keys, rather than starting another.
- A day is delivered again, under a newer version, only when its rows changed: a provider restated
  the day, or the workspace's integrations or accounts changed. A day whose rows did not change is
  not written again.
- A day restated to nothing gets a newer version whose manifest lists no parts and `"rows": 0`.

Costfluent never deletes an older version. Keep them for history or remove them yourself.

### The manifest

`manifest.json` says what the version folder holds:

```json
{
  "focusVersion": "1.4",
  "workspaceId": "wsp_…",
  "billingDate": "2026-10-25",
  "version": "20261026T080000Z",
  "format": "csv",
  "rows": 1834,
  "columns": ["BillingPeriodStart", "BillingPeriodEnd", "BillingCurrency", "…"],
  "parts": [
    {
      "key": "finops/costfluent/wsp_…/focus-1.4/billing-date=2026-10-25/version=20261026T080000Z/part-00000.csv.gz",
      "rows": 1834,
      "bytes": 51840,
      "sha256": "…"
    }
  ]
}
```

- `format` is `csv` or `parquet`, and `columns` lists every column of the parts in their order.
- `rows` is the day's row count, the sum of its parts' rows.
- Each part's `sha256` is the lower-case hex SHA-256 of the bytes uploaded, the compressed file for
  CSV, so a consumer can check that it read the whole part.

### The files

The rows are the workspace's cost rows for the day, as Costfluent stores them: from the
integrations the workspace holds, within the accounts it covers. The columns are the FOCUS 1.4
columns, named as FOCUS names them (`BilledCost`, `ChargePeriodStart`), followed by Costfluent's own
columns, which carry the `x_` prefix FOCUS gives custom columns (`x_AmortizedCost`). Amounts are in
the billing and pricing currencies FOCUS defines; the workspace's currency settings, segments and
virtual tags do not apply.

- **CSV**: gzip-compressed UTF-8, as RFC 4180 defines it: a header row of the column names, lines
  ending in CRLF, a field quoted only when it holds a comma, a quote or a line break, with its quotes
  doubled. Amounts are written exactly as stored, with up to 12 decimal places, timestamps as
  `yyyy-MM-ddTHH:mm:ssZ` in UTC, and `Tags` and `AllocatedTags` as JSON objects. An empty field is
  a missing value.
- **Parquet**: the same columns, typed: amounts as `DECIMAL(38, 12)`, timestamps as UTC timestamps,
  everything else as strings, the tag columns as their JSON objects. A row group holds at most
  100,000 rows.

:::note
The files are a data feed, not a spreadsheet. Values are written exactly as stored, so a value that
starts with `=`, `+`, `-` or `@` is kept as it is, where the CSV export from Cost Reports puts an
apostrophe before it so a spreadsheet does not run it as a formula. Load the files into a warehouse
or a script; to open costs in a spreadsheet, use [Export to CSV or
Excel](/cost-reports/explore#export-to-csv-or-excel).
:::

## How it behaves

- Delivery runs every four hours. Each run delivers at most 31 days to a destination, oldest first,
  so a new destination fills its history over several runs.
- The days run from the first day the plan's history keeps to the workspace's data-through day, the
  latest day every connection of the workspace has delivered, the day reports show as
  "Data through". A day after it waits until every connection has delivered it, and a day whose rows
  are still being processed waits for the next run, so a day is never delivered half.
- A day without cost rows gets no folder: a missing day means Costfluent holds no cost for it, not a
  cost of zero.
- The destination shows how far delivery reaches: not delivered yet; delivered, with the day it is
  delivered through; behind, when that day is more than two days before today, as during a backfill
  or while a connection's data has stopped arriving; or failing, with the reason of the last
  refused write. A failing destination is tried again on the next run.
- Changing the prefix or the format starts delivery over from the first day under the new folder or
  in the new format. The files already delivered stay where they are. A different bucket is a
  different destination: delete this one and add another.
- Deleting a destination stops its delivery and deletes its stored key pair or SAS. The files stay
  in the bucket. Remove the role, the grant or the key on your side as well; Costfluent's service
  account for Cloud Storage remains while your organization exists, so its grant outlives a deleted
  destination until you remove it.

## Related

- [Export to CSV or Excel](/cost-reports/explore#export-to-csv-or-excel)
- [Connect AWS](/connect/aws)
- [Connect GCP](/connect/gcp)
- [Power BI and Excel](/developers/power-bi)
- [Plans and billing](/plans-and-billing)