Copy page

Connect a source

Connect GCP

Connect GCP

Costfluent reads Google Cloud cost from your BigQuery billing export. It creates a service account for your organization in its own Google Cloud project, and you grant that account read access to the export dataset. No key is created or exchanged, and nothing is created in your cloud except that one grant.

Connect a billing account, not a project. Its export already carries the cost of every project the billing account pays for, so one connection covers them all. Connecting imports up to the previous six months, as far back as the export holds data.

What Costfluent is granted

BigQuery Data Viewer on the export dataset, and nothing else. Costfluent runs its queries in its own project and pays for them there, so it needs no role in your project and cannot see any other dataset, run jobs on your bill, or change anything.

Prepare the export

1. Pick a project to hold the billing data. A dedicated one keeps access narrow.
2. Create a BigQuery dataset in the US or EU multi-region.
3. Under Billing, then Billing export, enable the Detailed usage cost export into that dataset. The standard export lacks the per-resource fields Costfluent reads. Google can take a few hours to write the first rows.

Share the dataset

Open Settings, then Integrations, select Google Cloud Platform, and choose Add Billing Account. Costfluent shows the service account it created for your organization, with a button to copy it. The email is the same every time you open the dialog.

Grant that email BigQuery Data Viewer on the export dataset, in the dataset's sharing settings. If your organization restricts sharing to known domains, allow the organization ID and customer ID Costfluent shows beside the email first.

The published Terraform module makes the same grant:

Code
module "costfluent" {
  source  = "costfluent/cost-access/gcp"
  version = "~> 0.2"

  project_id                       = "<project holding the export>"
  billing_dataset_id               = "<export dataset>"
  billing_account_id               = "<billing account ID>"
  costfluent_service_account_email = "<the email from Add Billing Account>"
}

With the Costfluent Terraform provider, the costfluent_gcp_service_account resource returns the same email, so the whole connection can live in code.

Add the connection

In the same dialog, enter the billing account ID (012345-ABCDEF-678901), the project ID and the dataset name, then Test Connection or Connect. Costfluent validates the query it will run before storing anything, so a connection that saves is a connection that can read your cost.

If the grant is missing, the test says the dataset is not shared with the service account yet. A new grant can take a few minutes to apply, so try again shortly after granting it. A dataset that exists but has no export table yet gets its own message: Google has not written the first rows.

After connecting

The Manage tab lists the connection by billing account ID, with its project and dataset. Open the gear icon beside it to see whether the dataset is still readable, which workspaces the connection feeds, and which billing months have been imported.

Run checks on the connection page runs the checks again on request and reports each one separately, so a failure names the step that failed.

Rotating or removing access

There is no key to rotate. To retire the connection, remove it in Costfluent and remove the service account's grant on the dataset. When a Costfluent organization is deleted, its service account is deleted with it.