Copy page

Connect a source

Upload cost to a custom provider

A custom provider holds cost that Costfluent does not collect itself: a cloud account it cannot connect to, a SaaS bill, or any other source you can export as FOCUS CSV. You create it with a name, upload files to it, and its cost appears in Cost Reports under that name. There are no credentials and nothing to sync.

How Costfluent reads an upload

An upload is one run of an export: every file of that run, uploaded together. Costfluent checks every row before it imports any. If a single row is invalid, nothing is imported and the upload fails with a report of its errors.

A completed upload replaces the provider's cost for every day from its first charge to its last, including days inside that range it has no rows for. Uploading the same month again therefore corrects it rather than adding to it, and an upload that covers only part of a month leaves the rest of the month alone.

Prerequisites

• The Manage integrations capability, which owners and integration owners hold, to create a custom provider and to upload or delete files. Anyone with View integrations can see the uploads.
• Cost in FOCUS columns, as described under File format. Azure, AWS and Google Cloud all offer a FOCUS export.

Connect

Create the provider

1. Open Settings, then Integrations, and select Custom provider.
2. Select New custom provider.
3. Enter a Name, such as the account or tenant the cost comes from, and optionally a Description, then select Create provider.

The provider's page opens. A custom provider counts toward your plan's connections like any other.

Upload a run

1. On the provider's page, under Uploads, select Choose files and pick every file of one export run, as .csv or .csv.gz.
2. Select Upload.

The upload is listed as Pending, then Processing, and ends Completed or Failed. A completed upload shows the days it covers and what it billed per currency. Select Download template for a CSV with the columns and two example rows.

File format

The first row is the header. Columns are matched by name, in any order; columns Costfluent does not read, including every x_ extension column, are ignored.

Column	Required	Notes
ChargePeriodStart	yes	YYYY-MM-DD or an ISO 8601 timestamp.
ChargeCategory	yes	Usage, Purchase, Tax, Credit or Adjustment; Fee, Refund and Discount are also accepted.
ServiceName	yes
BilledCost	yes
BillingCurrency	yes	An ISO 4217 code, on every row.
ChargePeriodEnd	no	Exclusive, as in FOCUS: a charge for 1 September ends on 2 September. Without it a charge lasts one day.
EffectiveCost	no	Defaults to BilledCost.
ListCost	no	Defaults to BilledCost.
ConsumedQuantity, ConsumedUnit	no	A quantity needs its unit.
ServiceCategory, ServiceSubcategory	no
ResourceId, ResourceName, ResourceType	no
RegionId, RegionName	no
BillingAccountId, SubAccountId, SubAccountName	no
SkuId, ChargeDescription	no
CommitmentDiscountId, CommitmentDiscountType	no
LineItemId	no	Identifies a line across uploads.
Tags	no	A JSON object of string values, such as {"team":"platform"}.

Amounts are plain numbers with a dot for decimals, and may use scientific notation such as 1.2E-05. A currency symbol, a thousands separator or a unit makes the row invalid. A timestamp without a time zone is read as UTC, and one with an offset is converted to UTC, so each charge lands on its UTC day. A charge that spans several days is spread evenly over them.

Costs stay in the currency each row bills in; reports convert them as they do any other cost.

Data and freshness

Processing starts as soon as an upload is received and usually takes minutes; the provider's page refreshes until it ends. In Cost Reports the cost appears under the name you gave the provider.

Limits

• The files of one upload together: 90 MB as sent, 2 GB once decompressed, and 2,000,000 rows.
• At most 50 files in one upload.
• The error report keeps the first 1,000 errors and counts the rest.

Upload an Azure export

For an Azure tenant Costfluent cannot connect to, export its cost in FOCUS format and upload it:

1. In that tenant's Cost Management, create an export from the Cost and usage (FOCUS) template. Choose CSV with gzip compression, a monthly or a daily month-to-date schedule, and a storage account in the same tenant.
2. After a run, download every part_*.csv.gz file of that run. The run's manifest.json lists them.
3. Upload all of them together as one upload.

Azure can split one run into several parts, and a part can hold a single purchase on one day. Upload the parts separately and the second replaces the days of the first, which drops that purchase. A month-to-date upload replaces the days it covers, so upload the closed month again once Azure has finalised it.

Avoid double counting

Cost uploaded here is counted in addition to what your connections collect. Do not upload cost that a connected account already reports.

Use the Public API

A token holding Manage integrations can do the same through the Public API, as the API reference describes:

1. POST /v1/providers with key set to custom, a name and empty credentials creates the provider.
2. POST /v1/custom-providers/{providerId}/uploads takes the files of one run as repeated multipart fields named file, and answers with the pending upload.
3. GET /v1/custom-providers/{providerId}/uploads lists the uploads, and GET /v1/custom-providers/{providerId}/uploads/{uploadId} reads one, with its first errors when it failed. Reading takes View integrations.
4. DELETE /v1/custom-providers/{providerId}/uploads/{uploadId} deletes an upload.

POST /v1/custom-providers/{providerId}/costs is a different mode: it takes JSON entries checked by the same rules, with the Sync integrations capability. It adds and corrects lines and removes none, so it suits a feed of new charges rather than a full export.

Update or disconnect

Delete an upload from its row under Uploads. Its cost is removed from reports, except on days a later upload replaced, which keep that upload's cost. Removing the provider on the Custom provider page removes all of its cost.

Troubleshooting

Upload statuses

Status	Meaning
Pending	Received and waiting to be processed.
Processing	Being checked and imported.
Completed	Every row was imported.
Partly imported	Lines that share an identity but disagree were left out; the error report names them.
Failed	Nothing was imported. Download the error report, fix the file and upload it again.

Common errors

• A file is refused when you upload it. It is not .csv or .csv.gz, the files together are too large, or the parts do not share one header. Upload only the parts of one run.
• Missing required columns. The header lacks one of the five required columns.
• An amount is not a number. Remove currency symbols and thousands separators.
• A day holds too many rows. One day of an upload is limited in size; split a very large export by day.

Related

• Manage connections
• Currency