# Upload cost to a custom provider

A custom provider holds cost that Costfluent does not collect itself: a cloud account it cannot
connect to, a SaaS bill, or any other source you can export as FOCUS CSV. You create it with a name,
upload files to it, and its cost appears in Cost Reports under that name. There are no credentials
and nothing to sync.

## How Costfluent reads an upload

An upload is one run of an export: every file of that run, uploaded together. Costfluent checks
every row before it imports any. If a single row is invalid, nothing is imported and the upload
fails with a report of its errors.

A completed upload replaces the provider's cost for every day from its first charge to its last,
including days inside that range it has no rows for. Uploading the same month again therefore
corrects it rather than adding to it, and an upload that covers only part of a month leaves the rest
of the month alone.

## Prerequisites

- The **Manage integrations** capability, which owners and integration owners hold, to create a
  custom provider and to upload or delete files. Anyone with **View integrations** can see the
  uploads.
- Cost in FOCUS columns, as described under **File format**. Azure, AWS and Google Cloud all offer a
  FOCUS export.

## Connect

### Create the provider

1. Open **Settings**, then **Integrations**, and select **Custom provider**.
2. Select **New custom provider**.
3. Enter a **Name**, such as the account or tenant the cost comes from, and optionally a
   **Description**, then select **Create provider**.

The provider's page opens. A custom provider counts toward your plan's connections like any other.

### Upload a run

1. On the provider's page, under **Uploads**, select **Choose files** and pick every file of one
   export run, as `.csv` or `.csv.gz`.
2. Select **Upload**.

The upload is listed as **Pending**, then **Processing**, and ends **Completed** or **Failed**. A
completed upload shows the days it covers and what it billed per currency. Select
**Download template** for a CSV with the columns and two example rows.

## File format

The first row is the header. Columns are matched by name, in any order; columns Costfluent does not
read, including every `x_` extension column, are ignored.

| Column | Required | Notes |
|---|---|---|
| `ChargePeriodStart` | yes | `YYYY-MM-DD` or an ISO 8601 timestamp. |
| `ChargeCategory` | yes | `Usage`, `Purchase`, `Tax`, `Credit` or `Adjustment`; `Fee`, `Refund` and `Discount` are also accepted. |
| `ServiceName` | yes | |
| `BilledCost` | yes | |
| `BillingCurrency` | yes | An ISO 4217 code, on every row. |
| `ChargePeriodEnd` | no | Exclusive, as in FOCUS: a charge for 1 September ends on 2 September. Without it a charge lasts one day. |
| `EffectiveCost` | no | Defaults to `BilledCost`. |
| `ListCost` | no | Defaults to `BilledCost`. |
| `ConsumedQuantity`, `ConsumedUnit` | no | A quantity needs its unit. |
| `ServiceCategory`, `ServiceSubcategory` | no | |
| `ResourceId`, `ResourceName`, `ResourceType` | no | |
| `RegionId`, `RegionName` | no | |
| `BillingAccountId`, `SubAccountId`, `SubAccountName` | no | |
| `SkuId`, `ChargeDescription` | no | |
| `CommitmentDiscountId`, `CommitmentDiscountType` | no | |
| `LineItemId` | no | Identifies a line across uploads. |
| `Tags` | no | A JSON object of string values, such as `{"team":"platform"}`. |

Amounts are plain numbers with a dot for decimals, and may use scientific notation such as
`1.2E-05`. A currency symbol, a thousands separator or a unit makes the row invalid. A timestamp
without a time zone is read as UTC, and one with an offset is converted to UTC, so each charge lands
on its UTC day. A charge that spans several days is spread evenly over them.

Costs stay in the currency each row bills in; reports convert them as they do any other cost.

## Data and freshness

Processing starts as soon as an upload is received and usually takes minutes; the provider's page
refreshes until it ends. In Cost Reports the cost appears under the name you gave the provider.

### Limits

- The files of one upload together: 90 MB as sent, 2 GB once decompressed, and 2,000,000 rows.
- At most 50 files in one upload.
- The error report keeps the first 1,000 errors and counts the rest.

### Upload an Azure export

For an Azure tenant Costfluent cannot connect to, export its cost in FOCUS format and upload it:

1. In that tenant's Cost Management, create an export from the **Cost and usage (FOCUS)** template.
   Choose CSV with gzip compression, a monthly or a daily month-to-date schedule, and a storage
   account in the same tenant.
2. After a run, download every `part_*.csv.gz` file of that run. The run's `manifest.json` lists
   them.
3. Upload all of them together as one upload.

Azure can split one run into several parts, and a part can hold a single purchase on one day. Upload
the parts separately and the second replaces the days of the first, which drops that purchase. A
month-to-date upload replaces the days it covers, so upload the closed month again once Azure has
finalised it.

### Avoid double counting

Cost uploaded here is counted in addition to what your connections collect. Do not upload cost that
a connected account already reports.

## Use the Public API

A token holding **Manage integrations** can do the same through the Public API, as the
[API reference](/api) describes:

1. `POST /v1/providers` with `key` set to `custom`, a `name` and empty `credentials` creates the
   provider.
2. `POST /v1/custom-providers/{providerId}/uploads` takes the files of one run as repeated
   multipart fields named `file`, and answers with the pending upload.
3. `GET /v1/custom-providers/{providerId}/uploads` lists the uploads, and
   `GET /v1/custom-providers/{providerId}/uploads/{uploadId}` reads one, with its first errors when
   it failed. Reading takes **View integrations**.
4. `DELETE /v1/custom-providers/{providerId}/uploads/{uploadId}` deletes an upload.

`POST /v1/custom-providers/{providerId}/costs` is a different mode: it takes JSON entries checked
by the same rules, with the **Sync integrations** capability. It adds and corrects lines and removes
none, so it suits a feed of new charges rather than a full export.

## Update or disconnect

Delete an upload from its row under **Uploads**. Its cost is removed from reports, except on days a
later upload replaced, which keep that upload's cost. Removing the provider on the **Custom
provider** page removes all of its cost.

## Troubleshooting

### Upload statuses

| Status | Meaning |
|---|---|
| **Pending** | Received and waiting to be processed. |
| **Processing** | Being checked and imported. |
| **Completed** | Every row was imported. |
| **Partly imported** | Lines that share an identity but disagree were left out; the error report names them. |
| **Failed** | Nothing was imported. Download the error report, fix the file and upload it again. |

### Common errors

- **A file is refused when you upload it.** It is not `.csv` or `.csv.gz`, the files together are
  too large, or the parts do not share one header. Upload only the parts of one run.
- **Missing required columns.** The header lacks one of the five required columns.
- **An amount is not a number.** Remove currency symbols and thousands separators.
- **A day holds too many rows.** One day of an upload is limited in size; split a very large export
  by day.

## Related

- [Manage connections](/connect/manage-connections)
- [Currency](/currency)