rebase.energy

Upload / read timeseries

Upload measurements and forecasts to any record, then read them back windowed, downsampled, by run, as of a past instant, or as a full history.

Any record can carry named timeseries. A series is identified on its record by its type, a slug you choose such as power-production, and has a unit and one of two shapes:

  • singleindex: one row per valid_time. Use it for meter data, actuals, and prices. A row is a single value, or one field per named column.
  • multiindex: versioned data with named columns. Each point has a valid_time (what the value is for), a knowledge_time (when it became known, the forecast issue time), and one field per declared column. Use it for forecasts.

Under both shapes the store keeps every value it is ever given. Two questions that come up early have short answers:

Correcting and deleting

How do I correct a wrong value? Upload the right one. It becomes the current value and the old one stays in the history, still visible to as_of reads dated before the correction.

How do I delete? DELETE the series type with a list of points. A deletion is a retraction: no read returns the point from then on, but as_of and the history view still show what was stored.

Uploading

POST /objects/{object_name}/{record_id}/timeseries creates the series on first use and writes points. The body is discriminated by shape.

Measurements (singleindex)

r = requests.post(f"{BASE}/objects/sites/{site['id']}/timeseries", headers=HEADERS, json={
    "type": "power-production",
    "shape": "singleindex",
    "unit": "MW",
    "points": [
        {"valid_time": "2024-06-01T00:00:00Z", "value": 0.62},
        {"valid_time": "2024-06-01T01:00:00Z", "value": 0.71},
    ],
})
r.raise_for_status()
print(r.json()["points_written"], r.json()["points_skipped"])  # 2 0

Named columns on measurements

A singleindex series may carry several named columns, for example a met mast measuring wind speed at two heights. Declare columns and give each point one field per column instead of value:

r = requests.post(f"{BASE}/objects/met-masts/{mast_id}/timeseries", headers=HEADERS, json={
    "type": "wind",
    "shape": "singleindex",
    "unit": "m/s",
    "columns": ["wind_speed_100m", "wind_speed_10m"],
    "points": [
        {"valid_time": "2024-06-01T00:00:00Z", "wind_speed_100m": 9.4, "wind_speed_10m": 6.1},
        {"valid_time": "2024-06-01T01:00:00Z", "wind_speed_100m": 9.9, "wind_speed_10m": 6.5},
    ],
})
r.raise_for_status()

Reads then return one row per valid_time with a field per column, and a single column can be read directly at .../timeseries/wind/wind_speed_100m. A type keeps the form of its first upload: mixing the plain value form and named columns under one type is a 422.

Forecasts (multiindex)

Declare the columns once, then include every column on every point. A column may be null, but a point missing a declared column makes the whole request a 422.

r = requests.post(f"{BASE}/objects/sites/{site['id']}/timeseries", headers=HEADERS, json={
    "type": "power-forecast",
    "shape": "multiindex",
    "unit": "MW",
    "columns": ["q10", "q50", "q90"],
    "points": [
        {"valid_time": "2024-06-02T00:00:00Z", "knowledge_time": "2024-06-01T06:00:00Z",
         "q10": 0.2, "q50": 0.5, "q90": 0.9},
        {"valid_time": "2024-06-02T01:00:00Z", "knowledge_time": "2024-06-01T06:00:00Z",
         "q10": 0.3, "q50": 0.6, "q90": 1.0},
    ],
})
r.raise_for_status()
print(r.json()["points_written"])  # 6

Upload each new forecast run with its own knowledge_time. Runs accumulate rather than overwrite, which is what makes the read-side features below possible. Note that points_written counts value cells (rows × columns), so the two-point, three-column example reports 6.

Standalone series

POST /timeseries takes the same body and uploads a series that is not linked to any record. It is held at workspace level. Use it for data with no natural object, such as a market price curve.

What an upload does to existing data

The store keeps every value it is ever given, so an upload never destroys anything:

  • A point the store already holds with the same value is skipped and counted in points_skipped. Re-sending a window is free and idempotent.
  • A point with a different value becomes the current value. The previous value stays in the history and is still what an as_of read dated before the upload returns.
  • A null value records a gap: from now on the point is absent from resolved reads, and the history shows the null. Uploading a real value later brings it back.
  • Deleting is a separate call (below) and uses the same mechanism: a retraction.

Every stored value is stamped with the writing principal (changed_by) and the server's clock (change_time); both show in the history view.

Upload options

All optional, both shapes unless noted:

OptionMeaning
skip_unchangedDefault true: the dedup described above. Set false to append every row regardless, for instance to record a re-publication that happens to carry the same numbers.
retentionshort (180 days), medium (3 years), long (5 years) or forever. Bound when the series is first declared and immutable afterwards; a later upload naming a different tier is a 422. Defaults: singleindex forever, multiindex medium. Use short for high-rate observations such as 1 Hz grid frequency.
knowledge_timeSingleindex only. When the values became known, for the whole upload; default now. A backfill of old publications passes the original publication time so as_of reads stay truthful.
annotationPer point, up to 2000 characters. Free text stored with the values (a source note, a quality flag), visible in the history view.
unitA series keeps the unit it was declared with. An upload in a different but convertible unit (kW into a MW series) is converted on the way in; an incompatible one (EUR/MWh into MW) is a 422.
r = requests.post(f"{BASE}/objects/sites/{site['id']}/timeseries", headers=HEADERS, json={
    "type": "power-production",
    "shape": "singleindex",
    "unit": "kW",                                   # converted into the series' MW
    "knowledge_time": "2024-06-02T09:00:00Z",       # when this publication was known
    "points": [
        {"valid_time": "2024-06-01T00:00:00Z", "value": 640.0},
        {"valid_time": "2024-06-01T01:00:00Z", "value": None,
         "annotation": "meter outage"},             # a recorded gap
    ],
})
r.raise_for_status()
print(r.json()["points_written"], r.json()["points_skipped"])

Many series at once

A collector that has just parsed a whole document should not make one request per record. POST /timeseries/write takes many per-record upload bodies in one call and lands them whole or not at all. See Many series at once.

Listing

GET /objects/{object_name}/{record_id}/timeseries returns the series registered on a record:

{
  "items": [
    {"type": "power-production", "shape": "singleindex", "unit": "MW", "columns": []},
    {"type": "power-forecast", "shape": "multiindex", "unit": "MW", "columns": ["q10", "q50", "q90"]}
  ]
}

For many records at once, see the catalog.

Reading

GET /objects/{object_name}/{record_id}/timeseries/{series_type} returns the points plus metadata. The parameters compose.

Windowing

start and end bound valid_time, inclusive on both ends. If you give neither, the window is the last week of available data (end defaults to the series' last point, start to a week before end). The response always states the effective start/end and the series' full extent as series_first/series_last, so you can tell what you got.

r = requests.get(
    f"{BASE}/objects/sites/{site['id']}/timeseries/power-production",
    headers=HEADERS,
    params={"start": "2024-06-01T00:00:00Z", "end": "2024-06-30T23:00:00Z"},
)
r.raise_for_status()
data = r.json()
print(data["start"], data["end"], data["series_first"], data["series_last"])

Downsampling for charts

  • max_points: auto-downsampling. When the window holds more points than the cap, the server buckets to time-bucket means at a standard interval and reports the interval it chose as bucket_seconds in the response. Pass max_points=2000 for charting; omit it for raw exports.
  • bucket_seconds: an explicit aggregation interval (bucket means), applied regardless of density. Takes precedence over max_points.
r = requests.get(
    f"{BASE}/objects/sites/{site['id']}/timeseries/power-production",
    headers=HEADERS,
    params={"start": "2024-01-01T00:00:00Z", "end": "2024-12-31T23:00:00Z", "max_points": 2000},
)
r.raise_for_status()
data = r.json()
print(len(data["points"]), data["bucket_seconds"])

Only the resolved view (version=latest, the default) can be aggregated.

Versions, runs, and time travel

The store holds every value it was ever given. Four parameters choose what you see:

ParameterEffect
version=latest (default)For each valid_time, the newest known value. For a forecast this is the "best available" view across runs.
version=allMultiindex: every version of every point, each tagged with its knowledge_time.
version=historyThe audit view, see below.
knowledge_time=<ts>Multiindex: exactly one forecast run, only values issued at that instant.
as_of=<ts>Time travel. Resolve the series using only values that were known at that instant. "What did the forecast look like yesterday at 06:00?"

as_of applies to singleindex series too: every upload stamps a knowledge time (the write clock, or the knowledge_time it passes), so a late correction or backfill is invisible to an as_of read dated before it.

# The day-ahead forecast as it stood at 06:00 on June 1
r = requests.get(
    f"{BASE}/objects/sites/{site['id']}/timeseries/power-forecast",
    headers=HEADERS,
    params={
        "start": "2024-06-02T00:00:00Z",
        "end": "2024-06-02T23:00:00Z",
        "as_of": "2024-06-01T06:00:00Z",
    },
)
r.raise_for_status()
data = r.json()
print(data["runs"])

For multiindex reads, the response's runs lists the newest distinct knowledge_times in the window (with runs_truncated set if there were more than fit). This is what a run picker in a UI should show.

Latest vs as_of

version=latest answers "what is the best value we know now". as_of answers "what was the best value we knew then". Use as_of to evaluate forecast performance honestly, without letting later runs or corrections leak into the comparison.

The history view

version=history returns every stored row in the window, one per stored value: valid_time, knowledge_time, change_time, value (nulls included), changed_by, annotation, and name (the column) for types with named columns. This is the audit trail: who wrote what, when, and what was retracted. It cannot be aggregated (400 with max_points or bucket_seconds).

r = requests.get(f"{BASE}/objects/sites/{site['id']}/timeseries/power-production",
                 headers=HEADERS, params={"version": "history"})
r.raise_for_status()
for row in r.json()["points"]:
    print(row["valid_time"], row["value"], row["knowledge_time"], row["changed_by"], row["annotation"])

Read one column directly

When you already know the series and the column, skip the list step and read a single resolved {valid_time, value} timeline:

GET /objects/{object_name}/{record_id}/timeseries/{series_type}/{name}
r = requests.get(
    f"{BASE}/objects/sites/{site['id']}/timeseries/power-forecast/q50",
    headers=HEADERS,
    params={"max_points": 2000},
)
r.raise_for_status()
print(r.json()["points"][:2])

It accepts the same start, end, max_points, bucket_seconds, and as_of parameters. The read is always the resolved view.

Deleting points

DELETE /objects/{object_name}/{record_id}/timeseries/{series_type} with a JSON body naming the points to delete. A deletion is a retraction: from now on no read returns the point, while as_of reads at an earlier instant and the history view still show what was stored.

r = requests.delete(f"{BASE}/objects/sites/{site['id']}/timeseries/power-production",
                    headers=HEADERS, json={
    "points": [
        {"valid_time": "2024-06-01T00:00:00Z"},
        {"valid_time": "2024-06-01T01:00:00Z"},
    ],
})
r.raise_for_status()
print(r.json())
# {"target_id": "...", "type": "power-production", "columns": [],
#  "points_deleted": 2, "points_missing": 0}
  • Points are named one by one, up to 10,000 per request. There is deliberately no "delete everything in a range" form.
  • points_deleted counts the points that existed; points_missing the ones that were never stored or were already deleted. Missing points change nothing, so repeating a request is harmless.
  • Multiindex: a point without knowledge_time deletes the version a resolved read returns (the newest run at that time). Pass knowledge_time to delete exactly one version. Deleting the newest version hides the time from the resolved read; older runs stay visible under version=all and as_of.
  • columns (types with named columns only) deletes just those columns at each point, for example {"points": [...], "columns": ["q90"]}.
  • Uploading a value at a deleted time brings the point back, as a new version.
  • The series' series_first/series_last extent still counts deleted times.

Deleting points needs the timeseries:write permission, like uploads.

On this page