# Touchline Analysis Studio: observation import formats

The Studio imports match data as **observed evidence**: every plotted position keeps its source,
clock and uncertainty, and nothing missing is filled in. Formats are recognised from file content,
never from file names. All parsing runs in a background worker that streams the file in 64 KiB
slices, hashes the original bytes, and stores bounded chunks on this device. Each format below
becomes one or more **sources**; aligning them to the match (fixture, sides, identities, clock and
which end each team attacks per period) is a separate, explicit step in the Studio.

| Format | Files | What it is |
|---|---|---|
| Touchline normalized observations v1 | `.ndjson`/`.jsonl` or `.json` | Open contract below, for tracking or events from any provider |
| Mapped CSV | any comma-separated table | You declare every column, unit and axis in the Studio |
| Opta F24 + F7 | two XML files | F24 event feed with its F7 lineup metadata |
| Stats Perform MA3 + MA1 | two JSON files | MA3 event feed with its MA1 fixture/lineup metadata |
| Stats Perform MA25 + MA1 | MA25 tracking text file + MA1 JSON | MA25 (Opta Vision / "OPT") tracking for both teams, with its MA1 metadata |
| Metrica tracking | CSV | Native tracking (one file per team) |

StatsBomb events/lineups JSON and Metrica event CSVs keep the existing reconstructed replay.

**Event feeds are not tracking.** Opta F24 and Stats Perform MA3 describe events. Only the acting
player's supplied event location is plotted. The ball, pass recipients, possession and every
off-ball position stay unknown. Stats Perform MA25 is tracking and has its own section below.

## Limits (all formats)

| Limit | Value |
|---|---|
| Original file | 1 byte to 512 MiB (536,870,912 bytes) |
| Sources per batch | 32 |
| One record (NDJSON line, JSON element, XML event) | 65,536 characters |
| One CSV row | 65,536 characters including its line ending |
| Roster/metadata companion (F7, MA1) | 8 MiB |
| One MA25 line (frame) | 65,536 characters |
| MA25 tracked objects per frame (players, goalkeepers, referees) | 64 |
| MA25 frames per file | 524,288 |
| Samples per source | 524,288 |
| Normalized storage per source | 256 MiB in at most 1,024 chunks of up to 512 samples / 2 MiB |
| Entities (players/tracks) per source | 256 |
| Teams per source | 32 |
| JSON nesting / XML nesting | 16 levels |
| XML attributes per element | 64 |
| Event qualifiers / extra fields per event | 128 / 48 |
| Identifiers | 1 to 128 characters, no control characters, not `__proto__`, `constructor`, `prototype`, `.` or `..` |
| Text values | at most 1,024 characters |
| Periods | 1 and 2 (halves), 3 and 4 (extra time); at most 4 hours each |
| Clocks | exact whole milliseconds, never negative within a period |
| Coordinates | finite; after conversion inside x -20..125 m, y -20..88 m, height 0..40 m. Never clamped. |

A file that breaks any rule is refused with a reason, and nothing from it becomes active. A failed
or cancelled replacement keeps the source it was replacing.

## Touchline normalized observations, version 1

The canonical frame is a 105 x 68 reference pitch, origin at the top-left corner, x along the
length, y growing toward the bottom touchline. You declare your own axes; the Studio converts.

### Framing

* **NDJSON**: line 1 is the header object; every other non-blank line is one sample object.
* **JSON document**: `{ "header": { ... }, "samples": [ ... ] }`. `header` must come before
  `samples` (the file is streamed, not loaded whole). Other top-level keys are ignored, and a
  top-level key may not appear twice.

### Header

| Field | Type | Meaning |
|---|---|---|
| `format` | `"touchline-observations"` | required |
| `version` | `1` | required |
| `kind` | `"tracking"` or `"events"` | tracking may use complete frames; events may not |
| `provider` | identifier | who produced the data |
| `fixture` | `{ id, date, homeTeamId, awayTeamId }` | each may be `null`; `date` is `YYYY-MM-DD` or ISO 8601 |
| `teams` | `[{ id, side }]` | `side` is `"HOME"`, `"AWAY"` or `null` (unknown) |
| `entities` | `[{ id, teamId, playerId, trackId, label }]` | `playerId: null` means an anonymous track; `label` is a short tag such as a shirt number, never a person's name |
| `coordinates` | `{ units, pitchLength, pitchWidth, origin, yAxis }` | `units`: `metres`, `centimetres`, `yards`, `normalized` (0..1) or `percent` (0..100). Physical units need `pitchLength`/`pitchWidth` in the same units. `origin`: `top-left`, `bottom-left` or `centre`. `yAxis`: `down` or `up` |
| `clock` | `{ unit, origin }` | `unit`: `seconds` or `milliseconds`; `origin`: `period-start` (resets every period) or `first-observed` (each period starts at its first sample, which is not known kick-off) |
| `frequencyHz` | number or `null` | native sample rate; `null` for events |
| `capabilities` | `{ ball, height, possession }` | a sample may only use what is declared |
| `sourceTeamId` | identifier, optional | the one team this file covers, if it covers only one |
| `capturedAt`, `updatedAt` | ISO 8601 or `null`, optional | |
| `description` | text, optional | provenance shown with the source |

Unknown header fields are refused.

### Sample

| Field | Type | Meaning |
|---|---|---|
| `period` | 1 to 4 | |
| `t` | number | clock in the declared unit and origin |
| `complete` | boolean | `true`: everyone visible is listed and anyone absent was not visible; `false`: no claim about anyone omitted |
| `entities` | `[{ id, x, y, z?, confidence?, uncertainty? }]` | `x`/`y` both `null` = present but position unknown; `z` needs `capabilities.height`; `uncertainty` in metres (0 to 150) |
| `ball` | `{ x, y, z?, confidence?, uncertainty? }` or `"unknown"`, optional | omitted in a complete frame means unknown; omitted in a partial sample means not reported |
| `possession` | `{ teamId, entityId }`, optional | only an explicit provider statement |
| `event` | `{ id?, type, outcome?, teamId?, entityId?, qualifiers?, attributes? }`, optional | events feeds only; `qualifiers` are `[id, value-or-null]` pairs and `attributes` are `[name, value]` pairs, kept verbatim |
| `sequence`, `revision` | integers, optional | default: file order and 0 |
| `originalTimestamp` | string or number, optional | kept verbatim; defaults to `t` |

Examples: [`normalized-tracking.ndjson`](normalized-tracking.ndjson), [`normalized-events.json`](normalized-events.json).

## Mapped CSV

The first row names the columns (unique, non-empty). Comma-separated, RFC 4180 quoting, UTF-8.
One row is one player, track or the ball at one time. In the Studio you declare:

* **Columns**: period, time, player/track, x, y (required); team, height, event type (optional).
* **Player column**: provider player IDs, or anonymous track IDs (identities stay unknown), plus
  the cell value that means the ball, if any. With a team column, players are told apart by team
  and ID, so both teams may use the same number.
* **Clock**: seconds or milliseconds, counted from the start of each period, from kick-off as a
  match clock (second half starts at 45:00, extra time at 90:00 and 105:00), or from the first
  row of each period.
* **Coordinates**: units, pitch size for physical units, origin and y direction.
* **Missing values**: blank cells and any listed markers mean no value; anything else must parse.
* **Row meaning**: tracking frames (rows sharing period and time form a complete visible set and
  must be sorted by time), partial observations (same grouping, no claim about omissions), or
  events (one row per event). For event rows you may list extra columns to keep as event fields;
  no other column is stored, and a column holding person names should never be kept.

Nothing is preselected: clock unit and origin, coordinate units, origin and y direction must each
be chosen before the mapping can be used.

The file is read twice inside the worker: once to list its teams and players, once to import.
A file that changes between the two reads is refused. Example: [`mapped-tracking.csv`](mapped-tracking.csv)
(metres, origin at the centre spot, y up, 105 x 68 pitch, milliseconds from period start,
ball value `ball`, blank cells missing, tracking frames).

## Opta F24 + F7 (XML)

Select the F24 event file and its F7 file together; they are paired by match ID (`Game id` and
`SoccerDocument uID`), never by file name. The F24 `Game` home and away team IDs must match the
F7 `TeamData`, and every event player must be in that team's F7 lineup.

* Each team's events become one source, because Opta gives every location in the acting team's
  attacking frame (0..100, toward the opponent's goal, y = 0 on its right touchline).
* Period clock = `min:sec` less the conventional period start (45, 90, 105 minutes); stoppage time
  continues the count. The supplied `timestamp` is kept.
* `x = 0, y = 0` placeholders and deleted events (type 43) are not plotted. Periods other than
  1 to 4 (pre-match, shoot-out, post-match) are not represented.
* Every `Q` qualifier and every event attribute is kept verbatim, except fields whose name ends
  in "name", which are dropped. The F7 file's name and SHA-256 are recorded with each source;
  names in F7 are never read.
* Any `DOCTYPE`, entity declaration or processing instruction is refused.

## Stats Perform MA3 + MA1 (JSON)

Select the MA3 event file and its MA1 file together; they are paired by `matchInfo.id`. MA3
`matchInfo` must come before `liveData.event`, and its home/away contestants must match MA1.
Clock, frame and plotting rules are the same as for Opta. Every event field is kept as an event
field and every `qualifier` as a qualifier, except fields whose name ends in "name" (such as
`playerName`), which are dropped; nested objects in an event are refused. The MA1 file's name and
SHA-256 are recorded with each source.

## Stats Perform MA25 + MA1 (tracking)

MA25 is the text tracking file (often called the OPT file) that the MA25 Match Tracking Feed links
to. Select it together with the match's MA1 JSON. The tracking file contains **no fixture ID**, so
it is paired with the one MA1 in the selection, whose `matchInfo.id` is read from content (never
from a file name). The worker then checks the content: every tracked player ID must be in that
MA1's `liveData.lineUp`, on the side the frame gives. Otherwise the file is refused as belonging
to a different fixture, as having its sides swapped, or as paired with an MA1 exported before a
late squad change (download the MA1 again after the match).

Each line is one frame, in the layout documented by kloppy 3.19.1
(`kloppy/infra/serializers/tracking/statsperform.py`):

```
frameId;clockMs,period,status:side,playerId,jersey,x,y;side,playerId,jersey,x,y;...;:ballX,ballY,ballZ;
```

| Field | Meaning |
|---|---|
| `frameId` | the provider's frame stamp; kept verbatim as the original timestamp |
| `clockMs` | whole milliseconds since the start of `period` (it resets every period); kept as the period clock |
| `period` | 1 and 2 (halves), 3 and 4 (extra time). Later periods (shoot-out) are left out |
| `status` | 0 = ball in play, otherwise dead. Validated but not stored |
| `side` | 0 home, 1 away, 3 home goalkeeper, 4 away goalkeeper, 2 referee (not imported); anything else is refused |
| `playerId` | provider player ID, which must be in the MA1 lineup on that side |
| `jersey` | any whole number; not used. Labels come from the MA1 shirt number |
| `x`, `y` | metres, origin at the top-left corner, y toward the bottom touchline (kloppy's SportVU system) |
| ball `x,y,z` | the same axes, height in metres. A height outside 0 to 40 m is kept as unknown, never clamped |

* **Pitch size is your declaration.** Neither MA25 nor MA1 states the venue size, so the Studio
  asks for the pitch length (80 to 130 m) and width (45 to 100 m) before importing. Nothing is
  preselected. The declared size only scales the metres to the 105 x 68 reference. It is recorded
  in provenance as user-declared, and the source's physical scale stays unconfirmed.
* **One source for both teams.** Players get their MA1 provider IDs and shirt-number labels, and
  are never matched to people until you align identities. Names in MA1 are never read.
* **Nothing is filled in.** A frame without a ball section has an unknown ball. A frame listing no
  players keeps only its ball and makes no claim about the players. Player height is unknown.
* **Native rate** = 1000 / the most common clock step between lines, as kloppy computes it (read
  in a first pass inside the worker). Fewer than three frames leaves the rate unknown.
* **Provenance** keeps the tracking file's SHA-256, the MA1 file name and SHA-256, the declared
  pitch size and the reference version.
* **Strict lines.** Each line needs exactly the frame header, a players section ending in `;` (or
  just `;`) with exactly five fields per object, and an optional `x,y,z;` ball section. Empty
  sections and extra sections or fields are refused. Numbers must be finite, and positions are
  never clamped. A malformed line refuses the whole file and names the line number. A last line
  with no line ending must have its ball section, because a line cut at a `;` would otherwise look
  complete. A cut that happens to leave a complete-looking, newline-terminated line cannot be told
  apart from real data. A failed or cancelled replacement keeps the previous source.
* **MA1 is JSON.** The MA1 XML variant is not supported.

A synthetic example with three frames (invented IDs `pH1`, `pH8` home, `pA9` away; the second frame
has no ball):

```
1767279600000;0,1,0:3,pH1,1,5.25,34.1;0,pH8,8,40.5,30.25;1,pA9,9,60,40;2,ref-main,0,52,20;:52.5,34,0.11;
1767279600040;40,1,0:3,pH1,1,5.3,34.1;0,pH8,8,40.6,30.2;1,pA9,9,59.9,40.1;2,ref-main,0,52,20.1;
1767279600080;80,1,0:3,pH1,1,5.35,34;0,pH8,8,40.7,30.2;1,pA9,9,59.8,40.2;2,ref-main,0,52.1,20.1;:53.1,34.2,0.3;
```

## References

Provider layouts follow the open-source kloppy loaders
(<https://kloppy.pysport.org/reference/providers/opta/>,
<https://kloppy.pysport.org/reference/providers/statsperform/>; MA25 per kloppy 3.19.1). Support for a file layout is not
a claim of a provider licence or a live feed. All example files here are synthetic.