Project status¶
Datum is being built, specification first. A complete pass runs end to end for all nine resource types.
| Part | State |
|---|---|
| Document parsing and discovery | Implemented |
| Matchers, composition and precedence | Implemented |
| Substitution of declared label values | Implemented |
| Effective manifests and their digests | Implemented |
| The resource graph, cycles and duplicate targets | Implemented |
| Per-type field validation | Implemented |
datum render, datum explain, datum validate, datum affected |
Implemented |
| Observation, diffing and planning | Implemented |
datum observe, datum diff, datum plan |
Implemented |
The File, Directory and Symlink provider |
Implemented |
| The program runner providers execute through | Implemented |
Package through apt, tested against Debian |
Implemented |
Package through dnf, tested against Fedora |
Implemented |
Provider selection from os-release |
Implemented |
Sysctl through /proc/sys and /etc/sysctl.d |
Implemented |
User and Group through the shadow utilities |
Implemented |
| Drift a provider declares uncorrectable | Implemented |
Service through systemd, tested against a booted systemd |
Implemented |
| Applying, verification and failure propagation | Implemented |
| The pass lock and pass reports | Implemented |
datum reconcile, datum status |
Implemented |
datum agent, datum config check |
Implemented |
Repository through apt and dnf |
Implemented |
| The agent as a resident process, scheduling, metrics on a port | Implemented |
| Fetching from a remote, with a hardened checkout | Implemented |
| Signature verification and the descendant check | Implemented |
| Last known good, so a bad revision does not stop a host working | Implemented |
datum revision, datum version |
Implemented |
| Refusing resources that target Datum's own files | Implemented |
Packaging as a .deb and an .rpm |
Implemented |
| Grouping resource types into domains | Implemented |
| Secret resolution and reboot handling | Not started |
Package and Repository through apk and pacman |
Not started |
trust.strictPaths, and the provider path-safety rules it switches on |
Not started |
The --revision and --json flags |
Not started |
datum init and datum migrate |
Not started |
| Enrolment, so a host works out its own identity | Not started |
Applying is Linux-only. Reading a host works anywhere, since the safety rules the writing path depends on have no portable equivalent.
The order follows from where a mistake costs least. Resolution is a pure function of the repository, so it can be built and tested without a machine to break, and it is the half of the system every other part depends on. Reading a host came next, since it changes nothing, which left applying as the last part to build.
Everything above the provider boundary is portable, so the observer, the differ and the planner are tested without a host at all. A provider is the only part that touches an operating system, which is the separation ADR-0002 records.
The specification came first because the decisions that are expensive to change later are the ones made early. The shape of a resource, the way identity works, where the provider boundary sits and how composition resolves are all choices that an implementation would otherwise make incidentally, and would then be stuck with.
What the documentation is for¶
This site serves two audiences at once. For somebody evaluating Datum it explains what the system does and how it is meant to be used. For somebody implementing Datum it is the specification to build against, which is why pages describe observable behaviour, failure handling and edge cases instead of staying at the level of overview.
Where something is undecided, the documentation says so. A specification that fills in an answer nobody has thought through reads as settled when it is not.
How this site labels design maturity¶
| Label | Meaning |
|---|---|
| Accepted | Decided, and recorded in an architecture decision record. Implementation should follow it, and changing it means superseding the record. |
| Proposed | A concrete design that has not been accepted. Detailed enough to argue with, likely to change in the detail, and not safe to depend on. |
| Planned | Accepted in principle but not specified. It needs to exist and how it behaves has not been worked out. |
| Open question | A known gap with no resolution yet. Stated so that it is visible rather than discovered during implementation. |
A label describes how settled a design is, not whether it is built. The table at the top of this page and the support matrix record what actually works, and an entry appears there only once the behaviour exists and is tested. A page can be labelled proposed and implemented at the same time, which is the normal state of an alpha schema.
Most configuration examples are proposed. The datum: v1alpha1 marker at the top of
every document says the same thing more formally, because the alpha suffix means field
names, defaults and semantics can change without a migration path until the schema
reaches a stable version.
What is settled so far¶
Fourteen decisions are accepted, each with a record explaining what it was weighed against and what it costs.
| Decision | Record |
|---|---|
| Desired state is expressed as typed resources | ADR-0001 |
| Resource types are distribution neutral, providers are not | ADR-0002 |
| Git is the source of desired state | ADR-0003 |
| Host identity is separate from host classification | ADR-0005 |
| Dependencies are declared, never inferred | ADR-0006 |
| The effective manifest is the engine's only input | ADR-0007 |
| Resource references are separate from target identities | ADR-0008 |
| Only declared resources are managed | ADR-0009 |
| Datum does not manage its own trust anchors | ADR-0010 |
| Desired state never causes a command to run | ADR-0011 |
| Only declared label values are substituted into desired state | ADR-0012 |
| Secrets are referenced in the repository and resolved on the host | ADR-0013 |
| The implementation is written in Go | ADR-0014 |
| Resource types are grouped into domains | ADR-0015 |
The reconciliation model itself is settled. Desired state comes from Git, observed state comes from the host, the two produce a plan, and the plan is applied and verified. The five phases and their order are not up for negotiation, because everything else in the design assumes them.
What is not settled¶
The fleet composition model is proposed, not accepted. Labels and matchers are the intended mechanism, and the precedence and conflict rules are written down, but they have not survived contact with a real repository yet and are expected to move.
Every resource type schema is proposed. The common behaviour they share is close to settled and the individual field sets are not.
More than sixty questions are recorded as unresolved, and they are collected in open questions. Several of them would otherwise be answered by accident during implementation, which is the main reason the list exists.
What the specification is held to¶
A code change that makes a statement on this site false is not finished until the statement is fixed. The documentation is the specification and not a description written afterwards, so the two moving apart is a defect in both.
That cuts the other way as well. Where the implementation found a gap the specification had not thought through, the answer goes into the documentation as a decision instead of staying in the code as an accident. The fourth pass outcome arrived that way, because an observe-mode pass that found work to do fitted none of the three that had been written down.
Versioning¶
The version of the agent is separate from the v1alpha1 schema version in every document, and the
two change on their own schedules. Schema
versions covers the rules for the second.
A build from a checkout carries the version 0.1.0~dev, which sorts below any released version, so
a machine cannot end up with a build claiming to be a release it is not. A release tag sets the
version the binary reports and the version its packages carry, and a tag with a suffix such as
v0.1.0-alpha.1 is published as a pre-release.
This page records what is built rather than what is intended, so it is updated with each release.