Getting started
Ten minutes, three ways in - Rust, Python, command line - all on the same file:
package/Lutetia.epc (with its Lutetia.h5) from the reference model
energyml-lutetia.
Lutetia is a synthetic, documented, multi-domain model written in RESQML 2.2,
EML 2.3, WITSML 2.1 and PRODML 2.x. Download the two files side by side in one
folder: an EPC names its data file relatively, so they must stay together.
git clone https://github.com/geosiris-technologies/energyml-lutetia
ls energyml-lutetia/package # Lutetia.epc Lutetia.h5
An EPC is a zip of XML parts (the objects) plus relationship files; the big numeric arrays live next to it in an HDF5 file. Reading the arrays needs the
hdf5feature (see Troubleshooting if it does not build).
Install
| You want | Do |
|---|---|
| the command line | cargo install energyml-cli --features hdf5 (installs energyml; enyml is an alias), or a pre-built binary from the GitHub releases |
| the Rust libraries | cargo add energyml --features all (or only the versions you read, for example --features resqml22; the energyml crate turns no schema version on by itself) |
| Python | pip install energyml-rust (imported as energyml.rust) |
| a container | docker pull ghcr.io/geosiris-technologies/energyml, see Docker |
Command line
energyml list package/Lutetia.epc # one line per object: uuid, type, title
energyml check package/Lutetia.epc # three independent checks, exit code 1 on errors
energyml geometry package/Lutetia.epc # what can be drawn, how many points, which CRS
energyml export package/Lutetia.epc --format geojson -o out/ # WGS 84 GeoJSON
energyml export package/Lutetia.epc --format vtu --properties -o out/
A section view of the model, as SVG (see Section view):
energyml profile package/Lutetia.epc \
--track "650700,6861600;651300,6861750;652100,6862100" \
--left 300 --right 300 --cross-every 300 --svg long.svg
Every option of every command is in the generated
cli-reference.md. The overview is The command line.
Rust
[dependencies]
# Lutetia is RESQML 2.2, EML 2.3, WITSML 2.1 and PRODML 2.x: name the versions you read, or "all".
energyml = { version = "0.1", features = ["all"] }
#![allow(unused)]
fn main() {
use std::sync::Arc;
use energyml::storage::EpcStore;
// Which schema versions this program understands is the application's choice:
// `energyml::schemas::all()` registers the versions whose feature is on.
let registry = Arc::new(energyml::schemas::all());
let store = EpcStore::open("package/Lutetia.epc", registry)?;
for summary in store.resources() { // no object XML is opened
let header = store.header(&summary.uri)?; // stops at the end of the citation
println!("{} - {}", summary.qualified_type, header.display_name());
}
for diagnostic in store.diagnostics() { // what the package gets wrong
eprintln!("{diagnostic}");
}
}
Reading a representation as something you can draw:
#![allow(unused)]
fn main() {
use energyml::geometry::{crs, geometry};
let object = store.get_object(&uri)?;
let shape = geometry::of(object.as_reflect(), &store)?; // patches of points and topology
let (vertices, indices) = shape.merged();
let placed = crs::of(object.as_reflect(), &store)?; // its coordinate reference system
}
The long form, with arrays, editing and exporting, is The library.
Python
from energyml.rust.epc import Epc
pkg = Epc.open("package/Lutetia.epc")
for one in pkg.objects():
print(one.qualified_type, one.uuid, one.title)
from energyml.rust.resqml import geometry_of, mesh_of
surface = pkg.find("TriangulatedSetRepresentation")[0]
points, triangles = mesh_of(pkg, surface.uri, "metres") # flat numpy arrays
No h5py is needed: the arrays come out of the same HDF5 backend the command
line uses. See Python.
What next
- Make your own objects: Creating objects. The Lutetia
package itself is replayed from a recipe by
energyml create --recipe. - Check what the library decided where the specification is silent: House rules.
- Something does not build or does not open: Troubleshooting.
The library
Reading, editing and writing EnergyML packages from Rust: objects, value arrays, geometry, coordinate systems, exports. The command line and the Python bridge are thin faces over exactly what is described here.
An application that wants all of it takes one dependency, energyml, which
re-exports the crates below under short module names (energyml::storage,
energyml::geometry, …); one that wants a single layer depends on that crate
directly.
The layers
graph TD
gen["generated crates: one per package and version"] --> core["energyml-core: reflection, XML/JSON, registry"]
core --> storage["energyml-storage: EpcStore, the EnergymlStorage trait"]
storage --> domain["energyml-domain: units, CRS, value arrays, properties, colours, tables"]
domain --> geometry["energyml-geometry: representations to points and topology"]
geometry --> export["energyml-export: OBJ, GeoJSON, VTK, CSV (registry)"]
domain --> proj["energyml-proj: reprojection over proj4rs"]
geometry --> build["energyml-build: creating objects"]
geometry --> section["energyml-section / energyml-sheet: SVG drawings"]
section --> views["energyml-views: the views of a package as pictures"]
build --> cli["energyml-cli, Python bridge"]
export --> cli
views --> cli
energyml-corehas no knowledge of any schema and no native dependency (it builds for WebAssembly). It holds the reflection traits, the XML reader and writer, the JSON encoding and the typeRegistry.energyml-schemasregisters every generated version in one place; the storage layer receives a registry and never builds one.energyml-storagereads and writes EPC packages behind theEnergymlStoragetrait.energyml-domainholds what does not change between versions: units, CRS, value arrays in every shape, statistics, property kinds, colours, lookup tables.energyml-geometryturns representations into points and topology.
Choosing versions
The generated crates are heavy, and most applications read one release. Each
schema version is therefore a Cargo feature of energyml-schemas, relayed by
energyml, spelled as Energistics spells a qualified type:
# RESQML 2.2 and the EML 2.3 it is built on. No 2.0.1, no WITSML, no PRODML.
energyml = { version = "0.1", features = ["resqml22"] }
eml20 … eml23, resqml20 (the 2.0.1 crate), resqml22, resqml22dev3,
witsml20, witsml21, prodml20, prodml22, prodml23; the families eml,
resqml, witsml, prodml; and all. A version feature brings the EML
version its crate is built on, so resqml22 brings eml23 and resqml20
brings eml20. The energyml facade turns nothing on by itself;
energyml-schemas keeps default = ["all"] for the tools that want everything.
energyml::build and energyml::providers are features too, and build
turns on the four versions it writes (RESQML 2.2, EML 2.3, WITSML 2.1,
PRODML 2.3).
Whatever the spelling in a document - 2.0, 2.0.1, resqml20. or
resqml201. - RESQML 2.0.1 is read by the one resqml20 feature; the trailing
1 is not needed. A version that is not compiled is a named refusal, never
a half read: energyml::schemas::missing_feature("resqml", "2.2") says “RESQML
2.2 is not compiled in: enable the resqml22 feature of energyml-schemas (or of
energyml)”.
Reading an object
#![allow(unused)]
fn main() {
use energyml_core::prelude::*;
use energyml_core::{path, registry};
use energyml_resqml_2_2 as resqml;
let mut reg = registry::Registry::new();
resqml::register(&mut reg);
energyml_eml_2_3::register(&mut reg);
// You know the content type (an EPC gives you one per part).
let object = reg.read_xml(
"application/x-resqml+xml;version=2.2;type=TriangulatedSetRepresentation",
bytes,
)?;
// Reach into it without knowing its type...
let title = path::get_string(object.as_reflect(), "Citation.Title")?;
// ...or downcast to the concrete one.
let repr = object.as_reflect().downcast_ref::<resqml::TriangulatedSetRepresentation>();
}
Reading a field as a plain value: one door
To read one field of a reflected object as a string, a number, a flag or a
nested object, use energyml_core::reflect::fields rather than matching on
FieldValue yourself. Its policy is the same for every reader:
- an absent or unset field is
None(an emptyVecfor the plural readers), never zero orfalse; - a blank string is absent for
text, and the text is returned as written; - a field of the wrong shape is
None; a string is never parsed as a number unless you ask for the_lexicalreader, and a double is an integer only when it is whole; require_text,require_number,require_integer,require_unsignedandrequire_flagrefuse by name (MissingRequired,TypeMismatch) when the caller cannot go on without the value.
#![allow(unused)]
fn main() {
use energyml_core::reflect::fields;
let title = fields::text(citation, "Title"); // Option<String>
let count = fields::unsigned(array, "Count"); // Option<u64>
let leaves = fields::objects(parent, "Patch"); // one or repeated
let first = fields::require_text(object, "Uuid")?; // a named refusal
}
See ADR 0046.
Opening a package
#![allow(unused)]
fn main() {
use std::sync::Arc;
use energyml::storage::EpcStore;
// `energyml::schemas::all()` registers the versions whose feature is on
// (see "Choosing versions": `features = ["all"]`, or the ones you read).
let store = EpcStore::open("model.epc", Arc::new(energyml::schemas::all()))?;
for summary in store.resources() { // no object XML is opened
let header = store.header(&summary.uri)?; // stops at the end of the citation
println!("{} - {}", summary.qualified_type, header.display_name());
}
for diagnostic in store.diagnostics() { // what the package gets wrong
eprintln!("{diagnostic}");
}
}
An object whose uuid is the nil UUID (00000000-0000-0000-0000-000000000000)
is read and listed as usual, and gets a warning in those diagnostics, whether it
is a part or held by value in another object: it identifies no instance
(RFC 4122 section 4.1.7; Energistics Identifier Specification v5.0, section
2.3.1). energyml check shows it in its package section.
Reference checking is a free function over objects in memory, so a client that does not hold an EPC can reuse it:
#![allow(unused)]
fn main() {
let issues = graph::check_references(&summary, object.as_reflect(), &|uuid| store.find_by_uuid(uuid));
}
The Energistics JSON encoding goes both ways:
#![allow(unused)]
fn main() {
let text = energyml_core::json::write_string(object.as_reflect(), true)?;
let document = energyml_core::json::read_string(&text, ®istry)?;
}
Finding an object: one door
An object is found by EpcStore::lookup. A bare uuid is accepted at the edge of
the library (lookup_uuid, resolve) and is turned at once into a full
identity; internally everything is keyed on the full URI, version included. See
ADR 0003.
An EPC as a named dataspace
EpcStore::open makes the file the default dataspace: every URI is
eml:///<type>(<uuid>). A server that serves several EPCs at once (a folder of
them, each a dataspace named after its file) needs the dataspace in the URI, or
two distinct objects of two packages would share one:
#![allow(unused)]
fn main() {
let store = EpcStore::open_in_dataspace("Lutetia.epc", registry, "Lutetia")?;
// eml:///dataspace('Lutetia')/resqml22.BoundaryFeature(<uuid>)
assert_eq!(store.dataspace().uri, "eml:///dataspace('Lutetia')");
}
EpcStore::from_reader_in_dataspace does the same for bytes in memory. The name
is an ETP dataspace path (Lutetia, projets/demo); one the URI grammar refuses
is an Error::BadUri. Every URI the store builds then carries the dataspace, and
the identity is the full URI: a URI of another dataspace, or without one, is
Error::NotFound (a bare uuid is still accepted at the edge and answers the full
URI). A reference names no dataspace and resolves inside the dataspace of the
object that writes it, so the target of references(), edges() and graph()
is spelled in its source’s. open and from_reader are unchanged.
Reading policy
Strict is the default. ReadMode::Lenient (CLI: --lenient) tolerates missing
required elements and reports one diagnostic per invention. Parts no parser can
read are listed by energyml repair. See ADR 0004.
The members of a wellbore
What does this well have? Its trajectories, the frames that sample along it, its marker
frames (the tops), and the properties that hang on them. The question needs no array and no
geometry, only the type, the uri and the title of each object and the references between
them, so the answer is a pure function in energyml_domain::well_members, and a grid of
wells and the depth coverage walk the same members.
#![allow(unused)]
fn main() {
use energyml_domain::well_members::well_members_of_index;
for well in well_members_of_index(&index) {
println!("{}: {} trajectories, {} frames, {} properties",
well.title, well.trajectories.len(), well.frames.len(), well.properties.len());
}
}
well_members(objects, outgoing) is the same walk over any inventory: objects are
ObjectFacts (uri, qualified type, title) and outgoing(uri) gives the uris an object
references, so a dataspace of a server can give its own edges. It is asked once per
relevant object.
- A well is seeded from both ends. A feature is a row even with nothing on it, and a representation that reaches no well makes its own row. A representation reaches its well by its interpretation and the interpretation’s feature; a WITSML trajectory by its wellbore and the feature that names that wellbore.
- The kinds are a table (
MEMBER_KINDS), read by the local name of the type without theobj_of release 2.0.1, so one concept is the same in every release. - A property is categorical when it references a table of categories; the entry lists those apart.
- The walk does not judge. Whether a member can be drawn, the wording of a state and the telling apart of two wells with one title are the business of the face. The result is sorted by lower-case title, then by key.
See ADR 0040.
Editing and saving
#![allow(unused)]
fn main() {
use energyml_storage::SaveOptions;
let mut store = EpcStore::open("model.epc", Arc::new(registry))?;
energyml_core::path::set(
store.edit(&uri)?, // marks the object modified
"Citation.Title",
energyml_core::value::OwnedValue::Str("Renamed".into()),
)?;
store.remove(&other_uri)?;
store.save(&SaveOptions::default())?; // atomic: temp file, then rename
}
Nothing touches the archive until the save, and the save rebuilds the
package: [Content_Types].xml, every .rels file in both directions and
docProps/core.xml. A description that no longer matches the contents is worse
than none.
A transaction (start_transaction, then commit_transaction or
rollback_transaction) is a layer over everything the package holds that a
save would write: the objects, the bytes staged for a part, the media links,
the parts removed or superseded, the names of data files and the content types
of attachments. A rollback restores all of them, not only the objects.
Value arrays
An object does not contain its numbers: it names a file and a path inside it.
Reading arrays needs the optional hdf5 feature.
#![allow(unused)]
fn main() {
for location in store.arrays_of(object.as_reflect()) {
let data = store.read_array(&location)?; // one copy, into a typed Vec
if let Some(points) = data.f64() { // borrowed: no copy, no conversion
draw(points);
}
}
// Or straight into memory you already own:
let shape = store.array_metadata(&location)?;
let mut vertices = vec![0.0f64; shape.dimensions.iter().product::<i64>() as usize];
store.read_array_into(&location, &mut ArrayBuffer::F64(&mut vertices))?;
}
Writing is two calls on purpose, because they fail for unrelated reasons:
#![allow(unused)]
fn main() {
store.write_array_to("model.h5", "/resqml22/<uuid>/points", &data)?;
store.stamp_external_file("model.h5", proxy_uuid)?; // the root `uuid` attribute
store.register_external_file(proxy_uuid, "model.h5"); // so `save` names it
}
ArrayBackend is the seam: HDF5 is one implementation, and a WebAssembly
application supplies its own without changing anything else. The MIME type of
an array picks the backend. Without a suitable one, only the array read fails,
and it says which capability is missing (FeatureNotEnabled) rather than
returning an empty array.
Any array, whatever form it takes
An EnergyML value array is a choice of about twenty concrete types, and only a
third of them touch a file; the rest are constants, ranges, lattices, inline
values or derived from other arrays. energyml_domain::values resolves all of
them:
#![allow(unused)]
fn main() {
use energyml_domain::values;
let values = values::read(array, &store, "Values")?; // whatever form it is
values.integer(); // Some(&[i64]) if that is the family
values.boolean();
}
read widens to one type per family; read_native keeps the file’s own type;
read_into allocates nothing.
What read refuses, by name. A computed array that lacks what it computes
from: a constant with no Value or no Count (a boolean constant is never
false by default, a string constant never ""), a lattice step or a range with
no Count. A Count above 2^30 values, or a range that would pass i64::MAX:
the file asked for an allocation the host cannot honour, so it is refused before
anything is allocated. An inline XmlArray whose elements are not all of the
array’s type: no element is dropped, because dropping one shifts every value after
it. An integer array reads exactly: 64-bit values are never widened through
f64, and a u64 above i64::MAX is refused. values::count agrees with
stored_dims and read on an external part that designates a window
(StartIndex and Count) of its dataset.
Strings. A string dataset in HDF5 (fixed or variable length, ASCII or UTF-8)
is read as ArrayValues::Text, and values::read widens it to Values::Text.
An empty string is a value, not a gap. A backend that does not decode strings
makes the column refuse by its name; it never answers an empty column or numbers
in place of text (ArrayValues::Text has no bytes to lend, and as_bytes answers
an empty slice). The HDF5 backend reads fixed-length strings up to 1024 bytes and
refuses longer ones by name; it does not write strings.
Tables. A table column that declares ValueCountPerRow greater than 1 reads
its values per row: Rows::per_row says how many, Rows::values_of(row, column)
gives the values of one row and Rows::row_values(row) those of a whole row.
Rows::cell answers only for a scalar column, and says None for the others
instead of a value that belongs to another row. It reads a cell as ColumnCells
does: an integer equal to the array’s NullValue and a NaN are None, never the
number -999 or NaN (values_of is None when any value of the row is absent).
The keys of a DoubleTableLookup
keep their fractional part (Values::Floating); those of a StringTableLookup
stay integers.
Facets. A property or a table column carries qualifiers, read by one function
whatever the release: RESQML 2.0.1 writes the kind in Facet and the value in
Value, EML 2.3 and RESQML 2.2 write the kind in Kind and the value in Facet.
Facet.value is a FacetValue, Standard when the schema lists it and
Extension when it is the producer’s own
(ADR 0024).
Reading a block. values::read_block reads a block of an array by its
stored dimensions (start and count per dimension) and not the rest; a
source that hands back a different number of values than the block holds is
refused by name. values::element_block and values::slab_block compute the
block for elements, or for some slabs of the slowest free dimension (a time step
of a series, one realization), from the shape of the property: they are the one
block arithmetic, and the description of a property by slabs
(property::chunks::PropertyChunks) reads through them. The page
The shape of a property gives the shape, the choice of a
component and the description of a property.
From a representation to something you can draw
#![allow(unused)]
fn main() {
use energyml_geometry::{crs, geometry};
let surface = store.get_object(&uri)?;
let shape = geometry::of(surface.as_reflect(), &store)?; // patches of points
let (vertices, indices) = shape.merged(); // one buffer of each
if let Some(crs) = crs::of(surface.as_reflect(), &store)? {
let placed = crs.project_all(&vertices); // local -> projected
}
}
An EnergyML point array is not a file offset: of five concrete point-array forms only one means “read the HDF5”; a lattice carries an origin and spacings and its points are computed. Every form is resolved, including parametric and borrowed-from-another-representation arrays; a form that a representation still cannot take is refused by name instead of returning nothing. Supported: triangulated and polyline sets, point sets, 2D grids, IJK grids (faults, K gaps, parametric pillars, local grids), unstructured grids, wellbore trajectories and frames, sub-representations.
Readers are a registry (energyml_geometry::Readers) keyed on the
version-independent concept name, so a new release that keeps a concept needs no
new reader.
Directional surveys
A survey is read in energyml_geometry::stations, for a WITSML 2.1 Trajectory and for a RESQML 2.0.1
DeviationSurveyRepresentation, in four steps that each answer on their own.
- Read:
stations_oflists the stations of the file, the values it declares at them (Tvd,DispNs,DispEw,VertSect,Dls) and the vertical section it declares (AziVertSectand its origin), and places nothing. - Judge:
diagnoseruns ten checks in a written order and reports each as passed, failed with a typedFinding, or not reached.readreplays the same steps and stops at the first failure with the same sentences. - Place:
read_declaringplaces the survey from its datum. A convention the file does not state is a parameter of the call (SurveyDeclarations): the north whenAziRefis absent, and whether rejected stations are left out. The result carries where its north came from (NorthSource: stated, corrected from the file’s convergence and declination, declared, or not needed) and the stations it kept.survey_pathgives the line between the stations as aPlacedPath. The points are in the units of the CRS the object is in. - Compare:
energyml_analysis::survey_qcworks on a read survey.SignificanceCriterionsays whether an azimuth carries information,SectionAzimuthgives the azimuth of the vertical section,comparesets a survey against another path,rotation_fitfits the rotation between two surveys of one well,stated_against_computedcompares the values the file declares with those recomputed, andpairingspairs the surveys of a well.
#![allow(unused)]
fn main() {
use energyml_geometry::stations;
let diagnosis = stations::diagnose(&survey, &store, &declarations);
for (check, finding) in diagnosis.failures() {
println!("{check:?}: {finding:?}");
}
}
The choices the registry makes for a survey are in House rules and ADR 0034.
One CRS, whichever version wrote it
A coordinate system is one object in RESQML 2.0.1 and three joined by references in EML 2.3. One reader covers both:
#![allow(unused)]
fn main() {
let crs = crs::of(surface.as_reflect(), &store)?.unwrap();
crs.projected.epsg(); // Some(23031)
let placed = crs.to_metres(point)?; // placed, and in metres
}
to_projected applies the local placement (rotation, offset, depth/elevation
flip). to_metres also normalises units. Units convert on the string the file
contains, never on a generated Rust variant (LengthUom::Mm meant the megametre
in one release and the millimetre in another). A unit outside the table is an
error, not an assumption. Resistivity, conductivity and angle per length (dogleg
severity) convert too; see
Statistics, scales and measured depth.
crs.axes says what the file states about the order of the two horizontal axes
of the placed frame (Stated, NotStated, Unusable or Contradictory; it never
guesses, and a missing statement never stops a point from being placed). Whether
that frame is direct or indirect decides which side is right around a well.
direction_to_projected is the door for vectors as to_projected is for points:
the same rotation and vertical flip, without the offset. See
Around the well.
Reprojection goes through the reproject::Reprojector trait; energyml-proj
is the shipped implementation, pure Rust over proj4rs (so it builds for
WebAssembly). It refuses a definition that carries no datum rather than applying
one silently.
Writing it out, and adding a format
#![allow(unused)]
fn main() {
use energyml_export::{Exporters, Options};
use energyml_geometry::scene;
let object = scene::object_of(surface.as_reflect(), &store, &uri)?;
let mut exporters = Exporters::builtin();
exporters.push(Box::new(MyVtkExporter)); // tried before the built-ins
exporters.get("vtk")?.write(&object, &mut file, &Options::new())?;
}
Built-in formats: OBJ, GeoJSON (always WGS 84, as RFC 7946 defines it), CSV, and
three VTK flavours (legacy, XML PolyData, XML UnstructuredGrid). A SceneObject
carries the shape, the CRS and the identity chain (uuid, URI, title, and the
interpretation and feature above it).
Metadata is a file, not a code change
What an export carries beyond geometry is a JSON map of key to dotted path, and a path crosses references transparently:
{
"title": "Citation.Title",
"contentType": "contentType",
"featureTitle": "RepresentedInterpretation.InterpretedFeature.Citation.Title"
}
The same keys come out as GeoJSON properties, VTK FieldData and OBJ
comments. A path that finds nothing yields no key; a key that matched nowhere in
the whole package is reported.
The storage trait
EpcStore implements EnergymlStorage, a trait drawn after the ETP protocol.
The ETP client lives in a separate repository,
retp, and implements the same
trait for a remote dataspace: an application written against the trait changes
transport by changing the line that builds the store.
The objects a part holds by value (a WITSML Channel in a Log) are data
objects of their own: resources() lists the parts, contents() the contained
objects with their edge to the container, and references() gives a reference
written inside a contained object to that object (ADR-0023).
Reading the package once per operation
An operation that touches many objects reads each shared object once. The caches that do it live no longer than the operation, so none can serve a stale answer:
property_kind::Resolverkeeps thePropertyKindobjects it found (and those it did not) for its own life: build one per operation, not one per property.well_strata::StrataCachekeeps the ranks, the merged colour sets and the measured depths of a package for the frames passed towell_strata::of_frame_cached(andstrata_by_md::of_representation_cached);of_frameuses a fresh one per call.integrity::runreads each table, and each link target, once for all the table checks of the run.energyml_cache::colours::decisions_ofanswers the colour of several objects in one walk; the views ask it throughColourDoor::paint_all.
Further reading
Creating objects · Geostatistics · Schemas · House rules
The shape of a property
A property is numbers attached to the elements of a representation: one value, or
a vector or a tensor, per node, per cell, per triangle. Reading the numbers is
done (Value arrays); this page is about what comes before and
after: what shape the numbers have, how to read part of them, and how to
describe a property without loading it whole. Everything here is in
energyml-domain, in pure Rust, and builds for WebAssembly.
The library refuses rather than guesses. A shape that does not add up, a block the store did not honour, a pick that does not fit: each is a named refusal, never an empty array that looks like a property with no values.
The order of the dimensions
RESQML TRG 2.2 §7.1 (p. 222): “First (left) given dimension is slowest and last
(right) given dimension is fastest.” The values of a property are structured, top
is slowest, as [RealizationCount] [TimeCount] [IndexableElementCount] [ValueCountPerIndexableElement] (§7, figure p. 221), and the Usage Guide 2.2
§6.6.3 (p. 65) gives the same four dimensions in the same order. The Usage Guide
§4.2.2 (p. 39) adds the rule that makes a flat store and an N-dimensional one the
same data: “when viewed as an equivalent 1D array, the HDF5 data storage ordering
and the RESQML index ordering are identical.” Flattening consecutive dimensions
does not change the order of the values.
property::shape::ElementShape is the shape of the components of one element,
slowest first, with their labels (LabelPerComponent, linearised like the values).
A scalar is [1]. ravel and unravel are the only multi-index arithmetic of the
library; a zero dimension is refused.
How many elements a representation has is property::element_count, and it
refuses a pair the guides do not define. A wellbore frame has NodeCount nodes and
NodeCount - 1 intervals; interval edges is refused on a frame: the Usage
Guide 2.2 §4.2.1 Table 1 (p. 38) lists intervals for “wellbore frames,
column-layer grids” and interval edges for “column-layer grids” only.
PropertyShape is the whole shape, [realizations][time][elements...][components...].
The elements are ElementAxes::Ordered when the guide states an order (an IJK grid
is [nk, nj, ni], a frame’s nodes [n]) and Unordered otherwise (triangles,
scattered points). PropertyShape::layout(stored) confronts the shape the document
declares with the one the store holds, and accepts only a regrouping of
consecutive dimensions: [n, 3, 2], [n, 6] and [n * 6] are the same property,
[6, n] is not.
Time
A property dated by a series (the 2.2 TimeOrIntervalSeries) has a time dimension;
one dated by a single index (2.0.1 TimeIndex) has none, and the Usage Guide says
that inherited index “should be ignored if also present”. With UseInterval, a
series of n instants has n - 1 steps (the library’s reading of an element the
EML TRG 2.3 leaves without text, p. 296; see
ADR 0029). A time dimension
that contradicts the series is TimeCountMismatch.
Choosing a component
ComponentPick is the same choice for every view: All, At (one component by its
multi-index), Along { axis, at } (every component along an axis, the others
fixed), Reduce (one value per element), and AlongReduced. check says before
anything is read whether the pick fits the shape; width and positions say what
it keeps; select applies it to one element’s values. A NaN chosen by All, At
or Along stays NaN.
use energyml_domain::property::shape::{ComponentPick, Reduction};
// The plane "second component of the first dimension fixed", every element.
let plane = ComponentPick::Along { axis: 0, at: vec![0, 1] };
Reducing a dimension
AlongReduced { axis, at, reduced, by } shows the dimension axis, fixes the
dimensions it does not name at at, and reduces the listed ones: a projection
of a matrix onto one of its axes. by is a Reduction (Mean, Min, Max, and
Norm for a whole vector only), whose ids ("mean", "min", "max", "norm")
are stable and are what a view document writes. Reduction::apply is the one
arithmetic: only finite values take part, and a mean over several reduced
dimensions is the mean of the finite values of the whole sub-block, not a mean
of means (see ADR 0042). The
drawing crate has its own reducers for rows of pixels; the two are written twice on
purpose, because drawing knows nothing of the domain
(ADR 0028).
ComponentPick::narrowed says which sub-block of the components a pick reads, so
that a store which keeps each component dimension apart can be read for that block
only. For a matrix of 301 by 81 by 81 values per element, one plane is 24 381 values
rather than 1 974 861.
Reading a block
values::read_block reads a block by the stored dimensions, and a source that
returns another number of values than the block holds is refused by name
(values::stored_dims gives the stored shape from the header, reading no value).
values::element_block is the block of a run of elements; values::slab_block is
the block of some slabs of the first free dimension, with the slower ones fixed (a
realization, a time step), and, when the store allows, only the components a
narrowed pick reads. They are the one block arithmetic: nothing else in the library
computes a start and a count.
What each dimension stands for
component_axes says what each dimension of a vector property stands for: labels,
a paired property, a table linked by the convention B (a qualifier facet whose
value is a full ETP uri, property::qualifier_link), a declared link, a WITSML
AxisDefinition, or ranks when nothing says. It proposes a table
(candidate_tables) and never applies it. series::matrix reads a matrix per
element along a well (MatrixChannel) and one plane of it (read_plane), in part
when the store allows.
Siblings and shape candidates
PackageIndex::siblings finds the properties that go together (the horizontal and
vertical sister, the time steps of one property, a pair radius and angle), with a
Kinship that says why. PackageIndex::shape_candidates gives the facts a
detection needs without reading any array.
Describing a property
property::chunks::PropertyChunks presents a property as the ValueChunks that
stats::describe::Describer reads, slab by slab, without loading the array:
use energyml_domain::property::chunks::{ChunkSelection, PropertyChunks};
use energyml_domain::stats::describe::{Describer, Plan};
use energyml_domain::stats::modes::StatMethods;
let property = property::read(object)?;
let chunks = PropertyChunks::new(&uri, &property, elements, instants, None,
&ChunkSelection::default(), &source)?;
let mut describer = Describer::new(&chunks, Plan::default(), StatMethods::builtin())?;
while !describer.progress().is_done() {
describer.step(&chunks, 16)?; // a few slabs, then ask whether to go on
}
let description = describer.finish()?;
The slowest free dimension is time when every step is kept (TimeChoice::All) and
the first dimension of the elements otherwise. ChunkSelection carries the pick,
the time step (TimeChoice::Step) and the realization, which must be chosen
when the values have one dimension of them. A pick that reads one plane reads only
that plane from a store that keeps its component dimensions apart; a flat store is
read whole, and reads_narrowed() says which happened.
The NullValue of an integer array is left to the describer, which counts the
nulls and leaves them out, unless the pick reduces: the null is then replaced by
NaN before the reduction, and nulls_before_reduction() says how many were met.
A categorical property (it names a lookup table), a boolean or text one, several
patches, and a property that declares several values per element and holds one
under a pick other than All, are refused by name (ChunksRefusal).
keep_ordered_axes drops from a description what it says of elements that have no
“along” (AxisId::Element { ordered: false, .. }): no variation is measured across
triangles.
A property and its sister
PairWalk reads a property and its sister by the same slab into the
PairAccumulator of stats::pairs: the density, the ratio y / x, the share that
satisfies a rule the caller gives, the correlation. Two properties that do not read
as the same dimensions are refused, never aligned silently. The kinship of the two
is not checked: the caller chose them. What it counts is a figure; that a share is a
warning on a directional ratio is the rule house/property/directional-ratio
(see Statistics, scales and measured depth).
What a property says of itself
property::observe::observations lists, typed and without sentence, what the
file declares about a property, reading no value:
Shape: the declared shape and how the store regroups it;StoredDisagrees, orDeclaredCountWrongwhen several values per element are declared and one held;ExtensionFacet,FacetValueOfOtherKind,FacetKindRepeated: a producer’s own facet value (EML TRG 2.3 §11.2 p. 284), a standard value under another kind than the schema documents for it (the tableFACET_VALUE_KINDS, copied from the documentation of the schema and confronted with it by a test; not applied to a 2.0.1 value, which is free text), several facets of one kind (§11.11 p. 294);Unit: the unit against the quantity class of the kind:Checked,Outside, orNotCheckablewith the reason (no unit, no kind, a kind that does not resolve, no class, a class this build does not convert). A class this build does not convert is never reported as checked;Kind: another issue of the kind (abstract, deprecated);Synthetic: thesyntheticmark of ADR 0019, on the property or on its supporting representation. Nothing is said of an object without the mark.
These sit beside the observations of the values (stats::describe::Observation),
and no fact is written twice. The decisions of interpretation are in
ADR 0042.
Statistics, scales and measured depth
Two small toolkits of energyml-domain give every view the same arithmetic:
stats describes values, along_md moves along a measured depth (MD). Both are
pure Rust with no native dependency, so they build for WebAssembly. The numbers
they share (three median steps, ten median steps, three decades, the 1st and
99th percentiles) are decisions recorded in
ADR 0027 and listed as the
stats theme of the house rules.
Describing values
stats::Accumulator takes values in one pass and can be merged with another:
the count, the finite values, the NaN, the infinite values and those equal to
the declared NullValue are counted apart, and a missing statistic is None,
never zero.
use energyml_domain::stats::Accumulator;
let mut acc = Accumulator::with_null_value(Some(-999));
acc.extend(&[1.0, 10.0, f64::NAN, 1000.0]);
assert_eq!(acc.finite(), 3);
assert_eq!(acc.not_a_number(), 1);
let decades = acc.decades(); // Some(3.0): the log scale is proposed
quantiles are exact up to 10 million values. Beyond, a mergeable sketch
(DDSketch, relative error 1 %) answers, and the result says that it is a sketch
and what its error is.
Scales
stats::Scale is the scale of the value (linear or log10, bounds, direction).
It is not color::Scaling, which places a value on a colour map. A log scale is
proposed (propose_log) when the values span at least 3 decades, and automatic
bounds (auto_bounds) are the 1st and 99th percentiles. A scale the file states
or the user chooses wins, and a disagreement with the file is said.
Describing by steps
stats::describe is a door by steps, on the model of the package index: a
Describer is created for a plan, then advanced slab by slab with step, and
finish returns the Description. A description that was cancelled is not a
partial one.
Along the measured depth
along_md receives MDs already in one unit and one datum (see
strata_by_md::comparable); it never compares references itself.
#![allow(unused)]
fn main() {
use energyml_domain::along_md::{
GapRule, MdStepper, StepMode, bracket, interpolate_row, Interpolations, Nature, LINEAR,
};
let nodes = [3000.0, 3001.0, 3002.0];
let b = bracket(&nodes, 3001.4, 1e-6).unwrap();
assert_eq!((b.lower, b.upper), (1, 2)); // between nodes 1 and 2, t = 0.4
let stepper = MdStepper::new(nodes.to_vec(), StepMode::Nodes).unwrap();
assert_eq!(stepper.step(3000.6, 1).md, 3001.0); // the node strictly beyond
let rule = GapRule::of_axis(&nodes).unwrap(); // at most 3 median steps
let registry = Interpolations::builtin();
let linear = registry.get(LINEAR).unwrap();
let row = interpolate_row(&[1.0], &[3.0], 0.5, 1.0, &rule, linear, &Nature::default()).unwrap();
assert_eq!(row.values, vec![2.0]);
}
bracketsays where an MD falls, with a tolerance for “on the node”; an MD outside the support is refused with its bounds, never clamped.nearest_nodebreaks a tie towards the shallower node.union_axismerges the nodes of several supports.MdSteppersteps by notches, from node to node or on a fine grid, snaps onto a node, and says when it was clamped at an end of the axis.Interpolationsis a registry (nearest, previous, linear, log_linear, angular); the additions of the caller are tried first. A method refuses where it has no sense: categories and booleans only take nearest and previous, values that hold over intervals only previous, a logarithm needs positive values, two opposite angles have no shorter way round. Linear on an angle is accepted with the noteAngleTakenAsLinear.interpolate_rownever bridges a missing value (aNaNneighbour gives aNaNcell, counted inmissing) and never bridges a gap wider than theGapRuleallows. On a node the value is the node’s.JUMP_RATIOis the companion constant: a jump of the value beyond ten median steps of its variation is signalled as a seam, not smoothed.
Units
Resistivity (ohm.m and the others of the schema), electric conductivity
(S/m) and angle per length (the dogleg severity, 1/30 dega/m, rad/m…)
convert through the one door, units::convert, and can be checked against their
quantity class. units::resistivity_reciprocal turns a resistivity into the
conductivity of an isotropic material and back, and says so when it cannot.
The command line
energyml is the library with argument parsing in front of it. Nothing the
command does is out of reach of an application: every check it prints is a call
you can make yourself. (enyml is installed as a short alias of the same
binary.)
Every option of every command is in the generated
cli-reference.md. This page is the how and the why.
Some commands exist only in some builds. Commands that read package arrays need
the hdf5 feature; profile --png needs png. ETP commands (talking to a
remote dataspace) belong to the separate
retp project. energyml mcp file.epc serves a package to an AI assistant as MCP tools; it is the separate
program energyml-mcp (cargo install energyml-mcp-cli, see
The MCP server). galaxy-tool
and cli-reference are hidden from --help (see Galaxy tools).
Global flags worth knowing: --lenient (read past a missing required element or
an unreadable value, reporting each), --strict (refuse; use it when the objects
will be written back), --h5 [NAME=]PATH (offer an HDF5 file whatever the
package says), --log (see Logging).
Looking at a package
energyml list model.epc # uuid, type, title; --filter grid, --quick
energyml check model.epc # three independent checks; exit 1 on errors
energyml check model.epc --errors-only --no-schema
energyml show model.epc <uuid> # one object; --json, --with-data, --raw
energyml arrays model.epc --summary # where the numbers live; --values N needs hdf5
energyml properties model.epc --check # what the numbers mean, and counts that disagree
energyml colors model.epc # what colour each object has, and which level said so
energyml tables model.epc # lookup tables behind discrete properties
energyml wells model.epc --out wells.csv # wellbores as a table, one row per node
energyml series model.epc <uuid> # a log, a time series, a flow test as series
energyml geometry model.epc # what can be drawn, how many points, which CRS
energyml graph model.epc --format dot -o graph.dot # the reference graph
energyml types --describe TrianglePatch # a schema type, field by field
energyml docs --out docs/ # the model's own documentation as JSON
geometry ends with what it could not evaluate, by name: a viewer that
shows nothing and a viewer with nothing to show look identical, so the forms a
build does not evaluate are said aloud rather than counted as zero points.
graph writes one node per object (uri, uuid, qualified type, the
version-independent concept, title) and one edge per reference. An object the
package references without holding gets present: false; dropping those would
draw a self-contained package where there is none.
Exporting
energyml export model.epc --list # the formats this build has
energyml export model.epc --format obj -o out/ # one file per representation
energyml export model.epc --format geojson --metadata metadata.json -o out/
energyml export model.epc --format vtu --epsg 3857 --properties -o out/
A directory export names files qualifiedType_uuid.ext: the uuid makes it
unique (titles collide far more often than expected) and the qualified type
makes the folder readable. One representation that cannot be written does not
stop the others: it is named with the reason and the batch carries on.
--as PATTERN=SHAPE chooses the form each type comes out in (points, lines,
surface, volume, auto, plus the summaries outline and centroid). The
pattern matches anywhere in the qualified type, case-insensitively; rules are
tried in order, first match wins. A shape an object cannot take is an error,
not a fallback:
| From | points | lines | surface | volume |
|---|---|---|---|---|
| point set | yes | no | no | no |
| polylines | yes | yes | no | no |
| triangulated set | yes | no | yes | no |
| 2D grid | yes | no | yes | no |
| IJK grid | yes | no | yes (its outside) | yes |
| unstructured grid | yes | no | no | yes |
An IJK grid asked for as a surface gives its outside - the faces used by exactly
one cell - not every face twice. GeoJSON is always WGS 84, and every feature
carries a bbox.
What a file says is true of what it holds:
- Reprojection. With
--target-epsgthe OBJ and VTK headers name the target system, not the source’s. On a time-domain object, a stated time equivalence and a target system are both applied (the equivalence on the vertical, the reprojection on the horizontals); GeoJSON places a time axis through the equivalence too, in WGS 84 and with--native-crs, and writes the equivalence as avertical_axisproperty. .vtp.PolyDatahas no volume cell: a hexahedron is six polygons and a polyhedron one per face, so each cell’s value is repeated over its polygons. A cell property that does not have one value per cell is refused by name. (.vtukeeps one cell per cell.)- GeoJSON. One feature holds one kind of geometry: patches of different topologies, or an index that names no node, are refused by name instead of being dropped.
Repairing, relabelling, renaming
energyml repair model.epc # the parts no parser will touch, and why
energyml repair model.epc --extract broken/ # their bytes, under their part names
energyml repair model.epc --apply broken/ --out fixed.epc
energyml relabel model.epc --package resqml --schema-version 2.2_dev3 -o fixed.epc
energyml h5 model.epc # where each array really is
energyml h5 model.epc --fix fixed.epc
energyml rename model.epc --to survey.h5 # the quote: fields, objects, relationships
energyml rename model.epc --to survey.h5 --apply # and the file itself
energyml extract model.epc --select choice.json --out part.epc --dry-run
repair is for the parts every other command must refuse (markup that does not
close, bytes that are not UTF-8, an empty part): bytes out, fix, bytes back, and
--apply says for each part whether it now parses. --lenient makes the read
levels a ladder: what repair lists is what that mode cannot read.
rename and extract always quote first (--dry-run / no --apply).
extract never writes onto the package it reads (the output is refused when it
is the source), and builds its output beside the final name, moving it into
place only once it is complete: a failure leaves whatever was at the output as
it was.
Selecting by region
energyml select model.epc --type WellboreTrajectory --outer zone.geojson --rings-epsg 2154 --test head
Prints the uuids (or --format uri|json) of the objects that fall in a region
drawn by outer rings, minus inner ones (--inner), or its complement (--invert).
Creating objects
energyml create is the command form of the operations engine; see
Creating objects.
energyml create --list # what can be made from nothing
energyml create --list --from export.geojson # from this file, refusals named
energyml create crs --set epsg=2154 --set vertical=5720 --plan # the quote
energyml create crs --set epsg=2154 --out crs.epc # the write
energyml create --recipe recipe.toml --plan
energyml create --recipe recipe.toml --out Lutetia.epc
Parameters are the operation’s to declare, so there is one generic
--set name=value rather than an option per parameter. --list prints each with
its kind, whether it is required, what it depends on or excludes, and what
leaving it out makes. A value is read as JSON when it is one (2154, true) and
as text otherwise. --now fixes the instant, so two equal runs write the same
package byte for byte.
energyml import file.json --plan reads a GeoJSON or a GeoSciML GeologicUnit
export and writes the objects it describes (quote first).
Public data
energyml sources # the APIs this build can query
energyml sources brgm-bss # collections, filters, suggested values
energyml fetch brgm-bss boreholes --area area.geojson --where codeSpace=BRGM # the quote
energyml fetch brgm-bss boreholes --area area.geojson --where codeSpace=BRGM --out bss.json
energyml terrain --list --area 651000,6860000,653000,6862000
energyml terrain --area 651000,6860000,653000,6862000 --step 25 --set vertical=5720 --out dem.epc
fetch and terrain quote first and download only when told to. Nothing about a
provider is spelled in the CLI: it is declared by the provider and read back as
data. See Public data sources.
Section view
energyml profile model.epc --track track.geojson --left 300 --right 300 --svg long.svg
energyml profile model.epc --track track.geojson --left 300 --right 300 --draw plan --svg plan.svg
energyml profile model.epc --track track.geojson --left 300 --right 300 --cross-every 300 --draw cross:2 --svg cross-2.svg
See Section view.
House rules
energyml rules --house # what this library decides where the spec is silent
energyml rules --house --theme colour --json
See House rules.
Exit codes
check exits 1 when it finds errors. A refusal that is a missing capability (a
build without hdf5) says which feature is missing; it never prints an empty
result in its place.
Python
A bridge over the Rust library: open, read, edit and write EPC packages and their HDF5 files, ask for geometry and properties, export, check. Like the command line it knows nothing the library does not; every answer is a call an application would make itself in Rust.
The wheel is built with every schema version (all): nothing to choose from
Python. The per-version Cargo features of the energyml crate concern Rust
applications only (see The library).
You do not need h5py: arrays come out of the same HDF5 backend the command
line uses. Binary wheels carry the HDF5 library, so the machine that installs one
needs neither HDF5 nor a Rust toolchain.
Name and install
The distribution is energyml-rust and the import is energyml.rust - a room in
the energyml.* namespace shared with the pure-Python generated model packages
(energyml-resqml2-2 and siblings, energyml-utils), not a competitor to them:
from energyml.resqml.v2_2 import resqmlv2 # generated classes (other distribution)
from energyml.rust.epc import Epc # this bridge
The wheel ships no energyml/__init__.py, so it never overwrites another
distribution’s files.
pip install energyml-rust
To build a wheel from the repository instead (needs Rust and HDF5):
pip install maturin
maturin build -m crates/energyml-python/Cargo.toml --release
The wheel is abi3: one wheel per platform, not one per Python version. On
Windows, a locally built wheel must be repaired so it carries HDF5 (see
Troubleshooting, DLL load failed).
Out of scope on purpose: ETP. For a remote dataspace use the retp project.
The map
| module | what is in it |
|---|---|
energyml.rust.storage | the abstraction above every store; Epc is its first subclass |
energyml.rust.epc | open, read, edit and write an EPC and its HDF5 files; ask its catalogue questions |
energyml.rust.model | the generated model by reflection: which types exist, what they hold, an empty object of one |
energyml.rust.resqml | geometry of a representation, values on it, export |
Opening a package
from energyml.rust.epc import Epc
pkg = Epc.open("package/Lutetia.epc")
for one in pkg.objects():
print(one.qualified_type, one.uuid, one.title)
The .h5 files are looked for beside the .epc. A summary is what is known
without opening the part; a part that cannot be parsed keeps its row with an
empty title (dropping it would say it is not in the package). Create an empty
package with Epc.create("name.epc") and tell it where data files go with
set_data_root(directory).
pkg.find("Representation") # by type, case-insensitive
pkg.find("Crs", "default") # and by title (both mean "and")
What points at what
for link in pkg.references(uri):
print(link["qualified_type"], link["title"], link["path"], link["present"])
pkg.referenced_by(uri) # the other direction (reads every object: no package indexes it)
needed, missing = pkg.needs([uri]) # the closure: what an extraction or a copy must carry
referenced_by and needs read every object, and refuse (an exception that
names the unreadable objects) when one cannot be read: the unreadable part might
be the one that points at the target, and “nothing depends on it” would be a
wrong answer to give before a deletion or a cut. Repair or remove the named part
first.
present says whether the package holds the target; a dangling reference keeps
its entry. A well-known PropertyKind from the Energistics dictionary is never
reported as dangling.
Taking objects into another package
small = Epc.create("small.epc")
small.set_data_root("out")
done = small.copy_from(pkg, [surface.uri])
done["added"] # what came along: interpretation, feature, CRS...
done["arrays"] # datasets that travelled
small.save_as("out/small.epc")
The set is closed on the way in: asking for a surface alone would give a package that opens and places it nowhere.
The model and making an object
from energyml.rust.model import describe, template, types
types("resqml") # every RESQML type this build carries
describe("resqml22.BoundaryFeature") # fields, cardinality, documentation
There are no Python classes, by decision: one class per type per release would be a second copy of the generated model. You get the reflection and a skeleton:
import uuid
obj = template("resqml22.BoundaryFeature", str(uuid.uuid4()))
obj["Citation"].update(Title="Top Chalk", Originator="me",
Creation="2026-09-19T12:00:00Z", Format="energyml-rust")
pkg.put(obj)
The uuid is always given, never invented: nothing defaults to a random one, because a caller that derives identities from its source must get the same objects on a second run.
Objects travel as JSON
obj = pkg.get(uri) # a dict
obj["Citation"]["Title"] = "Renamed"
pkg.put(obj) # back in; returns the URI it wrote
The $type key makes the JSON re-readable. A dict is checked against the schema
and a refusal names the field and value. pkg.get(uri) on an existing object of
the type you want to write is the best way to learn a shape.
Arrays
for ref in pkg.arrays(uri):
print(ref.file, ref.path_in_file, pkg.array_shape(ref))
values, dimensions = pkg.read_array(ref) # flat numpy array and the shape beside it
points = values.reshape(dimensions)
values, dimensions = pkg.read_array_part(ref, start=[1, 0], count=[1, 3]) # a slice
pkg.write_array(ref, np.asarray(points, dtype=np.float64).ravel(), [len(points), 3])
An HDF5 string dataset comes out as a list[str], not a numpy array.
Arrays are flat copies (a view would outlive the memory it points to). The dtype decides what is written (float32/64, int8-64, uint8-64); anything else is refused by name. A shape that disagrees with the number of values is refused, naming both counts.
Geometry, properties and export
from energyml.rust.resqml import geometry_of, mesh_of, properties_of, export_of, formats
shape = geometry_of(pkg, uri) # placed points and topology, by patch
points, triangles = mesh_of(pkg, uri, "metres")
values, problems = properties_of(pkg, uri) # values hanging on this shape, with colour ramps
open("model.obj", "wb").write(export_of(pkg, [uri], "obj"))
geometry_of returns what the file says, patch by patch, with a topology key
(points, triangles, quads, polygons, polylines, segments,
hexahedra, polyhedra, grid), and index, the patch’s number. An IJK grid’s
hexahedra add cells (8 node indices per cell), dimensions and cell_index,
the RESQML number of each cell - empty when every cell is there, in order: a cell
the file marks undefined is left out, and the cells after it no longer sit at
their own number. placement is local, projected (the
default) or metres; an unknown name is refused with the list, and metres is
refused when the unit cannot be converted. mesh_of merges and triangulates, and
says so in its name. properties_of returns the values that fit the shape,
checked against the geometry, and a second list of those that did not fit. A
colour ramp is empty when the package states none: no default is invented.
export_of returns bytes. The list of formats is the exporter registry’s own. A
format that cannot write an object refuses, naming it.
Judging and saving a package
r = pkg.check()
print(r["errors"], r["warnings"], r["sound"]) # the same numbers `energyml check` prints
diagnostics = pkg.save_as("copy.epc") # returned, never swallowed
check(schema=False) returns checked: False: not checked is not the same as
clean.
Renaming the data file
quote = pkg.plan_rename("survey.h5", "survey_2026.h5") # changes nothing
done = pkg.rename_data_file("survey.h5", "survey_2026.h5")
Quote first; the operation refuses as a whole rather than doing part.
Repairing the declared data files
files = pkg.array_files() # what `energyml h5` prints; changes nothing
# [{"declared": "survey.h5", "pointers": 12, "found": {"path": ..., "origin": ..., "agrees": False},
# "example": "/RESQML/...", "proxies": [...]}, ...] ("found" is None when the search failed)
pkg.pin_data_file("survey.h5", "/data/dataset_7.dat") # say where a missing file is, then ask again
done = pkg.repair_arrays("fixed.epc") # what `energyml h5 --fix` does; None = nothing to fix
print(done["renamed"], done["declarations"], done["lost"])
The bridge carries no logic of its own: the survey, the rewrite and the verdict
are energyml_storage::repair’s, the same ones the command line uses. lost is
read back from the written package; non-zero means the repair did not repair
(the package uses an externalResource relationship the writer does not
reproduce).
referenced_by(uri) answers for that uri: when a package holds two
versions of one uuid, an object that names version 2 is not reported as
pointing at version 1.
Examples
Worked examples live in the bridge’s examples/ folder,
crates/energyml-python/examples/, and are
run by its tests (tests/examples.rs): adding a surface to an existing package
(01_add_a_surface.py), and building a package from nothing (feature,
interpretation, point set, property: 02_a_package_from_nothing.py).
Creating objects
Writing a valid EnergyML object by hand means assembling a uuid, references
(DORs), a CRS, HDF5 paths and activities. energyml-build is the layer that makes
writing usable: “wells from this CSV”, “a surface from these points”, “a map
interpolated from these samples”. It has one principle.
An operation is a pure function from a bag of pieces to a plan. Only
applywrites.
graph LR
inputs["files, objects of a package, choices"] --> pieces["Pieces"]
pieces --> op["Operation (pure)"]
op --> plan["Plan: objects, arrays, template, activity"]
plan -->|quote| report["report: what would be made, what is refused"]
plan -->|apply| store["EpcStore: the only step that touches storage"]
Why pure
AnyOperation::plan takes no store. It cannot read one, so it cannot depend on
one, and a test reaches it with values alone; what the caller knows of the
package arrives already resolved. This gives you, for free:
- a quote before the invoice:
--planshows the objects, uuids and arrays an operation would produce and writes nothing; - the same operation behind every face: the command line, the Python bridge, a web page and Galaxy all call the registry; none of them contains a creation rule;
- reproducible output: uuids are derived (v5) from what the input owns - a
file and its features, an EPSG code - so a second run over the same source names
the same objects, and
--nowfixes the instant so the package is identical byte for byte.
A refusal is a sentence
Registry::offers(&pieces) answers what can I create with this? and lists the
operations that do not fit with their reason (“a set of polylines is not
written as a triangulated surface”), because absent reads as not written yet.
What is refused rather than dropped
An input or a setting a run cannot honour is refused by name, or kept and said, never ignored:
- a GeoJSON that holds a feature the reader cannot read (null geometry, a
GeometryCollection, a short ring) imports the rest, and each feature left out is named in the warnings ofmesh::read_placed; - a CSV cloud decides its separator once for the whole file, from its first row (a semicolon means decimal commas);
interpolaterefusesfaultswithmethod = mba, andouter/innerwith any other method; a ring holding several polyline runs is refused (one ring, one run);resurfacerefuses a grid whose nodes are not on the lattice of its first row and column, and keeps a node with no height as a hole (aholes-keptnote);extend-boreholesconvertsmaxDepthandstep(metres) into the frame’s depth unit, and refuses a unit that is no length;attachrefusesfileNameandtitlewith the part form, andpathwith the inline and external forms;- sizes are bounded before anything is allocated:
local-grid(child nodes),injection-test(samples),interpolate(nodes per axis); transform,project,clipanddelete-nodesplace a line or a surface in the target system the way they place a point set whensourceEpsgdiffers fromproject;sigeol-unitsrefuses a plan in which no group could be placed, and lists the groups it could not place indropped, with no row in the table;- a borehole with a
BoreholeIntervalwhose bound cannot be read is refused byBorehole::mds, naming the interval; stratigraphyrefuses a parent with only one age bound.
What an operation declares
| Declaration | Meaning |
|---|---|
needs() | what it still asks the caller: a number (with unit and bounds), an integer, an EPSG code, a flag, a closed choice, text, a CRS, an object of the package… Each is a Slot rendered as a CLI option, an interactive question or a form field |
takes() | the pieces it consumes: tables, features, a document, objects of the package |
gives() | the roles of what it writes, with types and cardinality |
reports() | what its activity records that is neither a choice nor an output |
The same declaration generates the command-line help, the web form, the
Galaxy tool and the canonical ActivityTemplate of the operation.
Every run records an Activity against that template, so the form and the record
are one declaration. Every canonical template declares input (the objects the run
read) and inputFile (a file it read, written <slot>: <path>), and a whole number
a run recorded is an IntegerQuantityParameter, as the template says; see
Provenance.
The operations
About ninety, grouped by what they do:
| Family | Examples |
|---|---|
| Foundations | crs, collection, representation sets, new version, edit, attach, media, colormap, table-link, along-well-view |
| Import | pointsets and polylines from GeoJSON, wells and properties from CSV/JSON tables (with OSDU column templates), WITSML wells and logs, terrain, GeoSciML GeologicUnit import (tested with SIGEOL exports) |
| Wells | well, well plan, blocked wellbore, computed tops, contacts |
| Surfaces | interpolate (IDW, kriging, MBA), triangulate, isocontours, surface maps (dip, curvature), remesh, densify, extend, clip, loft, outline, intersect, project, measure, thickness, zone statistics, isosurface (an iso-surface kept as a surface) |
| Structural | fault, fault throw, horizon law, boundary, stratigraphy, structural organisation, earth model, cross section |
| Grids and properties | IJK grid, faulted grid, local grid, grid properties, sub-representation, time series, combine (a property derived from two others) |
| PRODML and WITSML | fluid sample, pressure transient analysis, injection test, product flow model |
Run energyml create --list for the list your build has, and
energyml create --list --from your-file.geojson for what a given input allows.
Derived properties, links and view results
Four operations write what a computation or a view produced, only when asked: the result is first an artefact (a number, a mesh, a drawing), and promoting it into the package is a gesture, never a side effect.
combinewrites a property derived from two others, value by value: a ratio (for the anisotropy Rv/Rh of a resistivity), a difference, or the log of a ratio. The arithmetic isenergyml_analysis::combine(see Geostatistics); the operation adds the writing. The result keeps the shape of its parents (several values per element are written with theirValueCountPerIndexableElement, and theirLabelPerComponentwhen they have labels), and its unit is the computed one: a unit given against it is refused, never replaced. Two properties on two supports, or of two lengths, are refused; a result where every pair is absent or undefined writes nothing. The activity records both parents as inputs, the operation, and how many pairs were absent and undefined.isosurfacepromotes an iso-surface computed byenergyml_geostat::isosurfaceto aTriangulatedSetRepresentationwith its activity. A plan is pure and reads no array, so the operation does not compute the surface again: it writes the mesh as it was computed and records the recipe (the level, the spacing, the method, how many cells were set aside, the cube, its support) as the caller states it, and says so in a note. The mesh is written by the same writer astriangles.table-linkwrites the link of a column to a table, or of a property to the table of its axes, in the convention the library reads first; see Writing a link.along-well-viewstores a drawn geosteering section in the package: the SVG (and a PNG of that same SVG, when the caller gives one) as attachments of the frame, and anActivitywhose inputs are the objects the section was computed from. What the section was computed from is read from the drawing’s own metadata, an object it names that the package does not hold is refused by its uri, and a setting given that disagrees with the drawing is refused.stored_viewsreads such results back by the template of their activity, never by their title.
The decisions are in ADR 0037
and ADR 0039. Each of these operations
has its own canonical ActivityTemplate, like every other (see
Provenance).
Using it from the command line
energyml create crs --set epsg=2154 --set vertical=5720 --plan # the quote
energyml create crs --set epsg=2154 --set vertical=5720 --out crs.epc
energyml create polylines --from model.epc#<uuid> --set title="Copy" --into model.epc --out model+copy.epc
energyml create interpolate --from model.epc#<uuid> --set method=idw --set spacing=100 --into model.epc --out model+map.epc
--set name=value is generic; --list shows what each parameter is, which it
depends on (with), which it excludes (not with), and what leaving it out
makes ([empty: creates …]).
Using it from Rust
#![allow(unused)]
fn main() {
use energyml_build::{apply, operation::Registry};
let registry = Registry::new(); // the built-in operations
let offers = registry.offers(&pieces); // what fits, with refusals named
let plan = registry.plan("crs", &pieces, &choices, &identity, &when)?;
// ... inspect `plan`, then, and only then:
let made = apply::apply(&plan, &mut store, &when);
}
A caller can also start from Registry::bare() and register operations of its
own; a registered operation is tried ahead of the built-ins.
Recipes
A recipe is a versioned TOML file of ordered steps, each an operation with its
settings and its inputs. energyml create --recipe plays every step through the
same door into one package and writes it at the end. Played, the recipe is also a
workflow: an ActivityTemplate with one sub-activity per step, and a root
Activity that every step’s activity names as its parent. A step that took a file
records it as an inputFile parameter, so the workflow can be read back (and a
recipe drafted from it) without the original recipe; see Provenance.
energyml create --recipe recipe.toml --plan # what each step would make; writes nothing
energyml create --recipe recipe.toml --out Lutetia.epc # the package, and Lutetia.h5 beside it
A sketch (TODO(verify): the key names below are those of the current parser and are being translated to English; check them against the published version):
titre = "Lutetia" # the package title
creation = "2026-09-24T00:00:00Z" # stamped on every object: two runs, one package
auteur = "Geosiris" # every Citation's Originator
donnees = "Lutetia.h5" # the HDF5 file for steps that do not name another
[[etape]]
nom = "repere" # unique: later steps refer to what this one wrote by it
operation = "crs"
set = { epsg = 2154, vertical = 5720 }
[[etape]]
nom = "sol"
operation = "interpolate"
entrees = [ { creneau = "shape", etape = "repere", type = "PointSetRepresentation" } ]
set = { spacing = 25.0, title = "Ground" }
An input refers to what an earlier step wrote by that step’s name (narrowed
by type, titre or role), never by a uuid copied from a run. Everything is
checked before any step plays: unknown keys, unknown operations, duplicate step
names, settings the operation does not declare, inputs from a step that comes
later. A step marked as waiting is checked and reported, not played - never
commented out, which would leave nothing to report.
The reference model energyml-lutetia
is replayed from a recipe this way.
Logging
Spans carry the elapsed time of each operation, recipe step, read and write; see Logging.
Writing your own
See the design notes in ADR 0017. In
short: implement the operation trait with a plan that uses only its arguments,
declare needs, takes, gives, push it to a Registry, and the CLI, the
Python bridge and the Galaxy generator all find it.
Editing a scene
A viewer lets a person move a surface, cut it with a polygon, delete some of its
nodes, or draw a new surface between two lines. In this library each of those
gestures is an operation of energyml-build, the same kind of object that
creates wells or interpolates a surface. There is no separate editing engine, so
a gesture a page offers can be written in a recipe, run from the command line or
called from Python with the same result.
The rule they share
Like every operation, an editing operation is a pure function from a bag of pieces to a plan (ADR 0017):
- Plan. The operation reads the shape it is given and returns what it would write, the notes that say what it decided and what it counted, and the activity record. Nothing is stored and nothing is modified.
- Apply. Only
applytouches storage, and it only inserts: an editing operation produces a new object (a new representation of the same kind), and the object it started from stays untouched in the package. Whether a page shows the result as a copy or in place of the original is the page’s choice.
The one exception is edit-object, which is explicit about it: it rewrites an
object under its own identifier, or copies it under a new one, and says each
change (old value and new) in the plan.
Two more rules follow. A gesture that would silently do nothing, or silently do something else, is refused by name (an axis of length zero, a node that does not exist, a cut that removes everything or nothing). And the activity written with the result records which operation ran and with which settings.
The operations
| Operation | What it does |
|---|---|
transform | moves, turns and scales a shape, or some of its nodes, about one pivot: the scale first, then the rotation, then the translation. The topology is copied as it is |
change-frame | reads a representation in one frame of the package and writes it in another, the same nodes in the same order; depth to time takes a stated, sourced velocity law and marks the result synthetic (ADR 0019) |
clip | cuts a shape by an outer ring and inner rings; the cut follows the nodes, not the ring |
delete-nodes | takes some nodes out: a cloud loses them, a line is cut at them, a surface loses the triangles that name them |
densify | adds points between the vertices of a polyline and keeps every vertex |
loft | builds a triangulated surface between polylines taken in a declared order |
project | projects nodes vertically onto a surface or a level; a node with nothing above or below it is left where it was and counted |
remesh | redoes the topology of a triangulated horizon and keeps its nodes |
resurface | recomputes a shape’s own heights, keeping its topology; the rings only select which points inform the fit |
edit-object | sets fields of an object as told, by path in the JSON encoding of the object |
The computations live in energyml-analysis; the operation is the seam that
reads pieces, calls them and writes through the writer of the shape’s own kind.
What is refused
- A grid’s cells or a volume where the operation has no writer for it: writing a moved grid would mean choosing between a triangulated surface and a lattice with turned vectors, and that is a decision, not a default.
- A shape holding patches of several kinds: one representation holds one kind.
- Two shapes that name two different frames, when one of them is a grid: a grid is never moved to another frame to make them agree (ADR 0020).
To see what an operation would do without writing, ask for its plan (see Creating objects).
Section view
Long profiles and cross sections of a model along a trace: wells projected onto
the section, surfaces and IJK grids cut by it, strata colours, markers, a legend,
and an SVG you can open in a browser or edit in Inkscape. The computation is in
Rust (crate energyml-section); the drawing is the same one whatever asks
for it - the command line, a library call, or an application.
A long section of the Lutetia reference model, drawn by energyml profile (the
other views, and the exact command for each, are in the Gallery):
From the command line
# A long profile along a track (GeoJSON LineString, or "x,y;x,y;..." inline)
energyml profile package/Lutetia.epc --track track.geojson --left 300 --right 300 --svg long.svg
# A map view of the same section
energyml profile package/Lutetia.epc --track track.geojson --left 300 --right 300 --draw plan --svg plan.svg
# Cross sections every 300 m along the track; draw the second one
energyml profile package/Lutetia.epc --track track.geojson --left 300 --right 300 \
--cross-every 300 --cross-length 400 --cross-width 150 --draw cross:2 --svg cross-2.svg
# PNG (build with the png feature), short codes plus a legend instead of full titles
energyml profile package/Lutetia.epc --track track.geojson --left 300 --right 300 \
--labels codes --svg long.svg --png long.png --png-scale 2
# Several packages in one section: the first is s1, each --also the next
energyml profile model.epc --also wells.epc --also faults.epc --track track.geojson \
--left 300 --right 300 --svg long.svg
# Redraw a drawing from its own metadata (same packages: same bytes)
energyml profile model.epc --from-svg long.svg --svg again.svg
Without --svg, profile answers a plainer question: which wells have their head
in the band around the track, how far along it, how far off it, and on which side
(--json for the full answer, including the band as a polygon).
Wells are all trajectories and frames of all packages (or the --wells); surfaces
are all triangulated sets, 2D grids and IJK grids a section cuts (or the
--surfaces). A frame and the trajectory of the same wellbore are one line, the
frame that states the strata kept.
How it is built
graph LR
inputs["Inputs: packages, objects, trace"] --> read["read (per package; the only stage that touches a store)"]
read --> assemble["assemble"]
assemble --> catalogue["Catalogue (cacheable)"]
catalogue --> layout["layout (catalogue + View)"]
view["View: scale, colours, labels, styles"] --> layout
layout --> svg["svg::write"]
Three stages, three costs:
readreads one package’s wells, surfaces and grids, reprojects them into the scene’s system and cuts them along every section trace.assemblejoins the contributions of every package into aCatalogue: wells projected on each section, frames and trajectories of one wellbore folded into one line, and the fate of every object.layoutturns a catalogue and aViewinto numbers (scales, ticks, fills, labels);svg::writewrites them. Nothing is read or recomputed here.
Changing the view never rebuilds a catalogue; changing the inputs does.
model.rs holds the types every stage is written against; they travel as JSON
(camelCase), including in the SVG’s metadata. Callers use compute and draw,
not the stages one by one, so there is one drawer: what is exported is what
was shown. The crate has no native dependency and compiles to WebAssembly.
Rules the drawing follows
- Every refusal is a line, in English, naming the object. An object that cuts nothing is missed, not refused.
- Nothing that is not a number leaves a contribution. A well without an elevation or with fewer than two finite nodes is refused by name; a surface cut is broken at a hole, never joined across it.
- Fills go to the next horizon, abscissa by abscissa. For each trace, the nearest surface strictly above or below at each position bounds the polygon; where there is none, nothing is filled (no floor is invented). A fault never bounds a unit and is never filled. Two surfaces interpreting the same feature are one horizon drawn twice and do not fill toward each other.
- Grids are cut as envelopes: roof and floor stacked by column, a fault as a
vertical step, a K gap as a hole. The plan boxes of the cells are filed once in
the grid of buckets of
energyml_geostat::bucketswhen the grid is read; each section tries only the cells whose box a segment of its trace passes near, so a grid of millions of cells pays for the few hundred the trace crosses. - The section checks itself. Where two cut surfaces cross, both must give the
same elevation; otherwise a
crossingMismatchnote with both values (tolerance 0.5% of the vertical extent, at least one centimetre). Younger horizons passing under older ones are marked as erosions, but only when ages are stated in the model: no age is ever inferred. - Marks do not overlap. Labels are placed level by level, full text then short form; a mark without room keeps its line, loses its text, and the count is reported.
- The SVG is safe to embed. Colours are attributes, all text is escaped (a
package title is someone else’s text), and each object is a
<g>carryingdata-storeanddata-uri. Legends are separate groups marked as Inkscape layers so they can be edited. The SVG zooms itself in a browser (strokes keep their width, points and text keep their size);--staticturns that off.
Options of the view
- Strata legend (
--strata-legend): one entry per unit and colour. - Portions without strata (
--unstratified hidden|solid|dashed|hatched, with parameters such asdashed:colour=#999999,dash=6,gap=4): the library declares the available styles and their parameters, so an interface builds its form from that list instead of spelling them. - Markers (
--marker NAME:shape=...,colour=...,at=...): a named marker on a set of objects chosen bytype=andtitle=, with a legend. Shapes are read from the model (theSymbolenumeration ofGraphicalInformationForNodes); a value the drawer cannot draw is refused by name, never replaced by a circle. - Vertical exaggeration is written on the drawing.
From Rust
#![allow(unused)]
fn main() {
use energyml_section::{compute_packages, draw};
// `inputs`: what to cut and along which trace; `packages`: the opened packages;
// `reprojector`: how to move between coordinate systems; `view`: how to draw it
let catalogue = compute_packages(&inputs, &mut packages, &reprojector)?;
let svg: String = draw(&catalogue, &view);
}
The signatures, in crates/energyml-section/src/lib.rs:
compute_packages(inputs: &Inputs, packages: &mut [Opened<'_>], reprojector: &dyn Reprojector) -> Result<Catalogue, InputsError>;compute(inputs: &Inputs, read_one: impl FnMut(&Asked, &[SectionTrace]) -> Contribution) -> Result<Catalogue, InputsError>, the same calculation for a face that reads its packages itself (the front end holds one per session);draw(catalogue: &Catalogue, view: &View) -> String;redraw(svg: &str, packages: &mut [Opened<'_>], reprojector: &dyn Reprojector) -> Result<String, RedrawError>, which draws again a drawing from what its own metadata says it was made of.
See also Well sheet, the geosteering section (a section along a well, with a curtain of values either side of the hole) and ADR 0015.
Around the well
A well is a line, and a lot of what a geoscientist draws sits around it: a ring
of radii at each depth, a matrix of values carried by each node, an envelope, a
curtain, a vertical section. All of it answers the same questions: where is the
hole at this measured depth, which way does it go, where are its top and its
right, and how is it laid flat. This page is the one place of the library
(crate energyml-geometry) that answers them. It is pure Rust and builds for
WebAssembly.
Every position, direction, frame and offset on this page is placed: in
metres, with z up, in the projected frame of the coordinate system the file
names. Angles are in radians. Measured depths stay in the unit the trajectory
states (PlacedPath::md_uom) and are never converted silently.
A trajectory you can evaluate anywhere
PlacedPath is the one door to a trajectory that is placed and evaluable at any
measured depth. It has two sources:
#![allow(unused)]
fn main() {
// A RESQML WellboreTrajectoryRepresentation, placed through the CRS its geometry names.
let path = PlacedPath::of(trajectory.as_reflect(), &store)?;
// A WITSML survey: its stations joined by the minimum-curvature arc, placed by
// where the survey stands (datum and compass axes, in metres).
let path = PlacedPath::from_stations(&stations, &placement)?;
let here = path.at(1250.0); // point, and where the depth falls
let dir = path.direction_at(1250.0); // unit vector towards increasing MD
let tvd = path.tvd(1250.0); // below the MD datum, None without a datum
}
at says where the depth falls against the depths the path states
(MdRange::Within, BeforeFirst, AfterLast, NotANumber): a depth outside
the line gets the extended interval, as the line evaluator extends it, and says
so. Nothing is extrapolated without being flagged.
A trajectory without a coordinate system is refused (PathRefusal::NoCrs): which
way is up and where the well stands are not stated. If you know the system, declare
it with PlacedPath::new(path, crs, md_uom); the declaration changes the reading,
never the package. A line read without a system keeps depth downward and says that
it assumed it (Path::vertical_note), see ADR 0041.
The direction at a depth
Path::direction_at is the derivative of the very interpolant that Path::at
evaluates, kind by kind: no second curve, no difference of two evaluations. At a
knot of a linear line the direction is that of the interval that ends there, the
one in which the line evaluates that parameter (ADR 0031).
It is None where the derivative vanishes, not a zero vector.
The frame of the hole
#![allow(unused)]
fn main() {
let frames = frames_along(&path, &[1000.0, 1100.0, 1200.0], &FrameOptions::default())?;
for frame in frames.given() {
// frame.at, frame.tangent, frame.high_side, frame.right, frame.inclination
}
}
HoleFrame is the physical frame of the hole at one depth: the centre, the unit
tangent towards increasing MD, the high side (the upward vertical projected
across the hole), the right of someone looking down the hole, and the
inclination from vertical down.
Two things matter.
- The hand of the frame. The placed frame is not always direct: a system
written northing then easting is indirect, and a cross product on its coordinates
points to the other side. The library reads the order of the horizontal axes from
the file (
Crs::axes, in RESQML 2.0.1 and in EML 2.3) and givesrightas the physical right in either case. A horizontal hole going north has the east on its right in an (E, N, up) frame, and so does the same hole written in (N, E, up). If the file states no usable order, a frame is refused (AxisOrderUnknown) until you declare the hand inFrameOptions; a declaration never overrides the file (ADR 0032). - A vertical hole has no high side. Under 1 degree of inclination (or above 179)
the frame is refused at that node by name (
HighSideUndefined).Frameskeeps one outcome per depth asked, so a refused node is a hole you can count, not the end of the well. The threshold is the constantMIN_INCLINATION_DEGREES.
north_in_plane gives grid north seen in the plane across the hole, as
(right, high side), when an angle is counted from north. offset_point gives the
pose in space of a point moved along the high side. This frame is not the
transported frame of a tube (tube::sweep: minimal rotation, arbitrary origin);
the two stay, under two names.
Angles around the hole: a declared convention
Files give radii and angles without saying where angle zero is or which way angles
grow. The library does not guess. An AngleConvention is a parameter you declare
(origin: high side, low side, right, left or north; sense: clockwise or
anticlockwise; viewed from uphole or downhole; and an offset), and without it the
layer is refused (ConventionUndeclared).
#![allow(unused)]
fn main() {
let convention = AngleConvention {
origin: AngleOrigin::HighSide,
sense: AngleSense::Clockwise,
viewed_from: ViewedFrom::LookingDownhole,
offset: 0.0,
};
let ring = polar_points(&radii, &angles, Some(&convention), north)?;
// ring.points: one (right, high side) per radius, None for a counted hole
}
The canonical plane is the one an observer standing uphole and looking down the hole
sees: the right on the right, the high side up, a clockwise turn going from the top
to the right. A missing radius is a hole that is counted (holes), never filled; a
negative radius is shown as a hole and counted apart (negative). sector_bounds
gives the bounds of each sector, halfway to its neighbours, and to_space carries a
point of the plane into space by the frame.
A matrix carried by each node
Some tools carry a two-dimensional matrix at each node (an image across the hole:
81 by 81 values). Which dimension runs along the high side, which along the right,
with which sign and which offsets in metres is not in the file either, so a
TransverseGrid is declared too: one GridAxis for the slow dimension (the first
count, TRG 2.2 §7.1) and one for the fast.
#![allow(unused)]
fn main() {
let grid = TransverseGrid::new(slow, fast)?;
grid.checked(&shape)?; // the property's per-node shape must be [m, n]
let points = grid.slice(&frame); // m x n points of one node
let folds = grid.check_folds(&frames); // consecutive slices that cross
}
Values of the axes come from the registry of axis sources (see The shape of a property); this module only places them.
Envelope and ribbon
envelope builds a mesh whose section varies along the well: radii per sector and
per node, with the angles taken per node, shared, or evenly spread (Angles; a
choice of the caller, never a default). The triangles face outwards whatever the
sense of the angles. ribbon lays one column of a transverse cube along the well
(RibbonPlane::FixedFast or FixedSlow). Both give an EnvelopeMesh, which says
for each point the index of its value in the property’s own array, and counts
holes: a quad with a corner missing is not drawn and never bridged.
Laying a well flat
WellProjections is a registry (the caller’s additions are tried first) of ways to
lay a well flat. A point (md, dn) of the frame of the well, dn metres along the
high side, becomes a 2D point:
| id | x | y | x is a length |
|---|---|---|---|
md_bar | 0 | md | no |
md_tvd | md | TVD | no |
developed | plan length up to md (plus dn cos I) | TVD | yes |
vertical_section | distance along the section azimuth | TVD | yes |
plan | east | north | yes |
The TVD is counted below the MD datum, the point where the measured depth is 0,
not from the zero of the vertical system; a line that does not reach its datum is
refused (NoDatum) rather than counted from its first node
(ADR 0033). The vertical coordinate is the TVD
positive downward; the drawing turns it over.
#![allow(unused)]
fn main() {
let axis = WellAxis::new(&path, [0.0, 3000.0], 10.0)?; // sampled once
let projections = WellProjections::standard();
let md_tvd = projections.get("md_tvd").unwrap();
let line = locator_line(&axis, md_tvd, 0.5)?; // fine enough for 0.5 m
let mark = locator_mark(&axis, md_tvd, 1250.0)?; // where one MD falls
}
locator_line and locator_mark serve the small map of a well (the position of a
depth, the direction of the tick). A mark is always PlacedPath::at(md) projected,
never a length measured on the flat line: on the plan of a J-shaped well the line
is shorter than the measured depth. locator_md_at goes back from a point to an MD
and locator_spans cuts the line into the pieces inside given MD spans.
The rules, and how to read them
The four readings that are the library’s own choice are in the registry of house rules, theme well: the direction at a knot, the minimum inclination of a high side, the TVD below the MD datum, and the conventions declared by the caller.
energyml rules --house --theme well
They are declared rules: the code applies them, and a breach is a named refusal, never a drawing.
The geosteering section
A geosteering section is a vertical section along a well: the hole runs across the
page, and either side of it a cube of values (the resistivity of a deep-reading tool, for
instance) is laid out like a curtain. On top of the curtain sit the limits of the
tool’s depth of detection, the horizons the curtain cuts, and the tops of the hole. It
lives in energyml_section::along_well, next to the long section of the
section view and not in it: a section of a model along a trace and a
section along a well have different geometry, so each has its own model and its own
drawing. The decisions are in ADR 0038.
The module is pure Rust, reads no file of its own and builds for WebAssembly.
What it shows
| Layer | What it is | Where it comes from |
|---|---|---|
| curtain | one column per node of a wellbore frame, one row per offset, each cell a quadrilateral coloured by its value | a property with two dimensions per node |
| boundaries | the limits of the depth of detection above and below the hole | a property of radii and a property of angles |
| horizons | where each surface crosses the curtain, column by column | triangulated or gridded surfaces |
| tops | where the hole itself crosses each surface | the same surfaces |
| trajectory | the hole | the frame’s trajectory |
| legend, notes | the scale of the values, and the sentences that say what was assumed or left out | the model |
Every layer is optional: a request with no curtain still draws the hole, its boundaries, horizons and tops.
One frame for everything
Everything is computed in one frame, the hole’s. A point is (md, dn): md is the
measured depth, dn the offset in metres along the high side of the hole, positive
up. One function carries a point into the drawing, WellProjection::place (see
Around the well). The cells, the
boundaries, the horizons and the tops all go through it, so a boundary at radius r
and the centre of the row at offset r are the same point, to the bit.
The projection is chosen by its id in the request: md_tvd, developed or
vertical_section (the last needs the azimuth and the origin of the plane, given in the
request). The vertical coordinate is the true vertical depth below the MD datum
(ADR 0033).
Where the hole is too close to vertical for a high side to exist (under one degree of inclination, ADR 0032), the nodes there have no frame. They are counted and said in a note; they are never a refusal of the whole section.
The curtain
The curtain comes from a source, chosen by id in the request. The built-in
cube-on-frame reads a property carried by the nodes of a wellbore frame with a matrix
of two dimensions per node, each at least 2 (the UDAR Rh and Rv are [81, 81]). A
flat list of components is never factorised into a matrix.
What the file does not say, the request does (the rule of Around the well):
- The axes. The offsets of each dimension come from the links the caller declares, then
from the file’s convention B (a table or a paired property, see
Tables, keys and links). A rank is not an offset: when only ranks are
known the section is refused (
AxesUnknown) and says which dimensions to declare. An axis must be in a unit of length. - The roles. Which dimension runs along the high side, which is lateral, and the sign
of each, are the caller’s declaration (
roles). The note “the roles and signs … are the caller’s declaration” is written on every curtain made this way. - The lateral row.
lateral_indexpicks the row of the lateral dimension that is shown; left out, the view takes the middle row (row 0 is the edge of the cube, off the plane of the hole). A plane that is off the hole is drawn in the plane of the hole and says so (a note), and only inmd_tvdanddeveloped: in any other projection it is refused (LateralNotPlaced).
The cells of a node reach half way to its neighbour, and never more than half the widest
step the gap rule bridges (ADR 0027): a gap
in the data stays an empty band and is counted in a note, never bridged. A cell that folds
over the centre line of the section (in developed, where the hole bends sharply) is
marked, never straightened, and drawn where it falls. A stretch where the section
runs backwards is noted the same way. A value the file declares as NullValue is an
absence: the cell is left transparent and counted.
A source that is declared and not built (grid-property, lwd-image) refuses by name
(SourceNotBuilt), and so does an unknown id; neither draws an empty curtain. The
registry (CurtainSources) takes the caller’s own sources first.
As an image, the curtain is painted by curtain_image into the pixels of the plot, the
cells farthest from the hole first, so that where two columns overlap the nearer one wins.
The colour comes from a closure the caller passes: the crate holds no palette, and a
face resolves the colour through the one door of colours
(How a colour is found).
Boundaries
The boundaries of a depth of detection are rings of radii at angles. They are found by the value of the angle, never by its rank: an angle within 1e-6 rad of 0 is above the hole, one within 1e-6 rad of pi is below. The angles may be given once for all the nodes (the UDAR case) or node by node.
| Mode | What it draws |
|---|---|
in-plane | the radii at the angles 0 and pi only, the points that fall in the plane of the section |
projected | every angle, at r cos(theta) on the high-side axis (even in theta, so the sense of the angles is not needed) |
A radius that is NaN or negative gives no point and is counted. Two angles on one side of one node are refused by name (which radius is the boundary is not decided here), and so is an in-plane request where no angle is at 0 or pi. Radii and angles are converted from their own units; a unit that is not a length or an angle is refused.
Horizons and tops
No solver is written for the horizons. A column of the curtain is a line P + dn h, and
energyml_analysis::tops::tops finds where a line crosses a surface, dn playing the part
of the measured depth. Each crossing is placed by the projection, and the crossings of
neighbouring columns are joined when they pair (same number of crossings, same side). What
a column leaves undecided is kept, not smoothed: a fold of the surface, an unresolved
crossing, a gap, a touch. Places where crossings could not be paired are counted as breaks.
The range of offsets the horizons are looked for in is the request’s dnRange, else the
curtain’s rows, else the largest radius of the boundaries. With none of them, no horizon
is looked for and a note says so.
The tops of the hole are the crossings of the hole itself with each surface
(computedTops in the request), placed in the same drawing. The surfaces are given as
PlacedMesh values, read by PlacedMesh::read (the one door of the geometry, in metres)
or built by the caller.
Markers
A marker is a point of the hole with, perhaps, a dip. marker_ticks computes the apparent
dip in the plane of the section, tan(apparent) = tan(dip) cos(direction - azimuth),
where the azimuth is that of the hole at the marker’s depth. A dip counted from true or
magnetic north is drawn level, and says so, unless the caller declares the angle to grid
north; against the measured depth the slope is multiplied by sin(I).
There is no reader of markers in the library today (the well strata walk refuses a
marker set by name, and neither the RESQML marker frame nor the WITSML interval set has
one). So a request that names marker objects is not refused as a whole: each marker uri
is refused by name (NoReader, with the type the package holds), the section is drawn
without them, and a note counts them. A caller that holds markers by other means can pass
them to marker_ticks and put the ticks in Model::markers; the drawing does not
paint them yet, but the window takes them into account.
What a frame offers
along_well::offers(index, frame) answers, from the package index alone (no array is
read), what a frame could show: the properties that make a curtain and what each still has
to be told (Axes, AngleConvention, or a choice between several angle siblings), the
radius and angle pairs that make boundaries, and the trajectory the frame references. These
are facts, not a choice: a face decides what to draw from them.
Drawing, and drawing again
Three stages, three costs.
Request --> build (reads, once) --> Model --> layout / draw (View) --> SVG
+---> pick / pixel_of
buildis the expensive stage. It reads the frame, the trajectory, the columns, the boundaries and the maps of the surfaces, and places them.layoutanddrawonly arrange the numbers of aModelunder aView: the window, the vertical exaggeration, which layers are shown, the line colours, the legend. Changing the window or the exaggeration costs a drawing, not a reading. The display list is the shared one of Drawing, and the SVG is written by its one writer.pickandpixel_ofgo from a pixel to the data and back: the measured depth and the offset (through the projection’s way back), the cell and its value with its unit, the distances to the boundaries, and the nearest horizon of the column.pickgives the cell’s own value by default; an interpolation of the domain’s registry can be asked for, along the measured depth and never across the rows. Where a projection cannot be inverted (plan,md_bar),pickanswers nothing.
The legend scale proposes a base-10 logarithm when the values span at least three decades and takes the first and ninety-ninth percentiles as bounds, as everywhere else (Statistics, scales and measured depth).
The SVG carries the request and the view in its metadata (element along-well-view).
redraw reads them back and makes the same drawing again over the same package, byte
for byte. A PNG is always the SVG rendered by draw::png::render, never a second
drawing. With embedCurtain the curtain is written into the SVG as an image, in every
build.
An operation, along-well-view of Creating objects, stores the
drawing in a package on request: the SVG as an attachment of the frame and an Activity
that records what the section was computed from.
What is refused
Every refusal is one sentence that names what is wrong.
| Refusal | When |
|---|---|
NotAUri | a field of the request is not the full uri of an object (a bare uuid is turned into a uri at the edge, never here) |
NotInPackage | an object, or a surface that was asked and not given, is not there |
NotOnFrame | the frame is not a wellbore frame, or the property is not on it |
UnknownProjection, UnknownSource, UnknownInterpolation | the id is not in the registry; the refusal lists what is |
SourceNotBuilt | a source that is declared and not built |
NotTwoDimensions, FlatComponents | the property is not a matrix of two dimensions per node |
AxesUnknown, AxisUnit | the axes are ranks or labels, or not in a length |
LateralIndex, Curtain(LateralNotPlaced) | the lateral row is outside its dimension, or a plane off the hole where it cannot be drawn |
Boundaries(...) | the two properties are not on the same rings, no angle at 0 or pi, two angles on one side |
BoundaryUnit | a radius that is not a length, an angle that is not an angle |
NoFaces | a surface with neither triangle nor quadrilateral |
BadMetadata, Metadata | a drawing that does not carry this section’s request, or of another format |
Legend | a legend that was asked and cannot be drawn |
What is not a refusal is counted and written beside the picture: cells with no value, folds, backwards stretches, gaps, nodes with no high side, a lateral plane drawn in the plane of the hole, markers that were not read, a curtain whose file colours only one component.
From Rust
#![allow(unused)]
fn main() {
use energyml_geometry::around_well::projection::WellProjections;
use energyml_section::along_well::{
BoundaryAsk, BoundaryMode, CurtainAsk, CurtainSources, Request, View, build, draw,
};
fn run(store: &dyn energyml_domain::ArraySource, ask: CurtainAsk, boundaries: BoundaryAsk) -> Result<(), Box<dyn std::error::Error>> {
let request = Request {
frame: "eml:///resqml22.WellboreFrameRepresentation(...)".into(),
trajectory: None, // the one the frame references
curtain: Some(ask), // property, source id, declared axes and roles
boundaries: Some(boundaries), // radius, angle, mode
surfaces: vec![],
computed_tops: false,
markers: vec![],
md_range: None,
dn_range: None,
projection: "md_tvd".into(),
section: None,
step: None,
};
let projections = WellProjections::standard();
let model = build(&request, store, &[], &CurtainSources::builtin(), &projections)?;
let view = View {
size: [1000.0, 600.0],
window: None,
exaggeration: None,
show_curtain: true,
show_boundaries: true,
show_surfaces: true,
show_tops: true,
colours: Default::default(),
legend: None,
embed_curtain: false,
};
let svg = draw(&model, &view, &projections, None)?;
let _ = (svg, BoundaryMode::InPlane);
Ok(())
}
}
Every field of the request that names an object is a full uri. Every fact the file does not state (the roles and signs of the curtain, the axes when the file has none) is part of the request, so the same request gives the same drawing.
See also Around the well, Section view and Drawing.
Views as pictures
energyml-views draws the views of a package as pictures: SVG, and PNG on demand. One registry of views, one door,
the same picture for every face (the command line, a desktop window, a web page). The decisions are in
ADR 0043.
From the command line
energyml view list package.epc # what each view can show, from the index alone
energyml view about well-locator # its objects and its settings, with their defaults
energyml view draw well-locator package.epc \
--object "eml:///resqml22.WellboreFrameRepresentation(9ccdea13-638d-5d72-8c08-c82163596218)" \
--set mode=plan --set md=3050 \
--svg locator.svg --png locator.png --png-scale 2
energyml view draw along-well package.epc --proposed \
--object "eml:///resqml22.WellboreFrameRepresentation(9ccdea13-638d-5d72-8c08-c82163596218)" \
--object "eml:///resqml22.ContinuousProperty(13568c0a-a6f8-593a-a171-83828de41874)" \
--svg section.svg # a first drawing, the proposals said as notes
energyml view redraw package.epc --from-svg locator.svg --svg again.svg
--objecttakes a full uri, or a bare uuid that the package index turns into one.--set NAME=VALUEis read by the nature the view declares: a number, a word of a list,true, or JSON.--proposedfills the settings you did not give with what the view proposes (see Proposals); a setting given with--setis kept. Each proposal is written on stderr as a note,proposed: NAME=VALUE - why.--pngneeds a build with--features png. It is the SVG just written, rasterised.- A refusal says which object, which setting and why, and nothing is written. A part the view left out, with the rest drawn, is a note: on stderr, and in a band under the picture.
From Rust
#![allow(unused)]
fn main() {
use energyml_views::{ViewAsk, ViewKinds, draw_svg};
fn run(cx: &energyml_views::ViewContext<'_>) -> Result<(), energyml_views::ViewRefusal> {
let ask = ViewAsk::new("well-locator")
.object("eml:///resqml22.WellboreFrameRepresentation(9ccdea13-638d-5d72-8c08-c82163596218)")
.set("mode", serde_json::json!("md_tvd"))
.set("md", serde_json::json!(3050.0));
let picture = draw_svg(&ViewKinds::with_builtins(), &ask, cx)?;
Ok(()) }
}
The context (ViewContext) holds the package (an ArraySource), its index (PackageIndex), the colour door
(PackageColours over energyml_cache, or NoColourMaps) and the palettes. A face that draws the display list
itself asks draw_list; a view of the caller’s is registered with ViewKinds::register and tried first.
Proposals
Some values are not in the file and a view refuses to draw without them (the roles of the two dimensions of a
curtain, the table that holds its coordinates). A proposal is a value the view offers for such a setting, from the
package alone, so that a first drawing exists and can then be corrected: assist(&kinds, &ask, cx) returns the
proposals (Proposal { setting, value, why }) and the dimensions the view needs declared (DimensionAxes: the
dimension, its size, the settings that hold its role, and the numeric columns of the package’s tables that have
exactly that many rows, as AxisColumn).
A proposal is never presented as read from the file. It is always said where it shows: the command line writes a
note for each, a form marks the field as proposed and shows the reason, and a person’s own value replaces it. A view
with nothing to propose (the hole of along-well, with no property) answers an empty Assist.
For the curtain of along-well:
curtain_linksis proposed when the file links no axis and one table of the package goes with the curtain (one key column, one numeric value column per dimension, the right number of rows): dimensionitakes thei-th value column (ADR 0029). With two such tables nothing is proposed, and both are candidates for the person to choose.slow_axis=high_side,slow_sign=positive,fast_axis=right,fast_sign=positiveare a declaration: the file does not state the roles of the dimensions.lateral_index, left out, is the middle row of the lateral dimension (the one not along the high side), not row 0, which is the edge of the cube and off the plane of the hole.
axis_links_value writes the links setting from (dimension, column) pairs and axis_links_read reads it back, so
that a face never writes the JSON shape itself: [{"table": {"table": uri, "column": title, "dim": n}}, ...]
(a link may also be a paired-property or a witsml-channel).
For the target (target): md is the first node of the frame (frame_mds), origin=high_side, sense=clockwise and
viewed_from=looking_downhole are a declaration of the angle convention, and the sectors are regular_sectors=true
only when no property of angles pairs with the property on the frame; when exactly one does, it is proposed as
angles instead. With several, nothing is proposed: the offer names them as a choice. The drawing says the
proposals in its notes, like any other.
Names, parts and what an offer proposes
A view has a short readable name (ViewKind::name, ViewKinds::name_of(id): Along the well, Target), what a
face shows instead of the technical id. An Offer adds two fields to the objects and the title:
part: what the view shows of its objects when it is not the whole object (the hole,the curtain Rh);Nonefor the object itself. A face puts it after the name of the view, never the title of the object it is already looking at.proposed: what the request should give but thatassistproposes, in words. These entries are not inmissing, which keeps only what nobody proposes. An offer is made from the index alone, so it saysproposedonly for whatassistis sure to propose: the roles of a curtain, its links when one table of the package goes with it (the row count of each table must be known without reading an array), the convention, the sectors, and the first node asmdwhen the property hangs on a wellbore frame.
energyml view list prints the name, the part, then a proposed line and a needs line for each entry.
The views
| View | Objects | Shows |
|---|---|---|
well-locator | a wellbore frame or trajectory | the well laid flat (md_tvd, md_bar, developed, plan, vertical_section with its azimuth), the interval the frame carries data over, and a mark at md |
Text stays on the page
No text of a view leaves its picture: draw_list breaks a title, a label or a legend name wider than the page into
lines (energyml_sheet::draw::text::fit_to_page), and the notes under the picture are broken to the page’s width,
the band growing by its lines. md-coverage writes the name of a bar too short for it beside the bar, above it when
the band has the room, else below. property-insight writes the eight strongest seams of an array (the step over the
median step) and counts the rest in one line, and K more seams.
Colours
The map the package gives the object comes first, by the same walk as energyml colors. When it gives none, the
default palette of the value’s reading paints (Viridis for a value, Set2 for a category), and the legend says the
colours are not from the file. --set palette=NAME chooses a palette by name; it wins over the file. The palettes
are quoted from their sources: Viridis from matplotlib (CC0), Spectral, YlGnBu, Set2 and Paired from ColorBrewer
(Apache 2.0).
Provenance
Where an object came from: which activity wrote it, which objects and files that activity read, which step of which workflow it was. This page covers what the library writes (so that a package records it completely), what it reads (four releases of the schemas), and what it shows (the graph, the timeline, and the provenance that some converters keep in tables instead of activities).
Every rule below follows the same line as the rest of the library: a departure from what a template declares is signalled, never refused, and a statement the file does not make is not guessed (see ADR 0005 and ADR 0035).
What the library writes
Every operation records an Activity against the canonical ActivityTemplate
of that operation (see Creating objects). Three points are
worth knowing when you read such a package:
inputandinputFileare declared in every canonical template. The EML technical reference guide 2.3 (section 3.1, p. 40) says a parameter’s title must have an equivalent in the activity descriptor parameters.input(a data object, any number) andinputFile(a string, any number) are therefore declared by all of the canonical templates, so that a new package satisfies the rule.- A whole number is an
IntegerQuantityParameter. It has no unit in any release (section 3.8, p. 47). In Rust it isParameter::Integer, which is a breaking addition to theParameterenum. - A file a recipe step read is recorded. A step that took a file (an
inputs = [{ file = ... }]entry of a recipe) writes aStringParametertitledinputFile, whose value is<slot>: <path>, or just<path>when the recipe named no slot. The path is relative to the recipe, as the recipe wrote it.
What the library reads
energyml_domain::activity reads an Activity and an ActivityTemplate of any
of the four releases (RESQML 2.0.1, EML 2.1, 2.2 and 2.3) into one shape,
ActivityRecord and TemplateRecord.
- A parameter’s value is read by the field present, never by the class name:
DataObject,TimeIndex,ValuewithUom(a double), a textValue, or a whole numberValuewithout a unit. A shape this build cannot read is shown as unreadable, never blank. - A reference is kept as the file writes it (
DeclaredReference): the full uri when it names a type, otherwise the bare uuid, the one exception the library makes to “full uri inside”. - A
DataObjectParameterthat points to anActivityis read as thesubActivitykind, and aTimeIndexParameterastimestamp: neither kind has a class of its own in the schemas.
Roles
For each parameter, roles answers input, output, both, neither, sub-activity or
undeclared, and where the answer comes from:
- the template (the spec): its
IsInputandIsOutputflags give four states (TRG 2.3 section 3.10, p. 49), andsubActivityis the kind that names a child; - else a named rule of the registry (
RoleRules; the caller’s rules are tried first). The built-in rule,energyml-build:input-output, reads a parameter titled exactlyinputoroutputas such, only in an activity written byenergyml-rust; it exists for packages written before the templates declaredinput; - else
Undeclared. A role is never guessed from a title.
Conformance
conformance compares an activity with its template and lists what departs:
a title with no equivalent, a kind outside the allowed ones, fewer parameters than
MinOccurs, more than MaxOccurs (a value of -1 means unbounded), or no template
at all. Nothing is refused: the activity is still read.
The graph
energyml_domain::provenance_graph builds the graph a view draws, from activities
and, optionally, a package index.
- Edges say what an activity read (
Consumed), wrote (Produced), merely took part in (Participant, when the role is neither or undeclared), its sub-activities and parent (Parent), the representation, interpretation or feature that carries an object (Carrier, from named rules,CarrierRules), and a collection’s members (Member). Each edge keeps its basis: the parameter and the basis of its role, a reference field, a verbatim source, or both an activity and a verbatim source. - Nodes are keyed by the full uri, a file by its path as written (
./a.lasanda.lasare two nodes). Two activities that wrote the same object give one node with twoProducededges. - The tree of activities is checked against both of its writings: the child’s
Parentand the parent’s sub-activity parameters. They may disagree, and the disagreement is listed. A cycle or an absent parent is listed and the activities are shown flat; nothing loops. - Held steps. A workflow template declares its steps. A step declared but not
run (it has
MinOccurs0 and a reason in its constraint) is a node of its own, with its reason; one declared held that ran anyway is signalled. - Set aside. Objects of some types (a CRS, a property kind, an activity template) are not nodes: they are listed beside the graph with what named them.
- Absent outputs. An object an activity says it wrote and the package does not hold is signalled.
focuskeeps a node and what lies upstream and downstream of it, by a number of edges (at most 64).timelineorders activities by the instants the file states. When there are fewer than two distinct instants it refuses withNoMeasuredOrderand says which order it shows instead; it does not invent a time.- Collections.
PackageIndex::expand_collectionlists the members of aDataobjectCollectionfrom the association sets that name it. A collection that no set names is refused: it states no members, which is not the same as having none.
Provenance kept in tables (verbatim sources)
Some converters record provenance in tables rather than in Activity objects.
energyml_domain::provenance_verbatim reads such a convention as written and
compares it with the activities.
A VerbatimSource recognises a set of tables from the titles of their columns
only (no array is read to recognise), and reads them into VerbatimProcesses.
The sources are a registry (VerbatimSources), the caller’s first. The built-in
source is NwdfTables, for the NWDF v0.2 tables of the UDAR converter:
| Role | Columns |
|---|---|
| processes | process_uid, activity |
| links | source_uid, attribute, target, target_table |
| mapping (NWDF uid to EPC object) | nwdf_uid, role, epc_uri |
| references (optional) | path, reference_uid |
How they are read:
- One table per role. Two tables playing the same role are refused with
SeveralTables, naming them: the library does not guess which group a table belongs to. A missing or doubled column is named too. - A process’s inputs and outputs come from the links whose
source_uidis aprocess_uid, never from thesource_tabletitle. A targetexternal fileis a file (with the key a references table gives its path); any other target is resolved through the mapping table. - A key the mapping does not tie is
Unresolved, shown as such. It is never resolved by its title: a title only looks like the object. - The activity of a process is stated twice, by the
activitycolumn and by the mapping’seml activityrow. If they name different activities,ActivityLinkDiffersis signalled; the column wins. A process naming no activity draws no edge and is signalled. - Every other column of the processes table is shown verbatim. An empty text is a value; a missing number is an absent cell.
- The
dataset_uidslinks are acquisition context, not processing provenance, and are left to another view.
reconcile then compares what an activity says (the objects its parameters read
and wrote, by role) with what the tables say, and sorts each object into both,
only the activity, only the tables (files included) or roles differ (stated
in opposite directions). Two objects are the same by uuid and agreeing versions,
never by text.
From a workflow to a recipe
draft_from_workflow in energyml-build rebuilds a recipe draft from a workflow
that was played: the steps, their settings and the files they read (inputFile).
The draft is lossy and says what it left out. It refuses what it cannot restate
(for example a step whose operation it does not know) with a named
ReplayRefusal, rather than writing a recipe that would not reproduce the package.
The constraint of a step (follows <template>) is read by read_step_constraint,
which lives beside the code that writes it.
The MCP server
MCP (the Model Context Protocol) lets a language
model call tools that a program offers. energyml-mcp is such a program for an
EnergyML package: you register it in Claude Code, Claude Desktop or another MCP
client, and the model can orient itself in an EPC file, list and find objects, read
one as JSON, follow its references and read a bounded window of an array.
It is read-only. No tool writes, and the package is opened without any way to save it. It is also a face like the command line (ADR-0012): the tools call the storage traits and the library, and hold no EnergyML logic of their own.
graph LR
client["MCP client: Claude Code, Claude Desktop"] -->|"JSON-RPC on stdin/stdout"| prog["energyml-mcp (energyml-mcp-cli)"]
prog --> srv["energyml-mcp: server module (rmcp adapter)"]
srv --> reg["Registry of Tool"]
reg --> store["EnergymlStorage: EpcStore here, an ETP store in retp mcp"]
Installing and running
cargo install energyml-mcp-cli # installs the `energyml-mcp` program
energyml-mcp model.epc # waits for a client on stdin: not meant for a terminal
energyml mcp model.epc # the same, through the `energyml` command
energyml-mcp model.epc --h5 data.h5 # offer an HDF5 file for the arrays
energyml mcp works because the energyml command runs energyml-<word> when it
does not know a word (see The command line); the program is a package of
its own so that the MCP SDK and its async runtime stay out of the energyml
binary, its Docker image and the Python wheels.
Reading array values needs a build with the hdf5 feature (cargo install energyml-mcp-cli --features hdf5). Without it the arrays are listed as unreadable
and energyml_read_array_window refuses with feature_not_enabled: the data is
not absent, the means to read it is.
Registering it in a client
Claude Code
claude mcp add energyml -- energyml-mcp /path/to/model.epc
claude mcp add energyml-wells -- energyml-mcp /path/to/wells.epc --h5 /path/to/wells.h5
One server per package: add it once for each file you want the model to see. Claude Code starts the program itself and talks to it over its stdin and stdout.
Claude Desktop
Add the program to the mcpServers object of the configuration file, then restart
the application:
{
"mcpServers": {
"energyml": {
"command": "energyml-mcp",
"args": ["/path/to/model.epc"]
}
}
}
On Windows, write the path of the package with forward slashes or doubled
backslashes ("C:\\data\\model.epc"), and give the full path of energyml-mcp.exe
as command if it is not on the PATH the application sees. The file is
claude_desktop_config.json in %APPDATA%\Claude on Windows and in
~/Library/Application Support/Claude on macOS (and ~/.config/Claude on Linux).
The locations are those of the client’s own documentation
(the Claude Desktop quickstart of the Model Context Protocol),
not something a test here checks.
A limit can be given as an argument in args ("--timeout-seconds", "60") or as
an ENERGYML_MCP_* variable in an env object. A configuration file is readable by
anyone on the machine: put no secret in it.
For an ETP server
retp mcp wss://... serves the same tools over an ETP server instead of a file
(the connection options and the credentials, passed by environment variables, are
those of retp). It is part of the retp
project, which depends on this library and not the other way round. The tool names,
arguments, limits and refusals described here are the same.
The tools
All are read-only. Names carry the energyml_ prefix so that a client which joins
several servers does not see two list_resources.
| Tool | What it does | Calls |
|---|---|---|
energyml_store_info | Kind of store, capabilities, dataspace and type counts, whether arrays are readable, diagnostics, the limits in force. Call it first | store_kind, capabilities, dataspaces, supported_types, diagnostics |
energyml_list_dataspaces | The dataspaces with their URIs (an EPC file is exactly one). custom_data: keys only unless the operator shows values | dataspaces |
energyml_list_resources | The objects of a dataspace, or those reachable from an object, with type and URI. types: exact or family.*, case sensitive | resources_in |
energyml_find_resources | Case-insensitive substring search: title_contains, type_contains (not the same as types above), uuid_prefix | search::find, headers |
energyml_describe_object | One object as EnergyML JSON ($type on every object), bounded; XML when the store lists it, refused by name otherwise | get_object, header, references, get_raw |
energyml_get_references | Targets, sources or both up to a depth; references to absent objects are reported as dangling. Each edge carries its kind (primary, secondary) when the store states it, and kind_basis when that kind is not given by the specification: derived-xsd, derived-by-value, or default (no rule and no schema fact: primary by default, with the reason in kind_basis_detail; ADR-0022, ADR-0048); edge_kind_coverage is all or names the MLs it is stated for | resources_in, references, resources |
energyml_list_arrays | The arrays an object points at, with shape and element type, no values; an unreadable array is listed with the reason | get_object, arrays_of, array_metadata |
energyml_read_array_window | A bounded window of one array, with statistics and a preview; never the whole array | array_metadata, window::read_window |
tools/list shows them in this order. Every uri argument goes through the one
door that parses ETP URIs; a bare uuid is accepted at the entry of
energyml_describe_object and energyml_list_arrays only, and turned at once into a
full URI (an ambiguous one is refused with the candidates). Everything the tools
return carries full URIs, which the model copies into the next call.
Every answer is bounded and says so: it carries truncated and, when true,
truncation { reason, limit, how_to_get_more }; every list carries total,
returned and next_cursor. Tool answers are sent twice, as the specification
asks: as structured content and as the same JSON in a text block.
The tool hints (readOnlyHint, openWorldHint, title) are only hints: an EPC is a
closed world, an ETP server an open one. Nothing is protected by a hint; the
program simply has no tool that writes.
Limits
Settings of whoever runs the server, never of a tool call. A call argument can lower a bound and never raise it.
| Limit | Option | Default | Role |
|---|---|---|---|
max_response_bytes | --max-response-bytes | 65 536 | hard bound of any answer; a larger one is refused too_large, lists stop earlier and say so |
max_list | --max-list | 500 | ceiling of every limit argument |
max_header_scan | --max-header-scan | 2 000 | headers a title search reads when nothing narrows it |
max_window_elements | --max-window-elements | 10 000 | elements one array window returns |
max_object_bytes | --max-object-bytes | 262 144 | ceiling of max_bytes of energyml_describe_object |
call_timeout | --timeout-seconds | 30 s | time one call may take |
Each option has an ENERGYML_MCP_<NAME> variable (ENERGYML_MCP_MAX_LIST, …,
ENERGYML_MCP_TIMEOUT_SECONDS); the command line wins. The program refuses a value
outside a sensible range at start-up (see its README). energyml_store_info reports
the limits in force.
The timeout. Each call runs on a blocking thread, because a store may block
inside its async methods (an ETP round trip, a file read), and is raced against
the timeout. A call that is too slow is answered with a timeout refusal at the
timeout. The thread cannot be cancelled: it ends on its own and its answer is
dropped.
Refusals
A tool that cannot answer returns a refusal, never an empty list: a tool result
with isError: true and {"refusal": {"code", "message", "store"?, "operation"?, "hint"?, "detail"?, "candidates"?}} as structured content (and as text). The model
can read it and correct itself.
| Code | Meaning |
|---|---|
not_found | no such object, dataspace or array; list first |
ambiguous | an identifier names several objects; candidates has their full URIs |
bad_uri | not a URI this store can use |
unsupported | the store does not offer this operation; do not retry |
feature_not_enabled | a capability is missing (HDF5 not compiled in): the data is not absent |
external_data | the data lives in a file or path that cannot be reached |
too_large | the answer would exceed a bound; narrow the request |
remote | the remote server failed or refused; retry later |
bad_arguments | an argument is missing, unknown, ill-typed or out of range |
timeout | the call took longer than the server allows |
other | anything else |
Only an unknown tool name is a JSON-RPC protocol error (code -32602); it is the one failure the model cannot act on.
Untrusted data
Titles, descriptions, server messages and custom data come from the package and may
contain instructions aimed at the model (“ignore previous instructions…”). The
tools return them only inside data fields (title, detail, candidates…),
as the whole value of the field, never inside a sentence written by the server. The
description of every tool, and the instructions sent at initialize, tell the model
that these fields are data and not instructions. Credentials that a remote server
echoes in an error are removed from detail.
None of this makes a package trustworthy: a model that reads a package it did not write is reading input from someone else. Treat the output of the tools as you would a document you were sent.
Standard output is the protocol
The program writes only JSON-RPC messages on stdout. Its start-up line and its errors go to stderr, and a failure to start (a missing file, a bad option) leaves stdout empty with a non-zero exit status. Clients show stderr in their logs.
Adding tools (for applications)
An application that embeds the library can offer its own tools next to these:
#![allow(unused)]
fn main() {
use std::sync::Arc;
use energyml_mcp::server::{Server, ServerIdentity};
use energyml_mcp::{Extensions, FixedStore, Limits, Registry};
let registry = Registry::builtin().with_tool(Box::new(MyTool));
let extensions = Extensions::new().with(MyAppHandle::new());
Server::new(registry, Arc::new(FixedStore::new(store)), Limits::default(),
ServerIdentity::new("my-app", "1.0"))
.with_extensions(extensions)
.serve_stdio()?;
}
Registry::with_tooladds a tool. Caller tools are listed first and tried first; one with the name of a builtin tool shadows it (the name is listed once and the caller’s tool answers).ToolContext::extensionscarries what the tool needs beyond the store (an application handle, the account of an HTTP request). The builtin tools never read it. A tool whose extension is missing should refuse with a named refusal, not panic.- A tool is a value implementing
energyml_mcp::Tool: name, title, description, input schema (additionalProperties: false), and anasync callthat answers aToolOutputor aRefusal. Its description must say that the contents of the fields of its answer are data.
The Tool trait and the registry do not mention any MCP SDK: the SDK (rmcp) is an
adapter behind the server feature of energyml-mcp, off by default, so that a
program that only wants the registry has no async runtime. See
ADR 0021.
Compatibility
The contract with clients is the tool names and their arguments. A tool name never
disappears or changes meaning; an argument added later is optional; a field added to
an answer does not remove an old one; an incompatible change is a new tool name and
the old one is deprecated. A test keeps a snapshot of tools/list and fails when one
of these moves without being written down.
The server answers both generations of the protocol (the initialize handshake and
the stateless server/discover of the 2026-07-28 revision); that is the SDK’s job.
Well sheet
A two-dimensional sheet of one well - depth column, stratigraphy, interval
properties laid out as tracks - computed in Rust and written as SVG, in the style
of borehole logs published by geological surveys. The crate is energyml-sheet.
A sheet of a Lutetia borehole, drawn by the library (the command is in the Gallery):
Status
| Part | State |
|---|---|
layout (layout::compute), text measurement (text::wrap) | done, tested in numbers |
SVG output (svg::write) | done |
reading a wellbore frame into a board (read::board_of_frame) | done |
| curves for per-node properties and channels | not started |
| PNG / PDF output | not started (they will be derived from the SVG, never drawn separately) |
| lithology patterns | study done, not coded; licences of pattern catalogues must be checked before any is bundled |
| command-line subcommand | not started (TODO(verify)) |
The model of a sheet
A sheet is an ordered list of tracks, each a source and a rendering.
| Source (where values along the measured depth come from) | Example |
|---|---|
| measured depth | the depth column |
| strata of a rank | chronostratigraphy, formation |
| interval property (categorical with its lookup table, discrete, continuous) | lithology by interval |
| per-node property, WITSML channel | temperature log (planned) |
| markers | picked boundaries (planned) |
| Rendering | For |
|---|---|
| flat fill with the model’s colour | strata, categories that carry a colour |
| wrapped text | labels, descriptions, values |
| curve with its scale in the header | per-node properties (planned) |
| graduation | measured depth |
Sources and renderings are held in registries, not match statements, so a
caller can add its own.
Layout rules
- Only interval tracks cut the sheet. Sections are the union of the bounds of the displayed interval tracks. A log with tens of thousands of stations never creates tens of thousands of sections.
- A gap is stated, never filled: an interval without a value is a visible blank. Two sources of strata that contradict each other on one well make two tracks, never a merge. Two frames with different datums are refused, naming both.
- Two proportion modes, selected by a registry:
- to scale: height proportional to depth, given a scale (for example 1:200) or a total height;
- readable (the default): each section gets at least the height its content needs, the rest is shared in proportion. The algorithm is a water-filling over levels that terminates in at most n rounds; if the minimums exceed the requested height, the sheet grows and says so rather than overflowing a text box. In this mode a to-scale ruler in the margin is joined to the stretched positions.
- The mode is written on the sheet and kept in the export.
- Text is measured, not guessed: the width table comes from the Liberation Sans font (SIL Open Font License), embedded so a layout is reproducible without any font installed. Each text column has a maximum number of characters, a minimum per line, and truncation with an ellipsis that is counted in the notes.
- Parameters such as scale, height and page width have no default: the crate does not choose them for the caller.
From Rust
#![allow(unused)]
fn main() {
use energyml_sheet::layout::{self, HeightGoal, Params};
use energyml_sheet::read::{self, Options};
use energyml_sheet::svg;
// A frame of a wellbore (a `WellboreFrameRepresentation`, as `&dyn Reflect`), the
// package's listing and an array source (an `EpcStore` is one) -> a board: one
// track per stratigraphic rank, one per interval property.
let read = read::board_of_frame(frame, &resources, &store, &Options::new("MD (m)"))?;
let params = Params {
proportion: "to-scale".to_string(), // or "readable"
height: HeightGoal::Scale(5.0), // mm of paper per metre of MD: 1:200
page_width_mm: 180.0,
text_size_pt: 8.0,
};
let sheet = layout::compute(&read.board, ¶ms)?; // pure: numbers only
let text: String = svg::write(&sheet); // the single exit
}
Options::new takes the header of the depth column, which the crate does not
choose. crates/energyml/examples/gallery.rs
draws a sheet from a real package, end to end.
The SVG is the only output: the sheet a person sees is the sheet that is downloaded. See also Section view.
Drawing: display lists, SVG, PNG, graduations and graph layout
energyml_sheet::draw is the drawing every view shares: what a view decides to
draw is a DisplayList, and the SVG, the PNG and the answer to “what is under
this point” all come from that one value. It is also not a viewer of
EnergyML: it names no EnergyML type, and a colour is a closure the caller
passes, never a colour map read here. The decision is
ADR 0028.
A display list
A DisplayList has a size and a unit (pixels, or millimetres on paper) and
layers, from back to front. A layer holds elements; an element is one thing a
user can point at, with an optional key (a full uri, a uri and a row…), a
kind, a tooltip and the primitives it is drawn with: a path, a thick polyline, a
rectangle, a polygon, a circle, a ring sector, a symbol, a line of text, a
raster. Everything is in user units, y down.
#![allow(unused)]
fn main() {
use energyml_sheet::draw::{
Colour, DisplayList, Element, Item, Layer, PageUnit, Paint, Rect, Stroke, hit, svg,
};
let mut list = DisplayList::new([200.0, 100.0], PageUnit::Pixel);
let mut layer = Layer::new("data");
layer.elements.push(Element::keyed(
"eml:///dataspace('d')/resqml22.MdPointSet(1)",
"bar",
vec![Item::Rect {
rect: Rect { x: 10.0, y: 10.0, w: 50.0, h: 30.0 },
corner: 0.0,
fill: Some(Paint::Solid(Colour::rgb(200, 80, 40))),
stroke: Some(Stroke::solid(Colour::rgb(0, 0, 0), 1.0)),
}],
));
list.layers.push(layer);
let svg_text = svg::write(&list, None)?;
let under_the_pointer = hit(&list, 30.0, 20.0).map(|h| h.key);
}
A list is checked before anything is written (DisplayList::validate): a
coordinate that is not a finite number, a negative width or a malformed name is
refused by name. Writing it as 0 would draw a wrong point.
The SVG
svg::write writes one group per element, with its key as data-key, colours as
attributes, everything escaped, and an optional <metadata> block the caller
fills (the body is the caller’s own text; this module never parses it).
svg::read_metadata reads it back, so that a view can be redrawn from its own
export.
The PNG, and the png feature
A PNG is always the SVG rasterised: png::render(svg, scale) is the only way
to one, and png::render_list is just render(&svg::write(list, None)?, scale).
The PNG therefore always shows what the SVG shows. It needs the png feature
(resvg, pure Rust, WebAssembly included, with the font embedded: no system
font, the same picture on every machine); without the feature, the call refuses
by name. The rasters a view embeds in its SVG are drawn through the raster-images
feature of resvg, with an image resolver that accepts only a PNG embedded in a
data: URL: any other image is refused by name (not a PNG, not embedded,
unreadable, too large), never left out of the picture. The feature also compiles
GIF, WebP and JPEG decoders, which are never reached.
Graduations
draw::ticks is the one rule for round numbers on an axis.
- Linear (
nice_ticks(a, b, target)): a step of 1, 2 or 5 times a power of ten, the smallest one that gives no more thantargetintervals (a ruler that crowds is worse than one that is sparse). Values arek x step, never accumulated, each the number nearest to its decimal (0.3, not0.30000000000000004), never-0. Bounds in either order; refusals by name: a bound that is not finite, equal bounds, a target below one. - Logarithmic (
log_ticks): 1, 2 and 5 times each power of ten if they fit the target, else the powers of ten, else one power in every few. Positive bounds only. labelandlog_labelwrite a graduation with the decimals its step needs.nice_step_at_mostandnice_step_at_leastgive the round step just under or just over a length, exact as a decimal (a scale bar).
This is the rule of the cross-section and of the studio. The desktop used to take the round step nearest to the ideal one, which draws more graduations on some ranges; it will draw fewer, and say so.
Spreading labels
spread::spread(positions, gap) pushes labels apart by at least gap, only
towards larger values and without changing their order. A position that is not
a finite number is not placed (it comes back as NaN) and moves nobody. Nothing
is clamped to a drawing area: the caller compares the result to its own height.
Measuring text
draw::text measures and wraps text against the table of Liberation Sans advances
(string_width_pt, wrap); energyml_sheet::text re-exports it. It works in the
unit of the size it is given.
A text has a weight (TextStyle::weight, FontWeight::Normal by default, Bold
written as font-weight="bold"; the PNG follows because it rasterises the SVG, and
embeds the regular face only). The table holds regular advances, so a bold text is
measured BOLD_WIDTH_FACTOR (1.1) times wider: it errs toward a line too short.
fit_to_page(&mut list) breaks every unrotated text that would leave the page
(past list.size[0] or before 0, against its anchor) into lines that fit, with
wrap; the first line keeps its baseline, the next ones are 1.25 sizes lower. A text
that fits is untouched. max_text_x(&list) is the measure a test holds a view to.
energyml-views applies it at the one door of a picture (draw_list), and breaks
the band of notes under the picture itself, so the page grows by its lines.
Rotated texts (axis titles) are not broken.
Hit testing
hit(&list, x, y) answers with the topmost keyed element under a point: last
layer, last element, last item, lines held within HIT_SLOP (3 units) of
their axis. A face that points with a mouse uses hit_within(&list, x, y, grab)
instead, where grab is the grab distance in units of the page (pixels for a
page in pixels; 0 or a value that is not a finite positive number means 0).
What hit_within holds: a fill (a rectangle, a polygon, a path, a sector, an
image) that contains the point, or a line, an outline, a disc, a symbol or a
text at no more than grab from the point (plus half the stroke width for a
line or an outline). The rule between several candidates:
- a line, an outline, a disc, a symbol or a text held within
grabbeats a fill that contains the point; - then the highest layer;
- then the nearest (a fill that contains the point is at distance 0);
- then the element drawn last.
An element without a key is never held; a layer’s clip holds nothing outside it.
Hit carries the layer, the element, the key and, on an image, the pixel.
hit itself is unchanged (plain order of drawing, its own slop): its answers
differ from hit_within(.., 0.0) when a fill is drawn over a line.
outline(&list, layer, element) gives the SVG path commands (M, L, Z, two
decimals, units of the page) that trace an element: rectangles, polylines,
polygons, flattened paths, discs and sectors in segments, symbols, images (their
rectangle) and texts (their box measured by draw::text, rotated like the text).
A face strokes it over the picture to highlight the element; None for an index
out of range or an element with nothing to trace.
Rasters
draw::raster paints value grids and quadrilaterals into an RGBA image through a
colour function of the caller, keeps gaps as gaps (a gap larger than a cap is
never bridged), and counts what it could not colour.
A layered graph
draw::layered places a directed graph in layers, deterministically: back edges
are found first, nodes are put in layers (two methods in a registry the caller can
extend, whose additions are tried first), ordered inside layers to reduce
crossings (never worse than the initial order), positioned, and the edges are
routed as cubics. Pins keep a node where the caller asked. A graph beyond its
bounds, an edge naming an absent node or a method that breaks the layer order is
refused by name.
What moves here later
The cross-section keeps its own list for now: its nested groups, zoom attributes
and byte-for-byte tests need more than a DisplayList offers today, so it
migrates in a lot of its own. Until then, a few second implementations remain
(XML escaping, the PNG renderer of the cross-section, the font file), and are
listed in the ADR.
Geostatistics and computation
energyml-analysis holds what is computed from EnergyML shapes, as opposed to
what a file states (which is energyml-geometry’s job). Interpolators, surface
meshing, structural and well computations all follow the same contract:
- inputs are plain slices of samples and query points; nothing here reads a package, so the crate builds for WebAssembly and works with any storage;
- a
NaNsample is set aside and counted, never averaged; - refusals are named: an interpolated
NaNand a malformed call are different news; - a function says what it measured and in which unit, and converts nothing the caller did not ask it to.
The neutral part (interpolation, meshing, small tools; no
energyml-*dependency) is its own crate,energyml-geostat, usable in hydrogeology, environmental or civil work; the domain part stays inenergyml-analysis. This page lists the modules by family. The modules of the tables below are inenergyml-geostat, exceptsurface,intersect,project,fault_throw,fault_sticks,horizon_law,t2d,survey,well_plan,tops,profile,well_logs,fake,step_rate,hall,flow_law,ion_balance,water_diagramsandcombine, which are inenergyml-analysis.
Three pictures from the Lutetia reference model, with the call that draws each in the Gallery:
Citations: what is and is not referenced
A reference is listed only when the source code already cites it, copied as
written there. Where the method is standard but the code names no publication,
the entry says TODO(reference): a citation is added only after the work has
actually been checked against it. Some entries in the code say the source was
“cited from memory, no copy in this repository”: those are kept as written and
flagged for verification.
Interpolation
| Module | What it does | Reference |
|---|---|---|
idw | Inverse-distance weighting (Shepard’s method): the value at a point is the weighted mean of the samples, with an exponent and optional radius. Deliberately the plain interpolator; exact at samples | TODO(reference): Shepard’s method is named in the code without a citation |
variogram | The empirical variogram (mean pair distance per class, semivariance, pair counts) and fitted models, spherical and exponential. range is the practical range for the exponential model (about 95% of the sill) | TODO(reference) |
kriging | Ordinary kriging with the kriging variance as the model’s statement of uncertainty. The dense system is LU-factorised once; each query is a back-substitution | TODO(reference) |
rbf | Radial basis function interpolation: thin-plate spline, multiquadric, linear, Gaussian kernels, with the polynomial term each kernel requires decided by the kernel and not exposed | TODO(reference) |
mba, mba_frame | Multilevel B-spline Approximation: extends a surface while keeping its general shape where points are sparse, with a least-squares plane removed first (the paper’s own initialisation). mba_frame fits in a tilted local frame for surfaces MBA cannot fit in world coordinates | Lee, Wolberg, Shin, “Scattered Data Interpolation with Multilevel B-Splines”, IEEE Transactions on Visualization and Computer Graphics, vol. 3, n° 3, 1997, p. 228-244 (DOI 10.1109/2945.620490) |
solve | A dense LU factorisation with partial pivoting shared by kriging and RBF (the Lagrange border of kriging and the polynomial border of RBF make the systems symmetric but not positive definite) | - |
barrier | Faults as barriers: a sample on the far side of a fault says nothing about this side | - (original method) |
Surfaces and meshes
| Module | What it does | Reference |
|---|---|---|
triangulate | 2D Delaunay and constrained Delaunay triangulation of a point cloud (faults as constraints), each point keeping its own z. Built on the spade crate for its exact geometric predicates, because floating-point predicates give overlapping triangles over nearly collinear points such as an alignment of boreholes | spade library; TODO(reference) |
isocontour | Iso-lines of a per-node scalar over a triangulated surface. Inside one triangle a linear field crosses a level along one segment or not at all, so there is no ambiguous case; segments are then chained into polylines | TODO(reference) |
isosurface | The iso-surface of a scalar over a structured lattice of ni x nj x nl nodes, by marching tetrahedra: each cell is cut into six tetrahedra by the Kuhn subdivision, which all share one diagonal, so two neighbouring cells cut their common face the same way (no crack) and a tetrahedron has no ambiguous case (no table of 256 cases). Vertices are keyed by the edge of the lattice they lie on, so the mesh is welded and the same lattice gives the same mesh bit for bit. The value may vary linearly or linearly in its base-10 logarithm along an edge. When nothing is cut the answer says why (Crossing::AllBelow, AllAbove, Grazed, InSetAsideCells), never a silent empty mesh. A lattice that folds back on itself gives a wrong surface and the function cannot see it: the caller refuses first, with the fold check of the well geometry | Kuhn (1960) for the subdivision, Doi and Koide (1991) for the method; TODO(reference): neither has been re-read |
surface | Dip, dip azimuth and curvature (mean, Gaussian, principal) of a triangulated surface, with the cotangent Laplacian over the mixed area | Meyer, Desbrun, Schröder and Barr (2003), Discrete Differential-Geometry Operators for Triangulated 2-Manifolds. TODO(reference): venue and DOI are not in the code |
resample, densify | Resample a polyline at a spacing, or add points between vertices while keeping every vertex | - |
loft | A surface between polylines (two lines, or successive sticks of a fault) | - |
outline | The boundary of a triangulated surface, or the convex hull of anything else | - |
extend | Extend a polyline or a surface past its border | - |
intersect | Intersection of two triangulated surfaces, as polylines | - |
project, heights | Vertical projection of nodes onto a surface or a level; exact heights of a surface under a plan position (triangles linear, quadrilaterals bilinear) | - |
nodes | What erasing nodes or moving them does to a shape | - |
cells, ijk, ijk_faults | Cells in an IJK box or crossed by a wellbore path (a column that leans or is no parallelogram is refused by name only where the path meets it, as blocked-wellbore reads only the cells near the path: an irregular column far from the path refuses nothing); an IJK grid between horizons (pillars, K gaps); an IJK grid cut by faults (split coordinate lines, stated throw) | - |
measure, thickness, groups | Areas, lengths and volumes of shapes; thickness between two horizons at every node; values split by a key (zone, stratigraphic unit) | - |
samples (private) | What the point interpolators (idw, kriging, rbf, variogram, mba) share before computing: the set-aside of samples with a non-finite value or position, the Euclidean distance, and the refusal of coincident samples | - |
buckets | The one grid of buckets for “which of these many things lie near here?”: boxes filed once in a regular grid sized to about one per bucket. query returns exactly the boxes a linear scan with the same closed-box test returns (ascending ids), nearest the lowest-id minimum of the caller’s own distance, candidates_at the ids filed in the bucket of a position. A box with a non-finite bound is left out and counted, never lost silently. Used by the surface extension (the band of a border node), the blocking of a path in IJK cells, the faces of heights, the projection of nodes on a fault trace, the cells of a grid cut by a section and the crosslines of a fence; each confronts it with the scan in a test | - |
vector, vector::plan | The one place the 3D and plan-view vector arithmetic is written: add, subtract, scale, dot, cross, norm, distance, the unit vector with the caller’s floor, the distance of a point to a chord or a segment, and the exact intersection test of two segments. energyml_geometry::vector re-exports it, so every crate reaches the same functions. See ADR 0044 | - |
Structural computations
| Module | What it does | Reference |
|---|---|---|
fault_throw | A horizon triangulated with fault traces as constraints, torn along each trace, hanging wall offset by a throw that dies out at both ends. It also holds the one reading of a fault trace (orient, read_trace) and the one hanging-wall rule (hanging_wall_side), which ijk_faults (a grid cut by its faults) calls too, so a torn horizon and a cut grid cannot disagree on which side goes down | - |
fault_sticks | Fault sticks drawn from a plane and the orientation of the surface they make | - |
horizon_law | A horizon laid by a law over a reference surface | - |
t2d | Analytic time-to-depth with a linear velocity-depth law V(z) = V0 + k z; k = 0 is a legitimate constant-velocity law | TODO(reference) |
Derived values
| Module | What it does | Reference |
|---|---|---|
combine | Two sets of values combined cell by cell: a ratio, a difference, the log of a ratio. The combinations are a registry (Combines, the caller’s own tried first). The second input is converted to the unit of the first when they are two units of one family, and refused otherwise. A pair where an input is absent (NaN) and a pair whose result is undefined (a division by zero, the logarithm of a value <= 0) are counted apart, and the cell is NaN, never an infinity. The result is a list of values: writing it back as a property is the combine operation of Creating objects | - |
Wells
| Module | What it does | Reference |
|---|---|---|
survey | A survey read back from stations: inclination, azimuth and dogleg by the minimum curvature method, inverted | API Bulletin D20 (Directional Drilling Survey Calculation Methods and Terminology, 1985); Sawaryn & Thorogood, A Compendium of Directional Calculations Based on the Minimum Curvature Method (SPE 84246, SPE Drilling & Completion 20(1), 2005) - as cited in well_plan |
well_plan | A wellbore from a drilling plan: straight holds and circular arcs, surveyed by minimum curvature | same as above |
tops | Where a well crosses a surface (computed tops) | - |
profile, wells_profile | Wells projected on a long profile: which heads fall in the band around a track, where along it, how far and on which side | - |
well_logs | Values given by a law of depth along a wellbore (geothermal gradient, hydrostatic pressure) | - |
fake | Fake dated data for tests: vertical polylines from a point cloud | - |
Tests and hydraulics
| Module | What it does | Reference |
|---|---|---|
step_rate | Which steps of a step-rate injection test have settled, and the injectivity index they give | - |
hall | The Hall plot of an injection (cumulative pressure against cumulative volume; the slope is the inverse of the injectivity index under steady radial flow) | Hall, H.N. (1963), How to Analyze Waterflood Injection Well Performance, World Oil - cited from memory in the code, to be verified |
flow_law, time_law | A flow rate over time and what it delivers between two instants; a value that drifts with time | - |
Water chemistry
| Module | What it does | Reference |
|---|---|---|
ion_balance | Whether a water analysis is electrically neutral: 100 (sum cations - sum anions) / (sum cations + sum anions) in meq/L. The module computes; the tolerance is the caller’s | molar masses: IUPAC standard atomic weights (abridged, CIAAW 2021), as cited in the code |
water_diagrams | Piper and Stiff diagrams. An analysis that does not balance is refused before anything is drawn; an absent ion is never drawn as zero (the caller may declare an ion absent, and the result carries the declaration) | Piper, A.M. (1944), A graphic procedure in the geochemical interpretation of water-analyses, Transactions, American Geophysical Union 25; Stiff, H.A. Jr. (1951), The interpretation of chemical water analysis by means of patterns, Journal of Petroleum Technology 3; Hem, J.D. (1985), Study and Interpretation of the Chemical Characteristics of Natural Water, USGS Water-Supply Paper 2254 (all three “cited from memory” in the code, to be verified) |
Testing the algorithms
Tests are unit tests over fabricated numbers. Where a publication gives a worked numerical example, the plan is to reproduce it in a test with the numbers typed in and the citation in a comment; nothing reads a file. TODO(reference): add such tests as references are verified. A provenance audit (no code ported from a licence incompatible with Apache-2.0) is part of the release checklist.
Public data sources
energyml-providers queries public APIs only and turns what they answer into
input for the operations engine. It never needs a secret.
Check each provider’s own terms before reusing data: the licence column below is
what the shipped catalogue states, not legal advice.
Principle
Building a request and reading a response are pure. The round trip is not.
Everything that can be wrong - encoding a filter, axis order of a bounding box,
reading a server’s refusal - is pure and tested without a network. The round trip
is a Transport trait; the native implementation (ureq) is behind the native
feature, and a browser would supply its own fetch.
A provider declares what it offers: its collections and, for each, the filters it accepts with their type, whether they are required and a suggested value. A command-line tool or a form is drawn from that declaration without spelling a provider, a collection or a filter name.
Axis order matters and was measured: the same WFS wants lat, lon in a bbox
and lon, lat in a CQL polygon, and the wrong one answers “nothing here”
without complaining. That is each provider’s business and is held in its
definition.
energyml sources # what this build can query
energyml sources brgm-bss # collections, filters, suggestions
energyml fetch brgm-bss boreholes --area area.geojson --where codeSpace=BRGM # the quote
energyml fetch brgm-bss boreholes --area area.geojson --where codeSpace=BRGM --out bss.json
energyml terrain --list --area 651000,6860000,653000,6862000
energyml terrain --area 651000,6860000,653000,6862000 --step 25 --set vertical=5720 --out dem.epc
fetch and terrain quote first and download only on request. Reading is
streamed, so a harvest of tens of thousands of features stays within tens of
megabytes of memory.
Boreholes: the BSS
| Provider | brgm-bss: boreholes of the French Banque du Sous-Sol (BSS), published by the BRGM through a WFS, GeoSciML 4.1 Borehole, plus intervals |
| Endpoint | https://data.geoscience.fr/api/gsmlbh/ows |
| Attribution | BRGM - Banque du sous-sol (BSS) |
| Licence | not stated. The service declares no fees and no access constraints, which is not the same as stating a licence; the catalogue therefore shows none rather than inventing a credit. TODO(verify): confirm the licence with the data provider |
A harvested page feeds the import operation its collection declares: energyml create --list --from bss.json, then energyml create sigeol-boreholes --from bss.json --set project=2154 --out bss.epc.
(The operation reads GeoSciML; it is tested with SIGEOL exports.)
Terrain and imagery catalogue
Sources are a catalogue, not a list in code: each entry states its protocol,
coverage boxes, systems, resolution, licence and attribution
(crates/energyml-providers/catalogue/terrain.json). A protocol is code; a
source is data - another service speaking a protocol already read is one more
entry, no rebuild. --catalogue mine.json puts your entries first; one with the
same id replaces the shipped one.
| Id | What | Resolution | Licence | Read today |
|---|---|---|---|---|
ign-rge-alti | IGN RGE ALTI, bare-earth terrain model (France), WMS raster | 1 m | Licence Ouverte / Etalab 2.0 | yes |
ign-lidar-hd-mnt | IGN LiDAR HD, bare-earth model | 0.5 m | Licence Ouverte / Etalab 2.0 | yes (coverage partial, growing) |
ign-rge-alti-mns | IGN RGE ALTI surface model (buildings, vegetation) | 1 m | Licence Ouverte / Etalab 2.0 | yes |
ign-ortho | IGN orthophotos (BD ORTHO), WMTS | 0.2 m | Licence Ouverte / Etalab 2.0 | imagery: listed, not turned into terrain |
copernicus-dem-30 | Copernicus DEM GLO-30 (cloud-optimised GeoTIFF on a public bucket) | 30 m | Copernicus DEM licence | listed; protocol not read yet |
aws-terrain-tiles | AWS Terrain Tiles (terrarium PNG), worldwide | about 30 m | mixed open data (SRTM, 3DEP and others); see the attribution document of the tiles | listed; protocol not read yet |
s2cloudless-2016 | EOX Sentinel-2 cloudless 2016, WMTS | 10 m | CC BY 4.0 | imagery: listed |
Attributions as shipped:
- IGN sources:
IGN - RGE ALTI,IGN - LiDAR HD,IGN - BD ORTHO. - Copernicus: “(c) DLR e.V. 2021 and (c) Airbus Defence and Space GmbH 2021 provided under COPERNICUS by the European Union and ESA”.
- AWS Terrain Tiles: see https://github.com/tilezen/joerd/blob/master/docs/attribution.md.
- EOX: “Sentinel-2 cloudless - https://s2maps.eu by EOX IT Services GmbH (Contains modified Copernicus Sentinel data 2016 & 2017)”.
The grid a terrain run writes carries its source, service, layer, licence,
attribution and fetch date in the activity and the object’s description. Without
--source, the finest relief source that covers the area and whose protocol is
read is chosen; imagery, an unread protocol, an area nothing covers and a window
beyond the service’s size limit (5010 pixels a side for the IGN WMS) are refused
by name, the last with the step that would fit.
Coverage boxes are approximate: a box inside a source’s envelope is not a promise of data (the IGN layer’s envelope includes overseas territories, sea and foreign land).
Adding a provider
An API speaking a protocol already read (WFS, WMS raster) is an entry in a
declaration, not a code change. A new protocol is a Provider implementation
behind the Transport trait. Either way the new provider appears in
energyml sources and in any form built from the catalogue.
TODO(verify): provider list and catalogue entries at the time of the first release.
Galaxy tools
Every operation of energyml create can be offered to a
Galaxy server as a tool. The tool is never written
by hand: energyml galaxy-tool renders its XML from the operation’s
declaration (energyml_build::galaxy), and the tool runs the public energyml
image. An operation added to the registry has its tool without a line written
for it.
Galaxy -- docker run <image> /bin/sh -c "<command>" --> /opt/energyml/scripts/galaxy/energyml-galaxy.py
| (in the image)
v
energyml create <op> ...
Generating a tool
galaxy-tool is a hidden subcommand: it does not appear in energyml --help
or in the generated command reference.
energyml galaxy-tool contacts-from-frames --out out/galaxy-tools/energyml_contacts_from_frames.xml
energyml galaxy-tool contacts-from-frames # the XML on the standard output
energyml galaxy-tool --all --dir out/galaxy-tools # one file per operation of the registry
| Option | Meaning |
|---|---|
<operation> | the operation, as energyml create --list names it |
--out FILE.xml, -o | where to write; standard output when absent |
--all --dir DIR | every operation, energyml_<operation>.xml each (- becomes _); --all requires --dir |
--image IMAGE | the container the tool runs in. Default: ghcr.io/geosiris-technologies/energyml:latest. Point it at your own image to use a private build |
--runner PATH | the runner’s path inside the image (default /opt/energyml/scripts/galaxy/energyml-galaxy.py) |
Regenerate rather than edit: each file says so in a comment. The tool’s
version is the version of the crate that wrote it. A tool and its image must go
together, because the XML is written for the arguments of the runner of the image
it names.
What the XML holds
Inputs
What an operation takes becomes Galaxy data inputs:
| The operation takes | The tool offers |
|---|---|
| tables | a csv input (multiple when the slot takes several) and a required template input (json) saying what the columns mean |
| features | a json input (read as GeoJSON) |
| a document | a data input |
| objects of the package | two optional inputs: URIs typed in (text, comma-separated) and a file of URIs - what an upstream step’s URI output holds |
| anything else (a ramp, a lattice) | nothing: these are pieces a page builds, not files a workflow passes; the tool’s help lists them under Not offered here |
Beside them, epc_in and h5_in (the package to add to and its arrays) are
always there. The package is required only when a slot must be given objects of
it; otherwise the operation writes a new package.
What an operation asks becomes parameters: a number is a float (with min,
max, the unit in the label), an integer an integer, an EPSG code an integer
with min=1, a flag a boolean, a closed choice a select, text and
vocabulary terms text.
An optional parameter left unset reaches the command as None; every guard in
the generated command refuses both '' and 'None', so the operation’s own
default applies. A test refuses any tool whose guard does not.
Outputs
| Output | Format | What it holds |
|---|---|---|
epc_out | epc | the package |
h5_out | h5 | its arrays |
uris_out | txt | URIs of the objects made, one per line (filtered by uris_type) |
manifest | json | what the run wrote, by role |
uris_<role> | txt | one per role the operation declares: the URIs of that role, empty when the run wrote nothing under it |
log | txt | the command and its report |
A downstream tool takes uris_<role> as its file of URIs: that is how the
steps of a workflow chain. An operation whose declared outputs are empty is not
declared yet, not “writes nothing”.
The runner
Galaxy names every dataset dataset_N.dat, whatever it holds, while energyml
chooses a reader by extension. The runner energyml-galaxy.py (in the image at /opt/energyml/scripts/galaxy/, in the repository at scripts/galaxy/) does
what that requires and nothing the command could be asked instead; every
decision about an operation stays in energyml create.
python3 energyml-galaxy.py run wells-and-properties-from-table \
--input 'table:csv=/data/dataset_12.dat::passes.csv' \
--template /data/dataset_13.dat \
--set project=2154 --set units=STRA \
--epc-out /data/dataset_20.dat --h5-out /data/dataset_21.dat \
--uris-out /data/dataset_22.dat --log /data/dataset_23.dat
| Argument | Meaning |
|---|---|
--input SLOT:KIND=PATH[::NAME] | a file for a slot, staged under the extension its kind says; PATH may be a folder or comma-joined paths; ::NAME is the name Galaxy shows (kept because an operation may read a stage or date from a file name). Repeatable |
--object SLOT=URI[,URI...] or SLOT=@FILE | objects of the input package for a slot; needs --epc-in |
--template FILE | what the tables’ columns mean (JSON) |
--set NAME=VALUE | a choice of the operation; ignored when the value is empty or None |
--epc-in, --h5-in | the package to add to and its arrays (staged as copies) |
--epc-out, --h5-out | where the package and its arrays go |
--uris-out, --uris-type | URIs of the objects made |
--manifest-out, --role-out ROLE=PATH | the run manifest, and one file of URIs per role |
--log | the command and its report |
--energyml | the binary to call (default $ENERGYML, else energyml) |
It fails the step on any [refused] line of the report: create writes a
package and exits 0 even when arrays were refused, so a refusal is exit code 1
here. A role file is always written, empty when the role wrote nothing.
python3 energyml-galaxy.py --self-test checks the role files against a manifest
without Galaxy and without energyml.
Installing the tools in a Galaxy server
-
Generate the XML (
energyml galaxy-tool --all --dir out/galaxy-tools). -
Copy the files into your server’s tools folder, in a sub-folder of their own.
-
Declare them in
tool_conf.xml, in a section of their own:<section id="energyml" name="EnergyML"> <tool file="energyml/energyml_contacts_from_frames.xml" /> <!-- one line per file --> </section> -
Pull the image on the machines that run jobs and allow Galaxy to run Docker containers for its tools.
-
Restart Galaxy or reload the tool configuration.
TODO(verify): publication of the tools on the Galaxy ToolShed.
The run manifest
energyml create --manifest FILE.json writes, and the tool’s manifest output
holds, what a run wrote - from what apply wrote, not from the plan:
{
"format": "energyml-run-outputs/1",
"package": { "epc": "out.epc", "dataFiles": ["result.h5"] },
"steps": [
{ "name": "contacts-from-frames", "operation": "contacts-from-frames",
"outputs": [ { "role": "representation", "declared": true, "count": 8,
"types": ["resqml22.PointSetRepresentation"], "uris": ["eml:///..."] } ],
"refused": 0 }
]
}
Describing the operations
energyml create --list --json # every operation
energyml create --list --json contacts-from-frames # one
Nothing but JSON goes to standard output and no input is read. Each operation
carries name, describe, makes, category, needs, takes, reports and
gives. It is the description the Galaxy tools are written from, and what any
orchestrator reads to know what an operation takes and returns.
Compatibility
The runner’s arguments are a contract with every workflow engine that calls it. Changing what they mean breaks those callers silently: treat them as a public interface and version them.
The energyml image
Everything the energyml command does, without compiling anything: the CLI built
with HDF5, in a container, working on the files of a mounted folder.
Get it
docker pull ghcr.io/geosiris-technologies/energyml:latest
edge follows the main branch. latest is the newest release; every release is also published as X.Y.Z and X.Y
(for example 0.3.0, 0.3), for a chain that must not move under its feet. Images are built for amd64 and arm64 by the CI of the repository, never by hand.
Or build it from a checkout:
docker build -f docker/Dockerfile -t energyml .
--build-arg RELEASE_LTO=thin builds faster and with less memory.
Use it
The folder holding your files is mounted at /work, which is the working
directory, so paths are written relative to it:
IMG=ghcr.io/geosiris-technologies/energyml:latest
docker run --rm -v "$PWD:/work" $IMG list model.epc
docker run --rm -v "$PWD:/work" $IMG check model.epc
docker run --rm -v "$PWD:/work" $IMG export model.epc --format obj -o out
docker run --rm -v "$PWD:/work" $IMG create --recipe recipe.toml --out model.epc
An EPC and its .h5 must both be inside the mounted folder: the package names its
data file relatively.
PowerShell users write -v "${PWD}:/work".
What the image holds
| Path | What |
|---|---|
/usr/local/bin/energyml | the CLI, built with HDF5 and the PNG rasteriser of the section view |
/usr/local/bin/enyml | the second binary built by the CLI crate; the entry point runs it as it is |
/usr/local/bin/energyml-mcp | the MCP server over a package (MCP server), from 0.3.0; run it with docker run -i <image> energyml-mcp model.epc |
/opt/energyml/scripts/galaxy/energyml-galaxy.py | the runner of the generated Galaxy tools |
/opt/energyml/REF | the commit the image was built from |
The image runs as uid 1000, not root. On a Linux host, files it writes into the mounted folder belong to that user; Docker Desktop maps them to your user.
The ETP client (retp) is not in this image: energyml etp ... says which program to install, and retp publishes its own image, ghcr.io/geosiris-technologies/retp.
Logging
docker run --rm -e ENERGYML_LOG=info -v "$PWD:/work" $IMG create --recipe recipe.toml --out out.epc
See Logging.
Running under a workflow engine
The entry point is energyml, or the program given in its place: a first
argument that is a path (/bin/sh, ./run.sh) or sh, bash, python, python3, env, enyml
is run as it is; with no argument it prints the help. Galaxy runs a tool’s command as
docker run <image> /bin/sh -c "<command>", which an energyml-only entry point
would refuse as an unknown subcommand. See Galaxy tools.
Logging
The operations and the package writer emit tracing spans and events.
Nothing is shown by default: with no filter asked, no subscriber is
installed and every command prints exactly what it printed before.
A span reports how long it took (time.busy, time.idle on its closing line),
so the log answers “which step is slow” without a profiler. A log is not a
report: what you must know about a run (refusals, notes, what was set aside)
stays in the operation’s report; the log says where the time went.
From the command line
energyml create --recipe recipe.toml --out Lutetia.epc --log info
energyml create ... --log energyml_build=debug,energyml_storage=info
ENERGYML_LOG=info energyml create ... # the same, from the environment
energyml create ... --log info --log-format json 2> journal.jsonl
--log <filter>(orENERGYML_LOG) is atracing_subscriberEnvFilter: a level (error,warn,info,debug,trace,off) or a comma-separated list oftarget=level. Targets are crate names with underscores:energyml_build,energyml_storage.--log-format text|json(orENERGYML_LOG_FORMAT): human lines or one JSON object per line. The flag wins over the variable.- Always stderr, never stdout: what a script reads from stdout is untouched.
infoshows one line per span close (operation, recipe step, read, write) with its elapsed time;debugadds per-object detail. There is never an event per array element.
In Docker
docker run --rm -e ENERGYML_LOG=info -v "$PWD:/work" ghcr.io/geosiris-technologies/energyml create --recipe recipe.toml --out out.epc
docker run --rm -e ENERGYML_LOG=info -e ENERGYML_LOG_FORMAT=json -v "$PWD:/work" ghcr.io/geosiris-technologies/energyml ... 2> journal.jsonl
Changing the level while a program runs
energyml-log is built for an application that is already running:
#![allow(unused)]
fn main() {
// At start: install the subscriber, logs off.
let handle = energyml_log::init_reloadable(None, None)?;
// Later, from a menu:
handle.set_filter("debug")?; // on
handle.set_filter("energyml_build=trace")?; // one crate
handle.set_filter("off")?; // and off again
}
init installs nothing when no filter is given (a command-line tool);
init_reloadable always installs, starting off, so the level can be raised
later. An invalid filter is refused and leaves the current one in force.
energyml_log::dispatch builds the same subscriber over any writer without
installing it, which is how tests check the switch.
For library authors
Libraries depend on tracing only and never install anything. Spans on phases,
counts as fields, no event per element.
Troubleshooting
Problems a user can hit, by symptom.
Building with HDF5
Reading and writing arrays needs the hdf5 feature, and therefore the HDF5
library.
Windows: Unable to infer HDF5 library runtime version
The build script of hdf5-metno-sys fails with this message, which looks like a
PATH or HDF5_DIR problem.
- Cause: the installed
hdf5.dllis the Intel/Fortran variant, which importslibmmd.dll(an oneAPI runtime) that is not on the disk. The message never names the real cause.dumpbin /DEPENDENTS hdf5.dllshowslibmmd.dllwhen it is the case. - Solution: use only the official MSVC build of HDF5 1.14.6
(
hdf5-1.14.6-win-vs2022_cl.zip), installed outsideProgram Files, withHDF5_DIRpointing at it. Copy itshdf5.dllnext to the executable (the executable’s folder wins overPATH).
Invalid H5_VERSION: 2.2.0
The version check of hdf5-metno-sys only accepts a fixed set of minor versions
and does not know HDF5 2.x. Stay on HDF5 1.14.6. If a 2.x install is earlier on
PATH, test binaries load it: this works in practice but is not the version they
were compiled against.
h5dump / h5ls print nothing and exit 53
Some installed builds are unusable. Do not spend time on them: to check what an
HDF5 file really holds from outside the library, read it with h5py.
Binaries without HDF5
Where HDF5 cannot be compiled, a binary can be built without it. It then
refuses by name to read an array (FeatureNotEnabled) instead of returning an
empty one. An empty result with no error is never the library’s way of saying
“no HDF5”.
Python
ImportError: DLL load failed on Windows
import energyml.rust fails on a machine without HDF5, and putting HDF5 on
PATH does not help (Python 3.8+ no longer searches PATH for the dependencies
of an extension module).
-
Cause: a wheel built locally without bundling HDF5.
maturin --auditwheelis a Linux flag; on Windows it does nothing and does not say so. -
Solution: repair the wheel with
delvewheel, naming the namespace package so that it does not writeenergyml/__init__.pyover the otherenergyml.*distributions:delvewheel repair --namespace-pkg energyml --add-path "$HDF5_DIR/bin" -w wheels staging/*.whlReleased wheels already carry HDF5.
Another energyml.* package disappeared after install
Two distributions that both ship energyml/__init__.py overwrite each other. This
bridge ships none. If one is missing, reinstall the other.
Opening and checking packages
Arrays cannot be found after a rename or move
An EPC names its .h5 in every array pointer and in the .rels of parts that go
through a proxy. If you renamed or moved the file by hand, the package opens and
its arrays are unreachable.
- Find where each array really is:
energyml h5 model.epc; write the answer into the package withenergyml h5 model.epc --fix fixed.epc. - Offer the file explicitly:
energyml --h5 renamed.h5 ..., or--h5 model.h5=path/to/renamed.h5to pin a path to the name the document uses. - To rename on purpose, use
energyml rename model.epc --to new.h5(quote), then--apply: fields, relationships and the file change together.
The order in which the files are tried, and what the command says when it goes somewhere the package did not point, is in Where the arrays are looked for.
rename fails on Windows with a file in use
An HDF5 handle still open on a file locks it. The rename operation releases the handles it holds first; close any other program that has the file open.
An object is read in the wrong release, or as a development drop
Check the declared versions: energyml list model.epc. A package written against
a development drop but declaring the release reads as the release and silently
drops what only the drop has. energyml relabel model.epc --package resqml --schema-version 2.2_dev3 corrects the declaration without reading the content.
A part is refused
energyml repair model.epc lists the parts no parser will touch (markup that does
not close, invalid UTF-8, an empty part) with the reason. --extract writes their
bytes out, --apply puts repaired bytes back. --lenient reads past an absent
required element or an unreadable value and reports each; do not save what it
read, because it holds values nobody wrote (use --strict when writing).
check reports a dangling reference to a PropertyKind
A well-known PropertyKind of the Energistics catalogue is absent from a package
by design; such references are reported as catalogue references and count as
neither error nor warning. Any other dangling reference is real.
A coordinate looks plausible and is wrong
Check the unit and the placement: energyml geometry model.epc --points --metres. A file whose horizontal unit is feet read as metres is 3.28 times too
far from the origin with every number looking reasonable. A unit the library does
not know is refused, not assumed.
GeoJSON objects are missing, or land in the wrong place
GeoJSON is always WGS 84, so energyml export --format geojson needs the EPSG code
of each object’s CRS. An object whose CRS names none is skipped, never placed
anywhere: the command prints N skipped: geojson cannot write them, adding that
objects without a usable EPSG code are skipped. Check energyml geometry for the
CRS the package states; an eml 2.3 CRS is three objects and the code lives in the
third.
--native-crs writes those objects anyway, in their stored coordinates, with the
legacy crs member naming the EPSG code when there is one. The file is then not WGS
84, and the command says so. Small local coordinates read as degrees sit near
longitude 0, latitude 0: the Gulf of Guinea.
Exports
“evaluated but could not be written”
One representation that cannot be written does not stop the others. The line names the object and the reason, for example a travel-time CRS that needs a velocity model the package does not carry. This is the usual shape of a refusal: the geometry evaluated fine and the placement has no answer.
Docker
Files are written by uid 1000 inside the container. On a Linux host, set the
ownership of the mounted folder accordingly; Docker Desktop maps it to your user.
An EPC and its .h5 must both be inside the mounted folder.
Still stuck
Run the failing command with --log debug (Logging) and open an
issue with the output and the exact version (energyml --version).
Where the arrays are looked for
An EPC does not hold its big numeric arrays. It holds XML objects, and each
array pointer names an external file, usually an HDF5 file, and a dataset path
inside it. The name is a string, and strings go wrong: someone renames
model.epc and model.h5, a colleague sends only the .h5 under whatever name
their tool chose, a folder is moved. Every number is still on disk. Only the
name is lost.
So the library does not ask “which file has the right name” but which file holds this dataset. The name decides the order of the search; the content ends it.
The order
For every dataset, the places are tried in this order, and the first file that holds it wins. A file that exists but does not hold the dataset steps aside for the next one.
| # | Place | Where it comes from |
|---|---|---|
| 1 | A file pinned to the name the document uses | --h5 model.h5=path/to/renamed.h5 |
| 2 | A file offered without a name, in the order given | --h5 path/to/data.h5 |
| 3 | The name the document writes, absolute or relative to the package’s folder | the pointer and the .rels of its proxy |
| 4 | The file beside the package with the package’s name | model.epc gives model.h5 |
| 5 | Any other file in the package’s folder, in a stable order | the folder, not its subfolders |
Three details:
--h5is read once, by the library (ExternalFiles::apply_spec), for the command line and the MCP server alike. The text before the first=is a name only when it is a bare file name:--h5 /data/run=1/x.h5offers that path, it does not pin1/x.h5to the name/data/run.- An offer comes before the document’s own name, because offering a file is an instruction, and people give one precisely when the package is wrong.
- Another
.epcin the folder is never a candidate: a folder holding two packages is the ordinary case, and one is not a place to look for the other’s numbers.
A package whose names are correct is not searched: it resolves through its own declaration, and nothing is said.
What you are told
The search never goes somewhere unexpected in silence. What it says depends on what it did:
-
A file found by the search (4 or 5) is reported on stderr when the command ends, one line per file, with how many datasets came from it:
warning: 736 dataset(s) read from `folder/export.h5` instead of the file they point at (the package names `model.h5`): another file in the package's folderIt counts datasets, not array pointers: two objects can point at the same one, and it is counted once. A file you pinned or offered is used as you said, and is not reported when it holds the data.
-
A
--h5path that is not on disk is reported at once, and ignored:warning: --h5 `typo.h5` does not exist and is ignoredIt is dropped from the search because there is nothing to open, and a mistyped path used to look like a package with no data file.
-
Nothing holds a dataset:
energyml arrays --valuessays how many arrays could not be read on stderr, and each failing array names what was tried, including an offered path that is not there:`/resqml22/…/values` is in none of the 0 candidate file(s) for `model.h5`; offered by the caller and not found on disk: `typo.h5`arraysstill exits 0 when some arrays could not be read. A script that needs the arrays to be there should askenergyml h5, which exits 1 when a file is not found.
The command that answers the question: energyml h5
energyml h5 model.epc # where each array's numbers really are
energyml h5 model.epc --fix fixed.epc # write the answer into a package
h5 prints one line per file the package names, with the file that was found
for it and why, or NOT FOUND, and exits 1 when a file is not found. --fix
writes a package whose declarations name the files the search found, so that
the next reader does not pay for the search again.
To rename on purpose, use energyml rename, which changes the fields, the
relationships and the file together.
A rename never replaces a file: if a file named like the new name already sits beside the package and the package does not name it, the rename is refused (in the quote, and again when it is applied, since a quote can be stale), because moving a file onto another destroys the other. And it does nothing by halves: every object is rewritten on a copy first, so a rename that cannot be carried out leaves the package exactly as it was.
An array split over several ExternalFileProxy parts (eml 2.1 and 2.2) comes
back as one location per part, each carrying its own flat StartIndex and
Count. A reader that cannot take a flat range of a multi-dimensional dataset
refuses by name; it never reads the whole dataset once per part.
From the library
ExternalFiles::offer(path)andExternalFiles::set(name, path)(throughEpcStore::external_files_mut) are--h5’s two forms.EpcStore::array_departures()lists, once the arrays have been read, the fallbacks the search took: the name the document wrote, the file read, why, and how many datasets came from it. Empty for a package that was right.ExternalFiles::missing_offers()lists the pinned and offered paths that are not on disk.EpcStore::resolve_array(location)says, for one array, which file holds it and what put that file in the search.
Which file was read is a fact the library knows and a face tells: the library has no terminal. See also Troubleshooting for the symptom “arrays cannot be found”.
Schemas and supported versions
Supported versions
The library reads and writes these Energistics data schemas. Every version is a
separate generated crate under generated/, and energyml-schemas registers
them in one Registry (energyml_schemas::all(), or energyml::schemas::all()
through the energyml crate) - those whose Cargo feature is on.
all() registers every generated type on each call, which is a cost to pay once, not per request. A program
that answers many requests takes energyml_schemas::shared() instead: the same registry, built on the first
call and handed out as an Arc afterwards (the compiled-in crates cannot change during a run, so it is never
stale). Keep all() when you need an owned registry to add to.
| Standard | Versions |
|---|---|
| RESQML | 2.0.1, 2.2 (and the development drop 2.2_dev3, see below) |
| EML (Energistics common) | 2.0, 2.1, 2.2, 2.3 |
| WITSML | 2.0, 2.1 |
| PRODML | 2.0, 2.2, 2.3 |
Choosing the versions you compile
Each version is a Cargo feature, spelled as Energistics spells a qualified type:
| Standard | Versions | Features |
|---|---|---|
| EML | 2.0, 2.1, 2.2, 2.3 | eml20, eml21, eml22, eml23 |
| RESQML | 2.0.1, 2.2, 2.2_dev3 | resqml20 (the 2.0.1 release), resqml22, resqml22dev3 |
| WITSML | 2.0, 2.1 | witsml20, witsml21 |
| PRODML | 2.0, 2.2, 2.3 | prodml20, prodml22, prodml23 |
The families eml, resqml, witsml, prodml and all switch several on. A
version brings the EML version it is built on (resqml22 brings eml23,
resqml20 brings eml20). The energyml crate turns no version on by
itself, so a dependency says what it reads:
energyml = { version = "0.1", features = ["resqml22"] } # or ["all"]
energyml-schemas itself defaults to all (for the tools that want
everything); the energyml command and the Python wheel ship every version. A
version that is not compiled is a named refusal (“RESQML 2.2 is not compiled
in: enable the resqml22 feature”), never a half read. The façade’s build
feature (and providers, which implies it) turns on the four versions it
writes: RESQML 2.2, EML 2.3, WITSML 2.1 and PRODML 2.3.
Everything above the generated layer - units, coordinate systems, value arrays, geometry, exports - is written once for all versions. A concept keeps its version-independent name, so a reader for a concept serves every release that keeps it; a release that respells the inside of a type is handled by a reader that accepts both spellings.
The development drop resqml 2.2_dev3 is registered so that files written
against it before the 2.2 publication still open. It is read, never offered: no
new object is created in it, because most of its types carry the very qualified
type of a released one, and the rest exist only in the drop and no released 2.2
reader knows them. energyml relabel can correct a package that declares the
release while holding drop content.
Adding an Energistics release costs a generated crate, one dependency, one
feature, one row of KNOWN_VERSIONS and one register line in
energyml-schemas; the storage, domain and geometry layers name no version. A test checks that the registry and the list of versions the
code is generated from agree.
Where the generated code comes from
The Rust under generated/ is produced mechanically from the official
Energistics XSD files by an internal tool that is not part of this repository.
Generated files are committed and are never edited by hand; a robot proposes
changes to them as pull requests. generated/schemas.lock records the SHA-256 of
each input schema so a reviewer can tell which input a file was generated from.
It contains no Energistics schema content.
The generated code is public because it is a derived work, produced under the Energistics Product License Agreement; the XSD files themselves are obtained from the official publisher (The Open Group) and are not redistributed here.
Each generated crate carries a NOTICE. TODO(verify): final wording and any
condition The Open Group or Energistics attaches to naming crates after their
standards.
Notice
This product includes software developed by Geosiris. The following Energistics(R) products were used in the creation of this work: RESQML(TM) Data Schema v2.0.1 and v2.2, WITSML(TM) Data Schema v2.0 and v2.1, PRODML(TM) Data Schema v2.0, v2.2 and v2.3, Energistics common v2.0-v2.3, Energistics Transfer Protocol (ETP) v1.2. Energistics(R), WITSML(TM), PRODML(TM) and RESQML(TM) are trademarks or registered trademarks of Energistics Consortium, Inc. All rights in the Energistics standards, or any portion thereof, which remain in this work remain with Energistics and are subject to the Energistics Product License Agreement (https://www.energistics.org/product-license-agreement).
TODO(verify): this text follows the licence’s requirement for derived works; confirm it against the NOTICE file of the repository, which is authoritative.
Disclaimer
This project is not affiliated with or endorsed by Energistics or The Open Group. RESQML, WITSML, PRODML and Energistics are trademarks of their owners. “EnergyML” is used here as the common name of the family of standards.
Related
The ETP (Energistics Transfer Protocol) implementation lives in a separate
repository, retp.
ADR 0001 explains why generated code
is committed.
How a colour is found
RESQML 2.0.1 has no colour at all; from 2.2 on, colour is carried by graphical information objects that apply to an object, to a collection of objects, or to a whole model. An object that carries nothing itself can therefore still be drawn in a colour that was chosen for something it belongs to. This page states the order in which the library looks, and what it answers.
The walk lives in energyml-domain (inherit for the order, which knows nothing
about colours; color_inherit for the colour question). It reads relations from
the package index, so no object is opened
to follow a link. The rule that there is one colour decision for every face is in
ADR 0009.
The order
The first rank that has an answer wins.
| Rank | Where the colour is looked for |
|---|---|
| 1 | on the object itself |
| 2 | on the collections that reference it, then on the collections of those collections |
| 3 | on its parents |
| 4 | on the collections that reference those parents, nested |
| 5 | on the parents of the parents, with the same rules, one stage up at a time |
| 6 | on the model that references the object or one of its parents |
A parent is reached through the three fields a representation, an
interpretation and a feature climb by: RepresentedObject,
RepresentedInterpretation and InterpretedFeature (the feature /
interpretation / representation chain, “FIRP”). A collection is a membership
written by a CollectionsToDataobjectsAssociationSet: a member is in the
collection of the association with the same index.
Within one rank the shortest path wins; at equal length the first in document
order wins, and the other candidates are kept and shown as ties, not hidden.
Rank 5 is read stage by stage, so a collection of a grandparent does not race a
great-grandparent. Some things are noted but not followed: the collections of the
model, and any collection that is a project.
A property has a shorter path: the property, its property kind, the kind’s parents, a collection at each step, and nothing beyond. A property never takes the colour of the representation it hangs on, nor of a model.
What the answer carries
An answer is not just a colour. It carries:
- the level it was found at (the object, a collection, a parent, the model), read from the last step of the path;
- the path from the object to the one that carries the colour: each step says where it leaves, where it arrives, by what kind of link (parent, collection, model) and through which field;
- the ties that lost, and the notes about what was skipped and why (a cycle, a duplicate association, a colour on a collection that is not followed).
A face that shows a colour can therefore also show why it is that colour.
Nothing found is no colour
When no rank answers, the answer is none. The library does not invent a default: choosing what to draw then is the decision of the view, which is the one that knows whether to derive a hue from the identifier, draw grey, or hide the object. This follows refuse rather than draw.
Placing a value on the map
Finding the colour map is one question; placing a value on it is another.
ColorMap::sample takes the value and a Scaling (the ColorInformation of the
object) and does this, in order:
- A missing value or a
NaNhas the map’s null colour, whatever the scaling. MinMax, when given, is the range of values that spans the map; a value outside it takes the extreme colour. WithoutMinMax, the map’s own range stands for the range of the values.- Under
UseLogarithmicMapping, the log of the value is what is placed between the logs of that range (RESQML TRG 2.2 section 4.5, p. 56). InterpolationMethod(linearorlogarithmic, section 4.7 p. 58) is a different thing: the spacing between two stops of a continuous map. The two may be combined, and neither stands for the other.
A range that reaches zero or less has no logarithm. ColorMap::check_scaling
refuses it by name, try_sample returns that refusal, and sample answers
None (call check_scaling once to say why). The null colour is still given to a
missing value. See ADR 0025.
Cycles and duplicates
A collection that holds itself, near or far, is a cycle: it is named, and the walk does not follow the edge that closes it. The same collection named in two associations of one set counts once, as does an object listed twice in an association, as the common schema’s rules on collections require; both cases are counted in the notes.
Reading a colour map once
A colour map shared by thousands of properties is read and validated once by a Cascade, then kept for the
life of that cascade (and for the one source it was read from). Nothing invalidates it: a caller whose package
changed builds a new cascade, as it already does for the set. The stops of a map are sorted by index, and
sampling a value bisects them instead of walking them from the first.
Tables, keys and links
A table is columns of values: a category table (facies 3 is sandstone), a
relative permeability table, the tables an application keeps beside a package
(indexes, activities, files). This page is about what a cell holds, how a value
names a row, how a column links to another table, and what the library says about
the links, the rows and the depths of a table. Everything here is in
energyml-domain, in pure Rust, and builds for WebAssembly.
The library refuses rather than guesses. A link that leads nowhere, a table it cannot read, a key it cannot use: each is a named refusal or a finding, never an empty cell that looks like a table with nothing to show.
What a cell holds
A cell has four states, and none of them is turned into another:
| State | What it is |
|---|---|
| a value | a number, a boolean, a non-empty text |
| absent | an integer equal to the NullValue of its array, or a NaN |
| empty | an empty string: a value, not an absence |
| unreadable | the column could not be read; the refusal says why |
An empty string is a value because a StringXmlArray has no NullValue: nothing
in the file says that "" means nothing. It sorts as "". It names no row when
it sits in a link column.
ValueCountPerRow is the number of values of a row (common TRG 2.3 §9.1 p. 190:
if it is greater than 1, the fastest dimension of the array is this value; a
column of 10 rows of 3 is [10, 3]). A column of 7 values with a count of 3 is
refused, naming the column, rather than cut to 2 rows and a half. A row of such a
column is a vector of its values, where each component can be absent.
use energyml_domain::table::{CellState, Table};
let table = Table::read(object)?;
let columns = table.read_cells(&arrays); // one entry per column
let cells = columns[1].as_ref().expect("readable");
match cells.cell(0) {
Some(CellState::Value(cell)) => println!("{cell}"),
Some(CellState::Absent) => println!("no value"),
_ => {}
}
Rows are positional: row i is index i in every column (common TRG 2.3 §9.1: “Ordering matters since it will be the same in all columns”). Columns that do not hold the same number of rows are reported with the rows they share, never truncated in silence.
Which row a value names: the key, else the rank
A KeyColumn makes a table a lookup table. The standard does not say how a value
names a row; the library reads it one way, in one place, row_named_by:
- a table with one key column: a value names the row whose key it equals;
- a table with no key column: a value names a row by its rank, from 0;
- a table with several key columns: a value names no row (it is not a tuple), and the refusal says so.
A table with a key column is never read by rank: a key that finds nothing would
land on an unrelated row. Numbers compare exactly (-0.0 is 0.0, NaN is never
a key), and an empty key names nothing. When the same key is held by several rows,
the first is used and the duplicates are reported.
use energyml_domain::table::{Cell, Lookup, RowMatch, row_named_by};
let lookup = Lookup::read(object, &arrays)?;
let index = lookup.key_index(); // built once per table
assert_eq!(row_named_by(&index, &Cell::Number(20.0))?, RowMatch::One(1));
The category check of a property (house/discrete-value-in-lookup) reads its
values the same way.
A wellbore table (wellbore_table::of_frame) writes the label of a discrete
property, never its code. A code no row of the table names, and a code equal to
the array’s NullValue, are empty cells; the first are counted in the table’s
notes. A property whose lookup is missing or unreadable is left out with a note
that names the reason, rather than written as codes under a label’s heading.
The findings of energyml check are filed under the object’s uri: the report
items (SchemaIssue, PropertyIssue, CountIssue, RuleFinding) carry the uri
beside the uuid, so two versions of one object are told apart.
Links between tables
The specification has no column that names another table. The library reads one convention, and two that producers use, and says which one it read (ADR 0030).
B, one qualifier facet (ADR 0029).
The column has the kind identifier (its values are keys of the target) or
explicit index (they are ranks), and one qualifier facet whose value is the
complete ETP uri of the target table:
<eml:Column>
<eml:Title>index_uid</eml:Title>
<eml:ValueCountPerRow>1</eml:ValueCountPerRow>
<eml:PropertyKind>
<eml:Uuid>311dcaba-6512-4144-aa7c-79a9de19b4ab</eml:Uuid>
<eml:QualifiedType>eml23.PropertyKind</eml:QualifiedType>
<eml:Title>identifier</eml:Title>
</eml:PropertyKind>
<eml:Values xsi:type="eml:StringXmlArray">
<eml:Values>i1</eml:Values>
<eml:Values>i2</eml:Values>
<eml:Values>i1</eml:Values>
</eml:Values>
<eml:Facet>
<eml:Facet>eml:///eml23.ColumnBasedTable(7179e925-fba7-50be-9032-5462a75484da)</eml:Facet>
<eml:Kind>qualifier</eml:Kind>
</eml:Facet>
</eml:Column>
The target is the one that decides how a row is named: its one key column, or its
ranks when it has none. If the kind and the target disagree, the target wins and
the disagreement is reported. A qualifier facet that is not a uri (cumulative)
is not a link.
A, the description. key of the table <title>: <uri>, KeyColumn <key> (or
key of this table (self reference): <uri>, KeyColumn <key>). Read, followed and
reported as a convention of producers. Text that starts like this and does not
follow the form is refused, not guessed.
D, a title in a sibling column. A column source_uid with a column
source_table that names, row by row, the table of each value. Read, followed and
reported: a title is not an identity. A title that names no table of the package is
an external reference; one shared by two tables is refused.
E, uris as values. A column whose every non-empty value is a complete ETP uri
names objects, not rows. Its values are object chips (source::object_chip): the
object’s type, its title when the package holds it, or the fact that it does not.
A uri of another dataspace is not looked up.
When the facet and the description name two different tables, neither is followed
and the refusal names both. The conventions are tried in order, the caller’s first
(LinkReaders::push), and one can be dropped (LinkReaders::without).
use energyml_domain::table::{LinkReaders, TableSheet, resolve_kinds};
use energyml_domain::property_kind::Resolver;
let sheet = TableSheet::of(object)?; // no array is read
let kinds = resolve_kinds(&sheet, &Resolver::new(&objects));
for (column, declaration) in LinkReaders::builtin().declarations(&sheet, &kinds) {
println!("{column}: {declaration:?}");
}
Writing a link
The library reads convention B; the table-link operation of
Creating objects writes it, on an explicit gesture and never as a
side effect. It takes the object that holds the link (a table whose column links, or a
property whose component axes the target holds) and the target table, and writes what the
reader above reads first:
- The facet. One
qualifierfacet whose value is the complete ETP uri of the target, its version included when it has one, never a bare uuid. A column that already has aqualifierfacet of the same value has nothing to write; one of another value is refused, because a column links to one table. - By key or by rank.
by = keyneeds the target to have exactly one key column and the column to have the kindidentifier;by = ranktakes a column of kindexplicit indexorclassification index, and a target with no key column is accepted. The kind is checked with the same rule and the same constants as the reader, and never changed silently: a column of another kind is refused, and the refusal says to set the kind first withedit-object. - The description. For a link by key, when the column has no description, the
operation also writes convention A in the exact grammar the reader accepts
(
key of the table <title>: <uri>, KeyColumn <key>, orkey of this table (self reference): ...when the target is the table itself, as for a tree), for a human reader. A description that is already there is kept, and refused when it names another target than the facet. A link by rank has no form A and gets no description. - A property. For a property, no column is named; the facet is written on the property itself.
The write is the one of edit-object, and the operation records an Activity with the
column, the way, and the target’s uri. What was written reads back as convention B with no
warning, which is how the operation is tested against the reader.
Checking the links
check_links follows every cell of every link column and counts what it finds:
rows found, dangling, ambiguous (the target key is held by several rows), external
(a title naming no table), empty, absent, and the cells that cannot be followed,
with why. check_links_of_set repeats it for a set of tables. energyml check
reports the same facts as findings, by the rules of the table theme
(energyml rules --house --theme table):
| Rule | Weight |
|---|---|
house/table/link-resolves | error |
house/table/link-target-key-unique | warning |
house/table/link-declared-by-facet | warning |
house/table/uri-value-resolves | warning |
A link whose target is not in the package, or that was given no array reader, is reported as uncheckable: not having looked is not having found nothing.
Big tables: a window
TableWindow keeps a sort permutation and a filter mask of 4 bytes per row, and
hands back a page of row numbers. Absent values come last in both directions
(an absence is neither the smallest nor the largest), an empty string sorts as a
value, and the sort is stable. A filter is typed (text, equality, range, only the
absent, only the empty, given rows, several together): one that cannot be built is
the caller’s refusal, never “every row”.
How tables hold together
table_structure turns declarations into structure, with indexes and uris and no
labels: the schema of the tables (which links to which, and by what), the columns
that link a table to itself, the tree of its rows (on the generic forest, which
names each cycle and each orphan, and never hangs on a loop), the roots grouped by
a column, and the key-value shape of a table of two columns.
Where a table or an object stands along a well
md_extent gives the measured-depth extent of what carries one: the interval a
trajectory, a geology object or a log declares, the minimum and maximum of a
frame’s nodes, and the rows of a table that have MD columns (found by their kind,
never by their title; a row without MD can inherit the extent of the row its link
column names, and the extent says through which column). An interval written from
the larger to the smaller is kept as written and flagged. Time and true vertical
depth are on other axes and are counted apart.
coverage puts the extents on one picture, one group per comparable MD
reference. Two measured depths are comparable only when they are counted the
same way (same trajectory, same datum, same unit family): the rest are separate
groups, with the reason. A table row, which says nothing of its reference, goes to
a band of its own.
The alignment is an assumption, and it is the person’s. An Alignment says
count this item’s depths as that one’s are counted. It places the item in that
group and marks it aligned; it is written on the picture and in the export, and
never in the package: the file did not say it.
Extents of WITSML 2.0 objects are not read yet; such an object answers that it states no extent.
House rules
Energistics standards leave questions open: a producer may omit a field, two readings of a sentence are possible, a unit string means different things in two releases. Where the specification is silent, ambiguous or only interpreted, this library makes a decision. A house rule is such a decision, when it changes what a user sees, reads or is refused and no specification dictates it word for word.
Those decisions are not left scattered in code comments. They are in a registry,
the energyml-rules crate, so that:
- a user can ask the tool what it decided, and why;
- code cites a rule by an identifier, and a misspelt identifier does not compile;
- an interface can show “this is a house rule” next to the answer it affects.
What a rule says
Each rule is a record in a TOML file, one file per theme:
| Field | Meaning |
|---|---|
id | house/<theme>/<name> |
status | active (decided and coded), designed (decided, not coded yet - says so on the button that shows it), superseded (kept for history) |
| stance | what the specification does: silent, ambiguous, interpreted, extends (the result is no longer strict standard) or restates (a cited text made executable) |
| level, text, reason | what is decided, at which level, and why |
| references | quoted strings: a specification document, a page or paragraph, or a path in this repository. They are never read at run time |
| anchors | where the rule shows (a view, an object, a command) |
English is the source and is always present. A translation is optional per rule; a rule without one answers in English and says so.
A rule that changes meaning gets a new id that supersedes the old one, which
becomes superseded: history is information. A new rule enters the registry in
the same change as the decision or the code that makes it.
The crate is a leaf with no runtime dependency (WebAssembly included): the TOML is compiled by a build script into statics, and nothing is parsed when a program runs. The loader refuses a malformed rule with a named error (file, rule, field).
Reading them
energyml rules --house # all of them
energyml rules --house --theme colour # one theme
energyml rules --house --id <rule-id>
energyml rules --house --anchor <anchor> # the rules that apply where something shows
energyml rules --house --json
Themes today include attachments, colour, inheritance of properties, series,
statistics, tables, the well, directional surveys, properties and provenance (energyml rules --house lists the ten themes of this release).
Statistics, scales and gaps
Along the measured depth, a gap wider than 3 median steps is never bridged, and a jump of the value wider than 10 median steps of its variation is signalled as a seam. A logarithmic scale is proposed when the values span 3 decades, and automatic bounds are the 1st and 99th percentiles. Read them with energyml rules --house --theme stats; the decisions are in ADR 0027.
Around the well
Where a trajectory is evaluated, the direction at a knot of a linear line is that of the interval that ends there, and a hole under 1 degree of inclination has no high side, so its frame is refused at that node by name. A true vertical depth along a well is counted below the MD datum, and a convention the file does not state (the sense of angles, the axes of a per-node matrix) is declared by the caller, never guessed. Read them with energyml rules --house --theme well; the decisions are in ADR 0031, ADR 0032, ADR 0033 and ADR 0041.
Tables and links
A column links to another table by one qualifier facet that holds the complete uri of the target (convention B, ADR 0029); the target table has one key column, by which a value names a row, or none, and then a value names a row by its rank from 0. A link that leads nowhere is an error, a key held by two rows, a declaration made in a description or a title, and a uri that names no object of the package are warnings, and a link that cannot be checked is said to be uncheckable, never passed. Read them with energyml rules --house --theme table; the decisions are in ADR 0030 and the way to use them in Tables, keys and links.
Directional surveys
A survey that states no north is placed only with a north the caller declares, which is carried in the result and never written into the package, and a declaration never overrules a north the file states. A survey in true or magnetic north is turned to grid north by the convergence and the declination the file states, and a station left out as rejected is left out only when the caller declares so. A RESQML 2.0.1 deviation survey is read from the north of its projected CRS. Read them with energyml rules --house --theme survey; the decisions are in ADR 0034.
Properties
Two properties of one quantity on one support, told apart only by a vertical and a lateral direction facet, are expected to keep the order a layered medium gives them: for a resistivity, the vertical value is not below the lateral one. When more than 5 % of their pairs have a vertical value under 0.95 times the lateral one, the check report carries a warning, never an error: the specification names the directions and says nothing of the order of their values, so the expectation is ours and a package that breaks it is kept. A direction written as an extension value (nwdf:vertical) is not guessed: the check is then unverifiable, unless the caller declares which property is vertical. Read it with energyml rules --house --theme property; the decision is in ADR 0036.
Provenance
An activity is held to the template its ActivityDescriptor names: each parameter has an equivalent of the same title, of an allowed kind and in the allowed numbers, and a departure is a warning in the check report, never a refusal, because the activity is read and drawn all the same. The role of a parameter (input, output, sub-activity) comes from its template or from a named rule, and is undeclared otherwise: it is never guessed from a title. What this library writes, it declares in its own templates, the files a step read included. Read them with energyml rules --house --theme provenance; the decisions are in ADR 0035.
The full list
The complete, current list is generated from the registry: run
energyml rules --house (add --json for a machine-readable form). It is
deliberately not copied by hand here, because a hand-copied list is out of date
the day a rule changes.
Why this matters
A data format that says nothing about a case still gets a producer’s answer, and a tool that quietly picks another one yields results nobody can explain. Registering the choice, with its reason and its status, lets you disagree with it explicitly. See also ADR 0005.
Gallery
Every image in this repository is reproducible: it is drawn from the
reference model energyml-lutetia,
at the commit pinned in tests/lutetia.lock, by the command
written beside it. The SVG files are what the command writes, with nothing edited
by hand; the same package gives the same bytes.
The commands below run from the folder that holds package/Lutetia.epc (the .h5
beside it is found by name). The three sections use one track, so the three views
show the same ground.
| Image | What it shows | Produced by |
|---|---|---|
images/section-long.svg | long profile along a track: wells, horizons, faults, strata fills | energyml profile package/Lutetia.epc --track "650700,6861450;651250,6861750;651850,6861350" --left 200 --right 150 --static --size 1400x700 --draw long --fill down --labels codes --svg section-long.svg |
images/section-plan.svg | map view of the same section | the same command with --draw plan and --svg section-plan.svg, without --fill and --labels |
images/section-cross.svg | a cross section, the second of three | the same command with --cross-count 3 --cross-length 300 --cross-width 100 --draw cross:2 --fill down and --svg section-cross.svg, without --labels |
images/well-sheet.svg | well sheet of the wellbore whose columns show the most different colours (five, all of the Cenozoic stages): depth, stratigraphy, formation name | the gallery example below, energyml_sheet::read::board_of_frame then layout::compute and svg::write |
images/isocontours.svg | iso-lines of the depth surface “H6 - top of the Chalk, in depth”, every 1 m and heavier every 5 m | the gallery example, energyml_geostat::isocontour |
images/triangulation.svg | constrained Delaunay triangulation of the 14 borehole collars, with the traces of faults F1 and F2 as constraints | the gallery example, energyml_geostat::triangulate |
images/kriging.svg | ordinary kriging of the collars’ elevation on a 20 m grid, the estimate and its variance side by side | the gallery example, energyml_geostat::kriging and variogram |
The four last images have no command-line subcommand, so one program draws them with the libraries themselves, from the same pinned package:
cargo run -p energyml --example gallery --features all,hdf5 -- package/Lutetia.epc --out docs/images
cargo run -p energyml --example gallery --features all,hdf5 -- package/Lutetia.epc --out docs/images --check
The second line writes nothing and fails when a file under docs/images/ differs from
what the code draws now. It is not run by CI: run it after changing a drawing
library, with Lutetia at the commit of tests/lutetia.lock. The drawing is a pure
function of the package: no clock, no random number, every number written with a fixed
width. Two runs give the same bytes, and a test in energyml-geostat pins the one
place where that once failed (the order in which isocontour::chain returns lines).
The long section and the cross section carry notes under their plot, as every drawing does: how many objects were refused (a surface in time with no time factor, for example) and how many wells fall outside the band. Nothing is hidden to make the picture tidier.
The well sheet shows few colours because the package gives few: only the stratigraphy column has any (the stage colours of the BRGM chart, all of them close oranges for the Cenozoic), and no well of Lutetia shows more than five. The formation and lithology columns carry text only, since the file states no colour for them and the library does not invent one.
The well sheet is drawn in the readable proportion mode: the depth axis is distorted so
that every interval is tall enough to read, which the sheet says in its own header. At
the very bottom the last depth label overlaps the one above it; that is the layout of
energyml-sheet as it stands, left visible rather than hidden.
Screenshots of interactive applications are not part of this repository.
Testing
What the public test suite stands on
No public test depends on files nobody versions or reviews. The suite is built from four kinds of data:
| Source | Covers | How |
|---|---|---|
| A. The reference model Lutetia | the library’s functions on RESQML 2.2, EML 2.3, WITSML 2.1 and PRODML 2.x: geometry, properties, colours, tables, section view, export, Python, CLI | the model is a versioned repository, energyml-lutetia, read at a pinned commit (tests/lutetia.lock); in CI scripts/fetch-lutetia.py fetches it into target/lutetia, locally it is a neighbouring folder or ENERGYML_LUTETIA_DIR; with neither, its tests skip (a CI job runs the whole suite that way) |
| B. Pinned reference packages from other tools | interoperability and RESQML 2.0.1: packages written by other software keep the cross-checks honest | fetched in CI only from their original repositories, each listed in tests/fixtures.lock with repository, commit, path and SHA-256. A file whose hash differs fails CI; it is never accepted silently. No test goes on the network |
| C. Data fabricated inside the tests | edge cases and refusals: absent values, mistyped values, byte order, … | built by the test itself |
| D. (optional) Packages written during CI by other libraries | widening B without storing files | only if A, B and C leave a measured gap |
The packages in B are small example packages: those written by the example
programs of FESAPI (RESQML 2.0.1 and 2.2, in the Geosiris energyml-python
repository) and those of resqpy (RESQML 2.0.1, MIT licence). The list, the commits
and the licences are in tests/fixtures.lock.
python scripts/fetch-fixtures.py # the same as CI does, on demand, into target/fixtures/
cargo test --workspace --test tests # the end-to-end binary
cargo test --workspace <name-filter> # one test by name, across the workspace
A test whose data is missing fails in CI instead of skipping. Locally it may skip, and the summary says how many suites did.
Rules the tests follow
- The shape of an assertion comes from the specification, the values from the file. A real producer may write non-conforming data, and pinning a measured value turns that defect into a rule the suite then defends. Where a test states a rule, a comment names the document, page and paragraph; the document itself is never opened at run time.
- A test must be able to fail. When adding one, ask what you would have to break for it to turn red; if the answer is “nothing”, break something on purpose to find out. Empty sets pass any assertion.
- Triangulate. Never ask a component to check itself: each cross-check
confronts two independent sources of the same fact, at least one not written
here - the bytes of the HDF5 file, the part names of the zip, the counts the XML
declares. Cross-checks live in
crossaudit.rs. - One crate, one test binary. Test files are modules of a single
tests/main.rsper crate; a new test file is onemodline, or it does not run. A file that touches process-wide state keeps its own binary. - Do not run
cargo test -p <crate>: feature resolution for a single crate differs from the workspace and rebuilds everything. Filter inside the workspace.
What is not covered here
The suite above is honest about its limits: Lutetia is written by this project’s own operations, so on its own it checks functions, not interoperability - which is what B is for. A private campaign on real-world packages (round trips, cross-audits, the command line) also runs on the maintainers’ side; its files are not redistributable, and its results are not a public guarantee.
Releasing
How the crates and the Python wheels of this repository are published. Nothing is published from a workstation once the first release is out: publication runs from the CI of this repository, on its default branch, with credentials that only protected workflows can read.
Making a release
Two branches: dev, where changes are integrated, and main, which holds what
was released. Nothing is published by a push: a published GitHub release
publishes everything at once, and it is created by a maintainer.
A release (vX.Y.Z), from main:
- Merge
devintomain(a pull requestdev→main). - Merge the release pull request.
release-plzkeeps one open onmain, opened by the release bot app: the version bumps, the requirements between crates and everyCHANGELOG.md, written from the conventional commits. Merging it publishes nothing. Whenmainstill carries a pre-release version (0.2.0-alpha.4),release-plzdoes not move to the final version on its own: the version is set by hand in a pull request tomaininstead, with theCHANGELOG.mdentries (one commitchore: X.Y.Z). It may meanwhile propose the next pre-release (chore: release vX.Y.Z-alpha.N+1): close that pull request without merging it when the final version is coming. - Create a GitHub release (Releases > Draft a new release): a new tag
vX.Y.Zonmain, the workspace version ofCargo.toml; titlevX.Y.Z; notes (“Generate release notes” gives a start). Publish it.
A pre-release (vX.Y.Z-alpha.N, -beta.N, -rc.N), from dev:
- Set the version by hand on
dev: one commitchore: X.Y.Z-alpha.Nthat changes the workspace version and the requirements between crates. - Create a GitHub release on
devwith the tagvX.Y.Z-alpha.N, and tick “Set as a pre-release”. A release fromdevthat is not marked as a pre-release is refused.
Cargo never picks a pre-release unless it is asked for by name
(energyml = "0.2.0-alpha.1"), and pip only with --pre.
Publishing a release or a pre-release starts, each from the release:
| What | Where | Workflow |
|---|---|---|
| the library crates (see the order below) | crates.io | release-plz.yml, which also tags each crate <crate>-vX.Y.Z |
energyml-rust wheels and sdist (imported as energyml.rust) | PyPI | python.yml |
the energyml archives, one per target, with SHA256SUMS | attached to this release | release-cli.yml |
the energyml image: X.Y.Z, X.Y and latest for a release; X.Y.Z-pre and dev for a pre-release | ghcr.io/geosiris-technologies/energyml | docker.yml, once IMAGE_PUBLISH is true |
| not published | energyml-tests, energyml-roundtrip, energyml-python as a crate (it ships as wheels) |
Each workflow first checks that the tag names the version of the manifests, and stops before uploading anything otherwise.
Between releases, every push to main or dev that touches the crates
publishes a development build of the wheels to TestPyPI (X.Y.(Z+1).devN), and
every push to main the image edge. Neither is a release.
Dependency updates (Dependabot) target dev, like any other change.
Publication order
A crate can be published only when everything it depends on is already on
crates.io. The order below comes from the manifests: normal and build
dependencies decide it. Dev-dependencies on a crate of this repository carry no
version: cargo publish drops them from the published manifest, so they do not
constrain the order (a dev-dependency with a version is resolved against the
registry, and on 14/10/2026 that stopped a release at energyml-storage, whose
tests use the generated crates release-plz had not published yet). Crates of
one level do not depend on each other and can go in any order.
| Level | Crates |
|---|---|
| 1 | energyml-derive, energyml-build-derive, energyml-log, energyml-geostat |
| 2 | energyml-core |
| 3 | energyml-eml-2-0, energyml-eml-2-1, energyml-eml-2-2, energyml-eml-2-3 |
| 4 | energyml-resqml-2-0-1, energyml-resqml-2-2, energyml-resqml-2-2-dev3, energyml-witsml-2-0, energyml-witsml-2-1, energyml-prodml-2-0, energyml-prodml-2-2, energyml-prodml-2-3 |
| 5 | energyml-schemas |
| 6 | energyml-storage |
| 7 | energyml-domain |
| 8 | energyml-geometry, energyml-proj, energyml-sheet, energyml-rules |
| 9 | energyml-analysis, energyml-cache, energyml-export |
| 10 | energyml-section |
| 11 | energyml-build |
| 12 | energyml-providers |
| 13 | energyml-cli, energyml |
Why energyml-schemas is so early: energyml-storage and the others use it in
their tests. Looking at normal dependencies alone, energyml-schemas would come
after energyml-storage, and energyml-storage could not be packaged.
Everything that depends on the generated crates must wait for all of them
(energyml-schemas lists every release). The ETP stack lives in the
retp repository and depends
on energyml-core, energyml-storage and energyml-schemas: publish those
three first, then follow retp’s own order.
release-plz computes this order itself once the crates are on the registry.
The order matters for the first publication of each crate, which is done
once, in order, by a maintainer with a short-lived API token; trusted publishing
is then configured per crate on crates.io (owner geosiris-technologies,
repository energyml-rust, workflow release-plz.yml, environment crates-io)
and every later release goes through the workflow. Until its first publication,
a new crate is marked publish = false in release-plz.toml, so that the
workflow leaves it alone; the mark is removed once trusted publishing is
configured. The crates-io environment only accepts deployments from main
and dev.
The publication switch
The repository variable RELEASE_PLZ_PUBLISH is the switch for crates.io:
release publishes on a published GitHub release; unset (or any other value),
nothing is published, which is how a new repository starts. A release’s tag must
name a version of the manifests, vX.Y.Z for the workspace version or
<crate>-vX.Y.Z, and its commit must be on main (or on dev for a
pre-release). PYPI_PUBLISH, TESTPYPI_PUBLISH and IMAGE_PUBLISH are the
switches of the wheels and of the image.
The dev-dependency cycle between energyml-rules and energyml-domain
energyml-rules uses energyml-domain in its tests (it crosses the rules of
the registry with energyml_domain::integrity::checks()). Today
energyml-domain does not depend on energyml-rules, so there is no cycle
at all, and the order above puts energyml-rules after energyml-domain.
The risk is the day energyml-domain starts to depend on energyml-rules:
crates.io refuses a dependency cycle only between normal dependencies, but a
dev-dependency with a version is kept in the published manifest and must
be found on the registry when the crate is packaged, so neither crate could be
published first. The dev-dependency of energyml-rules therefore has no
version (energyml-domain = { path = "../energyml-domain" }): cargo publish removes such a dev-dependency from the published manifest, and the two
crates are independent at publication whatever their order. Do not add a
version to it. The same remedy applies to any dev-dependency that would close
a loop.
Python wheels
The metadata of the wheels is in
crates/energyml-python/pyproject.toml
(distribution energyml-rust; the version is read from the crate). A GitHub
release vX.Y.Z builds the wheels for eight targets and the sdist, with HDF5
compiled in, and uploads them by PyPI trusted publishing (environment pypi).
The tag python-vX.Y.Z does the same for the Python package alone. Either tag
must equal the version of the crate; the version job fails otherwise.
Development builds go to TestPyPI (environment testpypi) from every push to
main or dev.
The documentation build
docs.rs builds each crate on Linux with the features of its
[package.metadata.docs.rs], with --cfg docsrs and a nightly compiler. The
docsrs job of the CI does the same, so a crate that would not build there is
found before the release, not after.
Architecture decision records
Short records of decisions that shape this library and the reasons behind them. Each one has three parts: Context (what forced a choice), Decision (what was chosen) and Consequences (what it costs and what it buys). They are written to be read in a minute; the code and the other pages are the detail.
A record is not edited to change its meaning. A decision that is reversed gets a new record that supersedes the old one, which keeps its text and a status line.
| # | Decision |
|---|---|
| 0001 | Generated code is committed, produced by an internal tool |
| 0002 | The storage trait is drawn after ETP |
| 0003 | Identity: full URI internally, one door per access |
| 0004 | Reading policy: ReadMode, version resolution, tolerance |
| 0005 | Refuse rather than draw; FeatureNotEnabled |
| 0006 | Registries rather than match |
| 0007 | core has no native dependency (WebAssembly) |
| 0008 | Inheritance and xsd:list |
| 0009 | Properties, tables and colours: one colour rule |
| 0010 | The package index and caches |
| 0011 | Tests follow the specification, not a corpus; triangulation; a test must be able to fail |
| 0012 | No face (Python, CLI) carries logic |
| 0013 | WITSML channels in HDF5 |
| 0014 | Horizons and units, attachments, business constraints |
| 0015 | The section view: three stages, model.rs as contract, one drawer |
| 0016 | One crate, one test binary |
| 0017 | An operation is a pure function; only apply writes |
| 0018 | A synthetic reference model rather than an external corpus |
| 0019 | Synthetic data is flagged and sourced |
| 0020 | A grid is never reprojected |
| 0021 | Our own Tool trait; the MCP SDK is an adapter behind a feature |
| 0022 | The kind of an edge comes from the specification, labelled; the schema facts are generated |
| 0023 | The data objects held by value are listed apart, and write their own references |
| 0024 | Facets are read in every version, and say whether the schema lists them |
| 0025 | A logarithmic mapping places the log of the value |
| 0026 | A head portion says whether the file describes it, and is in the line’s frame |
| 0027 | Gaps, seams, scales and quantiles: the numbers the views share |
| 0028 | One drawing for every view: draw knows no format, one display list, one rule for graduations, a PNG is always the SVG rasterised |
| 0029 | A link to a table is one qualifier facet holding the uri of the table, read once; the axes of a property come from the table it names |
| 0030 | Table links: a column links by key or by rank, the target decides, one key column or none; description and title are read with a warning; an empty string is a value; the rules house/table/* |
| 0031 | The direction of a line at a knot is that of the interval that ends there |
| 0032 | The frame of a hole is the physical one, whatever hand the placed frame has |
| 0033 | A true vertical depth along a well is counted below the MD datum |
| 0034 | The north of a survey and its stations: a north the file does not state is declared by the caller, the corrections the file states are applied, a rejected station is left out on declaration, a RESQML 2.0.1 deviation survey is read |
| 0035 | Provenance, written completely and read from four releases: the role from the template then a named rule, input and inputFile declared, integers as integers, verbatim sources and the NWDF tables |
| 0036 | Two directional properties of one quantity keep the order a layered medium expects: a warning above 5 % of the pairs under 0.95, unverifiable on an extension direction unless the caller declares it |
| 0037 | A result of a view is an artefact first and is stored on demand: one operation per sort of view, the drawing is the input and the recipe is read from it, an iso-surface promoted as it was computed |
| 0038 | The geosteering section: the curtain of a cube along a well, one frame and one door for placing, folds marked and gaps never bridged, declared conventions, angles found by value |
| 0039 | Several values per element are written with their shape: ValueCountPerIndexableElement and LabelPerComponent, the dataset [elements, d1, ...], a scalar written as it always was |
| 0040 | The members of a wellbore are walked in the domain, from the inventory and the references alone |
| 0041 | A convention the file does not state is declared by the caller; a vertical without a CRS is signalled |
| 0042 | What a property says of itself: dimensions reduced together, scalars for a wrong declared count, facets of 2.0.1 outside the 2.3 list, the synthetic mark, observations beside those of the values |
| 0043 | The views of a package are drawn through one door, as a display list written to SVG |
| 0044 | The vector arithmetic and the small plan geometry live in energyml_geostat::vector; energyml_geometry::vector re-exports it |
| 0045 | Text and numbers written by hand (XML, JSON, base 64, calendar, numbers) have one copy each |
| 0046 | Reading a field of a reflected object goes through one door, energyml_core::reflect::fields, with its policy for absent, blank and mistyped fields |
| 0047 | A view may propose a value the file does not state, for a first drawing; a proposal is always said as one |
| 0048 | Every edge has a kind: EML and PRODML by the WITSML rule, primary by default otherwise |
Decisions about the ETP protocol implementation are recorded in the
retp repository.
0001 - Generated code is committed, produced by an internal tool
Status: accepted
Context
The Energistics standards are published as XSD files. A Rust type per element (about twelve crates, one per package and version) cannot be written by hand and kept in step across releases. The schemas themselves are distributed by their publisher under a licence that allows derived works but not redistribution of the schemas unmodified; the generator and its inputs are therefore not part of this repository.
Decision
- Rust types are generated from the XSD files by an internal tool and the
output is committed under
generated/, one crate per package and version. - Nobody edits generated files by hand. A robot proposes regenerations as pull requests; CI of the producing side checks that committed output equals a fresh generation.
generated/schemas.lockrecords the SHA-256 of every input schema. It carries no schema content.- Each crate carries a notice crediting the Energistics products used.
Consequences
- Building this repository needs no XSD, no generator and no network.
- A reviewer sees exactly how a schema release changes the Rust types.
- Contributors cannot fix a generated type here; the fix goes to the generator and arrives as a regenerated crate.
- The generated code is a derived work and carries the publisher’s notice. See Schemas.
0002 - The storage trait is drawn after ETP
Status: accepted
Context
Data lives in EPC files today and will live behind the Energistics Transfer Protocol (ETP) in remote dataspaces. If the storage abstraction were drawn around a zip file, adding a network transport would force a rewrite of everything above it.
Decision
EnergymlStorage is designed from the ETP v1.2 requests: addressing by URI,
plural operations, partial responses. An EPC fits inside that shape - a file is a
dataspace.
- The trait is async;
EpcStorealso has a full synchronous API, because a WebSocket cannot block in WebAssembly and a program reading local files must not pay for async. A test compares both faces so they cannot drift. - The trait only does I/O. Graph building,
.relsregeneration and array location are free functions over objects already in memory, so no logic has a sync and an async copy. - Three levels of materialisation: identity from the index, citation from a parser that stops, then the whole object. A listing of a multi-gigabyte package inflates no part in full.
- Tolerant and vocal: departures from the specification are accepted and recorded as diagnostics, never swallowed.
- Local-only capabilities (raw attachments and
.rels) are a separate trait, because they make no sense in a dataspace.
Consequences
- An application written against the trait changes transport by changing the line
that builds the store. The ETP client (
retp) implements the same trait. - Some conveniences of a local zip are deliberately not on the trait.
0003 - Identity: full URI internally, one door per access
Status: accepted
Context
An object can be named by a bare uuid, a short URI or a full URI that carries its version. A uuid alone is ambiguous when two versions of an object coexist, and a library with several parsers for the same identifier ends up with several answers to “which object is this?”.
Decision
- Inside the library, only full URIs are used as keys and in comparisons, and
the URI carries the version when the object declares one. A URI is built from an
object or a reference in exactly one way (
ObjectId::of_reflect, thenetp_uri). - A bare uuid or a short URI is accepted only at the edge (adapters such as
lookup_uuidandresolve) and turned into a full identity at once. - One door per access. A URI is parsed by
EtpUri::parse; an object is found byEpcStore::lookup; a colour is decided byenergyml_cache::colours. A second implementation, even private, is a defect: two faces would end up answering two different things. - A view that shows an object’s type and title also shows its version when it is neither empty nor absent.
Consequences
- Several versions of one object can live in a package without being confused.
- Adding a feature never adds a second identity parser; known exceptions are tracked and removed.
- Convenience for users remains at the entry points.
Addendum: what the door covers
ObjectId::from_etp_uri(andObjectId::parse) keep no reading of a URI of their own: they callEtpUri::parseand build the identity from its answer. So what the door refuses they refuse - a uuid that is not a uuid, a type with a character a type cannot carry - and the uuid comes back in lowercase. A URI that names a dataspace and no object, or that goes on below the object, is refused by name, because an identity cannot hold it.- The qualified type and the content type each have one grammar
(
identity::parse_qualified_type,identity::split_content_type); the ownedQualifiedTypeandContentTypeofidentifiersare wrappers around them. - Two addresses are compared by
identity::same_address: same uuid, versions that do not contradict. An address that names no object (an empty text, a dataspace) is the same as nothing, not even as itself. - A reference whose
ContentTypesaysversion=2.0.1and the object it points at have the same uri:Registry::canonical_keyandRegistry::canonical_qualified_typenormalize a version to the schema version the registry declares (2.0), and refuse a spelling the registry does not know, by name.EpcStoreresolves through them. - A quote inside a dataspace name is written twice in a URI (ETP 1.2 section
25.3.10), as for the object version:
EtpUri::dataspace_uri,objectandDisplaydouble it,EtpUri::parseundoes it. A dataspace namedit'sis a usable name.
0004 - Reading policy: ReadMode, version resolution, tolerance
Status: accepted
Context
Real packages break the specification: a required element left out, a value outside a closed enumeration, a part declared in one release and written in another. Refusing everything makes such packages unreadable; accepting everything silently makes wrong data look right.
Decision
- Absence and nonsense are different news. A
ReadModestates what to do about each. Strict is the default; leniency is opt-in and reports one diagnostic per invention (with the document’s own string when a value was repaired). - Leniency is about cardinality and unreadable values only. Malformed XML, bad
encoding and an
xsi:typeon a purely abstract choice stay refused in both modes: there is nothing to fall back to. - The version of a part is resolved from what it declares, over ordered candidates, never by guessing from content. A development drop is registered so files written against it still open, but is never offered for creation.
- The levels form a ladder: strict, lenient, and finally
repairfor parts no parser will touch. Whatrepairlists is what the selected mode cannot read. - Anything intended to be written back should be read strictly: a tolerant read holds values nobody wrote, and saving puts them on disk.
Consequences
- Users choose, per command, between reading everything possible and being told the truth about what was invented.
- Every departure is visible as a diagnostic.
0005 - Refuse rather than draw; FeatureNotEnabled
Status: accepted
Context
An empty result looks exactly like a program that works and has nothing to show; a wrong result looks exactly like a right one. A point placed in feet read as metres, a surface drawn from loosely connected lines, a missing array backend reported as an empty array: each produces a plausible output that nobody questions.
Decision
- Where the specification is ambiguous or the data insufficient, the answer is a named refusal, not a guess. When a guess is finally settled, it is settled by evidence written next to the code.
- Absence and error are distinct: a missing capability (an absent array backend, a
build without HDF5) returns
FeatureNotEnabled, never an empty array. A reader that cannot read something returns an error naming the type, never nothing. - A batch carries on past one refusal and reports each, with the object and the reason.
- Choices the library makes where the specification is silent are recorded in the house rules registry, with their reasons.
Consequences
- Users sometimes get an error where a lenient tool would give a picture. The error names the cause, and a rule or an option exists to go further.
- No refusal is silent: counts of set-aside samples, unlabelled marks or skipped objects are part of the answer.
0006 - Registries rather than match
Status: accepted
Context
Several parts of the library have a list that will grow: array backends,
representation readers, exporters, operations, proportion modes of a well sheet,
types registered per schema version. A match over a closed list forces every new
entry to change the library.
Decision
Where the list will grow, define a trait and a registry. Built-ins are pushed by default; entries added by the caller are tried first, so that a built-in is replaceable and not only extensible.
Consequences
- A new export format, a new array backend, a new operation or a new schema release is an addition, not an edit of existing code.
- Interfaces build themselves from the registry (the list of export formats, of operations, of provider filters), so they show a new entry without change.
- Ordering matters and is part of the contract: the first match wins.
0007 - core has no native dependency
Status: accepted
Context
The library must run in a browser (WebAssembly) as well as natively. Native libraries such as HDF5 or PROJ cannot be linked in a WebAssembly build.
Decision
energyml-coreand every crate below the storage layer’s array backends are free of native dependencies. WebAssembly is a hard requirement checked in CI.- HDF5 and similar libraries live in the storage layer, behind optional features
(
hdf5), behind theArrayBackendtrait. A WebAssembly host supplies its own backend. - Coordinate reprojection is pure Rust (
proj4rs), not a binding to PROJ.energyml-projrefuses a definition that carries no datum rather than applying one silently. - Computation crates (interpolation, section view, well sheet) compute over plain
slices and return
Strings or numbers; they read no files.
Consequences
- One code base serves the command line, Python and the browser.
- Anything needing a native library is optional and must say so when missing (see ADR 0005).
- A pure-Rust reprojection must be validated against PROJ; it agrees to the millimetre on the systems tested.
0008 - Inheritance and xsd:list
Status: accepted
Context
The schemas use type extension and restriction heavily, and xsd:list types that
store a whole array as whitespace-separated text in one element. Rust has no
inheritance, and writing a list as one element per value produces a document no
other tool reads.
Decision
- Inheritance is flattened at generation time. Chains of extensions are copied
base-first into each concrete struct, across crates, so a RESQML object carries
the fields of its Energistics-common base while the field types still point at
that crate. A
baserelation is emitted for each abstract ancestor in the same crate, so a question like “is this anAbstractRepresentation?” is answered by reflection. The chain stops at a crate boundary by design. - Restriction overrides inherited fields rather than duplicating them.
xsd:listis one element: the whole array lives in the text of a single element. The generator marks such fields as lists, which stops the writer from emitting one element per number.- What an object inherits that is not structural - a colour, a graphic information - follows an explicit, ordered lookup (see ADR 0009).
Consequences
- Generated types are plain structs; reflection supplies the hierarchy.
- A test guards that every abstract ancestor is reachable, because a missing one would answer “false” silently for every surface.
0009 - Properties, tables and colours: one colour rule
Status: accepted
Context
A property’s values are numbers in an array; what they mean depends on a kind, a unit, a lookup table for discrete values and, for display, a colour. The specification defines none of the priority between several colours that may apply to one object (its own, its property kind’s, its interpretation’s, a collection that references it, a parent’s).
Decision
- Property kinds, lookup tables (discrete and categorical properties) and colour maps are resolved once, in the domain layer, for every schema version.
- A property’s value count is checked against the geometry it hangs on; a property lined up against the wrong elements is reported, not drawn.
- One colour rule for every face. The colour of an object is decided by one
cascade, in one place (
energyml_cache::colours), which returns the level that answered and the object it came from. Order of search: directly on the object; then collections that reference it (recursively); then its direct parents; then parents through collections; then up the parents; finally the model and its collections. Among several paths, the shortest wins; on a tie, the first. - The decision is visible: a decision path can be shown rather than only a result.
- No default colour ramp is invented when the package states none.
Consequences
- The command line, Python and any viewer give the same colour for the same object.
- The rule is a house rule, listed in the registry, and can be disagreed with explicitly.
0010 - The package index and caches
Status: accepted
Context
Several questions cost a full read of a package: “who references this object?”, “what colour does it have?”, “which attachments does it have?”. Packages can hold hundreds of thousands of objects. Computing everything up front makes opening slow; computing nothing makes every question slow.
Decision
- A package keeps what it knows of itself, layer by layer (index of objects
and links, colour walks, attachment listings), in
energyml-cache. - Links and summaries are kept; objects are re-read on demand. The index is invalidated by an edit through an overlay of what was changed.
- There is one implementation: switching caching off means a slot that is thrown away, not another code path.
- The colour rule is the same one for every face (ADR 0009).
- The command-line cache commands are a fixed list.
- Costs are visible: a call that reads everything (like asking who references an object) says so in its name, and work is counted in tests, never timed.
Consequences
- Opening stays cheap; expensive questions are asked deliberately.
- An edit never leaves a stale answer behind, because invalidation is part of the write.
0011 - Tests follow the specification, not a corpus
Status: accepted
Context
A real package can contain non-conforming data. Pinning a value measured from one turns its defect into a rule the suite then defends. Separately, a component that checks itself agrees with itself, and a test that cannot fail proves nothing: checks that passed on empty sets have been found in this project.
Decision
- The shape of an assertion comes from the specification; the values come from the data. Only the first has authority. A test that states a rule cites the document, page and paragraph in a comment.
- Reference documents are cited, never read at run time. Specifications and schemas are not opened by tests; a test restates the rule instead.
- Triangulate. Never ask a component to verify itself. Each cross-check confronts two independent sources of the same fact, at least one of which is not code written here: the bytes of the HDF5 file, the part names of the zip, the counts the XML declares.
- A test must be able to fail. When adding one, ask what would have to be broken for it to go red; if the answer is “nothing”, break something on purpose to find out.
- An unexplained discrepancy is pursued, not pinned.
- Real files remain the only real test, so they are exercised in a separate private campaign; the public suite does not depend on them (ADR 0018).
Consequences
- Tests stay valid when a producer is fixed.
- Writing a cross-check costs more than writing an assertion; that is the point.
0012 - No face carries logic
Status: accepted
Context
The library has several faces: a command line, a Python bridge, a WebAssembly module and applications. Logic written in a face exists only for that face. When the same decision was written in three places, the copies drifted - colours resolved by one and not the others, a type filter present in three versions.
Decision
- The command line only parses arguments and prints; the Python bridge only converts values. Every check, decision and operation lives in the library.
- The order of work is fixed: write it in the library first, then bridge it. The bridge carries no feature the Rust library lacks.
- A face that knows something the library does not is a defect in the split.
- The only place a schema version is chosen is the face that starts the program; the storage layer receives a registry and never builds one.
- Some wrappers are intentionally thin and tested natively: the Python bridge’s
sessionmodule is native Rust and tested as such; only the conversion layer is Python-specific. The same holds for the WebAssembly module.
Consequences
- The command line, Python and applications give the same answer to the same question, and a check written once protects all of them.
- Replacing the storage (EPC by ETP) changes nothing in the faces.
0013 - WITSML channels in HDF5
Status: accepted
Context
WITSML 2.1 describes the bulk data of a log in a ChannelData structure: either
an inline data blob in the XML or a URI of an external file (the reference
format named in release notes is not HDF5). Large logs do not fit comfortably in
XML, while RESQML and the rest of an EPC already keep large arrays in HDF5 beside
the package.
Decision
Supporting HDF5 for channel data is an extension of the standard, and is treated as one:
- the external file is named in
FileUrias a relative URI, as the standard intends, and is the same string as the file of every other array in the package; - the path of the dataset inside that file goes in
Data; - the form is recognisable without ambiguity, and a reader that does not know the extension refuses rather than misreads;
- the writer produces it only on request; the default stays values in the XML.
Consequences
- Large logs can live with the other arrays in one HDF5 file.
- A package using the extension is not strict WITSML; it is flagged as such by the house rules.
- Interoperability with tools that do not know the extension is by refusal, not silent misreading.
0014 - Horizons and units, attachments, business constraints
Status: accepted
Three small decisions about links the standard leaves implicit.
Horizons and stratigraphic units
Context
RESQML expresses the relation between a horizon and the stratigraphic units it separates in exactly one place: a contact. If nothing writes that link, a viewer has nothing to read for the contact between units, and a fallback of “colour a surface with the colour of its unit” has nothing to follow.
Decision
The structural link between a horizon and its units is written through the contact the standard provides, and readers follow it. The final fallback of the colour cascade (ADR 0009) may use the unit’s colour; which side keeps which colour when two units differ is a library decision, recorded as a house rule.
Attachments
Context
Files (images, documents) can sit inside the zip or in an Attachment object.
Decision
A file is tied to an object by a relationship, never by its content type. The content type says what a part is; the relationship says which object it belongs to. Attachments are a local-package capability (a separate trait from the dataspace trait, ADR 0002) because a dataspace transports objects and arrays, not files. The declared type of a file is never guessed, and a file attached to nothing is written and reported.
Business constraints
Context
The schema cannot type the target of a reference, and the legitimate associations are written only in the technical guides.
Decision
Business rules (what a reference may point at, what two linked objects must say together) are a layer above schema validation, reference integrity and count coherence, and are executable. They are separate checks with named findings, not mixed into the schema check.
Consequences
- Links the standard implies are written and followed, not inferred.
- Each check reports which layer found the problem.
0015 - The section view: three stages, model.rs as contract, one drawer
Status: accepted
Context
A section along a trace draws wells, surfaces and grids from one or several packages. Reading and cutting are expensive; drawing, scaling and labelling are cheap and change on every interaction. A drawing computed in several places diverges.
Decision
- Three stages with three costs.
read(per package; the only stage that touches a store),assemble, then aCataloguethat can be cached; thenlayout(catalogue plus aView) andsvg::write. Changing the view never rebuilds a catalogue; changing the inputs does. model.rsis the contract. Every module is written against its types, which travel as JSON (also embedded in the SVG metadata). A new field is added there, with its reason.- One drawer. Screen, download, command line and package export go through
the same
drawwith the same view: the export is what was shown. - Nothing about scene colours lives in the catalogue; the view supplies them.
- Every refusal is a line naming the object; a non-finite number never leaves a contribution; the SVG escapes all text, carries colours as attributes and keys each object for the interface.
- The drawing can be recomputed from its own metadata, giving identical bytes on the same packages.
Consequences
- Views are cheap to change; large packages are cut once per line.
- The same geometry and colour rules apply everywhere.
- The crate has no native dependency and builds for WebAssembly.
See Section view.
0016 - One crate, one test binary
Status: accepted
Context
Every file in a crate’s tests/ directory is a separate test binary, and each one
links the generated crates. In a workspace with dozens of such files, a full test
run exhausted the disk of a CI runner with linker failures (“no space left on
device”) and made builds slow.
Decision
- Crates that have many test files keep their tests as modules of a single
tests/main.rs(withautotests = false). A new test file is onemodline; without it the file does not run. - A file that touches state of the whole process (for example a counter of opened HDF5 handles) keeps its own binary, declared explicitly.
- Tests are run through the workspace with a filter, never crate by crate: feature resolution differs for a single crate and forces a full rebuild.
- Generated code is never reformatted with a workspace-wide formatter; hand-written crates are formatted by name.
Consequences
- Disk use and link time drop considerably.
- Contributors must remember to register a test module.
0017 - An operation is a pure function; only apply writes
Status: accepted
Context
Creating an object correctly means assembling identity, references, CRS, array paths and activity records. Operations had to be offered by a command line, a page, a workflow engine and a recipe, each wanting to preview what would happen before anything is written.
Decision
An operation is a pure function from a bag of pieces (Pieces) to a Plan.
apply is the only step that touches storage.
plantakes no store, so it cannot read one; what the caller knows of the package arrives resolved.- An operation declares what it needs (slots), takes (pieces), gives (roles) and reports; interfaces, the Galaxy generator and the activity template are all generated from that one declaration.
- Fitness returns a reason, not a boolean;
offerslists unfit operations with their reason. - Identities are derived (version-5 UUIDs) from what the input owns, so equal inputs name equal objects.
- Every run records an
Activityagainst the operation’s canonical template. - A recipe is an ordered list of operations played through the same door into one package; each step refers to earlier ones by name.
Consequences
- Every operation can be previewed (“quote”), tested with values alone and replayed to an identical package.
- Adding an operation is one registration; every face gets it.
- All creation rules live in one crate, not in faces.
See Creating objects.
0018 - A synthetic reference model rather than an external corpus
Status: accepted
Context
A public test suite that depends on files nobody versions or reviews, or whose licence is unknown, cannot be relied on and cannot be reproduced by a contributor. Real files remain the best test of interoperability, but not all of them can be redistributed.
Decision
- The public suite is built from: (A) a synthetic, documented reference model (Lutetia) kept in its own repository and read at a pinned commit, (B) small reference packages written by other tools, fetched from their original repositories in CI only and pinned by commit and SHA-256, (C) data fabricated inside tests, and optionally (D) packages written by other libraries during CI.
- No test goes on the network; a missing data set fails CI instead of skipping.
- A model written by this project’s own operations checks functions, not interoperability; interoperability is the job of (B).
- A private campaign on real packages runs separately, and the public documentation says so.
Consequences
- Contributors can run the whole public suite from a clean clone.
- The reference model is itself a public asset: documented, citable and usable by other implementations as a test set.
- Some behaviours are only exercised privately; the testing page lists the gap.
See Testing.
0019 - Synthetic data is flagged and sourced
Status: accepted
Context
Some operations of energyml-build write objects nobody measured: a horizon laid
at a stated thickness from another one, a fault drawn from sticks, a property
filled by a law of depth, a throw, a velocity law between depth and time. Once
written, such an object looks exactly like a measured one, and a number with no
origin looks exactly like a fact. A reader downstream (a viewer, a check, another
tool) has no way to tell, and a plausible constant that nobody can trace becomes
a value other people rely on.
Decision
Two rules, applied by every operation that invents something.
- An invented object says so. It carries an
ExtensionNameValuenamedsyntheticwith the valuetrue(the field exists on every eml 2.3AbstractObject), whoseDescriptionsays what is invented and how, for example “laid byhorizon-from-lawat a stated thickness from another horizon, not measured”. The object’sCitation/Descriptionalso carries a sentence stating the law and its source; it stays within the 2000 characters that field allows, and a source that would not fit is refused. - No constant without a source. A value that is not measured comes with a
sourcesetting that is required, and a blank one is refused by name. The source is either a document with its page or paragraph, or the statement that the value is plausible and why. A value presented as real cites its document; a plausible one is declared synthetic. The source text is written in the object, but it takes no part in any derived identity: rewording or translating it never renames an object.
The plan reports the mark in its notes (a note with the key synthetic), so the
person sees it before anything is written. Operations that invent only when told
to (a synthetic parameter) write the mark only then. The operations that are
meant to produce synthetic models are grouped under the synthetic category in
listings.
Consequences
- A synthetic object can be found by any reader that lists extension values, and filtered out of anything that must be measured.
- Every invented number can be traced to a sentence a person wrote.
- An operation that invents must ask for a source; one more required setting is the price, paid once per recipe step.
- The mark describes the object, not the package: a package mixing measured and synthetic objects stays readable object by object.
See Creating objects and ADR 0018 for the reference model that uses these operations.
0020 - A grid is never reprojected
Status: accepted
Context
A structured grid (a 2D grid of heights, an IJK grid) is not a bag of points. Its nodes sit on a lattice written in the numbers of a frame: counts, steps, an origin, an order (fastest axis first). Reprojecting its nodes to another frame would give points that no longer lie on a lattice in either frame, and the grid could not be written back. Worse, a lattice read in one frame and filled with values read in another puts every node somewhere else, with no error.
Decision
A grid keeps the frame it was written in. No operation reprojects a grid; when two inputs name two frames and one of them is a grid, the operation refuses by name, with both frames in the message, and says what to do instead:
- interpolation onto a grid whose frame is not the samples’ is refused (“a grid is never reprojected: interpolate onto a grid of the samples’ frame, or bring the samples into the grid’s frame first”);
- the frame written for a result on a grid must be the grid’s own, so a
crsnaming another frame is refused; - the lattice must be the house one (along +x then +y of its frame), or it is refused as turned or reversed.
Things that are not grids (points, polylines, triangulated surfaces) can be moved
between frames with change-frame, which keeps their nodes and topology.
Consequences
- Nothing is resampled without being stated: putting values on another lattice is a different operation, interpolation onto a new grid, and it is named as such.
- A user with samples in one frame and a grid in another must bring one to the other explicitly.
- The rule is enforced by refusals in
interpolate, and stated ingrid-properties,ijk-grid-from-horizonsand the horizon operations; a test checks that theinterpolatemessage says “never reprojected”.
The operations whose source states the rule, found by searching the public tree for
“never reprojected”: interpolate, grid-properties, ijk-grid-from-horizons,
horizon-from-law, fault-from-sticks and the fault-throw operation.
0021 - Our own Tool trait; the MCP SDK is an adapter behind a feature
Status: accepted
Context
The MCP server offers read-only tools over the storage traits. Rust has an
official SDK for it, rmcp, which defines tools with macros on a handler
(#[tool_router], typed parameters) and brings tokio.
Three facts weigh against building the server on those macros:
- The SDK moves.
rmcpwent through three major versions in five months (1.0 in March 2026, 2.0 in June, 3.0 in July) and keeps tightening behaviour between minors. The MCP specification itself changed shape (a session with aninitializehandshake, then a stateless revision), and clients still speak both. - The tool list will grow, and not only by us (ADR-0006). An application
embedding the library adds the tools that drive its own views; a server adds the
tools its users’ rights allow; a later version adds writing, validation,
export. A set of tools fixed at compile time by macros on one
implcannot be extended by a caller, and cannot put the caller’s tool before a builtin one. - tokio must not reach the libraries. Neither workspace has tokio today, the
ETP client is blocking, and
energyml-coreis wasm-clean (ADR-0007). A crate that only wants the registry (a desktop application, a test, a different transport) must not pay for an async runtime.
Decision
ToolandRegistryare ours, and SDK-independent. A tool is a value with a name, a title, a description, a JSON Schema for its arguments, hints, and anasync call(&ToolContext, args)that answers aToolOutputor aRefusal. The registry lists tools in registration order and tries what the caller registered first (Registry::with_tool). What a tool needs beyond the store travels inToolContext::extensions.StoreProvideris the one point where the store of a request comes from (a fixed store for stdio, a per-request store behind HTTP).rmcpis an adapter inenergyml-mcp’sservermodule, behind a Cargo featureserverthat is off by default. It implementsServerHandlerby hand (list_toolswalks the registry,call_toolcalls it) and nothing else. Normcptype appears in a public signature ofenergyml-mcp; errors carry the SDK’s messages as text.- tokio is confined to the feature.
serve_stdiobuilds its own single-threaded runtime and blocks the calling thread, so callers see a plain function. Each tool call runs on a blocking thread aroundfutures_executor::block_on, because the ETP store blocks inside itsasyncmethods, and is raced againstLimits::call_timeout; a timeout answers the refusaltimeout. - Two kinds of failure, as the specification has them. An unknown tool is a
JSON-RPC protocol error. Everything else a tool declines is a tool result with
isError: true, carrying{"refusal": ...}(ADR-0005): the model can read it and correct itself. Never an empty list. - The tool schemas are written by hand as
json!values, not derived from Rust types, so that a snapshot test oftools/listguards the contract clients see and a refactor cannot move it silently.
Consequences
- Replacing the SDK, or writing the JSON-RPC loop by hand for stdio, rewrites one
file and touches no tool and no test of a tool. The adapter is a few hundred
lines; a new
rmcpmajor costs an afternoon in that file. - The adapter is tested with the SDK’s own client over an in-memory duplex stream, so the server and the client are not both written by us.
- The server can be handed a registry with the caller’s tools first, at no cost to
the builtin ones; a hosted server (
retp mcp, a future HTTP server) shares the same tools by providing a differentStoreProviderand transport. - We write the tool schemas ourselves; a schema derivation crate is not a dependency.
- A call that times out is abandoned, not stopped: a blocking thread cannot be cancelled. It ends on its own and its answer is dropped. At exit the process gives a stuck call a short grace period and leaves anyway.
- Features of the SDK we do not use (resources, prompts, subscriptions, tasks) are
not offered; the server advertises the
toolscapability only. Offering one later is an addition to the adapter and, if it needs a trait, to this crate. - The contract with clients is the tool names and their arguments, not the Rust API: a tool name never disappears, an argument added later is optional, an incompatible change is a new tool name.
0022 - The kind of an edge comes from the specification, labelled; the schema facts are generated
Status: accepted. The Unknown row (PRODML, EML on its own, any other ML) is superseded by
ADR-0048: every edge now has a kind.
Context
ETP 1.2 calls each relationship between two data objects (a data object
reference, or by-value containment) primary or secondary (section
8.1.1.1.2, p. 157), and a discovery can follow only one kind (navigableEdges)
or widen its answer to the secondary neighbours. Which kind a relationship has
is left to “the ML-specific ETP implementation guide”, written by the ML of the
object that holds the reference (or contains the other).
Those guides are not published for ETP 1.2. The schemas do not label the kind either; the specification says future versions of the data models will. A store that derives its edges from the objects (an EPC) therefore stated no kind, and a discovery by kind over it was refused for the whole dataspace, which also refused every such request a server built on the storage traits relayed.
The specification does state some rules:
- RESQML: “In RESQML, all relationships are primary” (section 8.2.2, p. 167-168).
- WITSML, three examples: a
ChannelSetcontainsChannels and aLogcontainsChannelSets (primary, by value); the reference from aChannelto itsWellboreis secondary (p. 157). - The “common characteristics” (p. 157): a primary relationship has one end that is mandatory, or is by-value containment; a secondary one has both ends optional.
Decision
- The facts are generated. For each top-level type of each schema version,
the generator writes where the type holds data objects by value and where it
writes references, with the cardinality of every step
(
energyml_core::relations, onerelations.rsper generated crate). The facts are checked against the schemas by an independent reader and against the reflection of the generated types (ADR-0005: never one source alone). - The policy is written once, by hand, and every answer says what it rests on
(
energyml_storage::kinds,KindBasis):EtpSpecfor RESQML and the three WITSML examples, with the section and page;DerivedXsdfor another WITSML reference (primary when the whole path to it is mandatory, secondary otherwise) andDerivedByValuefor other WITSML containment, both labelled as derived, never presented as the ML’s rule;Statedfor what a server sends;Unknown, with the reason, for PRODML, EML on its own, and any other ML. - An edge with no kind is refused where it is met (
Error::UnstatedKind), not dataspace-wide: a request by kind answers as long as its walk meets only edges that have one. - The kinds are given in one place, the provided
EnergymlStorage::edges, through the hookkind_rules;Capabilities::edge_kind_coveragesays of which MLs the kinds are stated.
Consequences
- An EPC holding RESQML answers every discovery by kind; a server built on the storage traits relays it, and a client reads the same edges back with their kinds (the triangulation of such a server agrees on every context of its matrix).
- A derived kind can be wrong for an ML whose guide would say otherwise. It is labelled as derived wherever it is shown, and replaced when a guide, or a version of the schemas that labels its relationships, says otherwise.
- A request by kind that meets a PRODML edge is refused by name until a rule for PRODML exists.
0023 - The data objects held by value are listed apart, and write their own references
Status: accepted
Context
Some data objects hold others by value: a WITSML 2.1 Log holds ChannelSets,
which hold Channels; an EML 2.3 LocalEngineering2dCrs holds its
OriginProjectedCrs. Each contained object is a data object of its own, with a
uuid, and ETP 1.2 lists it as one (section 9.1.3, p. 178: it “MAY be included in
one or more container data objects”). The relationship between the two is a
by-value containment whose source is the contained object (section 8.1.1.1.1,
p. 156), primary (section 8.1.1.1.2, p. 157).
An EPC holds only the container as a part. Reading it as before, the contained
objects did not exist, and every reference written inside one was given to the
part: a Channel’s reference to its Wellbore read as the Log’s, which states
an edge nobody wrote and judges its kind on the wrong type (Channel to
Wellbore is the specification’s secondary example).
Decision
resources()stays the parts. The objects held by value come fromEnergymlStorage::contents()(provided, empty), with one by-value edge from each to its direct container.EpcStorereads the container parts once, on demand, never when listing.Capabilities::contained_objectssays whether a store lists them; discovery adds them when it does.- A contained object is a resource:
ObjectSummary::contained_innames its containers,get_objectsandheadersanswer for it from the part that holds it. The same object in two containers is one resource with two edges; a contained object that is also a part is the part. - A reference has for source the object that writes it.
EnergymlStorage::referencesgives each DOR written inside a contained object to that object, its path relative to it (graph::attribute_to_writers), andgraphdraws the contained objects as present nodes with their containment edges.graph::buildkeeps giving every reference to the part: what a part depends on is what anything inside it names. - Writing refuses by name what it cannot do faithfully: deleting a container
whose contents would not survive, writing a contained object on its own.
Pruning a container’s contents is a separate capability
(
Capabilities::prune_contained), false for an EPC.
Consequences
- A WITSML
Logread from an EPC and the sameLogread from an ETP server should answer the same edges, with the same sources and kinds: the contractcontract_witsmlis written against the trait, to be run against either. - A package lists more objects than it has parts, and a graph drawn from it shows them.
0024 - Facets are read in every version, and say whether the schema lists them
Status: accepted
Context
A property or a table column carries facets: qualifiers such as the direction
(I, J, K), the side, the statistics, the net-to-gross. The releases write
them with the same two element names, swapped:
- RESQML 2.0.1 writes the kind in
Facetand the value inValue, a free text (RESQML TRG 2.0.1 section 6.36, p. 77). - EML 2.3, and so RESQML 2.2, write the kind in
Kindand the value inFacet, an open enumeration,FacetExt(EML TRG 2.3 section 11.11, p. 294).
A reader that knows one of the two reads nothing, or the wrong thing, in the
other, and there were two copies of it, one for properties and one for table
columns. The open enumeration also lets a producer write a value the standard
does not list: the Other hole of an EnumExtensionPattern, whose pattern is
.*:.* (EML TRG 2.3 section 6.28, p. 104 and section 11.2, p. 284). What such a
value means is the producer’s, and a check that compares it to the standard’s
literals would judge something nobody defined.
Decision
- One function reads the facets of any object that has a
Facetfield, property or column, whatever the release. It tells the two forms apart by the field that exists (KindorValue), and returns the facets in document order, without inventing, sorting or dropping any. - A facet is a kind and a classified value.
Facet { kind, value }, withvalueaFacetValue:Standard(text)when the schema lists it,Extension(text)when it does not.FacetValue::as_stris the one way to read the text, andis_extensionthe one way to ask the question. - Standard or extension, per release. In EML 2.3 and later the schema says:
a value of the generated enumeration is
Standard, theOtherhole is anExtension. In RESQML 2.0.1Valueis free text, so the rule of the pattern.*:.*is applied: a value with a namespace prefix (nwdf:vertical) is anExtension, any other isStandard. This is an interpretation, written here and not in the standard. - A table URI written as a facet value is an extension. The convention that
links a column to another object through a
qualifierfacet carries a URI, which has a colon: it is read as anExtension, which is what it is. - Reading is not judging. The business rule “do not use several facets of the same kind for the same property” (EML TRG 2.3 section 11.11, p. 294) is not checked by the reader; the rules that need it decide.
Consequences
- A facet on a 2.0.1 property and on a 2.2 property read the same, and the directions of a grid property, its side or its qualifier are available to every view without knowing the release.
- A rule that wants the standard’s own value (a direction
K, a ratio) can say “not checkable” on an extension instead of guessing; this is ADR 0005 applied to a vocabulary. Facet.valuechanged from a text to a type: a caller that read the text usesfacet.value.as_str(). No caller in this repository read it before.
0025 - A logarithmic mapping places the log of the value
Status: accepted
Context
Two things in a colour map are called logarithmic, and the code treated them as
one: ColorInformation.UseLogarithmicMapping and
ContinuousColorMap.InterpolationMethod = logarithmic. The reading was “either one
turns on a logarithmic spacing between the two stops around the value”. For a map
whose first index is 0, that flattened the colours: the lower bound was lifted to the
smallest positive number, and every value fell near the top of the range.
The Technical Reference separates them (RESQML TRG 2.2):
UseLogarithmicMapping“indicates that the log of the property values are taken into account when mapped with the index of the color map” (section 4.5, p. 56).MinMaxis “the range of values of the associated property which will result in the minimum color and the maximum color”, and a value outside it keeps the extreme color (same page).InterpolationMethodis “the method for the interpolation between color map entries”,linearorlogarithmic(section 4.7, p. 58 and section 4.23, p. 74).
The standard gives no base for the logarithm, and does not say what the range of the
values is when MinMax is absent.
Decision
UseLogarithmicMappingplaces the log of the value between the logs of the range, then the map is read as it is written. The base does not matter: placing a log between the logs of two bounds needs none.- The range is
MinMax, or else the map’s own range (its first and last index). This is the stand-in the linear mapping already uses, where placing a value on the map’s range is the identity. The other reading, that the index of the map is the log of the value, needs a base the standard does not give: it is not taken. InterpolationMethodstays the spacing between two stops and nothing else. The two can be combined.- A range that reaches zero or less is refused by name.
ColorMap::check_scalingsays why, once, for every value;try_samplereturns the refusal;samplekeeps its signature and answersNone. A missing value keeps its null colour, before any refusal. A value outsideMinMaxis clamped before the log, so a value of zero or less takes the extreme colour, never aNaN. - A house rule records it,
house/colour/log-mapping-is-log-of-value, with its references.
Consequences
- Every map that uses
UseLogarithmicMappingchanges colours: this is the fix. - A logarithmic map with no
MinMaxwhose first index is 0 or less no longer colours. A caller askscheck_scalingonce and shows the reason; this is ADR 0005. - The writer already refuses a
MinMaxof zero or less; it does not yet know the map’s indices whenMinMaxis absent, and may write a map the reader now refuses.
0026 - A head portion says whether the file describes it, and is in the line’s frame
Status: accepted
Context
A well frame starts at its first node, below the measured depth (MD) 0 of the trajectory. The stretch between MD 0, the datum, and that first node is the head portion, and a view draws it. Two defects hid in its calculation:
- The result did not say what it was. The doc stated that no object describes the interval, which is false when the trajectory’s own line covers MD 0. Whether the portion is the file’s line or a supposition decides how it is drawn.
- Two frames were mixed. When the trajectory has a
GeometrywhoseMdMinis above 0, the portion was drawn from the datum, already placed (projected, elevation upwards), while the nodes of the frame are on the line, in its own local CRS as stored. A caller places both by the same placement, so the datum was placed twice, far from the well and upside down, and the portion did not join the frame.
The datum is at MD 0: “for purposes of the trajectory the MdDatum within the MdInterval is mandatory” (RESQML TRG 2.2 section 12.17, p. 314). The line is the trajectory and its parameter is the MD. Outside its nodes it extends its first or last interval (RESQML TRG 2.2 section 3.8).
Decision
head_portionreturnsHeadPortion::Described(points)orAssumed(points).Described: the trajectory’s line covers MD 0 to the first node (MdMinat or below 0), the points are samples of that line, and the datum is not even read.Assumed: the portion contains something the file does not state.- One frame, the line’s own, as stored. Every point of the portion is expressed
there, and the caller places them with the same call as the nodes of the frame. The
only point that comes from elsewhere, the datum, is brought into that frame by the
exact inverse of the CRS’s placement (
Crs::from_projected), never placed apart. - The straight chord from the datum to the start of the line is assumed. When
MdMinis above 0 the file says where MD 0 is and where the line starts, not the path between: the portion is the datum, the chord, then the line itself, drawn as an assumption. A chord is the one shape that invents no curvature. Without aGeometry, the vertical segment between the datum’s two anchors is assumed the same way. - A datum that cannot be expressed in the line’s frame is refused by name. The same object as the line’s CRS needs no calculation. Otherwise the datum’s CRS is read, and its projected and vertical systems must be provably the same as the line’s (same EPSG code, same WKT, same authority); horizontal units that contradict each other, a unit stated on one side only, a CRS in time or absent are refused with the reason. Choosing one would be guessing.
- The coordinates of a datum are read in one place, which also reads the
RESQML 2.0.1 form, where
Locationis the point andLocalCrssits beside it.
Consequences
- A view draws a
Describedportion like the rest of the well and anAssumedone in a style that says so; the studio’s dashed grey portion is that second style. - A trajectory of RESQML 2.0.1 whose line reaches MD 0 now has its head portion; one that starts below it is joined to the datum in the frame of its own nodes.
- A refusal names its reason (the datum or its CRS missing, another system, a CRS in time, contradictory units), and does not fall back on a drawing.
- The chord cannot be longer than the interval it spans; a check of that between two independent sources, the datum and the line, is left to a later warning rule rather than a refusal here.
0027 - Gaps, seams, scales and quantiles: the numbers the views share
Status: accepted
Context
Several views read values along a measured depth (a log column, the 3D paint, a section, an object’s statistics). Each needs the same few answers: may two neighbouring nodes be joined, is a sudden step of the value a splice, on which scale does a quantity read, which bounds does a colour map take when the file states none. The standards say how a value hangs on a node or an interval and how a colour map maps its values; they say nothing about these four questions. Left to each view, each would answer a little differently, and a number on screen would depend on the view that drew it.
Decision
- A gap is never bridged beyond 3 median steps (S7). The limit is
GAP_RATIO = 3times the median step of the axis (along_md::GapRule). A caller may be stricter (stricter), never looser: the limit is private. A gap is drawn empty, and an interpolation across it refuses withGapTooWide. A node is not a bridge: on a node (tof 0 or 1) the value is the node’s, whatever the gap. - A seam is signalled beyond 10 median steps (S7). A step of the value
larger than
JUMP_RATIO = 10times the median step of its variation is aJump, reported with its place and its ratio. The values are kept as written; the drawing is not smoothed across it. The two thresholds sit in the same module so that they are read together. - The logarithmic scale is proposed from 3 decades (S8). When the finite
positive values span at least
LOG_DECADES = 3decades the log scale is proposed. It is a proposal: a scale the file states or the user chooses wins, and a disagreement with the file is said. Angles taken as linear are accepted with the noteAngleTakenAsLinear, not refused (detect and signal, do not forbid). - Automatic bounds are the 1st and 99th percentiles (S8). Over the finite values (the positive ones on a log scale). The minimum and the maximum stay available.
- Quantiles are exact up to 10 million values, then a sketch. The exact
definition is the linear one of Hyndman and Fan, type 7 (the one most tools
call the default percentile). Beyond
EXACT_QUANTILE_LIMIT = 10 000 000values the answer comes from a mergeable DDSketch (Masson, Rim and Lee, 2019) of relative error 1 % (SKETCH_RELATIVE_ERROR), and the result says that it is a sketch and what its error is. TODO(verify): the number of the theorem of the paper that states the guarantee. - On a tie, the nearest node is the shallower (the smaller MD, then the smaller index): a written rule, stable between views.
- Opposite angles are refused. Interpolating between two angles takes the
shorter way round; when the two are opposite (within 1e-12 rad) neither way is
shorter, and
angularrefuses withOppositeAnglesrather than choose a direction.
The four rules of the registry (house/stats/*) state these numbers; they are
declared rules (no check on a package), and a test crosses their text with the
constants of the code (tests/stated_numbers.rs).
Consequences
- One arithmetic for every view: a number that differs between two views is a defect, not a taste.
- A caller that needs a stricter gap says so; none can draw across a hole the rule refuses.
- The numbers are conventions of ours, not of the standards. They are in the registry with their reason, so that anyone can disagree with them explicitly (ADR 0005).
- Changing a number is changing a constant and its rule text together; the cross test goes red until both agree.
0028 - One drawing for every view
Status: accepted
Context
Several views were specified in parallel, and each came with its own display list:
a box, path and text list for the table schema and the provenance graph, a scene
for the target view, a layout with an image shape for the cross-section, panels
and layers for the composed views, figures for the statistics. The cross-section
already has a list of its own (energyml-section), written out as SVG and read
attribute by attribute by its script, the web studio and the desktop. Three
rules for graduations exist, in the cross-section, in the studio and in the
desktop, and they disagree (the desktop picks the round step nearest to the
ideal one, the others the smallest one that gives no more than the target). A
workflow engine will later need the same layered graph placement as the
provenance graph.
Decision
- One module,
energyml_sheet::draw, and it knows no format. It names no EnergyML crate (core, domain, storage) and no module ofenergyml-sheetoutside itself (crate::appears only ascrate::draw). A test holds it (tests/draw/no_format.rs), so that the module can be moved into a crate of its own without a rewrite. - One
DisplayList. Keyed elements in ordered layers, in pixels or millimetres, y down. Its SVG writer, its hit test and its PNG all read this one value. A coordinate that is not a finite number is refused by name, never written as 0. - One rule for graduations.
draw::ticks: a step of 1, 2 or 5 times a power of ten, the smallest one that gives no more than the target number of intervals, exact as a decimal; logarithmic axes in three densities; refusals by name. The cross-section calls it; the desktop and the studio will call it. - A colour is a closure of the caller. Rasters are painted through a function
the caller passes;
drawreads no colour map, so the one door to colours stays with the caller. - A PNG is always the SVG rasterised. The only way to a PNG is
png::render(svg), byresvgwith itsraster-imagesfeature and an image resolver that accepts only a PNG embedded in adata:URL. The other ways were set aside: a finer feature does not exist; anImageHrefResolverwithoutraster-imagescan only hand back vectors, one rectangle per pixel; drawing the images ourselves would be a second renderer. The feature adds eight pure-Rust packages toCargo.lockand three decoders (GIF, WebP, JPEG) that are linked and never reached. Thepngfeature is off by default: without it, a call refuses by name.
Consequences
- The cross-section migrates to
drawlater, in a lot of its own: its nested groups, its zoom attributes, its layers for Inkscape and its byte-for-byte tests need more thanDisplayListhas today. Until then, three second implementations remain, and are known: the XML escaping inenergyml-sectionand in the sheet’s SVG writer, and the section’s own PNG renderer. - The font exists twice (
energyml-section/fontsandenergyml-sheet/fonts), because a published crate cannot embed a file of another crate. The width table stays where its generator writes it. - The desktop, which drew fewer graduations on some ranges, will draw the cross-section’s. This visible change is intended.
0029 - A link to a table is one qualifier facet holding the uri of the table
Status: accepted
Context
Two things need to point at a table the standard gives no field to point at:
- a column of a table that holds the key of another table, or the rank of one of its rows;
- a property whose components have axes (a length from -40 to 40 m, sixteen angles) that live in a table.
RESQML TRG 2.2 section 7.1 (p. 222) gives a property labels per component and nothing else; EML TRG 2.3 has no link from a column to a table. Left to each reader, the link would be read as many ways as there are readers, and each way is a guess about what a producer meant.
What the standard does say, and what a link can rest on:
- EML TRG 2.3 section 11.11
PropertyKindFacet(p. 294): “Don’t use several facets of the same kind for the same property.” Onequalifierfacet can carry one link; a second beside it cannot be a second link. - EML TRG 2.3 section 11.3
FacetKind(p. 285):qualifieris “used to capture any other context not covered by the other facet types listed here”. An ordinary qualifier (cumulative,measured) is context, not a link. - EML TRG 2.3 section 11.2
FacetExt(p. 284): the list of values is open, so a uri is a legal value.
Decision
- Convention B. A column, or a property, links to a table by one facet of
kind
qualifierwhose value is the full ETP uri of that table. The convention is this library’s own, written down here. - One reading.
property::qualifier_link(facets)is the only function that reads it. It returns the uri,Ok(None)when noqualifierfacet holds a uri (an ordinary qualifier is left alone), and refuses by name: a uri beside anotherqualifierfacet isSeveral { count }(the standard allows one facet of a kind), a value that starts like an ETP uri and is not the uri of an object (a dataspace uri, a truncated one) isMalformedUri. It does not say whether the object named is a table: each caller checks that against what it needs. The table side (column links,LinkReaders) and the property side (axes) both call it; neither reads a facet list a second way. - The axes of a property. The table the facet names holds exactly one key
column, and one value column per dimension of the components, in the order
of the document, the slowest dimension first. That order is an
interpretation: the standard states none for a table. A table has one row count
for all its columns, so a shape whose dimensions differ (
[3, 2]) does not fit a table and is refused by name. A table with another number of value columns, or of key columns, is refused by name. - A declared source comes before the file. What the caller declares (a table,
a paired property, a WITSML channel) is tried before what the file says
(convention B,
LabelPerComponent); the first source to answer for a dimension wins, and a dimension nobody answers for has its ranks and says so. A source that applies and fails refuses: an axis that was declared is never silently replaced by ranks. A table that merely looks right is offered (candidate_tables) and never applied. - Nothing is written to the package by this reading: a declaration the caller makes is a parameter, kept in a recipe or a view document.
UseIntervalis read asn - 1steps forninstants. The standard names the field without a text (EML TRG 2.3, p. 296,TimeOrIntervalSeries); the library has read a value per interval since before this decision (TimeRef), and the shape of a property keeps that reading.
Consequences
- A column or a property that links by convention A (the target written in a description) is not read by this function; the table side reads it, with a warning (see the decision on table links).
- A producer that writes two
qualifierfacets, one of them a uri, gets a named refusal rather than a silent choice of the first. - The order of the value columns is the one place a file written by someone else may mean the opposite; the refusal of unequal dimensions catches the case where it matters most, and a reordering is the caller’s declaration.
0030 - Table links: conventions, keys and ranks
Status: accepted
The convention that makes a column a link, one qualifier facet holding the
uri of the target table, is ADR 0029
and is not said again here. This record is the table side of it: what a link
names, how a row is found, what is read beside the facet, and what is reported.
Context
The specification has no column that names another table. Common TRG 2.3
sections 9.1 and 9.2 (p. 190-191) give a table columns, a KeyColumn that
“makes the table a lookup table”, and positional rows (“Ordering matters since
it will be the same in all columns”), and nothing that says that a value of a
column is the key of a row of another table. No property kind says it either:
identifier is “a number, label or code given to an object in an attempt to
uniquely identify it”, and explicit index is a token “used as a reference or
pointer to a value stored in some data structure”. Producers write the target
where they can: in the description of the column (the target uri and the name of
its key column), or by the title of the table in a sibling column.
Nor does the specification say how a value names a row of a lookup table, nor that keys are unique, nor what a key is when the table has none.
Decision
- A link is written as in ADR 0029: kind
identifierwhen the value is the key of a row of the target, kindexplicit indexwhen it is its rank, and onequalifierfacet that holds the complete ETP uri of the target table (the list of facet values is open, section 11.2 p. 284;qualifieris “any other context”, section 11.3 p. 285; one facet per kind, section 11.11 p. 294). - The target decides how a row is named. A target table has one key
column (a value names a row by its key) or none (by its rank, counted
from 0). A table with several key columns is refused by name for every
convention: a value is not a tuple. A table with a key column is never read
by rank, so a key that finds nothing never lands on an unrelated row. The kind
of the link column and the target can disagree; the target wins and the
disagreement is reported. This one rule is
table::row_named_by, andhouse/discrete-value-in-lookupreads a category value the same way. - Two other ways of declaring a link are read, with a warning. A: the
description of the column says
key of the table <title>: <uri>, KeyColumn <key>(or its self-reference form). D: a sibling column titled<stem>_tablenames the target table, row by row, by title (a title that names no table of the package is an external reference, said as such; a title shared by two tables is refused, because a title is not an identity). Both are conventions of producers, not of the specification, and the documentation says so. When the facet and the description name two different tables, neither is followed and the disagreement is refused by name. - A column of object uris is not a link to a row. A column whose every non-empty value is a complete ETP uri (convention E) names objects, not rows: its values are read as object chips, and a uri that names nothing the package holds is reported.
- Aliases are refused. A name held in an alias is not an identity
(
Identifieris a 64-character string, not a reference). - An empty string is a value, and names no row. A cell has four states: a
value, an absence (the array’s
NullValue,NaN), an empty string and a failure to read. Absence and emptiness are not the same thing, and none of the four is turned into another. - Keys compare exactly. Two numbers are the same key when they are equal
(
-0.0is0.0;NaNis never a key). A tolerance is not transitive and cannot back an index. - A property names the table of its axes the same way (ADR 0029).
Consequences
- The rules
house/table/link-resolves,house/table/link-target-key-unique,house/table/link-declared-by-facetandhouse/table/uri-value-resolvesare checks ofenergyml check, with their findings;house/table/row-rank-without-keyis the rule of reading the others cite, with no finding of its own. A link whose target is not in the package, or that cannot be followed, is uncheckable, never a pass. house/discrete-value-in-lookupkeeps its id: its text now says that a value names a row by the key, or by rank when the table has none. A table without a key column, which the check used to leave unverifiable, is now verified.- A facet is turned into a reference, which is the cost of ADR 0029: a reader of the package that ignores the library sees a qualifier it does not understand.
- Writers (
energyml-build) are to write convention B, and derive any description (convention A) from it, so the two never disagree. - Whether to propose the conventions to Energistics is open; until they are adopted they are ours, and the reports say so.
0031 - The direction of a line at a knot is that of the interval that ends there
Status: accepted
Context
Every view around a well asks the same question: in which direction does the hole
go at this measured depth? The line of a trajectory is evaluated by
Path::at, which for a linear line interpolates between two knots. At a knot itself
a polyline has two directions, one on each side, and the specification of a
parametric line states none. A frame, a ring or a projection drawn at a knot
would be different depending on who computed the direction, and a view that
mixes a point from one segment with a direction from the other draws a frame
that does not sit on the line.
Two ways to get a direction were on the table: a centred difference of at
(cheap, written everywhere), or the derivative of the very interpolant at
evaluates.
Decision
- The direction is the derivative of the interpolant that
Path::atevaluates, kind by kind (linear, natural cubic spline, Hermite, minimum curvature, vertical), inPath::direction_at. It is not a second curve and not a difference of two evaluations. - At a knot of a linear line, the direction is that of the interval that ends
there: the interval in which
atevaluates that parameter (Line::segment). At the first knot, where no interval ends, it is the direction of the first interval. Nothing more is coded: the rule is the onesegmentalready applies, which keeps a point and its direction on the same segment. - Past either end, the derivative of the extended interval, as
atextends it. - Where the derivative vanishes, is not finite, or the line does not exist, the
answer is
None, never a zero vector. - The rule is registered as
house/well/tangent-at-a-knotand is read by the frame of a hole, byPlacedPath::direction_atand by every projection.
Consequences
- A test can compare the analytic direction with a centred difference of
ataway from the knots: two independent sources of the same fact. At a knot the two disagree on purpose, and the test says which interval wins. - A view that wants the direction of the interval that starts at a knot asks for a parameter a hair after it; there is no second function for it.
- A change of how
atfinds its interval changes the direction with it, which is the point: they cannot drift apart.
0032 - The frame of a hole is the physical one, whatever hand the placed frame has
Status: accepted
Context
Around a well, a view needs more than the direction: where is the top of the hole, and where is its right. Rings, matrices carried by each node, envelopes and a curtain all hang on that pair. It is computed from a tangent and the upward vertical, and its sign depends on a cross product.
A cross product taken on coordinates points to the physical right only when the frame of those coordinates is direct. The placed frame (first projected axis, second projected axis, elevation) is not always direct: a system whose axes are written northing then easting is indirect, and a cross product on its coordinates points to the other side. The coordinate system local to a tool (east, north, depth) is indirect as well. Computing the frame in the local coordinates, or in an unchecked placed frame, gives a right that is wrong and looks right.
Decision
- The frame is computed in the placed frame (metres, z up), where the vertical has already been turned by the coordinate system; nothing is computed in the local coordinates.
- The hand is read from the file, once, in
Crs::axes(AxisOrder,AxisReading,Handedness,CompassAxes): the order of the horizontal axes of the projected system in RESQML 2.0.1,AxisOrderandHorizontalAxesin EML 2.3. Directions go throughCrs::direction_to_projected, the door for vectors asto_projectedis the door for points. rightis always the physical right of someone looking down the hole, written in the placed coordinates:tangent x high_sidein a direct frame, its opposite in an indirect one. Two writings of the same well, one in (E, N, up) and one in (N, E, up), give the same physical right.- The high side is the upward vertical projected across the hole. Below 1
degree of inclination (and above 179) it is not defined, and the frame is
refused at that node only (
HighSideUndefined), never turned to north in silence. The threshold is one named constant,MIN_INCLINATION_DEGREES. - A hand the file does not give is declared by the caller (ADR 0041). A
declaration never overrides what the file states: a disagreement is refused
(
DeclaredAgainstFile). Two statements of the same frame that disagree make the readingContradictory, and a frame that needs the hand is refused by name (AxisOrderUnknown). Reading the axes never makes the placement of a point fail: only what needs the hand refuses. - The frame of a hole is distinct from the transported frame of a tube (minimal rotation, arbitrary origin): both stay, under two names.
Consequences
- The right of a ring or of a matrix is the same in a placed frame of either hand; a test writes one well both ways and compares.
- A package whose system states nothing about its axes gets no frame until the caller declares a hand; it is a refusal with its reason, not a drawing.
Crsgains a field: a literalCrs { .. }in a test has to sayaxes: AxisReading::NotStated. A published release carrying it is a breaking change of the public type.- Which of the projected
AxisOrderand the localHorizontalAxesthe specification means to govern in EML 2.3 is not stated; the projected one is taken first, because the placed coordinates are those of the projected system.
0033 - A true vertical depth along a well is counted below the MD datum
Status: accepted
Context
A view that lays a well flat (a measured depth against a true vertical depth, a
developed length, a vertical section, a plan) needs a vertical coordinate.
“True vertical depth” has two origins in practice: the zero of the vertical
system (a depth below the sea level or the ground, read straight from z), or
the point where the measured depth is 0, the datum of the MD (the rotary table,
the kelly bushing). A well drawn from the first reads shifted by the height of
its table against a log drawn from the second, and the two columns of a sheet
then disagree by a constant nobody can see.
The same question comes up in several views. Two of them had their own projection for the same formula (a measured depth against a TVD, a section along an azimuth), under two names.
Decision
- The TVD of a projection or of a curtain is counted downward from the MD datum:
PlacedPath::datum().z - z. It is registered ashouse/well/tvd-below-md-datum. - A line that does not reach MD 0 has no datum here, and a projection that asks
for a TVD is refused by name (
NoDatum). Counting from the first node would be an extrapolation made silently. - One formula per projection, one registry:
WellProjections, a trait and a registry in which the caller’s additions are tried first, as for the other registries of the library. The standard entries aremd_bar,md_tvd,developedandplan;vertical_sectiontakes an azimuth and an origin and is built by the caller. Two earlier projections that were the same formula are merged: the unfolding by measured depth and the MD/TVD mode are one entry, the unfolding by azimuth and the vertical section are one entry (VerticalSection::abscissais its only formula). - The position of a mark is always
PlacedPath::at(md)projected, never a length measured on the flat line: on the plan of a J-shaped well the flat line is shorter than the MD. - The well is sampled once (
WellAxis: the ends of the range, the knots inside it and a step) and every projection reads that sampling.
Consequences
- A measured depth and a true vertical depth start together at the datum, as a driller reads them, whatever vertical system the package uses.
- A trajectory that starts below MD 0 can still be drawn by measured depth and in plan; only the projections that need a TVD refuse.
- Adding a projection is adding an entry, not a branch.
- The vertical axis of these projections is the TVD positive downward; the drawing turns it over.
0034 - The north of a survey and its stations: what the file states, what the caller declares
Status: accepted
Context
A directional survey is a list of stations (measured depth, inclination, azimuth), joined by minimum curvature and placed from a datum. Three facts it needs are left open by the standards, and a drawing made on a guess looks as plausible as a drawing made on the truth:
- The north of the azimuths. WITSML 2.1 makes
AziRefoptional “because of legacy data” while strongly recommending it. Grid and true north differ by the convergence, several degrees in places: the bottom of a 3 000 m hole moves by tens of metres. - The corrections. A survey in true or magnetic north states, in the file, the convergence and the declination its producer used. Whether to apply them, and in which order, is left to the reader.
- The stations the producer rejected. The guide says a rejected station does not influence the stations below it, but a drawing that silently drops measured stations shows a well the file does not quite describe.
RESQML 2.0.1 has the same question for its DeviationSurveyRepresentation, which
the library did not read at all.
Until now the library refused a survey it could not place, by name, and the check of the north came after the placement of the datum, so a survey with no north was reported as something else when its datum was also wrong.
Decision
- The north is checked before the datum is placed (L1). The check needs only
the stations read; its second half, “does this north sit in the frame of the
datum”, needs the placement and comes after. “The hole leaves the vertical” is
read on the kept stations (L2): a station whose inclination has a sine above
min_curvature::VERTICAL. - A north the file does not state is declared by the caller, never guessed
(S11).
read_declaringtakes a grid or plant north; the result carries it (NorthSource::Declared), nothing is written into the package, and a declaration never replaces a north the file states (NorthAlreadyStated). Without a declaration, a survey that leaves the vertical is refused (NorthNotStated). Registered ashouse/survey/declared-north. - The corrections the file states are applied (S12, L3). True north to grid
north: the convergence is subtracted. Magnetic north to grid north: the
declination is added, then the convergence subtracted. A station’s own value
(
StnGridConUsed,StnMagDeclUsed) comes before the trajectory’s (GridConUsed,MagDeclUsed). The result is a grid-north azimuth, placed as a survey stated in grid north. Quote, WITSML TRG 2.1 section 16.7 p. 352: “To convert a true north referenced azimuth to a grid north azimuth, the convergence angle must be subtracted from true north”; p. 353: “To convert a Magnetic azimuth to a True North azimuth, the magnetic declination should be added”. A correction the file does not state is refused by name (NorthNotApplied); the library takes none from elsewhere (a geodetic library, a magnetic model) because that would be a second source the file does not name. Registered ashouse/survey/north-corrections-applied. - Astronomic and compass north, or a value outside the list, stay refused (L4): the file has no field that corrects them.
- A rejected station is left out only when the caller declares so (S13, L7).
By default the survey is refused (
RejectedStation). Left out, the positions run over the other stations and the result lists the stations kept (WITSML TRG 2.1 section 16.12 p. 363: a rejected station “does not influence position computation for stations below it”). A refusal names a station by its number in the file, from 1, even when stations were left out (L8). Registered ashouse/survey/rejected-station-left-out-on-declaration. - A RESQML 2.0.1 deviation survey is read (S14, L5, L6). Its azimuths are
relative to the projected CRS north by its schema, even when the local CRS is
rotated; its inclinations are measured from the vertical; no method is stated,
so the stations are joined by minimum curvature.
FirstStationLocationis read and shown, never used to place, because the schema does not say in which CRS it is: the survey is placed from itsMdDatum, and refused by name (GridWithoutProjected) where that datum names no projected CRS to turn from. Registered ashouse/survey/deviation-survey-grid-north. The survey is entered in the registry of readers, so thatgeometry::of, the scene and the exports say the named refusal instead of “unsupported”. - Values the survey declares never make the reading fail (L9). Station
values (
Tvd,DispNs,DispEw,VertSect,Dls,RateBuild,RateTurn) are read for comparison; a missing or unknown unit is a fault on that field alone.RateBuildandRateTurnare read and never compared: the schema ties them to the radius of curvature computation, another method than the one recomputed (L10).DispNsandDispEware not compared whenGridScaleFactorUsedis present, since the table of units has no length per length family (L11). - The points a survey is placed with are in the frame and the units of the CRS
that
crs::ofgives it (L12, Q12).crs.to_metres(points[k])equals the placement of the local position of the kept stationk; without a CRS, the points are in metres. The conversions go through the existing system (units::metres,Crs::to_metres,Crs::from_projected); there is no private table and no factor written by hand. A survey in feet, or a datum that is aReferencePointInACrs, is no longer drawn with its units mixed.
Consequences
- A survey that states no north draws nothing until the caller says which north to use, and the drawing says it was declared. A file that states its north is never overruled.
- A true or magnetic north survey lands where the same survey in grid north does. A test can compare the two: the same fact from two sources.
- The rejected stations are a choice the caller makes and the result reports; the default is the refusal of today.
- The four rules are declared, not checked:
stationsapplies them, and the breach is a named refusal of the reading, whichdiagnoselists. No entry inintegrity::CHECKS. - Not decided here: the stations whose status is
open, which are treated as before.
0035 - Provenance, written completely and read from four releases
Status: accepted
Context
The library wrote provenance that did not satisfy the rule of the specification it
claims to follow. The EML technical reference guide 2.3 (section 3.1, p. 40) says
the title of an activity parameter must have an equivalent in the activity
descriptor parameters; yet the operations wrote parameters titled input that
no canonical template declared, whole numbers as doubles, and the file a recipe
step had read nowhere. A graph of provenance drawn from such a package had to guess
which parameter was an input.
Reading was the other half. Activities come in four releases (RESQML 2.0.1, EML 2.1, 2.2, 2.3), with a template that may or may not be in the package, and a role that is stated only in the template. Some converters, such as UDAR with its NWDF tables, keep provenance in tables outside any activity, and a drawing that ties a dataset to its objects by a regular expression on the title looks right and is wrong.
Decision
- A
DataObjectParameterthat points to anActivityis read assubActivity. No class of parameter has that kind in the schemas; it is the way the library’s recipes already write a workflow’s steps. - A
TimeIndexParameteris read astimestamp, for the same reason. - The role of a parameter comes from the template, then from a named rule, else it
is
Undeclared. A role is never guessed from a title. The one built-in rule,energyml-build:input-output, readsinputandoutputin an activity written byenergyml-rust; it exists for the packages written before this decision, and its id is shown beside the role it gave. Rules are a registry, the caller’s first (ADR 0006). - Every canonical template declares
inputandinputFile. AninputFileparameter is a string whose value is<slot>: <path>, or<path>when no slot was named. A new package satisfies the rule of section 3.1. - Whole numbers are written as
IntegerQuantityParameter, as the canonical template declares them (Parameter::Integer, a breaking change of the enum). - A verbatim source is a public trait with a registry; the NWDF reader is public.
Its tables are recognised from column titles, one table per role (two are refused
as
SeveralTables, not merged), the objects of a key come from the mapping table and never from a title, and a key nothing resolves is shown as unresolved. When a process’s activity is stated twice and the two differ, both are kept and the disagreement is signalled. - A recipe draft rebuilt from a workflow is lossy, and says so. What it cannot restate is refused by name.
Consequences
- A package written from this version satisfies section 3.1 without an exception in
the template check; the check that excepted
inputis gone. - A package written before it still reads: its
inputparameters get their role from the built-in rule, with the rule’s id shown, andconformancesignals the parameters the template lacks. Nothing is refused for being old. - The rule of the registry (house rules, theme
provenance) reports what departs from a template; it never blocks a read. - A reader of an activity cannot tell a written integer from a written double in a package of another software: a number is read by the field present, and the conformance reports a difference of kind against the template.
- Replaying the Lutetia model changes its templates and activities; its lock file moves in the same release.
- Two tables of one role (two converted boreholes in one package) are refused until there is a way to tell the groups apart, which is a future case, not a guess.
0036 - Two directional properties of one quantity keep the order a layered medium expects
Status: accepted
Context
A property can be told apart from its sibling by a direction facet alone: a
resistivity logged along the layers and the same resistivity logged across them
(Rh and Rv), or the kx, ky and kz of a permeability. The facet kind
and its values are in the specification: I and J are the lateral axes of a
grid, K is its “third local grid (vertical) direction”, X, Y and Z the
same on coordinates, each with a + and a - variant (common TRG 2.3 section
11.1 p. 281-282, FacetKind section 11.3 p. 285).
The specification says nothing about the order of the values of two directions. What a reader expects is physics: in a layered medium, the current crossing the layers meets more resistance than the current running along them, so the vertical resistivity is not below the lateral one. On a real package of about two million pairs, 38 % of the pairs had a vertical value under 0.95 times the lateral one: a package that breaks the expectation is not rare, and it is not necessarily wrong (noise, an inversion that models another medium).
Two more facts shape the rule:
- a direction can be written as an extension value (
nwdf:vertical), which the standard leaves to the producer (common TRG 2.3 section 11.2 p. 284): which axis it names is not in the specification; - the expectation is known for one quantity only. For a permeability it is weaker and unconfirmed.
Decision
- A warning, never an error (
house/property/directional-ratio,Gravity::Warning). The expectation is a reading of the house, not of the specification, so the finding says it and the package is kept. - Two thresholds, written as constants of
integrity: a pair breaks the order when the vertical value is under 0.95 times the lateral one (RATIO_TOLERANCE), and the rule warns when more than 5 % of the pairs do (RATIO_SHARE, compared with>: exactly 5 % is silent). A value equal to the array’s ownNullValue, a value that is not finite and a lateral value of zero are not pairs. - The siblings are found by the one rule of kinship
(
property::kinship::kinship): same support, element, kind, unit and shape,directionfacets that differ. A vertical is paired with each lateral of the same family of axes (the grid’sI,J,K, or the coordinates’X,Y,Z): a grid’sKagainst a coordinateXis not a comparison. The pairs are counted bystats::tally, the one counter. - A table of kinds, not a
match:DIRECTIONAL_EXPECTATIONSholds the resistivity (the dictionary uuid 7c0d1a5d-483e-4d77-81f3-9883527838d2) and every descendant of it, which the resolution of the property kind finds. Any other kind is not judged. Adding a kind is one line. - An extension direction is unverifiable, unless the caller declares it. The
rule never guesses “vertical” from the text of a value. A caller that knows
which sibling is vertical gives a
DeclaredDirectiontodirectional_ratio_declaring; the declaration serves that one run and is never written into the package. - A check that cannot look says so: no array reader, a kind that does not resolve, arrays that do not read or of different lengths, no pair of finite values: each is an unverifiable finding on the property concerned, never a silent pass.
The physical reference for the expectation is not a document of the registry’s closed list. TODO(verify): a petrophysics handbook to cite, with its page, for the anisotropy of a layered medium.
Consequences
- A package of an array resistivity logged in two directions can now carry a warning in the report of a check, and the view of a property can run the rule again with a person’s declaration.
- A package whose directions are written with a producer’s prefix says unverifiable until somebody declares them. That is more noise than a guess and less than a wrong answer.
- A threshold the caller wants otherwise is not a parameter: the rule is one statement of the registry, and a changed threshold is a changed rule.
- The text of the rule in
energyml-rules/rules/property.tomland the constants here are two sources of one fact; the numbers of the text are the code’s.
0037 - A result of a view is an artefact first, and is stored on demand
Status: accepted
Context
The views of the workspace compute things that are not objects of the package: a geosteering section drawn along a well, an iso-surface cut in a cube of resistivity, a trace with its cross sections. A view shows its result at once, and most of the time nobody wants it in the package: writing every drawing would fill a package with files nobody asked for. Yet some results have to be kept, with the recipe that made them, so that the same drawing can be found again and the objects it was computed from are known.
Two facts make the recipe delicate:
- An operation is a pure function. A plan reads no array and no store (ADR-0017). A cube of two million values, the positions of a lattice placed along a trajectory, the frame of a hole: none of that is in the bag an operation receives, so an operation cannot compute an iso-surface again.
- The template of an activity declares each of its parameters (common TRG 2.3 section 3.1 p. 40, ADR-0035). A generic template, one for every view, cannot say what each view was told.
Decision
- The result is computed and drawn as an artefact, and promoted when somebody
asks. The computation is the one of the library (
energyml_geostat::isosurface,energyml_section::along_well); the scene keeps the artefact; an operation ofenergyml-buildstores it on a gesture, never by itself. - One operation per sort of view result.
along-well-viewstores a geosteering section,isosurfacean iso-surface; the section view and the coverage follow the same model. Each has its own template, so that each parameter is declared. A view with no operation of its own is not stored. - The drawing is the input, and the recipe is read from it. The
<metadata>of the SVG holds the request and the view; the operation reads them back (the one reader,svg::read_metadata), and a setting given beside the drawing is held to it: a disagreement is refused with both values. A second statement of the recipe would be free to disagree with the first. - The objects the drawing names must be held, compared on the full uri. One that the package lacks is refused by its uri: the drawing was made from another package, or before that object was written.
- The iso-surface is promoted as it was computed. The operation takes the mesh
and the declared recipe (level, spacing, method, the cells set aside, the inputs)
and writes the mesh through the writer of
triangles, the one writer of triangulated sets. It does not recompute, and the notepromotedsays so. A lattice that folds on itself is refused by the caller before it computes; the operation cannot check it and does not pretend to. - The images are attached to the frame of the section, in the part-address form
of
attach, and named by the frame and what they are, so that drawing a section again replaces its own files. - Stored views are read by the template of their activity, never by its title
(
read::stored_views). The loop that finds the activities is one function, shared with the stored sections. - A PNG is the PNG of the SVG that is stored with it, made by
draw::png::renderfrom that SVG (ADR-0028). The operation checks that it is a PNG by its signature and that it has the proportions of the drawing; it draws nothing.
Consequences
- A package can carry the recipe of a drawing without the drawing being an object of its own: the activity names the inputs, the part holds the SVG.
- The recipe of a promoted iso-surface is as true as the caller’s declaration. A caller that declares a level it did not use writes a false activity, and nothing in a pure operation can catch it. That is a limit stated here, not hidden; a check that recomputes belongs to a face that holds the cube.
- A new sort of view result is an operation and a template, not a parameter of an existing one. That is more code than a generic store, and an activity a reader can understand.
- Two doors to a triangulated set (
trianglesandisosurface) write the same bytes for the same mesh and naming; a test compares them.
0038 - The geosteering section
Status: accepted
Context
A geosteering section lays a cube held on the nodes of a wellbore frame (the resistivity of a UDAR tool, for instance) either side of the hole, with the boundaries of its depth of detection, the horizons it cuts and the tops of the hole. Four things the files leave open, or that two implementations would settle differently, had to be decided once.
Decision
- The curtain lives in
energyml_section::along_well, next to the long section and not in it: it has its own model (Request,Model) and its own drawing (energyml_sheet::draw), and adds no dependency to the crate. - One frame, one door. Everything is computed in the frame of the hole: a
point is
(md, dn),dnthe offset along the high side, positive up. One function carries it into the drawing,WellProjection::place. The cells, the boundaries, the horizons and the markers all go through it, so a boundary at radiusrand the centre of the row at offsetrare the same point. - Folds are marked, never straightened; gaps are never bridged. A cell
that folds over the centre line in
developed(wheredn I'reaches 1) is flagged and drawn where it falls; the cells of a node reach half the way to its neighbour, at most half the widest gap the rule bridges (S7). - What the file does not state is declared by the caller. The dimension
along the high side and the sign of each dimension are the request’s
(S2); the axes are the caller’s links, then the file’s convention B, and a
rank is never an offset (refusal
AxesUnknown). A flat list of components is never factorised into a matrix. - Angles of the boundaries are searched by value, within 1e-6 rad of 0 (above) and pi (below), never by rank; two angles on one side of a node, or none in the plane, are refused by name.
- A node with no high side (S4) is counted, not a refusal of the section. A source that is declared and not built, and a marker (no reader exists), are refused by name and the section is still drawn.
- The drawing carries its request and its view in its metadata, and is made
again from them byte for byte. A PNG is the SVG rendered by
draw::png::render, never a second drawing.
Consequences
- A face asks
buildonce andlayout/draw/pickas often as the view changes; nothing is read again. - A new kind of curtain source is an entry of the
CurtainSourcesregistry, the caller’s tried first. - Reading the markers of a well is left to a future reader: the pure function
marker_ticksexists and is tested.
0039 - Several values per element are written with their shape
Status: accepted
Context
A property of the package can hold more than one value per element: a matrix per
node (the cube of an azimuthal resistivity tool holds [81, 81] per station), a
vector, a tensor. The standard says it in ValueCountPerIndexableElement, a list
of counts, slowest first, and in LabelPerComponent for the names of the
components (RESQML TRG 2.2 section 7.1 p. 222). The note of the figure p. 221 gives
the order of the dataset: the elements first, then the dimensions of one element, and
time slowest of all when the property is a series.
Until now the one writer of a ContinuousProperty wrote [1] and a flat array.
A property derived from such a cube (the ratio of two directions, the difference of
two passes) would have lost its shape, and a reader would have taken a matrix per
node for a long scalar column.
Decision
property::write_shapedis the one writer, andwrite_timedis a call to it with a scalar shape. There is no second function that writes a property.- The shape is the one of the domain (
ElementShape): its dimensions are written asValueCountPerIndexableElement, its labels, when it has some, asLabelPerComponent; without labels the field is absent, never an empty list. - The dataset is
[elements, d1, d2, ...], or[steps, elements, d1, ...]for a series: time the slowest dimension, the elements, then the shape of one element. - A scalar writes what it always wrote:
[1]and a flat array, the same bytes as before. A test holds it. - The nodes are counted in elements, not in values: a representation of three
nodes does not take eight values of a shape
[2, 2], and the refusal names both counts. A number of values that is not a whole number of elements is refused, never truncated. CountPerValuestays 1. The common TRG 2.3 does not define it for a floating point array (section 6.19 p. 95 names the attribute forBooleanExternalArraywithout a note); the shape is said byValueCountPerIndexableElementand by the dimensions of the dataset. TODO(verify): a statement of the standard onCountPerValueof a floating point external array.- The shape is not a parameter of the activity. It is in the object written, the
template of
propertydeclares no slot of shape. The operationcombine, which writes a derived property, asks for the shape of its parents (valuesPerElementandcomponentLabels) because a plan reads no object but the ones it is given.
Consequences
- A derived property keeps the shape of its parents, and the readers that already
understand
ValueCountPerIndexableElementread it back without a change. - A caller that states a shape different from the parents’ writes a property of that shape: the operation checks the number of values against it, not against the parents’ own declaration, which it is not given.
- The question of
CountPerValuestays open until the standard answers it; the value 1 is the one every writer of the workspace has written.
0040 - The members of a wellbore are walked in the domain, from the inventory and the references alone
Status: accepted
Context
Several views start from the same question: what does this well have? Its trajectories, the frames that sample along it, its marker frames, the properties that hang on them. A grid of wells asks it to show what each well holds; the depth coverage asks it to know which representations to measure. The question needs no array and no geometry: only the type, the uri and the title of each object, and the references between them.
The walk existed once, in a desktop application, as a port of a web studio’s
wellMatrix. A library that two faces and a coverage share cannot keep it in
one of the faces. The first design put it in energyml-cache, which holds the
package index. But holding the index is not reading it, and the depth coverage
(md_extent) lives in energyml-domain, which does not depend on the cache.
Decision
- The walk lives in
energyml_domain::well_members, beside the package index it reads, so that the cache, the faces and the coverage all call one function. - It is a pure function of facts:
well_members(objects, outgoing), whereobjectsareObjectFacts (uri, qualified type, title) andoutgoing(uri)gives the uris an object references. It is asked once per relevant object.well_members_of_indexis the same walk over aPackageIndex, whose edges are itsreferences_of. A dataspace of a server gives its own edges the same way. - A well is seeded from both ends: a feature is a row even with nothing on it, and a representation that reaches no well makes its own row. A representation reaches its well by its interpretation and the interpretation’s feature; a WITSML trajectory by its wellbore and the feature that names that wellbore. A reference to an object the list does not hold is ignored.
- The kinds are a table (
MEMBER_KINDS), not amatch: a type added later is a line. A type is read by its local name, without theobj_of release 2.0.1, so that the same concept is the same in every release. - A property is categorical when it references a table of categories
(
CATEGORY_TABLE_MARKERS). The marks are carried over with the doubt of the code they come from: that a continuous or a discrete property never references aColumnBasedTableis not verified. - The walk does not judge. Whether a member can be drawn, the wording of a state, and the telling apart of two wells of one title are the business of the face. The order of the result is by lower-case title, then key.
Consequences
- The grid of wells of the desktop application, the studio and the depth coverage walk the same members; a type added to the table is added for all.
- The domain gains a module and no dependency.
- Two wells of one title come out as two entries with the same title; a face that shows them must tell them apart.
0041 - A convention the file does not state is declared by the caller; a vertical without a CRS is signalled
Status: accepted
Context
Several readings around a well depend on a fact the files do not say: where an angle around the hole starts and which way it grows, which dimension of a per-node matrix is the slow one and the sign of its lateral axis, which hand a placed frame has. Two readings of the same array give two pictures, and both look plausible; only the producer knows which is right. A library that picks one draws something that looks like data.
Another case is a line read without any coordinate system. The conversion from
depth to z needs to know which way is up. Without a CRS the library has long
taken depth as downward, which is the usual case and also a guess.
Decision
- A convention the file does not state is a parameter, declared by the
caller (
AngleConvention: origin, sense, side from which it is seen, offset;TransverseGrid: the axis, the sign and the offsets of each dimension; the handedness of a frame inFrameOptions). Until it is declared, the layer that needs it is refused by name (ConventionUndeclared). There is no default and no “unknown” state: the caller choosesRegularor states a convention. It is registered ashouse/well/conventions-are-declared. - A declaration is never written into the package by these functions, and it never overrides what the file states (ADR 0032). Writing one is the business of an operation that edits a working copy, not of a reading.
- The canonical plane is fixed: as seen by an observer standing uphole and
looking down the hole,
rightto the right andhigh_sideup, a clockwise turn going from the top to the right. A convention is written against it. - A vertical without a CRS is kept as depth downward and said so.
Vertical::ofis unchanged;Vertical::readingandread_notedreturn aVerticalReadingwhoseVerticalBasisisCrs,NoCrs,UnreadableCrsorDeclared. The basis travels with the line (Path::vertical_basis,Path::vertical_note), andPlacedPath::newsets it withPath::with_verticalwhen the caller declares the system. A view says that the vertical was assumed instead of drawing it as if it were stated. - Placing a path without a CRS is refused (
PathRefusal::NoCrs); declaring one goes throughPlacedPath::new.
Consequences
- A caller with the producer’s documentation writes its convention once and gets the same picture every time; one without it gets a named refusal and not a plausible wrong ring.
- A convention such as the order of the axes of a vendor’s compound system is settled in a campaign on real files, never guessed by the library.
- A test that declares a convention states which reading it stands for.
- The signal on the vertical does not change a number: it adds a note that a view can show.
0042 - What a property says of itself
Status: accepted
Context
A view of a property shows its numbers (bounds, scale, variation along each
axis) and, beside them, what the file declares about it: its shape, its
facets, its unit, its mark of synthetic data. The standards describe the
dimensions of the values and the facets, but they leave several readings open,
and a library that does not choose lets each view choose for itself. Six of
those readings had to be settled for property::chunks and
property::observe.
Decision
- The dimensions reduced by
AlongReducedare reduced together. The pick keeps every component along one axis, fixes the other dimensions, and reduces the listed ones (mean,min,maxof the finite values, byReduction::apply, the one arithmetic of a reduction). A mean over two reduced dimensions is the mean of the finite values of the whole sub-block, not a mean of means: with a missing value the two differ, and the first is the mean of the values that exist. - A property that declares several values per element and holds one is
described as scalars, under
ComponentPick::Allonly. The rule is the one the scene already applies (declaration wrong, values usable). Under any other pick the property is refused (ChunksRefusal::OnePerElement): a component cannot be chosen among values that do not exist. The observationDeclaredCountWrongsays it. - The list of facet values of EML 2.3 does not bind a RESQML 2.0.1 value.
In 2.0.1 the value of a facet is free text (RESQML TRG 2.0.1 §6.36 p. 77).
In 2.2 a standard value found under a facet kind other than the one the
schema documents for it is said (
FacetValueOfOtherKind); in 2.0.1 nothing is said. Several facets of one kind are said (FacetKindRepeated, EML TRG 2.3 §11.11 p. 294), and not refused. - The mark
syntheticis read astrue,falseor something else. AnExtensionNameValuenamedsynthetic(ADR 0019) is a mark when its value istrue, no mark when it isfalse, andUnclearotherwise, with the value as written. It is read on the property and on its supporting representation. It is never deduced from a title, and nothing is said of an object that carries none: the absence of a mark is not a proof that the data is measured. - The observations of a property live beside those of its values.
stats::describe::Observationsays what the values are (counts, scale, modes, axes) and is computed while they are read.property::observe::PropertyObservationsays what the file declares and is computed without reading a value. No fact is written twice. The unit and the kind are carried byUnitCheck(checked, outside the quantity class, or not checkable and why) and are never reported as checked when the class is one this build does not convert. - The ratio of two sister properties is a figure, not a judgement.
PairWalkcounts the pairs, the density, the ratioy / x, the share that satisfies a rule the caller gives, and the correlation. That a share is a warning, and which sister is the vertical one, is the rulehouse/property/directional-ratio, not this code.
Consequences
- A view that shows a property has two lists to show, the observations of the values and those of the file, and neither repeats the other.
- A defect of the file (a repeated facet, a wrong declared count) is reported, and the property stays readable: the library helps repair such files rather than refuse them.
- A new kind of observation is a new variant (
PropertyObservationis#[non_exhaustive]), not a change to the observations of the values. - The reading of a 2.0.1 facet value stays open to a producer’s own words, at the price that a misplaced standard value there is not caught.
TODO(verify): whether the packages in the field carry thesyntheticmark; until they do, most properties are silent on it.
0043 - The views of a package are drawn through one door, as a display list written to SVG
Status: accepted
Context
The library computes much of what a view of a package shows: a well’s line laid flat and the depth on it, the
values of a property along a frame, the graph of the activities that wrote the package, the links between tables.
Only one view went as far as a picture (the geosteering section, energyml_section::along_well), and no command
drew it. A command line, a desktop window and a web page each need the same pictures; drawn in each face, a picture
would be drawn three ways, and the SVG exported by one would not be what another shows.
Decision
- A crate of its own,
energyml-views, above the crates that compute (domain, geometry, cache, sheet’s drawing, section): nothing below it depends on it, so no edge is added between the crates that compute. - One registry of views (
ViewKinds, the caller’s first): each view declares the objects it takes (slots of types, by their name without release), the settings it reads (nature, bounds, default, sentence), what it can show in a package from the index alone (offers), and draws aDisplayListfrom what the other crates compute. A view lays out; it does not compute a second time. - One door (
draw_svg): the request (ViewAsk) is checked against the view’s declarations - an object the package does not hold, of a type no slot takes, a setting not declared, out of its bounds or of another nature, each is refused by name - drawn, and written as SVG with the request, completed with its defaults, in the metadata.redrawreads it back and draws again: the same bytes over the same package. A face that draws the list itself asksdraw_list. - A PNG is the SVG rasterised (
png, behind thepngfeature), never a second drawing. - Colours by one door: the map the package gives the object (
energyml_cache::colours, the walkenergyml colorsprints), else a palette ofenergyml_domain::palettes, named in the legend as not from the file; a palette the request names wins. No colour is invented: a value the map has none for stays transparent. - The command line asks the same door:
energyml view list | about | draw | redraw,--svgwritten,--pngon demand. It holds no logic.
Consequences
- A view is one file and one registry line; a face gains it without a line of its own.
- The settings of a view resemble the slots of an operation of
energyml-buildwithout their provenance; they are a type of their own here, so thatenergyml-viewsdoes not depend onenergyml-build. Bringing the two together is left for later. - The well sheet of
energyml-sheetand the geosteering section keep their computing; their pictures move to the display list and the views’ metadata.
0044 - The vectors in geostat
Status: accepted
Context
The cross product, the dot product, the norm and the distance were written in
more than a dozen private places: four modules of energyml-geostat, three of
energyml-analysis, min_curvature in energyml-geometry, three crates for a
distance, and a few 2D variants. energyml_geometry::vector claimed to be the
one place, but energyml-geostat depends on no EnergyML crate (and must stay so)
and could not use it.
Decision
The module moves down to energyml_geostat::vector: 3D vectors on [f64; 3]
(add, sub, scale, dot, cross, norm, distance, unit_above, unit,
distance_to_chord) and, in vector::plan, the 2D companions on [f64; 2]
(add, sub, scale, dot, cross, norm, distance, segment_distance,
distance_to_segment, orient, on_segment, segments_meet).
energyml-geometry depends on energyml-geostat (pure Rust, no native
dependency, compiles to wasm) and energyml_geometry::vector is
pub use energyml_geostat::vector::*, so its public API does not change.
energyml-geostat still depends on no EnergyML crate.
A function of the same name that does something else (a three-point cross
giving an area, a vector product that is not the right-handed one) is not a
copy and keeps a name that says what it does.
Consequences
- One definition of each helper; a change of handedness or of a floor is made once.
energyml-geometrygains a dependency onenergyml-geostat; the reverse never happens.
0045 - Text and numbers written by hand have one copy each
Status: accepted
Context
A review of the workspace found the same small helpers written again in crate after crate, and the copies had
drifted. XML escaping existed in nine places: the SVG writers of energyml-section and energyml-sheet dropped the
control characters XML 1.0 cannot carry; the VTK, Galaxy, core.xml, [Content_Types].xml and AnyXml writers
did not, and wrote a document no parser reads. Two of them did not escape >. The XML unescape was written twice,
one copy cutting an error message at a byte offset that is not always a character boundary. Base 64 existed twice,
the civil-date arithmetic three times, the short number formatter in six views (one of them mapping -0 to 0,
five not, and the text goes into element keys such as #md=...), the plain number formatter twice, and a JSON
string escaper in two crates. One copy, format_number of the sheet reader, printed 1e20 as i64::MAX.
Decision
One definition each, in the lowest crate that serves every caller:
| What | Where |
|---|---|
| XML text, attribute, unescape | energyml_core::xml::{escape_text, escape_attr, unescape} |
| JSON string | energyml_core::json::quote (written by serde_json) |
| Base 64 | energyml_core::base64::{encode, decode} |
| Civil calendar | energyml_core::calendar::{days_from_civil, civil_from_days} |
| A number without useless zeros | energyml_core::text::plain_number |
A number written short, n decimals | energyml_sheet::text::short |
The rules the doors settle, where the copies disagreed:
- Text escapes
&,<,>; an attribute value also"and'(XML 1.0, 2.4 and 3.1). The SVG writers go throughescape_attrfor both, so their output does not change. - A character XML 1.0 cannot carry is dropped, in text and in attributes (production
Char, 2.2): a title from a file may hold anything, and a missing control character is better than an ill-formed document. unescaperefuses by name an&with no;, an entity XML 1.0 does not predefine, and a reference to a character it cannot carry (�).shortnever writes-0, and writes a non-finite value as Rust does; the SVG numbernum(a coordinate that is not a number is not an SVG) writes it0.plain_numberwrites an integer only below 1e15: beyond it the cast saturates.
The old names stay where they were published (energyml_domain::attachments::{encode_base64, decode_base64},
energyml_section::svg::escape, energyml_sheet::draw::svg::escape) and call the door.
Consequences
- A fix to escaping, base 64 or the calendar is made once.
energyml-coregains no dependency: base 64 and the calendar are small and written out;quoteuses theserde_jsonit already has behind thejsonfeature.- Not unified, on purpose: the rules crate keeps its own JSON escaper (it must stay dependency-free); the JSON
number policy for
NaNdiffers between the MCP tools ("NaN") and the series reader (null), each a contract of its consumer;energyml_domain::raw_xml::decodeis a lenient reader of the five entities for a byte-level scanner.
0046 - Reading a field of a reflected object goes through one door
Status: accepted
Context
Every reader of the model that walks a &dyn Reflect asks the same few
questions: what text does this field hold, which number, is the flag set, which
nested object, which objects of a repeated field. About forty private functions
answered them, one set per module (text, scalar_f64, flag, objects,
field_object, string_of, number_of…), in energyml-geometry,
energyml-domain, energyml-storage and energyml-build.
The copies did not agree, and the same field therefore read differently depending on which check looked at it:
- a blank string: kept by one copy, dropped by another when empty, by a third when only whitespace;
- a fractional double asked for as an integer: truncated by some, refused by others;
- a string asked for as a number: parsed by one copy, refused by the rest;
- a field of the wrong shape: sometimes
None, sometimes a default.
Decision
- One module,
energyml_core::reflect::fields. It has no native dependency (it builds forwasm32). The private copies are deleted; where a module kept a local name for readability it re-imports the door’s function under that name, never a second implementation. - The policy is stated once, in the module’s documentation:
- absent (no such field, unset, undeclared name) is
None, or an emptyVecfor the plural readers; never zero,falseor the empty string; - a blank string (empty or whitespace only) is absent for the single
textreader; the plural readers keep every element, because in a list the position is the information; - a field of the wrong shape is
None: the strict readers never parse a string as a number, and a double is an integer only when it is finite, whole and in range; - reading a number out of text is a named variant (
number_lexical,integer_lexical,flag_lexical), asked for only where a schema version keeps the value as text; - a caller that must refuse says so: the
require_*readers return a named error,MissingRequiredfor an absent field andTypeMismatchfor a field of another shape, instead of the caller inventing a default.
- absent (no such field, unset, undeclared name) is
- The reflection walks of
identity.rsshare one traversal.collect_refs,nested_objectsandcollect_refs_by_ownerare visitors of a singlewalk(references, kept-verbatim subtrees, entered objects). The dotted path is one buffer, pushed and truncated, not aformat!per field. - An error that names a type borrows it. The geometry readers passed
Box::leaked copies of&'static strtype names to their errors; the names are already'staticand are passed as they are.
Consequences
- A change of policy (what a blank means, what an integer is) is made in one place and applies to every reader.
- A reader that wants the old lenient behaviour on a field kept as text asks for the lexical variant by name; it does not get it by accident.
- Behaviour that changed where the copies disagreed:
textnow drops a blank string in every reader (it used to be kept byintegrity,property,valuesand the geometrywell/parentreaders), and returns the wire text of a non-string scalar in every reader (the geometry andvaluesreaders were string-only);- an integer read from a fractional double is now
None(it was truncated by the geometry,valuesandcrsreaders); a whole double still reads; propertyno longer truncates a non-negative fractional double to a count.
- A new reader of the model does not write a helper: it imports the door.
0047 - A view may propose a value the file does not state, and says so
Status: accepted
Context
Some views need a value the file never states: the roles of the two dimensions of a curtain, the table that holds its coordinates. Following ADR 0005 and ADR 0041, a request that does not give them is refused. That is right for a recipe, and unhelpful to a person who opens a view for the first time: a refusal shows nothing, and nothing says what to give.
Decision
- A view can propose (
ViewKind::assist, reached byenergyml_views::assist): values for settings the file does not state, each with the reason it is proposed (Proposal), and the dimensions it needs declared with the candidates the package offers for each (DimensionAxes,AxisColumn). It reads the package index and the shape of the tables only, never an array. - A proposal is not a reading. It is applied only to a setting the request does not give, only by a face that
asked for it (
energyml view draw --proposed, a form’s first drawing), and it is said wherever it shows: a note on the command line, a mark and the reason in a form. A value the person gives replaces it. - Several candidates are a choice, never a guess. A table is proposed for the links of a curtain only when it is the one table that fits (one key column, one numeric value column per dimension, the right rows); with two, both are candidates and nothing is proposed. The roles are a declaration, proposed as such.
- The default of an index is not a proposal.
lateral_index, left out, is the middle row of the lateral dimension, and the help says so; row 0, the edge of the cube, was off the plane of the hole. - The wire shape of the links has one writer (
axis_links_value) and one reader (axis_links_read), so no face writes the JSON itself.
Consequences
- A first drawing exists for a package with one table of axes, and what is proposed is visible and replaceable.
- A package with several tables never gets a silent choice.
- A view that has nothing to propose costs nothing: the default method answers an empty
Assist.
Addendum (16/11/2026): names, parts, and the offers
- A view has a name (
ViewKind::name,ViewKinds::name_of) and anOfferapartand a listproposed: whatassistproposes leavesmissing, so a face does not show as a gap what it will fill. An offer is built from the index alone and saysproposedonly for whatassistis sure to propose (it errs on the side ofmissing). targetproposes the first node of the frame asmd, the angle convention (high_side,clockwise,looking_downhole, a declaration) and, for the sectors, the one property of angles that pairs with the property (angles) or elseregular_sectors.