Datalove Literals

Datalove Literals is a typed declarative expression language for serializing, storing and transmitting common data types. We often refer to it as datalit, and its types as datalit types.

Datalove Literals is the data sublanguage of Datalove: every datalit expression is written the same way in Datalove, where it means the same value. Datalit has no names, no operators and no computation. Datalit files use the .dlt extension. The format is intended for use in the Datalove ecosystem and not as a general-purpose serialization format.

This document is an informal specification for Datalove Literals and serves as a gentle introduction to the data model and syntax of Datalove in general.

Contents

An example

{
  name = "datalove",
  version = (0, 1, 0),
  unit = (),
  enabled = true,
  score = 99.5,
  flags = 0xFF,
  tags = ["fast", "typed", "portable"],
  counts = %{ "a" = 1, "b" = 2, "c" = 3 },
  ids = #{ 10, 20, 30 },
  matrix = [| 1.0 0.0, 0.0 1.0 |],
  cube = [|
    1 2,
    3 4,,
    5 6,
    7 8
  |],
  config = some {
    retries = 3,
    timeout = 30.0,
  },
  metrics_table = {|
    name, value
    "latency", 0.5
    "throughput", 1000.0
    "duration", 25.0
  |},
  backup = : ?string / none,
  status = ok "healthy",
  failure = : !string / er error "oops",
  state = atom Ready,
  event = term Click (100, 200),
  kind = : enum {
    atom Observation,
    term Custom string,
  } / enum {
    term Custom "experiment",
  },
  payload = data [1, 2, 3],
}

Lexical structure

Source text is UTF-8.

Whitespace is any Unicode whitespace character. It separates tokens and is otherwise insignificant, with two exceptions: newlines separate table rows, and spaces separate the elements of a tensor row. A newline is \n; a \r before it is ordinary whitespace. A ; is a line break written on the line: it is a newline everywhere, separating table rows and otherwise insignificant.

Comments are either // to end of line, or /* ... */, which nest. Comments are whitespace.

Identifiers name struct fields, table columns, and atom, term and enum variants. An identifier is a run of ASCII letters, digits and _ not beginning with a digit. Strings and comments may hold any text, but names are ASCII: café and x² are errors.

These words start an expression:

true  false  none  some  ok  er  data  error  atom  term  enum

In type position the primitive type names, bool, int, string, u8 .. u64, i8 .. i64, index, offset, f32, f64, data, error, and atom, term and enum are types or start one.

No word is reserved everywhere. Datalit declares no names, and a struct field, table column or variant may be any identifier, since only a name can appear there: { none = 1 } is a struct.

Tokens that are written against each other are glued. Gluing decides how far a numeric literal reaches: a number is a maximal glued run of sign, digits, point and exponent, so -1.5e-7 is one literal and - 1, 1 . 5 and 2.5e - 10 are errors. Letters glued to the end of a number are an error, as datalove has no numeric suffixes: 1u8 is written : u8 / 1. Inside a tensor, gluing is what separates [| 1 -2 |], two elements, from [| 1 - 2 |], an error.

EBNF · Lexical

ws             = { whitespace | newline | comment } ;
whitespace     = ? any Unicode whitespace character ? ;
newline        = "\n" | ";" ;
comment        = line_comment | block_comment ;
line_comment   = "//", { ? any char except newline ? } ;
block_comment  = "/*", { block_comment | ? any char ? }, "*/" ;

ident          = ident_start, { ident_start | digit } ;
ident_start    = "a" .. "z" | "A" .. "Z" | "_" ;
digit          = "0" .. "9" ;
hex_digit      = digit | "a" .. "f" | "A" .. "F" ;

Expressions

All expressions may be prefixed with a type hint, : type / expr.

Expressions are typechecked with a simple bidirectional discipline, drawing their expected type from a type hint or synthesizing a type directly. Most expressions synthesize types unambiguously. Type hints are sometimes required, in predictable positions. For many uses datalit expressions are checked against an external type context.

// A single top-level type hint.
: {
  name: string,
  version: (u32, u32, u32),
  tag: (bool,),
  counts: %{string = int},
} / {
  name = "datalove",
  version = (0, 1, 0),
  tag = (true,),
  counts = %{ "a" = 1, "b" = 2, "c" = 3 },
}

// Embedded type hints.
{
  name = : string / "datalove",
  limits = {
    low = 1,
    // A deeply-embedded hint.
    high = : u32 / 3,
  },
  kind = : enum {
    atom ProcessData,
    term Custom string,
  } / enum {
    term Custom "experiment",
  },
}

Types are generally spelled the same way as their values, but with the type sigil, :, replaced with the value-assignment sigil, =.

: {
  retries: int,
  timeout: f32,
} / {
  retries = 3,
  timeout = 30.0,
}

A hint covers the one expression after its /. An expression carries at most one hint; : u32 / : u32 / 1 does not parse. Parentheses group without changing meaning: (1) is 1, and (: u32 / 1) is : u32 / 1.

A type hint may not name a type beyond the built-ins: datalit has no type aliases, so any bare name in a type is an error. Collections are written with their sigils, and tuple, list, map, set, table and tensor are not types.

A datalit document is a single expression. Nothing but whitespace may follow it.

Anything written between brackets and separated by commas, in a value or a type, may end with a comma: [1, 2,], { x = 1, }, : [u32,] / [], enum { atom A, }. The comma closes the last item and means nothing more, so one is allowed and two are not. Tensors are the exception, as their commas carry shape; see Tensor.

EBNF · Expressions

datalit        = ws, full_expr, ws ;
full_expr      = [ type_hint, ws ], expr ;
type_hint      = ":", ws, type, ws, "/" ;
type           = primitive_type
               | tuple_type
               | struct_type
               | option_type
               | result_type
               | list_type
               | map_type
               | set_type
               | table_type
               | tensor_type
               | atom_type
               | term_type
               | enum_type
               | dynamic_type ;
expr           = primitive_lit
               | group_expr
               | tuple_expr
               | struct_expr
               | option_expr
               | result_expr
               | list_expr
               | map_expr
               | set_expr
               | table_expr
               | tensor_expr
               | atom_expr
               | term_expr
               | enum_expr
               | dynamic_expr ;
group_expr     = "(", ws, full_expr, ws, ")" ;

Primitive types

index is an unsigned integer representing the addressable size of collection types, offset is the signed version of the same size, both 32-bit by default, 64-bit by compile-time option. They are distinct types from the fixed-width integers of the same size.

Integers

Bare integer literals synthesize as int, the big integer type, which has no range limit. Fixed-width integers require a type hint or checking context, and a literal out of the range of the type it checks against is an error.

Underscores may group digits and say nothing about the value. A separator goes between two digits: 1_000_000 is an integer, 1_ is an error, and _1 is an identifier.

A sign is written glued to the digits: -1. There is no + sign. -0 is zero.

Floats

A float literal is written with a point, an exponent, or both: 42.0, 4.2e1, 1e-7, 6.022E23. Digits are required on both sides of a point, so 1. and .5 are errors. The exponent marker is e or E, and its sign is optional. Separators group digits in any run: 1_0.000_1e1_0.

Float literals synthesize f64, and check against f32 or f64. An integer literal does not check against a float type, nor a float literal against an integer type.

A decimal literal becomes the float nearest its exact value, ties to even, converted directly to the target width. A literal too large for the target width becomes infinity: : f32 / 1e300 is positive infinity. -0.0 is negative zero.

Hex literals

Integers and floats can be created from hex literals, written 0x or 0X and hex digits of either case, which may be grouped with underscores. Hex literals synthesize int, but check as int, unsigned fixed ints, index, or floats if provided a type hint. Hex literals do not check against signed fixed-width types or offset. A hex literal takes no sign: -0x10 is an error. Write a negative number in decimal.

A hex literal checked against a float type is its IEEE 754 bit pattern, and must fit in the float's width: 32 bits for f32, 64 for f64. This is the only way to write bit-exact floats, NaNs and infinity.

: {
  foo: int,
  bar: u32,
  baz: f32,
  inf: f64,
} / {
  foo = 0x01,
  bar = 0x02,
  baz = 0x3F80_0000,         // 1.0
  inf = 0x7FF0000000000000,
}

Strings

A string literal is written between double quotes, and holds any characters but an unescaped " or \, including literal newlines. The value is a sequence of Unicode scalar values. No normalization is performed.

Any other escape is an error, as is a \u{..} naming a surrogate or a value above 10FFFF.

EBNF · Primitive types

primitive_type = "bool"
               | "u8" | "u16" | "u32" | "u64"
               | "i8" | "i16" | "i32" | "i64"
               | "index" | "offset"
               | "f32" | "f64"
               | "int" | "string" ;

primitive_lit  = bool_lit | numeric_lit | string_lit ;
bool_lit       = "true" | "false" ;
numeric_lit    = float_lit | hex_lit | int_lit ;
int_lit        = [ "-" ], digit_run ;
float_lit      = [ "-" ], digit_run,
                 ( ".", digit_run, [ exponent ] | exponent ) ;
exponent       = ( "e" | "E" ), [ "+" | "-" ], digit_run ;
hex_lit        = "0", ( "x" | "X" ), hex_run ;
digit_run      = digit, [ { digit | "_" }, digit ] ;
hex_run        = hex_digit, [ { hex_digit | "_" }, hex_digit ] ;

string_lit     = '"', { string_char }, '"' ;
string_char    = escape_seq | ? any char except '"' and '\' ? ;
escape_seq     = "\", ( '"' | "\" | "n" | "r" | "t" | "0"
                      | "u{", hex_digit, { hex_digit }, "}" ) ;
                      (* 1 to 6 hex digits *)

A numeric literal contains no whitespace or comments.

Collections

An empty collection leaves its element type open, so it does not synthesize a type: [], #{}, %{}, an empty tensor, and a table with no rows need a type hint or checking context, as none does. : [u32] / [] is valid, and [] alone is an error.

List

An ordered sequence of homogeneous elements.

[1, 2, 3]                      // [int]
: [u32] / [1, 2, 3]            // [u32] via type hint
: [string] / []                // [string], empty with hint
[: u8 / 1, 2]                  // [u8]

All elements have the same type. Without an expected type, the elements are typed together: literals go along with any element that has a definite type, and take their defaults if none does. [: u8 / 1, 2] is a [u8], [some 1, none] is a [?int], and [: u8 / 1, : u16 / 2] is an error.

Map

A key-value mapping, ordered by key.

%{ "a" = 1, "b" = 2 }          // %{string = int}
: %{u32 = string} / %{ 1 = "x", 2 = "y" }
: %{string = u32} / %{}        // empty map

Entries use = to separate keys from values, same as struct field assignment. All keys must have the same type, and all values must have the same type. Any type may be a key.

A map's entries are kept in the total ordering of their keys, not in the order they were written: %{ 2 = 0, 1 = 0 } and %{ 1 = 0, 2 = 0 } are the same map. A key written more than once keeps the last value written for it.

Set

A collection of unique elements, ordered by value.

#{ 1, 2, 3 }                   // #{int}
: #{u32} / #{ 10, 20, 30 }
: #{u32} / #{}                 // empty set

All elements must have the same type. Any type may be an element. A set's elements are kept in their [total ordering], and an element written more than once is held once.

Table

A data structure with named, typed columns.

: {| x: int, y: string |} / {|
  x, y
  1, "a"
  2, "b"
|}

The first row names the columns; subsequent rows provide data. Rows are delimited by newlines or semicolons.

: {| x: int, y: int |} / {| x, y; 1, 2; 3, 4 |}

Without a type hint, each column's elements are typed together, as a list's are, and the table synthesizes those column types: {| x; 1; 2 |} is a {| x: int |}. A table with columns and no rows leaves its column types open, so it needs a type hint. Checked against a table type, the column names in the literal must match the type's in order. Each column is named by exactly one identifier, and no two columns share a name. Each data row must have exactly as many values as there are columns. A table with no rows is written with just its header row, and the table with no columns, of type {| |}, is written {| |}.

A semicolon is a newline, so a run of them is blank lines, which are free: {| x, y;; 1, 2; |} is a table with one row. A comma with no column before it is an error: {| x,, y |} does not parse. A comma at the end of a row closes the value before it and is fine.

Tensor

A multi-dimensional array with fixed shape, typed by element type and rank.

[| 1 2 3 |]                    // 1D, shape [3]
[| 1 2 3, 4 5 6 |]             // 2D, shape [2, 3]
[| 1 2, 3 4,, 5 6, 7 8 |]      // 3D, shape [2, 2, 2]
: [|u32, 2|] / [| 1 2, 3 4 |]  // typed: element u32, rank 2
: [|u8, 1|] / [| |]            // empty tensor, rank 1, shape [0]
[| 2 3 | 1 2 3 4 5 6 |]        // shape [2, 3], given in a header
: [|u8, 2|] / [| 0 3 | |]      // empty tensor, shape [0, 3]

Shape is inferred from the multi-comma structure: spaces separate elements along the innermost axis, , separates rows (2nd axis), ,, separates slabs (3rd axis), ,,, separates blocks (4th axis), and so on. The rank is one more than the longest run of commas written. Whitespace between the commas of a run does not break it: , , is ,,. Every group along an axis must have the same shape, so the element count equals the product of the shape dimensions.

Higher-dimensional tensors benefit from multiline layout, using blank lines to visually separate the higher axes:

// 3D tensor with shape [2, 3, 3]
: [|f64, 3|] / [|
  1.0 0.0 0.0,
  0.0 1.0 0.0,
  0.0 0.0 1.0,,

  2.0 0.0 0.0,
  0.0 2.0 0.0,
  0.0 0.0 2.0
|]

A comma run goes between two parts and nowhere else. One with nothing before it or after it separates nothing, so [| ,1 2 |] and [| 1 2, |] do not parse: unlike every other bracketed list, a tensor takes no trailing comma, since its commas say its shape. Blank lines are whitespace here and mean nothing to the shape, which is what lets the layout above breathe.

Some shapes cannot be shown by separators: an axis of extent 0 above the innermost one has no parts to separate, and a leading axis of extent 1 has no separator to show it. These are written with a shape header, the extents as whole numbers and then a |:

[| 1 3 | 1 2 3 |]              // shape [1, 3]
: [|u8, 2|] / [| 0 3 | |]      // shape [0, 3], empty
: [|u8, 2|] / [| 2 0 | |]      // shape [2, 0], empty, and not the same value
[| 2 2 | 1 2 3 4 |]            // shape [2, 2], body written flat
[| 2 2 | 1 2, 3 4 |]           // shape [2, 2], body shaped as the header says

Any tensor may have a header. Under one, the body is written flat, with no commas, filling the shape in row-major order, or shaped exactly as the header says; a body shaped any other way is an error. A header is the simple way to write out a large tensor: the extents, a |, and every element.

A tensor has at least one axis. There are no rank-0 tensors, so neither [|T, 0|] nor an empty header, [| | 7 |], is accepted.

The type specifies element type and rank: [|T, N|]. Shape is not part of the type — two tensors of the same element type and rank but different shapes have the same type. All elements have the same type, and are typed together as a list's are. An empty tensor needs a type hint for its element type; its rank is its shape's, hint or not.

EBNF · Collections

list_type      = "[", ws, type, [ ws, "," ], ws, "]" ;
map_type       = "%{", ws, type, ws, "=", ws, type, [ ws, "," ], ws, "}" ;
set_type       = "#{", ws, type, [ ws, "," ], ws, "}" ;
table_type     = "{|", ws, [ type_field_list ], ws, "|}" ;
tensor_type    = "[|", ws, type, ws, ",", ws, rank, [ ws, "," ], ws, "|]" ;
rank           = digit, { digit } ;  (* not zero *)

list_expr      = "[", ws, [ expr_list ], ws, "]" ;
map_expr       = "%{", ws, [ entry_list ], ws, "}" ;
set_expr       = "#{", ws, [ expr_list ], ws, "}" ;
table_expr     = "{|", tbl_ws,
                 [ table_header,
                   { ws, newline, tbl_ws, table_row } ],
                 tbl_ws, "|}" ;
tensor_expr    = "[|", ws, [ tensor_header ], [ tensor_body ], ws, "|]" ;
tensor_header  = extent, { ws, extent }, ws, "|", ws ;
extent         = digit_run ;

expr_list      = full_expr, { ws, ",", ws, full_expr }, [ ws, "," ] ;
entry_list     = entry, { ws, ",", ws, entry }, [ ws, "," ] ;
entry          = full_expr, ws, "=", ws, full_expr ;

table_header   = ident, { ws, ",", ws, ident }, [ ws, "," ] ;
table_row      = full_expr, { ws, ",", ws, full_expr }, [ ws, "," ] ;
tbl_ws         = { ws | newline } ;

tensor_body    = tensor_row, { ws, comma_run, ws, tensor_row } ;
tensor_row     = full_expr, { ws, full_expr } ;
comma_run      = ",", { ws, "," } ;

Within a table, outside any brackets nested in it, ws does not include newlines, which separate rows; a newline inside a block comment is not one. Some rules for these forms cannot be expressed in EBNF:

• every table row has exactly as many values as the header has columns, and the column names match the type hint's in order;
• every group along a tensor axis has the same shape, so [| 1 2, 3 |] is refused;
• a tensor's rank is its longest comma run plus one, and shorter runs nest inside longer ones;
• a tensor header's extents multiply to the number of elements in a flat body, or equal the shape of a structured one.

Aggregates

The payload of some, ok, er, term, data and error is one expression, which may carry its own hint: some : u32 / 1.

Tuple

An ordered sequence of heterogeneous values. Two tuples are the same type if they have the same length and element types in the same order.

()                             // unit: the zero-element tuple
(true,)                        // 1-tuple (trailing comma required)
(true, 42)                     // (bool, int)
(1, "hello", 3.14)             // (int, string, f64)
: (u32, i32) / (1, -1)         // with type hint

Unit () is both a type and a value. A 1-tuple requires a trailing comma to distinguish it from a parenthesized expression. Types have no grouping parentheses, so in a type (T) is the same 1-tuple type as (T,). Each element synthesizes its type independently: (true, 42, 3.14) synthesizes as (bool, int, f64).

Struct

A collection of named fields. Two structs are the same type if they have the same field names, types, and order.

{ x = 1, y = 2 }               // {x: int, y: int}
: { x: u32, y: f32 } / { x = 1, y = 2.0 }
{}                             // the empty struct

Types use : between field names and types, while expressions use = between field names and values. Field order matters — {x: u32, y: bool} and {y: bool, x: u32} are different types. Checked against a struct type, a struct literal must give every field, in the type's order. Field names are unique within a struct. Each field value synthesizes its type independently.

Option

An optional value: either present or absent. The type is written ?T with a prefix ?.

some 42                        // ?int
some "hello"                   // ?string
: ?u32 / some 1
: ?u32 / none
: ??u32 / some none

some wraps a value; none represents absence. some e synthesizes as ?T where T is the type of e. none cannot synthesize a type — it requires a type hint or checking context. A value does not check against an option type without some: : ?u32 / 1 is an error.

Result

A success-or-failure value. The type is written !T with a prefix !. The failure side always holds an error.

ok 42                          // !int
: !u32 / ok 1
: !u32 / er error "oops"

ok wraps a success value; er wraps an error. ok e synthesizes as !T where T is the type of e. er cannot synthesize a type — it requires a type hint or checking context. The payload of er is checked against error, so it is written as an error expression, er error "oops", and is the error the result holds. An error is not a result by itself: : !u32 / error "oops" is an error.

Atom

A named unit type with no payload. An atom is both a type and a value — the syntax is the same in both positions.

atom Red                       // type and value
atom Blue

Two atoms are the same type if they have the same name.

Term

A named type carrying a typed payload.

term Foo 42                    // term Foo int (value)
term Bar "hello"               // term Bar string (value)

In type position, the payload is a type: term Foo int. In expression position, the payload is a value: term Foo 42. Two terms are the same type if they have the same name and the same payload type.

Enum

A closed union of atom and term variants.

: enum { atom Red, atom Blue, term Custom string } /
  enum { atom Red }

: enum { atom Red, atom Blue, term Custom string } /
  enum { term Custom "hello" }

The type lists all variants; the expression provides a single variant, which must be one the type lists. Variant names are unique within an enum. Enum type equivalence compares variants by name, regardless of the order they are declared. Enum literals require a type hint.

An atom or term checks against any enum that lists it, so the enum { } around a variant may be left off where the enum type is expected:

: [enum { atom Red, atom Blue, term Custom string }] /
  [atom Red, term Custom "hello", enum { atom Blue }]

This is not subtyping. An atom literal is a value of every enum that lists it, as 5 is a value of every integer type that holds it, and in both cases it is the literal that is flexible, not the type: an atom Red value is not an enum value, as a u8 value is not an int value, and in Datalove each is converted with @.

EBNF · Aggregates

tuple_type     = "(", ws, [ type_list ], ws, ")" ;
struct_type    = "{", ws, [ type_field_list ], ws, "}" ;
option_type    = "?", type ;
result_type    = "!", type ;
atom_type      = "atom", ws, ident ;
term_type      = "term", ws, ident, ws, type ;
enum_type      = "enum", ws, "{", ws, enum_variant_list, ws, "}" ;

tuple_expr     = "(", ws, ")"
               | "(", ws, full_expr, ws, ",", ws, ")"
               | "(", ws, full_expr, ws, ",", ws, expr_list, ws, ")" ;
struct_expr    = "{", ws, [ field_list ], ws, "}" ;
option_expr    = "none" | "some", ws, full_expr ;
result_expr    = ( "ok" | "er" ), ws, full_expr ;
atom_expr      = atom_type ;
term_expr      = "term", ws, ident, ws, full_expr ;
enum_expr      = "enum", ws, "{", ws, ( atom_expr | term_expr ), [ ws, "," ], ws, "}" ;

type_list      = type, { ws, ",", ws, type }, [ ws, "," ] ;
type_field_list= type_field, { ws, ",", ws, type_field }, [ ws, "," ] ;
type_field     = ident, ws, ":", ws, type ;
field_list     = field, { ws, ",", ws, field }, [ ws, "," ] ;
field          = ident, ws, "=", ws, full_expr ;
enum_variant_list = enum_variant, { ws, ",", ws, enum_variant },
                    [ ws, "," ] ;
enum_variant   = atom_type | term_type ;

Dynamic types

A data value holds a value of any type together with that type. data e synthesizes data, and its payload e must synthesize a type of its own, which is the type the data carries: data 1 holds an int, data : u32 / 1 a u32, and data none and data [] are errors.

An error is the same thing under a different type, used for the failure side of results. error e synthesizes error, and becomes a result's failure under er: er error e.

A value is only a data or error when written as one: : data / 5 is an error, and is written data 5.

EBNF · Dynamic types

dynamic_type   = "data" | "error" ;
dynamic_expr   = ( "data" | "error" ), ws, full_expr ;

Type equivalence

All datalit types are structural: two types are the same when they are spelled the same, with these rules for the aggregates.

Type synthesis and checking

Every expression either synthesizes a type from itself, or is checked against an expected type from a type hint, enclosing expression, or external type context. An expression that synthesizes can always be checked: it checks against a type if it synthesizes that type. There are no implicit conversions: an integer does not become a float, a value does not become an option, and a u8 does not become a u32. An expression with a hint has the hinted type, and checks only against that type: : {a: u32} / {a = : u8 / 1} is an error.

An expression synthesizes a type only when its form determines the whole type. A literal checks against every type that holds the value it writes, and synthesizes a default: 5 checks against u8 and synthesizes int. An expression whose form leaves part of its type open, like none or [], does not synthesize at all.

The table with no columns, {| |}, leaves nothing open and synthesizes {| |}.

So a type hint or context is required for none, er, enum literals, empty collections, and fixed-width numbers, index, offset and f32; and for anything containing one of these where it would have to synthesize, as in data [], some none, or a struct { xs = [] } with no hint.

Typing together

Without a hint, a collection's type comes from all of its elements together. Literals like 1, 2.0 and none go along with the rest: if any element has a definite type, such as a hinted one, the literals are read as that type. If nothing decides, the literals take their defaults, int and f64, and none or [] with nothing to go along with needs a hint. The order of the elements doesn't matter.

[1, 2]                         // [int]
[1, : u8 / 2]                  // [u8]
[some 1, none]                 // [?int]
[[], [1]]                      // [[int]]
[none, none]                   // error: nothing says what the option holds
[: u8 / 1, : u16 / 2]          // error: two different types

A map's keys go along with each other, and so do its values, and so do the elements of each table column.

Comparison and total ordering

Every datalit type has a total ordering, so any value can be a map key or set element, and any two values of the same type can be compared. Values of different types are not compared.

The ordering is the one the runtime uses to keep maps and sets, so it is also what decides whether two keys are the same key.

Floats are totally ordered by their bit patterns, so -0.0 and 0.0 are different values and different keys, and a NaN is a value like any other, equal to itself and to no other NaN bit pattern. #{ 0.0, -0.0 } has two elements.

The order among the types a data or error may carry is unspecified. Enum variant names order by UTF-8 byte.

Canonical forms

Datalove Literals has no canonical serialized form as might be used for consistent hashing. There is no canonical form for floats. Will be revisited.

Required type hints

Without an expected type, an expression's type is worked out from its parts. Parts in the same position get one type between them. These share a position:

• the elements of one list, set or tensor, the keys of one map, the values of one map, and the elements of one table column;
• and the matching parts inside those: the payloads of the some elements of one list, the first elements of the tuples in one list, and so on.

The type they share is, in order of preference:

1. the type of any of them that has a definite type: one with a hint, or one whose form fixes its type, as a string, a bool or a data does;
2. otherwise, the default of the literals among them: int for integers, f64 for floats and for hex beside a float, and an atom's or term's own type, atom A or term A T;
3. otherwise, none: the parts are only none, er e, empty collections and enum literals, and a hint is required.

A part with nothing else in its position gets its own definite type or its default, and needs a hint if it has neither.

So a value needs a hint, on the part or on anything containing it, wherever this gives a type other than the value's. That happens in exactly these cases:

• A type no literal defaults to: an integer that is not an int (u8 .. u64, i8 .. i64, index, offset), an f32, a float written only as hex bit patterns (as a NaN or infinity must be), an atom or term of an enum type, or an enum literal. These need a hint unless a part in the same position has that type definitely. A literal beside them does not help, since it brings only its default.
• A type nothing in the value says: none, er e, an empty list, set, map or tensor, or a table with columns and no rows. These need a hint unless something in the same position has a type, and then they take it, even if it is only a literal's default. The table with no columns, {| |}, needs none.

[1, 2]                         // [int]: defaults, no hint needed
[1, : u8 / 2]                  // [u8]: the hinted element decides
: [u8] / [1, 2]                // [u8]: or one hint on the list
[none, some 1]                 // [?int]: none takes 1's default
[none, some atom Foo]          // [?atom Foo]: and atom Foo's
[[], [1]]                      // [[int]]
[none, some "a"]               // [?string]
[: enum { atom A, atom B } / atom A, atom B]
                               // the hinted element decides; atom B goes along
[1.5, 0x7FF8000000000000]      // [f64]: the NaN goes along with the float
[atom A, atom B]               // error: two defaults, no enum
[none, none]                   // error: nothing has a type

The payload of data and of error is typed on its own, as a whole document is, so nothing outside it says anything about it: : [data] / [data 1] holds an int, and data : u8 / 1 is how to hold a u8.

A hint names exactly the type of what it is on. So a hint at the root of a value covers every part of it but the payloads of its data and error values.

Bugs and open questions

Bugs

• int is not arbitrary precision when values are built. The checker accepts any integer for int, but instantiation parses decimal through i128 and hex through u128, so 170141183460469231731687303715884105728 fails with "number too large to fit in target type".
• Deep nesting overflows the checker's stack. A list nested about 700 deep aborts the process instead of reporting an error.

Open questions

• Whether a float literal that overflows to infinity, or underflows to zero, should be an error. It is silently rounded today.
• Whether duplicate map keys and set elements should be errors rather than resolved last-wins.
• Whether nesting depth is limited, and if so to what.
• Whether a byte order mark is allowed. Nothing strips one.
• The value printer writes non-finite floats as nan, inf and -inf, which do not parse. Printed fixed-width values have no hints, so they read back as int and f64. Round-tripping through the value printer is only exact for values whose types synthesize.
• Where a printer should put the hints required above: once at the root, or on each part that needs one.
• What are the rules for printing disambiguating floats, etc?
• Should dupe map/set keys be an error?
• Should grouping parens actually be allowed? Are they needed?
• Should structs require identical field order?
• Need to think harder about float total order and ergonomics.
• Spelling the 1-tuple (T) without a trailing , seems inconsistent.