Posts

In my previous round of benchmarking I noted that Datalove's system library iterates collections in an awkward way. It keeps a loop induction variable of type index, increments it each time around the loop, and indexes into the collection to get the element. Each index imposes a bounds check. Datalove is missing a basic language feature: the for loop. Adding one sounds simple, but without first-class references we can't make iterators a general-purpose library feature. Instead we have to build them into the language and runtime, teaching the compiler directly about every type that can be iterated.

I've decided to close out Checkpoint 1. The last remaining item for some time was to write sufficient human docs. The task is never over, and eventually one just has to call it done. So it's done for now.

The human docs consist of:

Recently I've been proving the performance potential of the language model with some simple benchmarks and a simple representative demonstration app, the details of which aren't important. I have been focusing on the bytecode interpreter and the JIT, including compilation time and the runtime. As with writing the high-level docs for the website, this dogfooding exercise has been highly effective, finding lots of bugs and semantic gaps.

I'm trying so hard to finish "checkpoint 1" and publish the source and website. I just want the website front page, and the three overview docs, Datalove Literals, Datalove Functions Primer, and Datalove Principles to read cleanly and competently. All the rest can be placeholders.

Writing these has shaken out many bugs at all levels of the language and compiler. Write some examples; run them through the compiler; fix the bugs. Pretty happy with that.

Next big blocker is the ability to make runtime calls from std.

I have a runtime with an interface that is accessible to the compiler, and we have intrinsics via the icall keyword, but Datalove functions can't themselves call native code.

Typically at this stage one might introduce a simple mechanism that allows std to declare the existance of foreign functions, have the compiler always link in the runtime so std can call those functions. The runtime functions needed by std just exist in the ether.

Following on the initial compile-time function evaluation (CTFE) work, Datalove now supports const parameter specialization. Functions can declare parameters as const, and the compiler will specialize the function for each unique combination of const argument values.

fun scale(const factor: int, x: int): int
    ret x * factor
end fun

const TWO: int = 2
const TEN: int = 10

let a = scale(TWO, 5)   // specialized for factor=2
let b = scale(TEN, 5)   // specialized for factor=10

Until now both interpreter and AOT used some runtime tracking method to determine if linear types needed to be dropped.

Now most bindings and all temporaries are tracked precisely and the IR describes exactly when to drop them.

All let statements are precise. All var statements are dynamically tracked.

out parameters use dynamic tracking to avoid dropping before initialization.

More analysis could improve precise tracking, at the cost of more analysis. This is a nice and simple baseline for correctness and efficiency.

Datalove has a restrictive type system that enables lots of fun features.

Because functions are pure, deterministic with no side effects, and we have a working interpreter, and no pointers, it's almost trivial to evaluate functions at compile time.

Getting this feature in early to inform the architucture. Initial feature const-evals all const statements, in scripts, script functions, modules, and module functions. After that works fully I'll try to add const parameter specialization.

The simple specialization that requires will hopefully inform full generics.