Four small language models trained on RDF instead of English, over one Pokémon knowledge graph, plus a browser to poke at them. A curiosity, and my first look at what happens when a model reads and writes the Semantic Web.

The four experiments

They share a tokenizer, an ontology, and one rule (below). They differ in what the model is asked to do. Each has its own, more specific write-up:

How it runs

Everything is client-side. There is no server and no API call. An ONNX Runtime WebAssembly session loads the model and generates Turtle (an RDF text format), one token at a time, on your machine. The graph you see is that Turtle, parsed and laid out with a small force simulation. The models are byte-level BPE decoder-only transformers, the same architecture as a small GPT, only fed triples.

The one rule: the page holds names, never relations

Every page knows which IRIs exist, that pikachu and electric are things. None of them knows that Pikachu is Electric. That fact, and every other one, lives only in a model's weights. The line is easy to test: delete the model file and a page can still list subjects and lay out empty graphs, knowing nothing about any of them. One thing follows from it. Because the catalogue is the index of every known IRI, the Memoriser can constrain its decoding to a trie of real identifiers, so it cannot emit an IRI that doesn't exist.

The pokemonkg ontology

The data is PokéKG, the Pokémon Knowledge Graph, a community RDF dataset that models Pokémon, moves, abilities, types, egg groups, shapes, habitats and evolution chains as a Semantic-Web graph. Every entity is an IRI under pokemonkg.org, and the vocabulary is an OWL ontology: classes like Species and PokéType, properties like hasType, isAbleToApply and weakTo.

That ontology is what makes this a good toy. It is a small, closed, tidy universe: a few thousand entities, a couple of dozen relations, with real structure. It is small in the way tinyshakespeare is small. A model can do well inside it, which is the setting the experiments were built for, not a shortcoming.

The honest verdict

This began in a research program asking whether a model can reason over Semantic-Web structure rather than memorise domain facts. Measured properly, the big model does the memorising. That result is more useful than it sounds:

The Pokémon graph gzip -9s to 0.17 MB. The model that memorises it is ~80 MB, about 455× larger than the data, at worse recall than gzip. As a database, a neural net loses to a 1970s compression utility, and scaling makes the ratio worse, not better.

So why is it interesting? Because of what a much smaller model does on top of storage. The 11 MB Minter, given a brand-new IRI, invents a coherent typing, and through the Reasoner it derives the weaknesses from a rule it was only ever shown. Rules are cheap, retrieval is expensive. The same small model scores 0.917 applying the chart to species it invents and 0.385 recalling real ones. The extra capacity was buying the database, not the intelligence.