Skip to content

The graph-to-vector leap

Embedding spaces look like graphs — neurons, connections, pretrained networks. They're not. They're geometry. This page explains why the graph intuition is wrong, why the spatial one is right, and what it means to write programs in that space.

The intuition trap

When you build a connectionist system — a stack of artificial neurons trained on data — your starting metaphor is neurons + connections. That's literally how you describe the architecture. N neurons in this layer, connected to M neurons in the next layer, with weights on each connection. It is a graph in every sense: a vertex set, an edge set, edge weights.

So you train it. The neurons settle into a configuration. Now you ask the trained network "what does this word mean?" and you get an answer. Where, mechanistically, did the answer come from?

Your instinct says: "from the network of connections." From the graph. Some pattern of activations propagated through the edges and out the other side. Meaning lives in the connections. Connectionism.

That's the trap. Because if you actually look at what the network is doing at inference time, the meaning is not in the connections. The meaning is in the position of a vector inside a high-dimensional continuous space. The connections did the work of putting it there, during training, but at inference time the connections are frozen and what you actually compute on is a point in space.

A point in space is not a node in a graph. It is a coordinate. And the operations that work on coordinates are linear algebra, not graph traversal.

So a thing you started by describing as a graph — "neurons connected to other neurons" — becomes, when you actually use it, a thing where queries are dot products and answers are nearest-neighbor lookups. The graph metaphor was scaffolding for training. The runtime is geometric.

What "spatial" actually means

Spatial isn't a metaphor. It's the literal mechanism. Here is what is true about a real LLM embedding space:

  • Words are points. Every word, phrase, sentence, paragraph in your input becomes a vector — a tuple of (in mxbai-embed-large's case) 1024 real numbers. That tuple is a coordinate. The word lives at that coordinate.
  • Similar things are close. "Cat" and "dog" are nearer to each other than "cat" and "calculus." Distance in the space corresponds, very roughly, to semantic distance. There is no edge between "cat" and "dog" — they are just close.
  • Relationships are directions. The classic king - man + woman ≈ queen is not a metaphor either. The vector that goes from "man" to "king" is the same direction as the vector that goes from "woman" to "queen." That direction encodes the relationship "is-a-monarch-of-this-gender." You can add it to a vector to apply the relationship.
  • Meaning composes by arithmetic. Bundling two vectors (by adding them) gives you a vector that is similar to both — the geometric implementation of "AND" or "OR" depending on how you read it. Binding two vectors (by sign-flipping or rotating one in terms of the other) gives you a vector dissimilar to both — the geometric implementation of associating a key with a value.

None of these are graph operations. There is no traversal. There is no edge to walk along. There is no node to be at. You are at a point in space and the next operation moves you to another point in space. The thing under your feet is a floor, not a graph.

Why this is so hard to internalize

The reason most people struggle with this is that the language of computer science was built around discrete structures. Lists, trees, hash maps, graphs — all of these are discrete. They have "elements" you can "be at." Iteration is "step from one element to the next." Search is "is this element in here? yes or no?"

Embedding spaces have none of that. Iteration in the discrete sense doesn't apply. There is no "next element." You can move in any direction. Search is "give me the nearest stored points to this coordinate" — but "this coordinate" might be a point that was never stored, that was constructed by adding two other coordinates together, and it can still mean something.

The language for this exists, and it's been around since high school: linear algebra. Vectors. Dot products. Matrix multiplication. Subspaces. Projections. Distance. The math you might have used to figure out "if a ball is rolling down this incline, where will it be in 3 seconds?" is the same math that powers a semantic search query. That is wild and most people never get told about the connection.

The reason it's hard is that the path to embeddings goes through the connectionism story (neurons! networks! graphs!) before it lands on the geometric story (vectors! dot products! continuous space!). And the connectionism story is so vivid and so memorable that the geometric story feels like a demystification — like you're being told "actually it's not magic, it's just numbers." Which, yes, but the numbers are spatial coordinates in a high-dimensional manifold, and that is genuinely beautiful and powerful, and "it's just numbers" undersells it.

Sutra exists to give you a way to write programs in those coordinates.

Once you accept the leap, this is what becomes possible

If meaning is geometry and operations are linear algebra, you can do things you couldn't do with graph databases:

  • Logical inference on frozen embeddings. The Latent Space Cartography paper demonstrated that frozen general-purpose text embedding models (mxbai-embed-large, nomic-embed-text, all-minilm) encode 30+ relations as consistent vector displacements — relations the models were never trained for. You can do entity + (capital_of) and get back a sensible answer. This is logical inference, implemented as vector addition. Hours of GPU time become microseconds of arithmetic.
  • A programming language whose ops are O(1) instead of O(n) graph traversals. Bind, unbind, bundle, similarity — six microseconds per operation on a CPU. The expensive operation in conventional graph databases (the join, the path query, the recursive traversal) doesn't exist in Sutra because there are no edges to traverse. There is just geometry.
  • The same code running on radically different substrates. Sutra compiles for LLM embedding spaces, and also compiles for a simulated Drosophila melanogaster mushroom body — a real biological connectome. Same source file, deliberately different targets. The fact that this works is itself empirical evidence for the geometric view: the math doesn't care whether the underlying neurons are silicon transistors or simulated leaky-integrate-and-fire spiking cells.
  • A first-class spatial visualizer in the IDE. Because every value in the program is a coordinate, the debugger can literally show you where your program is in semantic space. The IntelliJ-based Sutra IDE has a planned 2D/3D visualizer pane that draws the embedding region the current code is touching. This is impossible in graph-database tools because graphs don't have a coordinate system.

Why "graph database vs. vector space" is the wrong framing

A persistent question this project keeps getting: "is Sutra competing with knowledge graph databases? With vector databases?"

Neither. Both miss the point.

Knowledge graph databases model the world as discrete entities connected by typed edges. They are correct for tasks where the world really is discrete — relational records, citation networks, bills of materials. But they are hostile to anything fuzzy, anything graded, anything where "this matches kind of" is the right answer.

Vector databases are what happens when somebody tries to add similarity search to the existing graph-database mental model. They store vectors, they let you do nearest-neighbor lookups, but the programming model around them is still discrete: insert a vector, query for k nearest. There is no operation more sophisticated than "give me the closest matches." A vector database is infrastructure, not a language.

Sutra is a language for programs that compute in the space. It uses a vector database (a lightweight bundled one is part of the planned vertical stack), but the programs are not "queries" — they are computations. They construct new vectors that were never stored. They bind keys to values via geometric operations. They run inference loops. They compose. The vector database is the storage backend. The language is the computational primitive layer.

Closest existing analogy: SQL is to a relational database what Sutra is to a vector database. SQL is not "look things up in PostgreSQL" — it is a language that runs computations over relational data. Sutra is the same thing for embedding spaces. The key difference is that the underlying primitive ("a row" for SQL, "a vector" for Sutra) is fundamentally different in kind: one is discrete, one is continuous.

The Sutra ecosystem

The vision the language is part of is bigger than the language itself. The full stack we're building, in rough priority order:

  1. The Sutra language — compiler, type system, semantics, the .su source format. (SDK in sdk/sutra-compiler/)
  2. The Sutra runtime — the silicon-substrate execution engine. Probes a target embedding space, fits correction matrices, exposes the three-tier operation API. (sutra-paper/scripts/sutra_runtime.py)
  3. The Sutra IDE — IntelliJ Platform plugin with syntax highlighting, completion, diagnostics, settings, and a planned 2D/3D embedding-space visualizer. (sdk/intellij-sutra/)
  4. SutraDB — the lightweight bundled vector database, brought into this monorepo as a subtree at sutraDB/. The SQLite-of-vector-databases idea: zero-config, embedded, optimized for the kinds of queries Sutra emits.
  5. The fly-brain substrate — a Brian2 spiking simulation of the Drosophila mushroom body that the Sutra compiler can also target, used as a stress test of substrate-adaptivity. (fly-brain/ + fly-brain-paper/)
  6. A bundled vertical stack installer — one download gives you the language, runtime, IDE, embedded vector database, default embedding model, and a small curated default corpus. Hello-world-without-config is the goal — the same affordance that made SQLite the most-deployed database in the world.

All of it is open source. All of it lives in github.com/EmmaLeonhart/Sutra.

  • Hello Sutra — write your first .su file, run it, see the geometric semantics in action.
  • Bind and unbind — the operation that makes the spatial view useful for real programs. Why Hadamard fails on natural embeddings and why sign-flip binding works.
  • Snap-to-nearest — the cleanup operation that makes long computations possible. How error correction works in continuous space.
  • The papers — the empirical evidence behind everything on this page.