← Tuesday, September 29, 2026Models & trainingDeep dive

LLM word embeddings form circular patterns in a high-dimensional analysis

jamiesimon.io 3 min read Editor score 7/10

The analysis finds that month embeddings project to a circle and have an approximately circulant Gram matrix, with similar patterns in word2vec. Engineers working with embeddings can use these geometric results to inform how they interpret representation spaces.

What

An analysis examines the geometry of word embeddings, including whether selected word vectors form circles in a PCA projection.

How

An iterative search found word2vec sets matching a target circulant Gram matrix and forming clear circles; embedding size did not enter the search.

Why it matters

Embedding plots can be shaped by how vectors are selected, so engineers should be cautious when interpreting geometric patterns as inherent structure.

Read on jamiesimon.io → Hacker News · 8 comments

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.