Hypertext, Transclusion, Xanadu: the Web that Could have Been

The Web as it currently exists is something of a shadow of its potential. Many thinkers, chief among them Ted Nelson, set out systems that could have provided for something more robust and human than the Web, and which would be subject to fewer of the issues that plague it at present: these ideas have not yet been made real. However, companies like Notion are working to create systems that bring the Web closer to par with what it could have been.

For the purposes of this article, it makes sense to define a few key terms. 1. The Internet refers to the global system of connections between business, home, wifi networks, etc. 2. The Web refers to the system of websites and other resources, accessible via URLs over the Internet.

Hypertext

To understand what the Web could have been, we need to examine a controversial visionary: Ted Nelson. Nelson’s ideas predicted and underpin numerous fundamentals of online communication, yet Nelson was doggedly anti-technical and himself implemented only a fraction of what he devised. 

One such idea is hypertext.

To internet users today, hypertext links are so fundamental to the online experience as we know it that most would find it difficult to imagine an alternative. The deeply embedded ubiquity of hypertext, however, obscures a long history of debate and innovation around the problem of how best to express relationships between information.

For centuries, librarians have used indexes that break down the topics a given book covers, and academics identify sources via citations — but what about the actual relationships between texts themselves and the concepts they cover? The following video, an excellent tech talk from Google, breaks down numerous ideas, including a library system that recorded the research path between papers taken by a reader, giving subsequent researchers a useful guide by which to direct their own research.

In 1967, Ted Nelson explained his vision for hypertext:

And I am trying to work out the forms of writing of the future. Hypertexts. 

Hypertexts: new forms of writing, appearing on computer screens, that will branch or perform at the reader’s command. A hypertext is a non-sequential piece of writing; only the computer display makes it practical. Somewhere between a book, a TV show and a penny arcade, the hyper text can be a vast tapestry of information. All in plain English (spiced with a few magic tricks on the screen), which the reader may attack and play for the things he wants, branching and jumping on the screen, using simple controls as if he were driving a car.

— Ted Nelson, “Hypertext Notes”, 1967

Shortly after Nelson published “Hypertext Notes,” Douglas Engelbart in 1968 gave the “The Mother of All Demos” of his and his team’s “oN-Line System.” It included windows, video conferencing, the mouse, hypertext links, and many other fundamentals of computing.

Tim Berners Lee took a significant step forward when in 1989 he invented the URL and hypertext transfer protocol (HTTP) and specified hypertext markup language (HTML) — the three of which, in combination, facilitate the Web as we know it. People quickly adopted the Web, which allows for an approximation of Nelson’s connected hypertext idea.

What the Web Could have Been

Tim Berners Lee beat Ted Nelson to the punch. Nelson had been working on his own hypertext project, “Xanadu,” since 1960. The project aimed to meet 17 goals, of which the following are particularly prescient.

• Every user is uniquely and securely identified. (3)

The Web uses back-formed means of identifying users, which are neither unique nor secure.

• Every user can search, retrieve, create and store documents. (4)

Word processors had not yet been invented.

• Every document can consist of any number of parts, each of which may be of any data type. (5)

This describes the modern, multimedia Web (before multimedia came to fruition conceptually in the 1990s).

• Every document can contain links of any type including virtual copies (“transclusions”) to any other document in the system accessible to its owner. (6)

Nelson imagined not just being able to link to other documents, but to include and attribute, seamlessly, content from them. This was implemented, partially, by the html <iframe> in 1997, but iframes are notoriously unstable and are not universally supported.

• Every document can contain a royalty mechanism at any desired degree of granularity to ensure payment on any portion accessed, including virtual copies (“transclusions”) of all or part of the document. (9)

• Every Xanadu service provider can charge their users at any rate they choose for the storage, retrieval and publishing of documents. (15)

Nelson imagined a micropayment method, baked into the infrastructure, as opposed to the Web, wherein publishers are scrambling to pay for their services with ads and paywalls.

• Links are visible and can be followed from all endpoints. (7)

Fundamentally, HTML does not actually link text, pages, or websites together — it merely directs the user to a Web address where the resource should exist. People routinely change the URL, thereby breaking the link. Web links are also unidirectional: which is akin to saying that Ford Motor Company is connected to Dearborn (where it is headquartered) but neglecting to add that Dearborn is connected to Ford.

Nelson responded to the popularity of the Web, thus:

I DON’T BUY IN

[…]

The original hypertext project, Xanadu®, has always been about pure document structures where authors and readers don’t have to think about computerish structures of files and hierarchical directories. The Xanadu project has endeavored to implement a pure structure of links and facilitated re-use of content in any amounts and ways, allowing authors to concentrate on what mattered.

[…]

Markup must not be embedded. Hierarchies and files must not be part of the mental structure of documents. Links must go both ways. All these fundamental errors of the Web must be repaired. But the geeks have tried to lock the door behind them to make nothing else possible.

Hopefully it’s now clear that the Web could have been much better. Unfortunately, Nelson is still working on Xanadu. His ideas and its standards, however, continue to be relevant.

Notion

There are signs of hope however, emerging from unexpected places. A few note-taking and project management tools are working to solve these issues, my favorite being Notion (notion.so). When I started using Notion, I immediately felt the realization of Ted Nelson’s ideas. 

Most work in Notion starts with a blank page: the user can insert text, formatting, images, embeds, just like in a word processor or CMS. More interestingly, the user can arbitrarily invoke multiple project management and data-visualization functions: kanban boards (like Trello and ClickUp), galleries, tables, to-do lists, calendars, with infinitely customisable views, fields and filters. 

To my delight, all of these functions are arbitrarily invokable and nestable: this is to say that one can create a kanban board and in one of the cards, create a kanban board, and so on.

Notion allows for several implementations and variations on Nelson’s transclusion idea. One is the fact mentioned above, that Notion is essentially whitespace in which the user can invoke functions: this means that the user can create a page with several nested calendars and kanban boards: one place that stitches together all these different functions.

Another implementation is what Notion calls a “linked database.” This arguably more Nelson feature allows you to create a database view in another or the same location as an existing database, and express it one of a variety of ways. 

For example, I could create a kanban board one one page, and create a linked database on another page and express the contents of the kanban board as a calendar — the data is the same, so an update to one is reflected in the other.

Notion’s linking is very powerful, also. Its are not fragile like normal URLs: you can delete parts of them, use and old version of the link, and they still work — you have to try very hard to break a link in Notion. When you link items, meanwhile, the fact that they are connected shows up in the navigation — this is to say not just that there is a part of the document that contains the address of another document, but that they are connected.

Notion is only just getting started, and will be releasing more and more powerful features in the future.

Conclusion

Ultimately, there’s no reason why a better idea should win out in practice; the market demonstrates this with the victory of Microsoft Windows over competitors like CP/M, MacOS and Linux; Facebook over MySpace, Bebo, etc. 

Making a better Web, however, seems more socially imperative, given the extent that it represents a repository for pretty much everything that we care about. The way the Web works directly influences the propagation and exchange of ideas — metaphorically, this means that humanity is running on something like Windows 98, when we could be on Unix at no extra cost. 

Companies like Notion are pushing to bring digital information in closer alignment with Ted Nelson’s original vision. For my part, it can’t happen soon enough.