Versioning wikitext

Before there can be a wikitext 2.0, or 1.1, or 0.9 or whatever version is “next”, we need a way to distinguish old from new stuff. The alternative is what we call a “flag day”, where we wake up one morning and say “today wikitext behaves like this, not that” and change the whole world at once. We’ve managed to do that a couple of times already, but it is slow and hard. I think–and you can disagree with me on this—that if we want wikitext to move more quickly, a good first step would be a way to mark some content as “new” so we can try new things out without breaking old things.

There are a number of obvious alternatives here. One which has been used in a lot of programming languages is just to put a flag at the top of an article page indicating either a version {{#version:2}}, a particular language {{#markup:wikitext-2.0}}, or something more granular to enable specific new features, like {{#usefeature:backticks}}. Articles can transclude other articles, and we need to decide whether that feature flag or language version necessarily applies to transcluded content as well, or not.

Another option would be to use an extra bit of metadata, a “slot” in the Multi-Content Revisions model, to record a version identifier or set of enabled features.

Each article page also has a “content model”. This is typically used to distinguish wikitext pages from, say, Scribunto modules, or JavaScript gadgets, or TemplateStyles CSS. But we could also use it to mark certain articles as “wikitext 2.0” without using inline markup or an MCR slot.

Either way I’d expect it would look like this in VisualEditor: a metadata dialog that would let you either select the wikitext version or toggle specific feature flags, which would be recorded either in the wikitext or in the MCR slot.

It’s worth noting that the UX for changing a page’s content model already exists. You could simply select “wikitext2” in the standard dialog.

But currently you need special permissions in order to change a page’s content model. I think setting a feature flag for a page is something an ordinary editor should be able to do for a page they are writing.

Grunge

So let’s say I can set a page as “wikitext 2.0” instead of “wikitext 1.0”. What would wikitext 2.0 look like?

One of the difficulties of current wikitext is the lack of a written spec, and the complexity of its implementation.

Grunge is an attempt to radically simplify wikitext, to something you could describe “on a single sheet of paper”, and which eliminates the excessive look-ahead and backtracking in the current parser.  It was designed to map to and from HTML, just like Parsoid does with “wikitext 1.0”, so that you can in fact use MediaWiki DOM Spec HTML as an intermediate representation to convert between wikitext 1.0 and Grunge.  An article stored as wikitext 1.0 can be edited as Grunge by converting first to HTML and then into Grunge, and vice versa. (This is a useful property for any "wikitext 2.0" proposal, not just Grunge.)

Here’s what it would look like.  I tried to make this “look like wikitext”, while also squashing ambiguities in wikitext 1.0.  For example, in current wikitext you can put 5 single-quotes in a row and create something which could resolve into either bold or italics or quotes.  In Grunge most constructs start and end with curly braces to eliminate the ambiguities.  We also add block headings and block list items, which are missing from wikitext 1.0.  We’ve made some changes to magic words and extension tag syntax, which we’ll return to later.

This ends up looking very much like wikitext.  Here’s an example of editing a page with Grunge: 

This is boring, because Grunge is exactly the same as wikitext 1.0 in this simple example.  But that’s kind of the point.

Markdown

If we can use HTML as an intermediate form (phab:T127329) to allow editors to work with an alternative language, wikitext 2.0 or Grunge, while still converting back to wikitext 1.0 for storage–or to use wikitext 2.0 for storage while letting editors still work with wikitext 1.0 if they prefer–you might logically wonder… why does the new thing need to look like wikitext at all?

Markdown is one alternative. It was created in 2004, shortly after wikitext's creation in 2002, and there are currently 166 million markdown files being parsed at a rate of 1,300 per second on github alone. In comparison, Wikipedia has 65 million articles written in wikitext with parsing happening at a rate of roughly 5 per second.

Markdown has a far larger user base, with a specification and a large number of independent implementations in a variety of programming languages, while wikitext is only used by MediaWiki.

However, Markdown has no (standardized) extension or transclusion mechanism.  Before Markdown can be used for wiki projects, a reasonable syntax for wikitext transclusions (templates, extension tags, parser functions, and probably media) needs to be invented.  If you can figure out how to make those into Markdown, then there’s nothing stopping you from adding "Markdown" as a MediaWiki editor alongside wikitext 1.0 and HTML.  We did a rough prototype of this, and it would look something like:

Again, the big catch here is: Markdown does not have a template or extension mechanism.  You need to invent some non-standard syntax for transclusions, and that probably includes images.

HTML-only wikis

So, if we can do Markdown, can we do nothing at all?

That is, store articles natively as HTML, use Visual Editor as an HTML editor, and not have any parser at all in core?

We’ve done this, too: it’s how Flow worked.  Flow was a minor disaster, and part of that disaster was the fact that we didn’t have a good solution for versioning the HTML we stored, but it did work.  And versioning was our item #1, so we’ve solved that already, right?

A more rigorous statement of this vision is written up as the “zero parsers in core” proposal.  And the goal wasn’t actually to remove all the parsers, but to make them generic and pluggable (phab:T114194), so that wikitext was just a parser you could load as an extension like any other parser.  The main architectural hurdle is system messages, which are hard-wired as wikitext.  Tagging system messages with a content type would fix that, and it would also help security as well because some messages aren’t wikitext, they are actually plaintext, and some others are raw html, and not being certain about exactly how your system message is going to be parsed is an opening for a heap of security issues.

An HTML-only wiki would just looks the same as VisualEditor except it wouldn’t have the “source” editing option:

Extension tag fragments

One of the extension mechanisms in wikitext is “extension tag syntax”. It is intended for embedding non-wikitext content into wikitext, and so syntax rules of wikitext, brace and bracket matching and such, don’t apply.  As a tradeoff, you can embed anything inside an extension tag.  Almost anything…

<ref>
  Extensions can contain anything.
  <ref>Citation needed.</ref>
</ref>

It turns out that the only thing which can’t appear inside an extension tag is the close tag sequence for that extension tag.  Which is kind of a bummer when you want to nest things.

In current wikitext, if you write this, your reference ends at the first close tag, leaving an unmatched open tag inside, and an unmatched close tag outside:

So here’s an easy syntax hack to fix that.  We don't want to try to do nesting directly, because the whole point of the extension syntax is that we shouldn't care about what's inside and it should be fast and easy to find the end of the extension content. Instead, we’re going to allow fragments after the tag name, which are completely ignored except that the close tag must match the fragment.  We can now safely embed anything, by carefully choosing a fragment which doesn’t appear in the thing we want to embed.

<ref#outer>
  Extensions can contain anything.
  <ref>Citation needed.</ref>
</ref#outer>

Syntax uniformity

Syntax uniformity is another thing you might have noticed in Grunge.  Wikitext has a large number of different forms for effectively the same thing.  We have behavior switches, magic variables, parser functions, templates, extension tags, hidden links, and media links.  Each has different rules for case-sensitivity, different ways of parsing (or forbidding) arguments, and different escape mechanisms for content.

So we’re going to add a standard “curly brace” form for each of these. Parser functions largely use this syntax already.  Templates are already in “curly brace form”.  Extension tags already have a curly-brace form with the {{#tag}} parser function.  We’re not going to remove the other forms, at least not yet (although Grunge removed them!) but we’ll always have a “standard” form to use if (for example) we need to pass an argument, or want to use a consistent escape mechanism or a long argument form.

Colon replacement

Speaking of parser functions: isn’t it annoying that the first argument to a parser function is separated by a colon, while the first argument to a template (and every other argument for both) are separated by a vertical bar? 

{{Template|arg1|name=arg2}}

but

We could fix that, or at least treat a vertical bar as a synonym for a colon. We propose to support a vertical bar as an alternative separator, and add built-in support for named arguments while we're at it:

Backticks for <code>

Backticks for <code> are often requested, similar to markdown.

<code>hello, world!</code>

becomes

`hello, world!`

(This is the first "inside wikitext" feature that wasn’t in Grunge. The Grunge equivalent would be {`hello, world!`}, since Grunge prefers consistent curly-brace delimiters.)

This would be a fairly straightforward addition to wikitext; the difficult part would be linting all of our existing content to ensure that existing content with backticks wouldn't be affected. It might be best to enable code backticks only on non-encyclopedic wikis, or in the Template: or Module: namespaces, or only as a shortcut in VisualEditor but not as wikitext syntax.

Syntax for Discussions

This is a bonus proposal, not included in the Wikimania 2025 talk! Instead this is based on a proposal drawn up during the Wikimania 2019 hackathon. It sits on the edge of "syntax reforms included in Grunge", because although Grunge includes block list items, the proposed syntax in Grunge was very different.

From the parsing perspective, discussions on talk pages are a big list (the : character used for replies technically introduces a HTML “definition list” item), and threads are separated by section headings. Signatures are entered with four tildes ~~~~, and typically expand to your username, a link to your talk page, and the current day and time, localized appropriately for the current wiki, although this can be customized in user preferences.^[2] This presents three main issues: first, it is difficult to reliably extract attribution from custom signatures and the signature date from the localized timestamp. Second, list item syntax is fragile and easily broken; identifying the boundaries of each individual comment in a thread is thus more difficult than it should be. Finally, individual list items don't have stable id attributes that can be used to link to them.^[3]

[[User:Username|Username]] ([[User talk:Username|talk]]) 17:24, 16 August 2025 (UTC)

{{#~:Username|2025-08-16T17:24}}

{{#~:Username|2025-08-16T17:24|[[User:Username|Username]] ([[User talk:Username|talk]]}}

: This is the parent comment. {{#~:Alice|2025-08-01T01:23}}
:: This is a long reply.  It goes on and on.
:: In fact it is still going.
:: Finally, Bob concludes. {{#~:Bob|2025-08-02T12:34}}

: This is the parent comment. {{#~:Alice|2025-08-01T01:23}}
:: <<< This is a long reply.  It goes on and on.

In fact it is still going.

Finally, Bob concludes. {{#~:Bob|2025-08-02T12:34}} >>>

:: id=my-comment class=important <<< This is my comment! >>>

== id=my-heading <<< This is my heading >>> ==

== Is this syntax okay? ==
I think it probably will be fine. {{#~:Alice|2019-08-17T10:45}}
: Will it, though? {{#~:Bob|2019-08-17T10:46}}

== id=rebuttal <<< Maybe we need a few extensions >>> ==
It’s possible we need to think this through more?
{{#~:Eve|2019-08-17T10:47}}
: id=denial <<< No, nothing’s wrong with it.

In fact, I love it! {{#~:Mallory|2019-08-17T10:48}} >>>
:: Agreed, everything is just fine. {{#~:Alice|2019-08-17T10:49|[[User:Alice|'''alice''']]}}

{{#media}}

You might have noticed {{#media}} on previous proposals.  It turns out that the existing [[File:...]] syntax is really gross.  One particular misfeature is that any unknown option is treated as a caption, so you can’t safely add or remove options without potentially affecting captions.

But it also turns out that “put this image exactly at this point and exactly XYZ pixels in size” isn’t really what you want with media, and that there’s a lot of semantic information we’d like to include but isn't present in the current syntax (phab:T90914).  Finally, we have non-media content like graphs, charts, maps, and pull quotes that we want to be able to style just like the “built-in” media.

So here’s some syntax for a new {{#media}} parser function to address those gaps.  We add a style option, where we can specify a semantic style for the image, like “this is a lead image” or “this image should be full width”, and we require an explicit named caption argument.

{{#media:src=Foo.jpg|style=thumb|caption=...}} <!-- matches current defaults -->
{{#media:src=Foo.jpg|style=largethumb|...}} <!-- some wikis have a "large thumb" style -->
{{#media:src=Foo.jpg|style=fullwidth|...}}
{{#media:src=Foo.jpg|style=leadimage|...}}

We could also add focal point information so that we can more intelligently crop and resize images for different platforms. This could have a default focus derived from annotations in commons, but an image depicting multiple subjects might need the focus overridden in an article:

{{#media:src=Eiffel Tower|focus=50 75}} <!-- percent width, height -->

There’s a sample catalog of styles collected by User:MGalloway (WMF), although the exact mechanism by which local wikis define, extend, and localize semantic styles is still an open question. The strawdog proposal in phab:T90914 uses data in the MediaWiki: namespace, but invoking a Scribunto module of a given name might allow better integration with the localization provided by the Plan for Scribunto proposal.

To address the issue of non-media content styled similarly to media, we also allow content= as an alternative to src=.

{{#media:content=<<<...>>>|caption=...}}

This allows user-generated content like maps and charts that can be styled the same way media is.

In Page Description Language we discuss further how media may be composed into the page, and perhaps {{#media}} is just a stopgap until we remove media from the page flow entirely. As a transitional step to the future, however, we should start considering the semantic information that would be necessary to place media in a page description language and allow editors to record that information in the wikitext.

{{#lang}}

Current wikitext has a single “target language” per page, but many of our pages contain content in multiple languages.  There are ways to manually set language and directionality in the output HTML, but they are cumbersome and often omitted.  Language affects internal MediaWiki functions like the {{int}} function and the operation of the Translate extension, and in non-Latin scripts it often affects font and glyph shaping. Broadly, “the letters don’t look right”” unless the language is correctly set.  {{#lang}} is a technical feature, but one that is very important for many world languages.  In addition, we have many wikis which embed “user interface content” inside the content area, which is intended to be rendered in the user language not the page language.  We don’t currently have a way of indicating this.

Here’s an example of a multilingual wiki, commons, with embedded content in English, Arabic, and “the user language”. 

== {{int:filedesc}} ==
{{Information
|Description    =
{{ar|1=صُورة [[:ar:أوروبا|لِأوروپَّا]] و[[:ar:أفريقيا|أفريقيا]] التقطتها كاميرا [[:ar:مرصد مناخ الفضاء العميق|مرصد مناخ الفضاء العميق]] التابع لِوكالة [[:ar:ناسا|ناسا]] يوم 29 تمُّوز (يوليو) 2015م.}}
{{en|1=
Africa and Europe from a Million Miles Away
...}}
{{es|1=Vista [[:es:Satélite artificial|satelital]] de [[:es:África|África]] y [[:es:Europa|Europa]] a una distancia de un millón de [[:es:milla|millas]].}}
}}

We can use {{#lang}} with the special target x-user to correctly set the language and directionality for this content in the user’s language:

== {{#lang:x-user|{{MediaWiki:filedesc}}}} ==

The {{ar}}, {{en}}, and {{es}} templates can also use {{#lang}} internally to ensure the parser’s target language is correctly set when processing its argument.

References blocks for non-European wikis are another extremely common case of multilingual content, since the citations are often to sources in European languages. The directionality can be a mess. Here is an example of using the {{#lang}} function to correctly reset language and directionality for embedded English-language content inside a reference in a right-to-left language.

Note that we already have a parser function named {{#language}} (which returns properties of a given language code) so {{#lang}} may not be the best name for this functionality. It is likely to be wrapped within the preexisting templates named after language codes ({{ar}}, {{en}}, {{es}}, etc) so having a short name may not be critical.

{{#balance}}

Here’s a short fragment of wikitext.  Is “bar” going to be in boldface, or not?

''' foo {{tpl}} bar ''' baz

This is a trick question! The answer depends on the contents of the template tpl:

Yes it will be bold if {{tpl}} expands to a or '''a''' or <b>

No it will not be bold if {{tpl}} expands to '''a or '''''a'' or </b>

We can address this unpredictability with a mechanism called “balance”.  Broadly, it ensures that any contexts opened in a fragment are closed at the end of the fragment.   In addition to providing more predictable behavior for editors and ensure that the dreaded “boldface spilling out all over the rest of the article” bug doesn’t occur, it also enables some key performance updates by allowing incremental update of parsed content.  And of course, it ensures good behavior of a large number of wikitext features, not just boldface. To get technical, {{#balance}} enables a principled and strongly-typed composition semantics for wikitext.

Returning to our example:

''' foo {{tpl}} bar ''' baz

If the template tpl opts-in to balance, we can be assured that bar will always be bolded; nothing will “leak out” of tpl into the outer context.

Long arguments

Let's look at an example from a Wikimania 2015 talk, "Future of structured documents: VisualEditor, Citations and Wikidata, oh my!", which shows a reference in the article Nahuatl. This is a template to create a citation region using Wikidata: this region of the text is supposed to be supported by reference 32412 stored with WikiCite. Here's the syntax:

{{cite id=“32412”|
First person plural pronouns in Isthmus-Mecayapan Nahuat:

:''nejamēn'' ({{IPA|[nehameːn]}}) "We, but not you" (= me & them)
:''tejamēn'' ({{IPA|[tehameːn]}}) "We along with you" (= me & you & them)
}}

But this doesn’t work: instead of displaying its first argument it displays nothing! why?

It’s these equal signs here:

It turns out that there are a number of “special” characters you can’t naively include in a template argument, and equals sign is one of them. Everything here to the left of the first equal sign is treated as the name of the first argument, and thus there is in fact no argument “1” defined here.^[6]

There are a number of such “gotchas” when trying to pass arguments to templates or parser functions.  I gave a talk at Wikimania 2015 (updated in 2024) going through a lot of them.  Long argument syntax is an effort to provide a “quote” syntax to make it easier to pass arguments without having to “properly escape” them, if you can figure out what “properly escape” means.

Here’s what it looks like, just triple angle brackets on either side of the argument:

Here’s an example that is difficult or confusing to write currently:

The workaround to date has been to have two separate “start” and “end” templates (tablestart and tableend here), in order to avoid the need to figure out how to properly escape the body.  But those two templates aren’t balanced (remember {{#balance}}?) and make it slower to reparse, hard to edit this table using VisualEditor, etc.  But with long arguments we can easily create a balanced “table” template and put the table cells in the argument.

Like extension tag syntax, long arguments allow anything to be passed except the magic >>> close sequence.  But if you need to pass that particular character, for example to nest quoted arguments, you can do that as well using the same mechanism we introduced for extension tag fragments.

At the risk of inconsistency with extension tag fragments, we might consider flipping this and making nesting of <<< >>> work by default, which would reserve fragments for the case where we explicitly want to pass an unbalanced <<< or >>>:

The goal is not to invent special escape syntax but instead to interpret characters literally and just allow changing the delimiters if necessary.^[7]

Going back to our first example, it makes it easy to pass arbitrary content to our citation template, without having to watch our for equals signs and other bad characters:

Variable-length/structured arguments

Have you ever had to stare at a template like this and try to figure out what in the world it is doing?

Well, most of what it is doing it just passing arguments through to its children!

Here’s something to help with that mess.  In wikitext, template arguments are unstructured strings which can make it difficult to manipulate them.  Recently we landed PFragment support in Parsoid and core (phab:T374616), which allows you to embed structured data in wikitext.  This allows us to add parser function constructors, which create structured data values:

{{#list:one|two|three}}

The output of the {{#list}} parser function here is not a plain string, but instead a structured list which can be passed to other templates or parser functions and further manipulated.  We can extract the first element of the list using {{#first}}, for example:

{{#first: {{#list:one|two|three}} }}

We can also create a key/value or named-argument list:

{{#kv: name=Moth | caption=Variable Drab Moth }}

This is a two-element list whose first element is the key name and value Moth, and whose second element is the key caption and value Variable Drab Moth.

Note that there is no new syntax added to wikitext so far, this is just internally creating a new structured-data output type for transclusions. And in fact you can do this even without changes to wikitext internals; the ArrayFunctions extension does this by passing around a base64-encoded JSON string value (to ensure it does not contain wikitext special characters) and decoding the value whenever access to the structured value is required.

Using these structured types, we now have a reasonable way to do variable-length argument lists.  In current wikitext, this would typically be done by suffixing the argument names: you’d have name1, caption1, name2, caption2, etc and the source of MyTemplate would have ten or twenty copies of identical code to process each argument, depending on whatever the maximum number of arguments was expected to be:

{{MyTemplate
|name1   = A
|caption1= Apple
|name2   = B
|caption2= Banana
}}

But now we can write this instead:

{{MyTemplate
|files={{#list:
 {{#kv:name=A|caption=Apple}}|
 {{#kv:name=B|caption=Banana}}
}}}}

And we can iterate over the files argument as a structured list of key/value pairs.

Note that #list is just syntactic sugar for #kv, since {{#list|one}} is the same as {{#kv|one}} which wikitext treats as shorthand for {{#kv|1=one}}. Yes, wikitext is 1-based, not 0-based.  Still no new syntax required.

If we’re willing to reserve the argument name ...,^[8] we can use these structured arguments to implement safe argument forwarding. We propose to make {{{...}}} evaluate to a structured key/value pair list of all of the current template’s arguments. We can then manipulate those using the methods operating on structured data. If a key/value pair list is passed to a transclusion as an argument named ... it is expanded into the argument list, allowing invocations like:

{{Template:Infobox
|type=movies
|...={{{...}}}
}}

where arguments are safely forwarded to a child. This solves a long-standing limitation of templates which has been the cause of a lot of unreadable template code, and which required the addition of parent frame access in the design of Scribunto.

This requires new semantics for the argument named ... but still no new syntax.

Annotations

Let's consider how we annotate content for translation on a multilingual wiki. This example is taken from mw:Help:Extension:Translate/Page_translation_example. We start with some article content:

Fréttinga is a small municipality in MungoLand, located on the BaMungo island.
It hosts a population of about 400 people. It has some agriculture and fishing.
Tourists like to visit it in the summer time.

== Services ==

It doesn't have many services. There is a shop, and a car ferry visits the island
from the mainland once a day.

The first step is to mark the annotated regions, which we do with explicit markup in the wikitext.

The translation UI then shows the translation regions:

The translator can then translate each of them:

Of course, this is Wikipedia, so the original article will quickly get changed.

The translator can update the translation units, as well as indicate whether the change was just a copy-edit (in which case the translation doesn’t need update) or needs further attention.  This notifies translators, who can then update the translations.

But let’s look again at our wikitext.  Even though we’re not encoding /all/ of the knowledge here in the wikitext – we’re storing the actual translations of these texts somewhere else – there’s still a lot of strange stuff cluttering up our article, which is often off-putting to editors.

This example isn’t unusual.  There are a lot of examples on our projects with information which is cumbersome to include inline in the wikitext:

• Translation regions from Translate extension
• Inline discussions
• Citations (especially citation regions)
• LanguageConverter rules -{...}-
• WikiSource (proof-reading, transcription regions)
• Maintenance tags {{citation needed}}

In addition there's other information the community would like to include with an article but are currently blocked on because the extra inline information would be too disruptive to “ordinary” editors:

• WikiSpeech pronunciations
• Fine-grained translation correspondences

There's a large list of use cases collected from the community at the Annotations session of the Wikimedia Developer Summit in 2017.

We are proposing a general annotation service to allow us to record all the information which doesn’t belong in wikitext.  Here are the important properties of annotated content on our projects:

• First, the annotation is generally transformative.  It’s an independent piece of content but with a relationship to the base. Both the source and the annotated content require independent maintenance.
• We wish the annotation to be decoupled from changes to the base text. The original source author or editor doesn’t need to update the annotated content in order to write or edit, in fact is likely unable to. Similarly, the annotation can be created or updated without changing the source or imposing on the source editor.
• It is associated with a specific revision of the base content, although that revision may not remain current long. This is in contrast to some annotation systems, which need to maintain fuzzy anchors to the source material because it may change unexpectedly. In our projects articles are versioned, so we can have stable anchors to a specific location in a specific revision. Of course the revision won't remain current forever, so...
• There is an explicit migration process to the latest revision.  We don’t assume the annotation will apply after edits have been made, although we can provide good tools to make it easy to migrate where possible. As with the translate extension, there is an explicit notion of a "fuzzy" annotation, which can be explicitly re-anchored to the current revision in a manual process done by a human.
• Display the latest revision by default, even if annotation migration is incomplete. As a matter of wiki culture our project has discovered that it’s best to make it very obvious when something needs to be fixed.  When an edit is made to a wiki, we generally make it visible immediately, so that if the edit is incorrect it will be similarly quickly noticed and fixed.  In our translation example example, if an edit invalidates a translation, the translated version typically reverts to the untranslated content, to draw attention to the fact that the translation annotation needs to be migrated (translation reviewed and updated).

Annotations should be transparent: the presence of annotated content should never prevent editors from being able to easily update the base content. There are many authors, many annotations, and many annotators.

What does editing annotated content look like?  At the top of this section we showed one example with the Translate extension, which used a separate editor for the annotated content (translations). But we can also look at the InlineComments extension for an idea of a more generic interface. Viewing and refining the annotation regions is done with a special mode in Visual Editor:

Annotations are anchored to a specific revision, and they can be specified as a range in either the wikitext source or in the HTML DOM; Parsoid allows mapping back and forth between these.

Just as we’re using VisualEditor to mark regions, I expect it will probably be useful to make a generic-but-extensible mechanism to edit annotation content alongside the source text.  This is a simple example from InlineComments:

Not every annotation type will use this simple form in the sidebar – the Translate extension uses a custom editor as we saw earlier – but providing a generic functionality of this sort will probably make it easier to develop new annotation types and provide some common user interface elements to make editing new annotation types easier to learn.

Visual Templates

We’ve made some proposals which facilitate creating and using wikitext-based templates.  But wikitext is not the only transclusion mechanism on our platform. In A Plan For Scribunto I briefly mentioned adding a "wikitext" slot to Scribunto modules to allow using VisualEditor to edit the presentation layer of Scribunto modules. The idea is to move the presentation layer out of Scribunto and use Scribunto only for computation and logic.  Instead, each Scribunto module has an associated wikitext slot to handle presentation. Wikitext in that slot is rendered in a “data context” provided by the Scribunto module, using parser functions similar to those introduced by the ArrayFunctions extension to access the data context. The data computed by Scribunto fills in the blanks in the wikitext, using a "logic-less" template system similar to Spacebars, Handlebars, or Mustache. This results in a simple block+placeholder structure compatible with creating/editing the wikitext slot using VisualEditor.

Here is an example of "logic-less" templates, in this particular case Mustache, with the Mustache syntax highlighted in red:

There are only two constructs: named placeholders, like {{name}} and {{value}}, and blocks, like the section between the {{#if}} and the {{/if}}.  Blocks have a main body and an optional “else” body (not shown in this example), and insert the content of their bodies one or more times into the output.

This template is evaluated in a data context such as the following, presented here in JSON format:

{
  "name": "Chris",
  "value": 10000,
  "taxed_value": 6000,
  "in_ca": true
}

And the Mustache output for this example would be:

Here’s that same example translated into wikitext syntax.

It’s very similar!  The placeholders use triple-brace syntax in wikitext, but are still referring to the data context for the module.  I used long argument syntax to pass block content.

In visual editor, the placeholders appear as editable elements, and the "if" and "else" arguments to blocks (aka the arguments to certain parser functions) are editable inline.

This is a constrained version of the normal wikitext template system, with the "data context" replacing template arguments.^[9]

This mechanism is also how HTML-only wikis could work with templates. In that case the "wikitext" slot would actually be stored as "HTML", but it could use a similar logic-less template mechanism integrated with Scribunto and editable (as MediaWiki DOM Spec HTML) with VisualEditor.

Page Description Language

Let’s consider some beautiful things!

As any designer will tell you it is quite hard to create beautiful content on Wikipedia in wikitext.  Our content is used on desktop, on mobile, in apps, in print, and in other places we're probably not even aware of, and it is a real challenge to make it look good in each of those formats.  Part of the problem is how we specify and place images and other media on the page, which is incredibly low-level (“give me a 200-pixel wide thumbnail exactly here”) and omits most semantic information about how the image is being used.

Can we have designers create different article layouts? Can we make "article style" into a first-class notion in MediaWiki?

For example, on desktop or in PDF form many articles are most readable in a two-column layout, but math-related articles tend to look better in a single column, since the mathematical expressions are often wide. How can we mark an article as "has wide content, looks best single column"?

What are the different categories of presentation? Can we expose "full width", "single column", "double column", "lead image", “side bar”, etc as properties authors can apply to media or content fragments? What others are needed?

The Multi-Content Revisions system lets us associate content in one or more "sidebar" slots with an article. How do we compose that with the rest of the page?

If you’ve ever used Desktop Publishing Software, you’re probably familiar with the frames those programs use to lay out a page.  Content can be poured into these frames to lay them out in columns, establish headings, pull quotes, sidebars, images, etc.

The big idea here, and I apologize for being a bit vague, is to associate a “page design” with a page or portion of a page—perhaps each section of the article has its own "design".  Within that "page design" we’d insert various bits of independent content stored in Multi-Content Revisions, including the main body text in the main slot, but also infoboxes, images, sidebars, pull quotes, footnotes, etc.^[10]

The page design would have variants for desktop, mobile, print, etc which use the content slots in different ways.

Here are some of the existing "page design" patterns in Wikimedia articles, from a larger set collected by User:MGalloway (WMF):

• Page design examples
• 
  Article Infobox
• 
  Illustrate and accompany content (large)
• 
  Give context to the mentioned article (medium, grouped)
• 
  Full-width image
• 
  Show views of the same object (grouped, share same description, fixed height)
• 
  To compare a few images (grouped, share the same title, fixed height)

The big idea here is that we’d be selecting from one of a library of templates like this, with associated semantic information and alternate layouts for mobile, etc, instead of trying to manually position content.

Native Script Editing

This is some text from the English Wikipedia article titled Elevator:

An elevator is a machine that vertically transports people or freight between levels. They are typically powered by electric motors that drive traction cables and counterweight systems such as a hoist, although some pump hydraulic fluid to raise a cylindrical piston like a jack.

If you speak British English, of course, this isn’t an “elevator” at all.  This is a Lift:

An elevator (American English, also in Canada) or lift (Commonwealth English except Canada) is a machine that vertically transports people or freight between levels. They are typically powered by electric motors that drive traction cables and counterweight systems such as a hoist, although some pump hydraulic fluid to raise a cylindrical piston like a jack.

American and British English vary on a number of terms and pronunciations, which co-exist somewhat uneasily on English Wikipedia.  And there are further variants found in Indian English, Kenyan English, and other places around the world.  There is a long policy on Wikipedia about where and how to use regional terms.  But, with a few exceptions, the different variants are mutually intelligible.

As it turns out, Serbian sides with the Brits. This is Lift on Serbian Wikipedia:

Lift služi za vertikalni ili strmi prevoz osoba i materijala, pa se i deli na putničke ili teretne liftove. Za prevoz (transport) se koriste otvorene ili zatvorene kabine s mogućnošću zaustavljanja na potrebnom broju stanica. Primenjuju se u stambenim i poslovnim zgradama, rudnicima, industrijskim postrojenjima, brodovima, na gradilištima i slično.

But as it turns out, there are multiple ways to write Serbian.  This is the same article, but written in Cyrillic script, Лифт:

Лифт служи за вертикални или стрми превоз особа и материјала, па се и дели на путничке или теретне лифтове. За превоз (транспорт) се користе отворене или затворене кабине с могућношћу заустављања на потребном броју станица. Примењују се у стамбеним и пословним зградама, рудницима, индустријским постројењима, бродовима, на градилиштима и слично.

LanguageConverter allows you to switch between these views of the article, depending on which script you prefer to read.  As I understand it, many people in Serbia can read both.  There are also spelling and vocabulary differences, orthogonal to the script choice.

This is the same article on Chinese Wikipedia, 電梯, as you would read it if you lived in Taiwan, Hong Kong, or Macau:

電梯（英語：Elevator），其他地區亦稱升降機。在香港、新加坡和馬來西亞俗稱「𨋢」（英語lift的譯音），是一種垂直運送行人或貨物的運輸工具。據統計，2002年全球電梯總數超過600萬部，是現代使用最多的垂直運輸工具。

It is written with what are called “traditional” characters.

But if you’re on the mainland, the title and appearance of the article is different, 电梯:

电梯（英语：Elevator），其他地区亦称升降机。在香港、新加坡和马来西亚俗称“䢂”（英语lift的译音），是一种垂直运送行人或货物的运输工具。据统计，2002年全球电梯总数超过600万部，是现代使用最多的垂直运输工具。

Compare the first character of the titles carefully: 電 versus 电. Mainland China uses “simplified” characters, which were introduced in 1956 and 1964 by the mainland People’s Republic of China government in an attempt to promote literacy.^[11]

Unlike Serbian, few people are equally fluent in both traditional and simplified characters.  There are thousands of characters which differ between the two orthographies, although note that not all of the characters change. (The second character in the article title 梯 does not change, for instance.)

This is the Hindi article, written in Devanagari script, उत्थापक:

उत्थापक, उच्चालित्र अथवा एलिवेटर (lift या elevator) एक युक्ति है वस्तुओं एवं व्यक्तिओं को उर्ध्व दिशा में चढ़ाने-उतारने के काम आती है। प्रायः किसी बहुमंजिला ऊँचे भवन, जलपोत एवं अन्य संरचनाओं में उत्थापक लगा होता है जो गोलों को या सामान आदि को एक मंजिल से दूसरी मंजिल या एक स्तर से दूसरे स्तर पर लाता और ले जाता है। उत्थापक प्रायः विद्युत मोटर द्वारा चलते हैं।

And this is the article on elevators in Urdu, رافعہ:

انتصابی نقل و حمل کی کل۔ جدید عمارتوں، جہازوں اور کانوں میں استعمال ہونے والی تمام کھلی اور بند ساختوں اور لگاتار چلنے والے ان پٹوں کو بھی رافع یا (انگریزی:Elevator/Lift) کہا جاتا ہے جو بھاری چیزوں کو ایک جگہ سے دوسری جگہ پہنچاتے ہیں۔ طاقت سے چلنے والے رافع جو عام طور پر بھاپ سے کام کرتے تھے انیسویں صدی عیسوی سے استعمال ہو رہے تھے جبکہ اس صدی کے اواخر میں برقی رافعہ عام ہو گیا۔

Urdu is written in Arabic script, right to left.  Urdu and Hindi are different dialects of the Hindustani language, written in very different scripts: Arabic script on the Pakistani side of the border, Devanagari script on the Indian side.^[12]

Punjabi is a similar case, with two smaller wikis split by the India-Pakistan border.  There are four scripts used to write Punjabi: Shahmukhi (closely related to Urdu script) and Gurmukhī are the most commonly ones used for writing Punjabi and are considered the official scripts of the language, but Devanāgarī and Latin scripts are occasionally used as well.

As I mentioned earlier, wikitext has a feature called LanguageConverter that automatically converts between words in different dialects, different scripts for the same language, and, if you squint a bit, it is used to translate between languages as well.^[13]  It is currently in use on 15 wikis, requested on more, and probably relevant to most of our wikis to some degree.^[14]

It lets you write markup like this to define word or script variants, and then those are automatically applied:

-{en-us:elevator; en-gb:lift}-

And there is markup to prevent the conversion in places where it isn't appropriate:

Please -{lift}- those boxes and place them in the lift.

This is a brief summary of more complete talks I've given in the past (MediaWiki Developer Summit 2015, Wikimania 2017, Wikimedia Language Engineering Community Meeting May 2024, Wikimania 2024). But to summarize, there are a number of features missing from LanguageConverter, including glossary support and a means to mark the source variant of a text. The markup used for language converter is cumbersome (although glossaries would help) and may benefit from annotations in order to move some of the markup out of the text.

Maintenance of LanguageConverter is also made difficult by the current architecture, which requires someone with the combined skills of a linguist and a PHP regular expressions expert in order to create or modify the converter for a language or script pair. In order to safely make changes to LanguageConverter engines, the canary system used by Parsoid could be applied to evaluate a proposed change on a large set of articles. With the aid of such a system to evaluate changes to LanguageConverter, we could consider moving some or all of the engines to a more maintainable system, for example the rules-based system used by libicu which uses transformation rules in a form familiar to linguists.^[15]

However, the biggest drawback of LanguageConverter can be seen by simply attempting to edit an article on Serbian wikipedia, for example Хемијски елемент:

== Списак 118 познатих хемијских елемената ==
Следећа табела садржи 118 познатих хемијских елемената.
* '''Атомски број''', '''име''', и '''симбол''' служе независно као јединствени идентификатори.
* '''Имена''' су она која су прихваћена од стране -{[[Међународна унија за чисту и примењену хемију|IUPAC]]}-; провизиона имена за недавно произведене елементе који нису формално именовани су дата у заградама.
* '''Група, периода,''' и '''блок''' се односе на позицију елемента у [[периодни систем|периодном систему]]. Бројеви група су у тренутно званично прихваћеној нотацији; за старије алтернативне нотације погледајте [[Група (периодни систем)|Група периодног система елемената]].
* '''Stanje materije''' ''(Čvrsto, tečno,'' ili ''gasovito)'' se odnosi na standardne uslove [[temperatura|temperature]] i [[pritisak|pritiska]] ([[Стандардни услови за температуру и притисак|STP]]).
* '''Pojavljivanje''' pravi razliku između elemenata koji se javljaju u prirodi, kategorisane kao bilo ''Praiskonski'' ili ''Prolazni'' (u smislu raspada), i ''Sintetički'' elementi koji su proizvedeni tehnološkim putem, i nisu prirodno poznati.
* '''Opis''' sumira svojstva elementa koristeći opširne kategorije koje su prisutne u periodnom sistemu: [[aktinoid]], [[alkalni metal]], [[Земноалкални метал|zemnoalkalni metal]], [[halogen]], [[lantanoidi|lantanoid]], [[metal]], [[metaloid]], [[plemeniti gas]], [[nemetal]], i [[Prelazni metali|prelazni metal]].

Notice that there's a mix of Cyrillic script at the top and Latin script at the bottom. Luckily, most users of Serbian Wikipedia can read and write both scripts for their language.  But this isn’t the case for most of our other language variants, who can read and write only one of the scripts for their language comfortably. This can be a huge barrier to editors, who can read the article (as output from LanguageConverter in their preferred script) but can’t read or write it once the editor starts up and the script changes.  This is also a barrier to the adoption of LanguageConverter for several wikis which could use it, who fear fragmenting their editor base.

Native Script Editing (phab:T17161, phab:T113002, phab:T87652) is a feature proposal which uses Parsoid’s selective serialization support to editing the article entirely in the user's preferred variant, with transparent conversion so that we don’t “dirty diff” unedited content.

To drive this home: LanguageConverter is used on Chinese Wikipedia, with a billion speakers, who are not biliterate and can not generally read both simplified and traditional characters. Lack of Native Script Editing blocks editor growth on Chinese Wikipedia.

One Wiki

As I did when I moved from wikitext 2.0 to markdown and then jumped to the extreme, no wikitext at all, I’m going to take the previous discussion of native script editing and take it much further: Transliteration is just Translation between languages that are really really similar.

If we can write in British English and read in American English, or write in Cyrillic and read in Latin script, or write in Arabic-script Urdu and read in Devanagari-script Hindi, why isn’t the Content Translation Tool and LanguageConverter different aspects of the same thing?

We have completely separate technical stacks for translation and transliteration, and we probably don’t need them.  In fact, everything I showed about native script editing could be generalized to native language editing, to allow you to edit in your native language and have that transparently translated.  And in fact, why don’t we allow editors and readers to work in multiple languages more generally?

For bilingual readers, or language learners, why not show parallel texts for bilingual readers?  This is a really crappy mockup, but it gets the point across:

Why can’t we track parallel texts between (say) Swahili and English, and prompt an editor to apply a parallel change to English Wikipedia when an article about Africa is changed in Swahili Wikipedia?

And why not facilitate multilingual discussions among our community, breaking down silos? We have really good translation technology now. There’s no reason why English editors operate in almost complete ignorance of edits and content on (say) Swahili projects.  Why can’t this article on Nairobi be maintained cooperatively between all the folks who speak the languages spoken in Nairobi and know this subject well? Why not facilitate cooperation between the editors of the English and Swahili articles?

In countries with minority languages and an "official government language", editors on minority language wikis often tell me they have trouble building their community because users keep finding content missing from their local wiki. They get in the habit of checking the "official language" wiki instead to find the missing articles, and then eventually just start going first to the official language wiki and are lost from the community of the local wiki. We can do a better job keeping readers on their local wikis. We could transclude content from the "official" language, but there's no reason to show folks content they don't understand. Why not automatically translate it, with an easy on-ramp to use Content Translation to start a local article?

I had a poster on this topic at Wikimania 2018 and it was part of my position statement for the 2018 Wikimedia Developer Summit. It builds on other topics presented in the Dozen Visions for Wikitext and Plan for Scribunto:

• Annotations to track correspondences and maintain parallel texts (allowing helpful cross-language editor nudges) without cluttering the wikitext for the monolingual reader/editor,
• Multilingual templates (related to Visual Templates), so that collaboration on technical content can also occur across language barriers, and
• even esoteric syntax bits like {{#lang}} to better support multilingual content on the same page, so we can render everyone's scripts properly when we're talking together.