Mavo: Creating web applications by authoring HTML • Lea Verou’s PhD thesis

Languages like HTML and CSS have characteristics that make them more natural [1, 2] to learn and use. They are declarative, reactive, robust and forgiving in terms of syntax. Authors assemble high-level concepts and constraints, rather than explicit instructions. Robustness is achieved in different ways across the two (HTML attempts to correct authoring mistakes, CSS to scope them tightly and ignore them), but both are designed with resilience and fault tolerance as a design principle.

These desirable properties have given rise to a large community of authors who are comfortable with HTML and CSS, yet not being comfortble with JavaScript or other traditional programming languages. While it is difficult to pinpoint the size of this community, it is likely large and growing. The ACM cites knowledge of HTML and CSS to be at the K-12 level of computer literacy [3].

Far more powerful than static pages are web applications that react dynamically to user actions and interface with back-end data and computation. Even a basic application like a to-do list needs to store and recall data from a local or remote source, provide a dynamic interface that supports creation, deletion, and editing of items, and have presentation varying based on what the user checks off. Creating such applications requires knowledge of JavaScript and/or other programming languages to support the necessary user interaction and to interface with a data management system, as well as understanding of some form of data representation, such as JSON or a relational database.

There are many frameworks and libraries aiming to simplify creation of such Web applications. However, all target programmers and still require writing a considerable amount of code. It is indicative that even implementing a simple to-do application similar to the one in Figure 3.1 requires hundreds of lines of code:

Framework	LOC
AngularJS	294
Polymer	246
Backbone.js	297
React	421

Many people who are comfortable with HTML and CSS do not possess additional programming skills¹ and have little experience articulating data schemas [4]. For these novice web authors, using a CMS (Content Management System) is often seen as their only solution. However, research indicates that there are high levels of dissatisfaction with CMSs [5]. One reason is that CMSs impose narrow constraints on authors in terms of possible presentation–far narrower than when editing a standalone HTML and CSS document. When an author wishes to go beyond these constraints, they are forced to become a programmer learning and modifying server-side CMS code.

The problem worsens when authors wish to present structured data [6], which CMSs enable via plugins. The interfaces for these plugins do not allow authors to edit data in place on the page; instead they must fill out forms. This loses the direct manipulation benefits that are a feature of WYSIWYG editors for unstructured content.

Finally, CMSs provide a heavyweight solution when many authors only need to present and edit a small amount of data. For example, out of the over 7,000 CMS templates currently provided in ThemeForest.net, a repository of web templates, 39% are for portfolio sites, while another 31% are for small business sites.

This chapter presents and evaluates a new language called Mavo² that augments HTML syntax to empower HTML authors to implicitly define data schemas and add persistence and interactivity. Simply by adding a few HTML attributes, an author can transform any static HTML document into a dynamic data management application. Data becomes editable directly in the page, offering the ability to create, update, and delete data items via a WYSIWYG GUI.

3.1.1 Implicit Data Schema Definition

While programmers generally prefer to keep their data schema logic separate from presentation definition, end-users may not have the same preferences, and may instead be frustrated by the need to think about data in two separate places. Indeed, with a certain category of applications, including most CRUD applications, how the data is laid out on the page can easily translate to how the data should be organized. For end-users who are seeking to build these sorts of apps, it may be easier to define a proper schema in tandem with defining the layout.

Mavo authors never have to articulate a schema separately from their interface or write data binding code. Instead, authors add attributes to describe which HTML elements should be editable and how, unwittingly describing their schema by example in the process. With a few attributes, authors quickly imply complex schemas that would have required multiple tables and foreign keys in a relational database, without having to think beyond the interface they are creating. As an added benefit, Mavo’s HTML attributes are part of the HTML RDFa standard [7] and thus contribute to machine-readable data on the Web.

Mavo is inspired by the principle of direct manipulation [8] for the creation of the data model underlying an application. Instead of crafting a data model and then deciding how to template and edit it, a Mavo author’s manipulation of the visual layout of an application automatically implies the data model that drives that application. In addition, Mavo does not require the author to create a separate data editing interface. Users simply toggle an edit mode in their browser by clicking an edit button that Mavo inserts on their webpage. Mavo then adds affordances to WYSIWYG-edit whatever data is in view, with appropriate editing widgets inferred from the implied types of the elements marked as data. Mavo can persist data locally or outsource storage to any supported cloud service, such as Dropbox or Github. Switching between storage backends is a matter of changing the value of one attribute.

In addition to CRUD functionality, Mavo also embeds Formula² expressions, allowing users to perform complex calculations on nested data with a natural syntax. Formula² has been described in detail in Chapter 4.

In contrast to the hundreds of lines of code demanded by the popular frameworks, Figure 3.1 shows how an HTML mockup can be transformed into a fully functioning to-do application by adding only 5 lines of Mavo HTML.

Our approach constitutes a novel way for end-users to transform static webpages to dynamic, data-backed web applications without programming or explicitly defining a separate data schema.

From one perspective, this makes Mavo the first client-side CMS, where all functionality is configurable from within the HTML page. But it offers more. In line with the vision of HTML as a declarative language for describing content so it can be presented effectively, Mavo extends HTML with a declarative specification of how the data underlying a presentation is structured and can be edited.

Fundamentally a language extension rather than a system, Mavo is completely portable, with no dependence on any particular web infrastructure, and can thus integrate with any web system.

Similarly, existing WYSIWYG HTML editors can be used to author Mavo applications. We offer Mavo as an argument for the benefits of a future HTML language standard that makes structured data on every page editable, persistent and transformable via standard HTML, without dependencies.

We conducted a user study with 20 novice web developers in order to test whether they could use Mavo to turn a static HTML mockup of an application into a fully functional one, both with HTML we provided and with HTML of their own creation.

We found that the majority of users were easily able to mark up the editable portions of their mockups to create applications with complex hierarchical schemas.

3.2 Related Work

Quilt [9] was a system that allowed users to link a Google spreadsheet with a webpage and provided simple syntax to bind GUI elements with particular cells in the spreadsheet. Similarly, Gneiss is a live programming environment that incorporates a spreadsheet editor and allows users to create bindings between GUI elements and spreadsheet cells [10].

Mavo combines ideas from three prior systems that addressed the downsides of CMSs. Dido [11] built on Exhibit [12], extending HTML with language elements that visualized and stored editable data directly in the browser. This approach allowed a web designer to incorporate Dido into any web design and made Dido independent of any back-end system. Quilt [9] extended HTML with a language for binding an arbitrary web page to a Google spreadsheet “back-end”, enabling web authors to gain access to lightweight computation without programming. Gneiss [10, 13] was a web application within which authors could manage and compute over hierarchical data using an extended spreadsheet metaphor, then use a graphical front end to interact with that data.

These three systems introduced powerful ideas: extending HTML to mark editable data in arbitrary web pages, spreadsheet-like light computation, a hierarchical data model, and independence from back-end functionality. But none of these systems provides all of these capabilities simultaneously. Dido had no computational capabilities, could not manage hierarchical data, and was never evaluated. Quilt was dependent on a Google spreadsheet back-end, which left it unable to manage hierarchical data. Gneiss was a monolithic web application that only allowed the user to construct web pages from a specific palette. It did not offer any way (much less a language) to associate an arbitrarily designed web page with the hierarchical data Gneiss was managing, which meant that a web author faced constraints on their design creativity. Gneiss and Quilt both required users to design their data separately from their web pages.

Mavo is a language that solves the challenge of combining the distinct positive elements of this prior work, which are in tension with one another. It defines a simple extension to HTML that enables an author to add data management and computation to any web page. At the same time, it provides a lightweight, spreadsheet-like expression language that is expressed and evaluated in the browser, making Mavo independent of any particular back-end. The editing and expression language operates on hierarchical data, avoiding this limitation of traditional spreadsheet computation.

The combination of these ideas yields a novel system that is particularly well-suited to authoring interactive web applications. In Mavo (like Dido), the author focuses entirely on the design of the web page, then annotates that page with markup describing data and computation. The web page implies the data model, freeing the author of the need to abstractly model the data, manage a spreadsheet, or describe bindings between the two. At the same time, our expression language provides lightweight computation (Quilt and Gneiss), even on hierarchical data (Gneiss) without relying on any external services (Dido). Because they are part of the document (Dido), Mavo expressions can refer directly to data elements elsewhere in the document, instead of requiring a syntactic detour through references to cells in the associated spreadsheet. Finally, because it is an HTML language extension (Dido and Quilt), Mavo can be applied to any web page and authored with any HTML editor, freeing an author from design constraints.

In sum, we believe that the combination of capabilities of Mavo align well with the needs and the preferred workflow of current web authors. In particular, the independence of the Mavo authoring language from any back-end system (or even from any particular front-end interpreter) means that Mavo prototypes a future for HTML and the web browser itself, where data interaction becomes as much a basic part of web authoring as paragraphs and colors.

3.2.1 End-User Web Development

There are many systems that assist novice web developers with building dynamic and data-backed web applications. The drawback to many of these systems, however, is that they often require using their own heavyweight authoring and hosting environments, and they provide pre-made plugins or templates that users can not customize without programming. Examples of such systems include CMSs such as Wordpress, Drupal, or Joomla. The growing community around static site generators, such as Jekyll [14] is indicative of the dissatisfaction with rigid, heavyweight CMSs [6]. However, these require significant technical expertise to configure, and offer no graphical interfaces for editing data.

In the previous section, we described three systems—Dido [11], Quilt [9], and Gneiss [10]—from which we draw key insights. However, this work solves challenges in combining those insights into a single system, incorporates additional ideas, and contributes useful evaluation of the resulting system. Most importantly, Mavo demonstrates that the often-hierarchical data model of an application can be incorporated directly into the visual design on which a web author is focused, making the data modeling task an automatic side effect of the creation of the web design. Supporting hierarchical schemas is critical because they occur naturally in many data-driven apps on the web (53% according to [6]). Our evaluation studies users working with such hierarchical schemas.

3.2.2 The Semantic Web and Web Data Extraction

There has been a great deal of work on both encouraging and extracting structured data on the web [15]. However, automatic scraping techniques often have errors because they must infer structure from unstructured or poorly structured text and HTML markup. Several efforts have been made to define syntaxes and schemas, such as RDFa [7] and Microdata [16], for publishing structured data in web pages to contribute to the Semantic Web and Linked Open Data [17]. However, novice users have had little incentive to adopt these standards—sharing data rarely provides direct benefit to them—and find them difficult to learn, potentially contributing to their limited adoption on the web. Mavo contributes to this line of work by using a standards-compliant syntax that is machine-readable. Authors typically do not care about theoretical purity and are motivated to add additional markup when they see a tangible benefit. With Mavo, they expend effort because it makes their static website editable or creates a web application. As a side effect, however, they enrich the Semantic Web.

3.3 The Mavo Language

A description of the Mavo language follows. Its expression syntax, Formula² is described in detail in Chapter 4, so here we will focus on how Mavo HTML embeds formulas, and not their syntax. Similarly, its storage location is specified as a Madata URL (discussed in Chapter 5), so here we will focus on how Mavo HTML interfaces with Madata, and not the specifics of where data is stored.

We chose to use declarative, HTML-based syntax instead of new syntax for Mavo functionality because our target authors are already familiar with HTML elements, attributes, and classes and because HTML is inherently fault tolerant (Section 1.4.4) Whenever possible, we reused concepts from other parts of HTML. Using HTML5 as the base language also means a WYSIWYG editor for Mavo applications can be easily created by extending any existing WYSIWYG HTML editor — in fact, we discuss an attempt at this in Chapter 9. But as discussed previously, we consider it a key contribution of Mavo that it is a system-independent language. For example, we expect most Mavo authors to frequently take advantage of the ability to “view source” and work with arbitrary HTML.

View source is an essential methodology for learning and adopting new elements of web design. It permits authors to copy and tweak others’ designs (even without fully understanding them) without worrying about new or conflicting system dependencies [6]. Source editing is essential to let authors circumvent any limitations imposed by graphical editing tools.

Per [18] and our Design Principles (Section 1.4.1), we want a low threshold (cost to get started) while allowing users escape the low ceiling (maximum achievable power) of GUI-based tool builders.

3.3.1 Creating a Mavo Application

To specify Mavo functionality on an HTML structure, the author places an mv-app attribute on the enclosing element. Its (optional) value provides an identifier that can be used to refer to this app’s data from other Mavo apps on the page, and is used in a variety of other places. If not specified, Mavo looks at the element’s HTML id attribute, and defaults to a generic identifier (e.g. mavo3) if that’s also not present.

3.3.2 Data Persistence

By default, Mavo does not store data anywhere, which can be useful for calculator-type applications. Authors can specify a storage location via the mv-storage attribute.

The syntax of this attribute is a thin abstraction over a Madata URL (Chapter 5) to make these easier to specify for novices, and to provide sensible defaults, since Mavo has more information about the use case at hand.

For example, to store data locally in the browser, the Madata URL is local:foo, where foo is a unique identifier for the data store. To alleviate novices from having to understand what a custom protocol is or from the cognitive tax of finding a suitable key, Mavo uses a local keyword instead, and defaults the key to the app’s identifier.

Similarly, to store data in a remote service, e.g. GitHub, Madata expects a URL that identifies a specific storage location (e.g. a repository and file path). These do not need to exist — for storage services with predictable resource URLs (such as GitHub), Madata will create any resources needed (e.g. files, repositories, etc.). However, simply having to choose suitable values can still be taxing to the novices who are be using GitHub as a backend because of its capabilities, but do not necessarily understand its constituent concepts. To make this easier, Mavo includes a set of defaults for each backend that facilitate underspecified URLs. To reuse the GitHub example, authors can specify a URL to their GitHub profile (e.g. github.com/leaverou), and Mavo uses defaults for the repository name (mv-data) and file name (<app-id>.json). That said, not every backend is conducive to this — for example a Dropbox URL does not have a predictable structure, and thus for Dropbox to be used, authors first need to upload an empty file and get a link to it.

3.3.2.1 Data Loading Overrides

By default, mv-storage specifices both the data source, and the data destination. If data does not yet exist, an empty dataset is rendered, and the data is created upon saving. However, there are certain common use cases that require more complex logic when reading data (storing data always goes to mv-storage).

Some applications need different backends for reads and writes. These include applications that transform data and store the result elsewhere, certain performance optimizations, or simply read-only applications. To enable these use cases, Mavo supports an mv-source attribute, whose syntax is identical to mv-storage, but it has precedence over mv-storage for loading data.

For many types of CRUD applications, starting from empty data provides a poor user experience. Using a certain default dataset if storage is empty can help users better understand the application’s purpose and functionality, or simply facilitate experimentation, while still allowing the user’s own data to take precedence once it exists. Mavo supports an mv-init attribute for this purpose, which is only used if the main data source (either via mv-storage or mv-source) is empty or unavailable.

3.3.3 Data Definition

Our approach to data definition means that end-users define their data by defining the way they want their data to look on the page. This is in contrast to many systems which expect their users to define their data model first and then map their model into a view. In the spirit of direct manipulation, Mavo users are manipulating their data schema by manipulating the way the data looks. We believe that our approach is more natural for many designers, permitting them to directly specify their ultimate goal: data that looks a certain way.

Once Mavo is enabled on an HTML structure, it looks for elements with property or itemprop attributes within that structure in order to infer the data schema. These elements are henceforth referred to as simply properties. If the HTML author is aware of semantic Web technologies such as RDFa [7] or Microdata [16], these attributes may be already present in their markup. If not, they don’t need to understand either technology — authors are simply instructed to use a property attribute to name their element in order to make it editable and persistent. An example of this usage can be found in Figure 3.1.

When an element becomes a property, it is associated with a data value. This value is automatically loaded from and stored to the specified mv-storage location. For many elements (e.g. <span>), the natural place for this value to be “presented” is in the element’s contents. In others, such as <img> or <a>, the natural place for a value is a primary attribute such as src and href. RDFa [7] defines a small set of such attributes (mainly href and src), which Microdata [16] extends with many more, which Mavo adopts.

3.3.3.1 Explicit Primary Attribute

There are many cases where the data value cannot be predicted by these heuristics. For example, consider the following Mavo HTML fragment:

Since this is an <a> element, its primary attribute is href, but the data value is in the element content. To handle such cases, both RDFa and Microdata use a content attribute as an escape hatch: If a content attribute is present, it overrides both the element contents, and the primary attribute. However, this requires duplicating data across where it actually lives and the content attribute.

Instead, Mavo adds an additional rule to these semantics: an mv-attribute which allows authors to specify the default primary attribute, with a none value for the element contents (which is what we would use above).

3.3.3.2 Objects

Properties that contain other properties become grouping elements (objects in programming terminology); this permits a user to define deep hierarchical schemas implicitly, simply as a natural consequence of spatial containment. For example, an element with a student property can contain other elements with name, age, and grade properties, indicating that these properties “belong” to the student.

3.3.3.3 Collections

A core value proposition of Mavo is automating the large amount of repetitive UI code that CRUD applications require to manage collections of data. Interactions like adding items, deleting items (with undo), reordering items via drag-and-drop, plus keyboard handling for efficiency and accessibility for all of the above, are all automatically handled.

To convert a property into a collection, all that authors need to do is mark what should be the collection item — i.e. the element that will be repeated — with an mv-multiple or mv-list-item attribute, or mark the collection itself with an mv-list attribute.

During editing, appropriate controls appear for adding and deleting new elements in the collection, as seen for the to-do items in Figure 3.1. Collection items can themselves be complex HTML structures consisting of multiple data-carrying elements and even nested collections. This enables the author to visually define schemas with one-to-many relationships.

To author a collection, the author creates one representative example of a collection item; Mavo uses this as the archetype for any number of collection elements added later.

As discussed earlier, this archetype can contain real data so it resembles actual output and not just a template, and can also provide default data values for new collection members.

3.3.3.3.1 Collection Syntax

Originally, there was no HTML representation for the collection itself — mv-multiple was added to the collection item and was the only way to define a collection. This was not an issue for purely CRUD applications with very simple computations — it was just framed as mv-multiple makes an element repeatable.

Regardless, putting the attribute on the collection container instead was a common slip, although easy to self-correct from feedback (see Section 7.1). However, as noted in Section 7.2, when authoring expressions for hierarchical schemas the lack of an HTML element to host the entire collection created some confusion.

Additionally, this design created some conceptual inconsistencies. Other Mavo attributes (e.g. mv-default or mv-value) had to heuristically determine whether they were being set on the collection itself or its items, with awkward escape hatches.

As a result, Mavo later adopted a more explicit syntax involving two separate attributes: mv-list and mv-list-item. However, per Mavo’s philosophy of flexibility and fault tolerance, collections can be defined with only one of the two, and the other will be inferred. If necessary, suitable container elements to hold the collection or its items will be crated.

3.3.3.4 Relationship to RDFa

The Mavo syntax for naming elements is based on a simplified version of RDFa we call Loose RDFa that is designed to prioritize learnability over generality.

3.3.4 Data Editing

Mavo generates UI (user interface) controls for toggling between reading and editing mode on the page, as well as saving and reverting to the last saved state (if applicable), as seen at the top of Figure 3.1. In editing mode, Mavo presents a WYSIWYG editing widget (called the property’s editor) for any property that is (a) not already a form control, (b) not a computed property, and (c) not explicitly defined as read-only.

In line with its philosophy of implicit data schema definition, Mavo leverages available HTML semantics to both optimize the editing interface and extract data type information. For instance, a <time> element is expected to hold temporal data, and will be edited via a date or time picker (depending on its datetime attribute), whereas an <img> element will be edited via a popup that allows specifying a URL or uploading an image (Figure 3.3).

3.3.4.1 Customization

These heuristics are intended to both reduce cognitive load on the author and to encourage the use of semantically appropriate HTML. However, per Mavo’s design principles, inferred information should be escapable (Section 1.4.3). Indeed, both the data type and the editing widget can be overridden.

The inferred data type can be overridden by using the datatype attribute. This is an RDFs attribute, and thus does not need an mv- prefix, though Mavo uses it more loosely by accepting plain identifiers like number rather than CURIEs like xsd:number.

The generated editing UI can be customized in a variety of ways. For small tweaks, such as overridding an attribute, authors can specify attributes with an mv-editor- prefix, and they are copied to the generated editing widget. For more extensive customization, authors can provide their own editing widgets by linking to an existing element anywhere on the page via the mv-editor attribute, whose value is a CSS selector. If no data type is explicitly specified, this new editor element will be used for inferring it, just like default editors.

For example, if a property only accepts certain predefined values, authors can express this by linking to a <select> menu, essentially declaring it as an enum. Any changes to the linked form element are propagated to the property editors. This way, authors can have dynamic editing widgets which could even be Mavo apps themselves, such as a dropdown menu with a list of countries populated from remote data and used across multiple Mavo apps.

3.3.5 Computation

The aforementioned three attributes—mv-storage, property, and mv-multiple (or mv-list/mv-list-item) — are sufficient for creating any CRUD content-management application with a hierarchical schema and no computation. However, many CRUD applications in the wild benefit from lightweight computation, such as displaying a count of items, summing certain values, or conditionally showing text depending on a data value.

To accommodate these use cases, Mavo embeds reactive expressions in brackets ([]) in the HTML, as well as raw within certain attributes (mv-value, mv-if, etc.), called directives. Originally, Mavo supported two expression interpreters: (a) raw JavaScript (executed in a sandbox environment where properties become read-only variables), and (b) MavoScript, a Formula² precursor. Authors did not specify the flavor they were using. Instead, expressions would be first parsed as MavoScript expressions, and if that failed, as JavaScript expressions. This contributed to author confusion, and restricted the evolution of Formula², so it was quickly abandoned.

Our approach to expressions only partially meets the declarative, direct manipulation goal we described in our motivation. It is challenging to specify computation, an abstract process, entirely through direct manipulation. The expression language is similar to that in spreadsheets—fully reactive with no control flow, which nods towards declarative languages. The widespread adoption of spreadsheets provides evidence that this type of computation is within reach of a large population. Furthermore, placing the expression in the document, precisely where its value will be presented, as opposed to referencing values computed in a separate model “elsewhere”, fits the spirit of direct manipulation in specifying the view. During our several user studies many participants volunteered observations that this was effective.

3.3.5.1 Embedding Formula² Expressions in HTML

The first is to use an mv-value attribute, containing the expression. This is more verbose than the second method, but allows providing a fallback value (which is also used before the expression is evaluated) and allows Mavo to provide visible feedback when the expression is invalid, as there is an element to apply CSS to.

The second is to enclose it in square brackets ([]), which allows placing it anywhere inside the Mavo instance, including in HTML attributes (but not element names — yet).

To avoid triggering unrelated uses of brackets on individual elements, authors can use the mv-expressions attribute to customize the syntax or disable expressions altogether (mv-expressions="none"). The setting is inherited by descendant elements, unless they have a mv-expressions attribute of their own. For example, for the double-brace expressions common in many templating libraries, authors can use mv-expressions="{{ }}".

The choice of brackets for delineating expressions was based on the observation that non-programmers often naturally use this syntax when composing form letters, such as email templates. In addition, many text editors automatically balance brackets.

An earlier version of Mavo used a more spreadsheet-like =(expression) syntax, but we found from preliminary user studies that few users realized the spreadsheet connection and found it difficult to determine where an expression terminated due to parentheses also being used inside expressions.

3.3.5.2 Named References

Instead of referencing mysterious row and column coordinates (a common source of errors in spreadsheets [20]), Formula² (and by extension Mavo) refers to properties by name. Every property defined in a Mavo instance becomes a (read-only) variable that can be used in expressions anywhere in the Mavo instance. The algorithm is described in detail in Section 4.4.1. These named references are necessary since Mavo has no predefined grid for row/column references. We consider this necessity a virtue.

We believe this will decrease bugs caused by misdirected references. Indeed, many spreadsheets offer named ranges to provide this benefit of understandable references (although no-one used them in our user studies, see Chapter 7). For spreadsheets, perhaps the main benefit of the row-column references is having formulas with “relative references” (e.g. to adjacent columns) to automatically update as they are copied down into new rows. But Mavo’s automatic duplication of templates in collections means copies are never made by the user, obviating the need for this benefit.

3.3.5.3 Dynamic Collections

Formula² is not restricted to primitive values, but can also operate on lists and objects (and combinations thereof). Mavo harnesses this via the mv-value attribute. We saw earlier how mv-value can be used to specify scalar expressions with a fallback. However, mv-value follows the schema of the element it’s being used on (adapting the data if necessary, see Section 3.6.2). When used on a collection, it renders its value as collection data and reactively re-renders the collection items whenever needed. Combined with Formula² list-valued semantics, this can produce powerful results with very little code. Furthermore, this dynamic collection can also be named and referenced elsewhere, creating a basic but powerful abstraction mechanism.

For example, suppose our To-Do app had a priority property for each task, and we wanted to display the number of tasks per priority (a pivot table). It could be as simple as:

3.3.6 Reactive Defaults

It is worth noting that in collections, the example item can be filled with real data, which makes the template really look like the output, unlike other templating languages where the template is filled with visible markup. In addition, this example data can easily become default values, by using the mv-default attribute without a value. Specifying a value for mv-default will set the default value to that, which is useful for static defaults.

However, mv-default becomes very powerful when combined with expressions, creating reactive defaults. To our knowledge, Mavo is the first novice programming language to provide reactive defaults.

When a property is set to a reactive default, it gets updated whenever the default value changes, unelss it has been edited by the user. This effectively provides a data/formula hybrid which enables many use cases that are otherwise cumbersome or even impossible.

The more obvious use case is facilitating smart defaults that are progresssively refined as other properties are edited. For example, suppose we have a calendar application whose events have start_date, end_date, start_time, end_time properties. end_date could default to start_date if end_time is after start_time, and to one day after start_date otherwise. It would look like this:

However, there is a large set of use cases where fields are essentially computed via an expression, but there are some few exceptions where the user might want to override the computed value. Using mv-default rather than mv-value in these cases preserves editability and allows the user to override the computed value.

3.3.7 UI Customization

We designed Mavo to be useful to HTML authors across a wide range of skill levels, including web design professionals. This presents a tension: Web designers want control, whereas novices want UI to be generated requiring as little involvement as possible.

The approach that Mavo follows is to generate a default UI, but expose hooks for customization at various levels of the ease-of-use to power curve. Smaller customizations can be done by simply styling the generated Mavo elements with CSS.

In addition, authors can provide their own UI elements that replace those generated by Mavo by using certain class names (such as class="mv-add-task" for a custom “Add task” button).

For the parts of Mavo most likely to be customized, such as the editing widgets or the toolbar, Mavo also provides specific attributes for high level customization with low effort (e.g. see Section 3.3.4.1 for customizing the editor).

An earlier version of Mavo also supported nesting form elements inside properties to implicitly define an editor without the indirection of linking to a form element elsewhere, akin to nesting a form element in a <label> to implicitly associate the two without using a for attribute. However, practice showed that this frequently conflicted with the author’s intent for the form element to be a part of the template, and not an editing hint, and thus it was removed.

3.4 Implementation

Mavo is implemented as a JavaScript library that integrates into a web page to simulate native support for our syntax. On load, Mavo processes any elements with an mv-storage attribute and builds an internal Mavo tree representation of the schema (Figure 3.4). It also inspects every text and attribute node on or inside every element looking for expressions, and builds corresponding objects for them. For every expression, a JavaScript parser is used to rewrite binary operations as function calls in order to enable array arithmetic.

Any remote data specified in the mv-storage attribute is then fetched via Madata and recursively rendered. Every time an object is created during editing or data rendering, it holds a reference to its corresponding node in the Mavo tree (Figure 3.4), which it uses as a template. This improves performance by only running costly operations (such as finding and parsing expressions) once per collection.

When the data in an object changes — via rendering, editing or expression evaluation — expressions within it or referring to it are re-evaluated to reflect current values. This occurs in a special execution context where current object data and Mavo functions appear to be global scope. ECMAScript 2015 Proxies are used behind the scenes to conditionally fetch descendant or ancestor properties only when needed, to allow for identifiers to be case insensitive, to make the variables read-only, and to allow identifier-like strings to be unquoted.

3.4.1 Extensibility

There are APIs in place for third-party developers to add new default editing widgets, new expression functions, and new storage backends. In addition, Mavo includes a hooks system for developing plugins that modify how it works on a lower level. For example, both the inline Mavo debugging tools and most directives are implemented as plugins and could be removed.

3.4.2 Debugging

3.4.2.1 Inline Debugging Tools

Mavo includes debugging tools that show the current application state, as expandable tables inside objects (Figure 3.5). This is enabled by placing a mv-debug class on any ancestor element or adding ?debug to the URL. These tables display current values of all properties and expressions in their object, and warnings about common errors. Expressions shown can be edited in place, so that users can experiment in real-time.

3.4.2.2 Mavo Inspector

While inline debugging tools may appear to work better with novices, they do produce a lot of clutter. Furthermore, our first user study reported low levels of author engagement with those early debugging tools.

We later also implemented a Google Chrome devtools extension (Figure 3.6) for inspecting the values of Mavo Nodes and experimenting with expressions. The extension adds a “Mavo” tab to the Element inspector of the built-in browser developer tools that displays current values, and allows one-off execution of expressions and data updates. Since property references in Mavo expressions are context sensitive, we used this inspector in our evaluation to demonstrate context sensitivity to our subjects, and later on in the study to allow them to quickly experiment with expressions in training tasks.

Later user studies used the Mavo Inspector instead, both to explore data, and to allow subjects to quickly experiment with expressions in training tasks.

3.5 Discussion

In this section we discuss various issues brought up in our design and study of the Mavo language, and other potential future directions.

3.5.1 Data Formatting

Currently, data formatting is automatic. The only type that supports custom formatting is the <time> element, which can optionally nest an expression for presenting the date/time to override the default:

For example, numbers are automatically formatted in a locale-aware way, with thousands separators and two decimal places. However, there are cases where authors might want to override this formatting. For example, in a recipe app, one would typically want to list ingredients with fractional quantities like “⅜ cups”, rather than “0.375 cups”. This is already possible by redirecting the number value to an attribute, and using a custom expression as the element contents:

However, this is essentially a workaround, as its closeness of mapping [21] is poor. A more direct way to specify formatting would be beneficial, e.g. an attribute dedicated to this, alongside a richer set of built-in formatters.

3.5.2 Direct Manipulation of Data Schemas

Mavo’s approach of designing schmemas by designing the presentation of the data from those schemas works well because the presentation of data usually reflects its schema. If we have a collection of objects with properties, we generally expect those objects to be shown in a list, with each object’s properties presented inside the space allocated to that object. This is understandable, as the visual grouping conveys relationship to the viewer. We are simply inverting this process, arranging for the visual grouping to convey information to the underlying data later. Mavo may not be suitable for creating presentations that conflict strongly with the underlying data schema, should such presentations ever be wanted.

3.5.3 Target Users

Mavo is aimed at a broad population of users. There is no hard limit to what it can do, since JavaScript can be used as an extension point. However, this is not the primary target. Our focus is increasing the power payoff for a given investment of effort/learning that is accessible to novices (Section 1.4.2). Currently, even small web applications require substantial skill and effort to build. Too often, designers of essentially static websites are forced to deploy them inside CMSs, only so that their non-technical clients can update the site content. Mavo frees designers from these CMS constraints by providing an automatic WYSIWYG content management UI for plain HTML. Plain CRUD apps only need mv-* attributes “entirely in HTML” without application logic.

For users who want more, expressions add power: lightweight computation for application logic at a conceptual cost similar to spreadsheets. More complex functions provide more power, like advanced spreadsheet ones. Our user study traced out this ease/power curve and showed that most users can work with such expressions.

Although we have focused on Mavo as a tool to support non-programmers, skilled programers can also benefit from the ability to rapidly build dynamic CRUD interfaces. Even for programmers Mavo brings some of the benefits of data typing to the construction of the interface: declaring data types enables the system to provide appropriate input and data management without demanding that the developer write special purpose code for the typed content.

3.5.4 Handling Large Amounts of Data

The existing Mavo backends save all data in a single JSON file. This is convenient for use cases involving small amounts of data, and allows using any popular file hosting web service as a backend. However, making multi-user apps possible will create a pressing need for handling larger amounts of data with Mavo, even to create entire social websites

Mavo already supports displaying and editing part of the data, and already keeps track of what data has been modified, to highlight unsaved changes. Therefore, it is easy to implement incremental saving for web services that support it. Implementing a backend adapter for a cloud database service (e.g. Firebase), will also allow for fetching partial data. Such backends usually also support server-sent events, which would enable incremental updates for true bidirectionality.

3.5.5 Encouraging Semantic Web Best Practices

Mavo encourages semantic web practices by providing incremental value to authors when they add semantic markup. For example, using suitable HTML elements will produce more suitable editing widgets, and using good property names is essential for an editing UI that makes sense, since they are used in a number of places in the generated editing UI: button labels, tooltips, and input placeholders to name a few.

Additionally, its storage functionality typically produces structured, machine-readable data, contributing to the Semantic Web through another avenue. In fact, if a vocab attribute is used in the markup, Mavo will produce valid JSON-LD [22].

It could be argued that because of its looser syntax (Section 3.3.3.4), data produced by Mavo app users are not useful. However, we believe that reducing the burden of creating structured data and increasing the author perceived value of doing so is the only way to encourage widespread adoption of structured data practices, and a large quantity of imperfect structured data is more useful than a small quantity of perfect data created by the few users with intrinsic motivation to contribute to the Semantic Web.

3.6 Future Work

3.6.1 Separate Name for Collection and Collection Items

Using plural instead of singular nouns and vice versa is a common slip of Mavo authors, since currently the same name is used for both the collection itself, and each item.

As noted in Section 3.3.3.3.1, the current syntax for defining collections consists of two attributes: mv-list and mv-list-item. While that was not the motivation for the change, it opens up the possibility of defining a separate name to refer to the collection and its items (currently an authoring mistake handled by precedence rules).

This would allow using a plural name for the collection itself, and a singular alias for the items. This would also allow referencing the collection from within each item without having to do propName.all.

3.6.2 Handling Schema Mutations

Mavo’s innovation of inferring schema from HTML presentation might be its Achilles’ heel. After Mavo is used to create data, changes to the HTML may result in a mismatch between the schema of the saved data and the new schema inferred from the HTML, which could lead to data loss (or perceived data loss, where data is retained but not displayed, creating the appearane of data loss). Similar mismatches can occur when connecting Mavo apps to third-party data with a pre-existing structure that was not created by the Mavo app. Mavo does automatically handle some changes, such as:

However, if the disparity between the data schema and the inferred schema is too great, Mavo will not be able to automatically reconcile them without author intervention. Such intervention includes:

More complete handling of schema changes and/or better author feedback about how they are handled are key open questions for Mavo.

3.7 Conclusion

This chapter presents Mavo, a system that helps end-users convert static HTML pages to fully-fledged web applications for managing and transforming structured data. Our user studies showed that HTML authors can quickly learn use Mavo attributes to transform static mockups to CRUD applications, and, to a large extent, use Mavo expressions to perform dynamic calculations and data transformations on the existing data.

We carried out a snowball sample of web designers using a Twitter account followed by 70,000 Web designers and developers. Of 3,578 respondents, 49% reported little or no programming ability. ↩︎
Open source implementation & demos available at mavo.io ↩︎

Chapter 3 Mavo: Creating web applications by authoring HTML

Contents

3.1 Introduction