WeChat, Meituan, TaoBao style mobile "SuperApps" are very popular in Asia. Not so much in western society where the Apps are more centered around doing one thing well. That being said, I've had to do a lot of business travel over the past year and many times landed in China where those apps are the norm from their core offering to even ordering food without table service. There is also a hugely untapped market in Asia with the rise of these SuperApps. Recently there was an online summit speaking about SuperApps with some forefront thinkers who are at the edge such as Hanno Stegmann and Anand Chawra.

As you can guess from the title, I wanted to see what it would take to build something similar. We always thought "oh it's just HTML and Javascript" but the amount of effort to building your own ecosystem and essentially App Store, requires a lot more thinking – interestingly enough, more thinking than I had originally imagined. Saying it is just HTML and Javascript is an understatement to the ingenuity these companies had with existing tools.

Just as a small note, I am learning Flutter as I go along here, typically I am very much Back end and DevOps dominant but this problem was interesting enough to just do for fun so please excuse the lack of Flutter fidelity.

What is a SuperApp?

A SuperApp starts with a single hero feature. Whether it's messaging or ride hailing or food ordering. Then when they scale up, they allow third parties to create "Mini Apps/Mini Programs" inside their application with a subset of their functionality. This enables the Mini Programs to access domain functionality such as the user base of the host program.

There are a couple of ways to go about building a SuperApp though. If the company wants to expand into many verticals but control the entire app and its development including the new verticals, they can build the whole thing itself as one monolith.

On the other hand, if the company wants to put the innovation and share some of the risks in a B2B2C play, it can enable third parties to develop these new verticals in the form of Mini Apps. The decision of either vastly changes risk, implementation and investment cases up front.

But first, let's ensure we're on the same page with the terminology.

Terminology

Host -  The host is referring to the application that was downloaded and installed from either the Play Store or App Store. The host is both the device and the application itself which 'hosts' the Mini Apps. Such examples would be Wechat, Meituan, Alipay.

Platform - In this case, the platform refers to Android or iOS.

Mini App / Mini Program - A third party application run by the host, independent of the Play Store and App Store.

Page - A single view of the Mini Program.

Architecture for Mini Programs

Source: W3.org

This diagram roughly indicates the major components that make up the SuperApp. We should make the distinction though that the host application actually runs the Logic Layer and the Renderer. The Mini App itself just pushes events when in the view. An alternative take on this with the Host is this diagram:

Mini App Internal Architecture

Everything but the View runs inside the host application. Before we get to the Flutter code, let's look at the components more closely. The boxes in purple represent what we'll PoC out in code.

The directory structure of the main code looks something like this:

miniapp_framework.js
lib/
├── miniapp/
│   ├── logic_layer/
|       └── message_handler.dart
│   ├── view_layer/
|       └── renderer.dart
│   ├── jsbridge.dart
├── main.dart
├── assets/
│   ├── miniapp/
│       └── app.js
│       └── app.json
│       └── index.html
│       └── index.js

Everything under lib/ is the Flutter code and everything inside assets/ gets loaded into Flutter as a file asset.  For PoC purposes, it's not loaded from a CDN. Let's pretend everything in assets was written by a third party Mini Program creator.

Web components & custom SDK

As you may have guessed, the Mini Apps run in a WebView. Luckily, almost all the mobile browsers support some fairly new features in the WebView. One of them being web components which is perfect for speed and templating. The reason why I chose web components is because of the fact that we can build custom markup tags for the SDK, and only render components that got updated instead of manipulating the entire DOM. If you don't know much about web components, I suggest you look it up. It contains a subtree called the Shadow DOM which gets changed and allows us to manipulate less things for a more performant UI. Here's the MDN reference to webcomponents: https://developer.mozilla.org/en-US/docs/Web/Web_Components

So why do we need an SDK? Well, given that we're using a WebView and allowing third parties to run custom code, we want to ensure that the user gets the best experience in terms of security, performance and usability. What that means is that we need to limit what functions can be called. We don't want ads all in the user's face, phishing sites etc to happen when the third parties are in control of navigation. We also don't want to expose all of the platform's APIs and user information to dodgy Mini Programs especially calls that trigger the phone's native functionality.

Let's take a look at building our SDK framework out for the developers. This is in miniapp_framework.js. I used Polymer as a way to generate reusable web components and have hooks into rendering.


import { LitElement, html, css } from 'lit-element';


// Allowed templates below
class NativeElement extends LitElement {

  render(){
    return html`
      <button @click="${this.handleClick}">Native Func Request</button>
    `;
  }

  handleClick(e) {
    console.log('--in native--');
    _superapp.callphone();
  }
}

class SimpleElement extends LitElement {
  static get properties() {
    return { data: { type: String } };
  }

  constructor() {
    super();
    this.id = '';
    this.data = 'No Change';
  }

  static get styles() {
    return css`
      div { background-color: light-gray; }
      * { color: blue; }
  `}

  render(){
    return html`
      <button @click="${this.handleClick}">Simple Element</button>
      <div>Response:${this.data}</div>
    `;
  }

  handleClick(e) {
    console.log('--in simple--');
    _superapp.mySDKFunc({target: this, callbackFnName: 'callback'});
  }

  callback(msg) {
    this.data = msg;
  }
}

customElements.define('native-element', NativeElement);
customElements.define('simple-element', SimpleElement);


/* Core lib that should allow native functionality and callbacks */


// Observer pattern
document._observer = {
  listeners: [],
  registerListener: function(callback) {
    this.listeners.push(callback);
  },
  notify: function(msg) {
    this.listeners.forEach(cb => cb.target[cb.callbackFnName](msg));
    this.listeners = [];
  }
};


var _superapp = {
  callphone: function() {
    SUPA.postMessage('phoneCall');
  },

  mySDKFunc: function(callback) {
    document._observer.registerListener(callback);
    SUPA.postMessage('onMyFunc');
  }
};

There are a few main parts. All it does is provide 2 button templates. One of them hooks into the host's native functionality – namely the phone dialler. The other is where we need to run the code in an isolated fashion and get the return value back to the view and only update that component.

Then the last two functions at the end have a simple observer pattern so that when Flutter calls back into the WebView, it will trigger the callback with the new data and we should see it update.

Ignore the SUPA.postMessage part for now, I'll get back to that in a minute.

The Renderer

In this PoC, I'll build the renderer to load from a static asset to keep things simple. What I'll need to do first is use the SDK above to build my view of approved templates. So my "Page" looks like this:

index.html:

<html>
    <head>
        <script type="module">
            <!--OMITTED for clarity-->
        </script>        
	</head>

    <body>
        <native-element></native-element>
        <br/>
        -----------------------------------
        <br/>
        <simple-element id="my-data" label="mylabel"></simple-element>
    </body>
</html>

You can see that I'm using my two templated buttons. But what goes into the OMITTED section?

For that, I needed to minimize and pack the SDK. I used the Parcel.js CLI and in the same directory as the SDK, ran this command:

NODE_ENV=production npx parcel build miniapp_framework.js

That will produce dist/miniapp_framework.js which can be copy and pasted into the OMITTED section.

In a production environment, I would load all assets from a controlled CDN. Keep in mind that it's just a PoC though. Again, I would also add restrictions on the Javascript code submitted.

What needs to be done now is to create the renderer that would load this code into a WebView. This is done in renderer.dart:

import 'package:flutter/widgets.dart';
import 'package:webview_flutter/webview_flutter.dart';
import 'package:superapp/miniapp/logic_layer/message_handler.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'dart:convert';
import 'dart:async';

class Renderer extends StatefulWidget {
  Renderer({Key key, this.url}): super(key: key);

  final String url;
  
  @override
  _RenderedViewState createState() => _RenderedViewState();
}

class _RenderedViewState extends State<Renderer> {
  final Completer<WebViewController> _controller = Completer<WebViewController>();

  @override
  Widget build(BuildContext context) {
    _loadHtmlFromAssets();

    return WebView(
      initialUrl: 'about:blank',
      javascriptMode: JavascriptMode.unrestricted,
      onWebViewCreated: (WebViewController webViewController) {
        _controller.complete(webViewController);
      },
      javascriptChannels: <JavascriptChannel>[
        logicMessageHandler(context, _controller),
      ].toSet(),
    );
  }

  _loadHtmlFromAssets() async {
    String fileText = await rootBundle.loadString('assets/miniapp/index.html');
    
    _controller.future.then((controller){
      controller.loadUrl( Uri.dataFromString(
        fileText,
        mimeType: 'text/html',
        encoding: Encoding.getByName('utf-8')
    ).toString());
    });
  }
}

The gist of the renderer is that it loads a WebView and enables Javascript. By default, Javascript is not enabled. What's happening here is that we load the index.html from the local assets. You will also notice that it loads something called a JavascriptChannel This is what enables IPC between Flutter and the WebView.

You should get something that looks like this:

Rendered view of the MiniApp

I know, not the prettiest -- but it works. I just wanted to demonstrate web components and the bottom component also has CSS as well which just makes everything blue.

Let's dive into the Logic Layer next which consists of the message handler for the WebView and the logicMessageHandler() method.

Logic Layer

Cool, now our logic layer has the following tasks:

  • Get the event from the Renderer/WebView
  • If it's not a native ask, we execute the logic specified by the user inside an isolate JS Engine. Return the value back to the Renderer and update the web component that changed
  • If it's a native ask, we execute the phone call functionality of the host application

Recall in the Javascript code we had something called SUPA.postMessage(). SUPA is the Javascript channel and object that gets injected into the WebView from the logic layer. We define this in message_handler.dart:

import 'package:superapp/miniapp/jsbridge.dart';
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';
import 'package:flutter_liquidcore/liquidcore.dart';
import 'package:flutter/services.dart';
import 'dart:async';


JavascriptChannel logicMessageHandler(BuildContext context, Completer<WebViewController> controller) {
  return JavascriptChannel(
    name: 'SUPA',
    onMessageReceived: (JavascriptMessage message) async {
      print(message.message);
      switch(message.message) {
        case 'onMyFunc': {
          JSContext _jsContext;
          String _jsContextResponse = '<empty>';

          _jsContextResponse = await _executeJavascriptEngine(_jsContext, 'onMyFunc');
          // Logic layer needs to run this registered js callback in miniapp/index.js
          print('Response from isolated js code was: $_jsContextResponse');
          // Return this to the renderer to call re-render
          
          controller.future.then((webviewController){
            webviewController.evaluateJavascript('document._observer.notify("$_jsContextResponse")');
          });
        }
        break;

        case 'phoneCall': {
          launchCaller();
        }
        break;

        default: {
          // do nothing
        }
        break;
      }
    }
  );
}

Future<String> _executeJavascriptEngine(JSContext context, String funcName) async {
  String _response = '<empty>';

  try {
    if (context == null) {
      context = new JSContext();
    }

  String _pageCode = await rootBundle.loadString('assets/miniapp/index.js');

  _response = await context.evaluateScript("""
    (function(){ 
      var _page = {};
      var Page = function(params){
        _page = params;
      }; 
      $_pageCode 
      return _page.$funcName(); 
    })();
    """);

  } catch(e) {
    print('Got script exception: $e');
  }

  return _response;

}

I'll break this logic into two different parts. The first one is the native call to the bridge to make a phone call. Ignore the Javascript engine code for now.

Native Bridge to the Logic Layer

In the code above, when the message passed is phoneCall, the logic calls the method launchCaller() which happens to sit inside our jsbridge.dart:

import 'dart:async';
import 'package:url_launcher/url_launcher.dart';
import 'package:webview_flutter/webview_flutter.dart';
import 'package:flutter/material.dart';

// Can plug in direct listener if wanted
JavascriptChannel callNumberChannel(BuildContext context) {
  return JavascriptChannel(
    name: 'NATIVE',
    onMessageReceived: (JavascriptMessage message) {
      launchCaller();

      print(message.message);
    }
  );
}

Future<Null>launchCaller() async {
  const url = "tel:1234567";   
  if (await canLaunch(url)) {
    await launch(url);
  } else {
    throw 'Could not launch $url';
  }   
}

This is fairly straight forward, it uses the url_launcher dependency to call our telephone activity. I fixed the number to 1234567.

When clicking the native function button in the WebView, we should see this:

The Flutter debugger shows the print out and the phone dialler activity gets called. Cool, that's our native hook!

Isolated Javascript Engine

Now, the slightly more involved one is running the Javascript code specified by the user in a safe and isolated environment. Up to this point I haven't yet addressed where the third party specifies their function callbacks and custom logic.

Well that happens to sit inside their index.js file:

Page({
    
    initialData: {
      text: "This is page data."
    },
    onLoad: function(options) {
    },
    onReady: function() {
    },
    onShow: function() {
    },
    onHide: function() {
    },
    onUnload: function() {
    },
    onPageScroll: function() {
    },
    onResize: function() {
    },
    // Event handler.
    onMyFunc: function() {
      return "Custom Page Handler called";
    }
  });

All my function does is return a static string: Custom Page Handler called. I left a bunch of them empty as a reminder that when building a true SuperApp framework, you need to consider the entire lifecycle of your host application and your Mini App. The vendor will most likely need to hook into the Page load, scrolling actions and the App load and unload events. I left app.js empty to remind myself of that fact.

Alright, so how do we actually load this into the engine? Well remember renderer.dart? It called this method here:

  _response = await context.evaluateScript("""
    (function(){ 
      var _page = {};
      var Page = function(params){
        _page = params;
      }; 
      $_pageCode 
      return _page.$funcName(); 
    })();
    """);

It does a function name replacement at $funcName() which in this case is onMyFunc.

Does this mean Flutter needs to hard code these function names? No, it doesn't have to. Just for demo purposes we do this. But if you were building your own version of the SDK, I would recommend that you limit what functions can be called and not just allow any arbitrary Javascript functions to be called.

The true magic here lies in the V8 engine that runs this function in isolation. I used LiquidCore to do that. It allows me to run the code in isolation and retrieve the results. Have a look into the documentation for more details.

Finally, when the data comes back from the JS engine, we use the WebView controller to pass it from Flutter into the WebView and execute its specified observer notifier function. We should see the view automatically get updated and just the component that changed:

TaDa! Success.

More Considerations

If you've read this far, thank you so much. There are just a few other considerations I would like to point out.

Keep in mind that you should control the page load and page views. For the best user experience you don't want the third party vendors to be in control of navigation. This means extra work in pushing and popping the WebViews and extra configuration on the Mini App development side.

Create your own SDK and framework. Strip out all tricky functions in Javascript and only white list the ones you want to allow third parties to use.

Create your own web component engine if you have to. WeChat did because they needed something more performant. Ensure that your re-rendering times aren't abysmal.

Don't underestimate the people needed to actually have an ecosystem. It's more than just the technical feasibility. You also need to consider an entire Mini App review process and create third party tooling so that people creating Mini Apps on your platform have a good experience too.

I had a lot of fun learning the various technology to put this together and a hilarious time using Google translate through some Chinese documentation to take a guess at how they did it.

I hope you had fun reading too.

Here is a link to the sample code: https://github.com/serinth/superapp