Overview

Adama is many things, and it is hard to nail down one precise thing. However, Adama is a fundamental rethink of how to build infrastructure that brings people together. The aspiration is that Adama is simple, becomes well understood, and changes the world. This is why Adama is 100% open source.

Since "What is Adama" is hard to pin down, you can gain insights by reading Adama as a ?. Adama has a history in board games which you can read more about, and you can learn more from the requirements that drove Adama's engineering.

You can also jump in today with the CLI by following the tutorial.

Concepts of Adama, the platform

There are two big concepts to deal with. First, the notion of a space and then individual living documents.

Spaces

A space is collection of living documents with shared Adama code and a public website.

TODO: outline limits

Living Documents

A living document is a JSON document with change of history that is under the influence of Adama code.

TODO: outline limits

Requirements driving Adama (the birth of Board Game Infrastructure)

Since answering "What is Adama" is hard, let's start with the requirements that drove Adama in the first place as that provides essential clues to why exists.

This all started after seeing TVs placed within tables to play games (like here , here , or here). These table are pretty neat, so I thought I would build one myself. Low and behold, I decided to focus on the software aspect to build a digital version of battlestar galatica (BSG).

The core thesis of having the TV be the board is that this enables easier setup and teardown, rule enforcement, rule automation, rich media integration, no lost pieces, and more. Furthermore, by lowering the setup cost, it enables a quick play through to teach the rules and tutorials can be used to speed up the players' understanding of the game.

Since there is private state within BSG, I would need to bring people into the TV with their device. Fortunately, everyone these days has a smart phone, so the TV would need to be a server. We must contend with networking to deal with multiple devices owned by different people of varying degrees of capabilities.

Requirement #1: We need to deal with networking up-front.

As an ideal, players should be able to continue play during a network outage. Power outages can be handled with a battery. We must allow players to be able to host locally without any cloud nonsense.

Requirement #2: Games should be hostable locally.

Sadly, home networking is a bit of a mess and outage are relatively a rare event. For easier game play start, the cloud is a fantastic option for 99% of the time.

Requirement #3: Games should be hostable on the cloud.

The cloud opens up many possibilities, and people can play with multiple TVs distributed across various homes. This would allow couples to share a TV while playing on their phones with friends in another state. Online play opens more opportunities for more players to connect without the hassle of geography or existing relationships.

Requirement #4: Games should behave like a virtual room.

Connectivity to the cloud or local network can be spotty, or players can take exceptionally long time per turn; thus, players should be able to jump in/out easily.

Requirement #5: Players should receive notifications when it is their turn to take an action.

Some players are more social rather than competitive, so mistakes are common and easy enough to do in person with a physical board.

Requirement #6: Undo/rewind are important for social-driven players

Since some games have minimum player requirements, it may be interesting to include a bot to learn the game mechanics. Furthermore, games are best when balanced fairly well, and there should not be any exceptionally obvious advantages of initial choices.

Requirement #7: Games should support A.I. to play and seek balance.

At this point, the user requirements are leaning us towards some technical decisions. For instance, we need a client/server model due to the cloud, and we skip any kind of decentralized architecture because of privacy.

Requirement #8: We need privacy between players such that game integrity is maintained.

The client/server model requires a protocol, and the state complexity of some games can be staggering. The mental model however can become much simpler if we see the controllers and TVs as thin/dumb clients similar to a virtual PC or game streaming service. When you have a thin/dumb client, the server is responsible for everything. Here, the server picks up the role of dungeon master and then asks players questions directly.

Requirement #9: Streams vastly reduce the client complexity.

Unfortunately, the modern cloud doesn't play well with streams for a variety of reasons. See woe of websocket for more details. Specifically, the cloud works really well with request/response and databases. A process within the cloud may terminate for a variety of reasons: deployment, kernel upgrade, machine migration, capacity management, host failure, etc.

Requirement #10: The stream must be reliable over failures.

Host failures can be accounted for in a variety of ways (VMs that float around), but the developer situation is important too. As a developer play tests a game and pushes the boundary on all the rules, it is desirable to be able to upgrade/hot-reload the code. This ability to hot-reload requires the need to rewind state to avoid the quadratic state build-up.

Requirement #11: The stream must be reliable over code changes.

Some games adopt house rules, and the server side should be fairly easy to mod or change in a social setting. The client side should be a mostly dumb projection of the client.

Requirement #12: The entire infrastructure should be represented by a single file.


That's a lot, and Adama addresses all these challenges.

The Basics of Adama

Hello there! Welcome to the introduction of the Adama Platform book. In this chapter, we are going to explore the core idioms and language of what Adama is and how it helps you.

So, what is the Adama Platform? Well, we could kick this off with some buzzword bingo by saying that the Adama platform is an open-source reactive server-less privacy-first document-oriented compute-centric key-value store acting like a platform as a service, but that doesn't communicate much (or, does it?). However, we have to start somewhere, so let's tear down those buzzwords with more words.

Let's start with document orientated key-value store. Adama stores documents, and documents are identified via a key (hence the key-value). Documents are organized by a space (which is similar to the bucket concept used by S3 except with a more mathematical feel). Adama has variants of the four big CRUD operations, but there are notable differences which make the Adama platform unique. Deconstructing the CRUD operations is the best way to teardown the buzzword bingo.

Creating

The first notable aspect is that Adama documents are created on the server via a constructor. This constructor is defined with code that is bound to the space holding the document. Wait, what? This is where the compute-centric comes into play as each space has code as config. All documents within a space share the same Adama (the language) code, and the Adama code defines the document schema, logic for transformation, access control, and more. This is why the name space was chosen over bucket because buckets can only have fixed config while spaces have infinite potential.

For a clear example, the below code illustrates valid Adama code which we will tear down.

// static code runs without a document instance
@static {

  // 1. a policy which is run to validate the given user can create the document
  create {
    return @who.isAdamaDeveloper();
  }
}

// 2. the document schema has a creator and an integer named x
private principal creator;
public int x;

// 3. the constructor is a message named by the document
message ConsXYZ {
  int x;
}

// 4. connect the constructor message to code
@construct (ConsXYZ c) {
  creator = @who;
  x = 100 + c.x;
}

Admittedly, this is a lot of ceremony to get a document that looks like:

{"x":123}

when constructed with a message like

{"x":23}

However, this document contains (1) an access control mechanism for who can create a document, (2) a document schema with privacy as a first class citizen, (3) a message interface for validating input structure, and (4) logic to construct the state of the document.

As a rule, documents can only be created once and race conditions go to first creator. It's worth noting that the document's state is different from the state used to construct it which enables developers to think in their domain rather than the document's schema. This is very similar to Alan Kay's original thinking around object-oriented programming and Carl Hewitt's actor model.

Reading

Once a document is created, documents can be read by connecting to the document. This requires further Adama code because there is a need to gate access to the document, and we append this code to the above code.

// 5. gate who can connect to the document
@connected {
   return @who == creator;
}

This will allow people to connect or not. The reason we say connect instead of read or get is because we establish a long-lived stream between the client and the document which allows changes to flow from the document to all clients with minimal cost; this explains the reactive buzzword.

Updating

Updates to the document happen by sending messages to the document via channels; channels are basically procedures exposed to clients. For example, we can open a few channels to manipulate our document in various ways.

message Nothing {}
message Param { int z; }

channel square(Nothing n) {
  x = x * x;
}

channel zero(Nothing n) {
  x = 0;
} 

channel add(Param p) {
  x += p.z;
}

Messages will hit a document exactly once, run the associated logic, change the document state, and all connected clients will reactively receive a change to update their version of the document. Access control is possible per channel, but it is worth noting that only connected clients can send to a document (by default).

Deleting

Deleting happens from within the document via logic within a message handler.

channel kill(Nothing n) {
  if (@who == creator) {
    Document.destroy();
  }
}

Or, there is an API "document/delete" which allows creation from the outside which will invoke the @delete event.

@delete {
  return @who == creator;
}

The core motivation for this is that access control for deletion requires business logic. Since Adama documents can run code based on time passing, this also enables documents to self-destruct.

Buzzword Bingo Summary

With the CRUD operations laid bare, we can analyze the buzzword bingo aspects in a table:

BuzzwordTranslation
open-sourceYes, all the source for the platform is hosted on Github
reactiveThe connection from client to server uses a stream such that updates flow to client as they happen
server-lessThe servers are managed by the platform, and developers only have to think about keys and documents
privacy-firstThe language has many privacy mechanisms that happen during document schema definition enforced during run-time. It's entirely possible for the document to hold state that is never readable by any human without hacking.
document-orientedAdama maps keys to values, and those values are documents
key-value storeAdama use a NoSQL design mapping keys to values
platform as a serviceAdama provides the trinity of compute, storage, and networking which is enough to build many products. Adama is designed to pair exceptionally well with a web browser.

So, how does this help? What is Adama's value proposition?

This is the big question. At core, Adama takes the trinity of the cloud: compute, storage, and networking and bundles them into one offering. The value proposition is thusly multi-faceted depending on various markets:

  • Jamstack developers are able to hook their application up directly to the Adama platform such that privacy, security, query, and transformation are provided out of the box.
  • Game developers can leverage Adama platform to act as a serverless platform for both a game lobby and a game server (The history of Adama starts with board games).
  • Any website can integrate Adama as a durable and reliable real-time service for chat, presence, web-rtc signalling, and more without managing servers.

Beyond making development easier, business operations is further aided by having a tunable history of changes to the document available which makes auditing changes easy as well as having a universal rewind.

Adama as a ?

by Jeffrey M. Barber

When I worked at Amazon S3, there was a meme on many office walls of the form "S3 as a _____" with a long list of answers. S3 is heavily abused because it is simple, and this is a fantastic thing. Well, it's also frustrating as that abuse could often lead to late night pages. I am aspiring to keep Adama simple, but I've had a tough time answering "What is Adama". Instead, I'll provide a tsunami of answers in the form of "Adama as a ____".

Adama as board game infrastructure

Board games is where Adama was spawned from which is why the most myopic use-case for Adama is first. However, it's worth thinking about as it sets context for all the other scenarios. The motivating question was how to get control of all the state and logic of building a fantastic board game: Battlestar Galactica (BSG).

This started by taking control of all the game state with a domain specific language. By code-generating the state structures, the state could be easily transactionable such that snapshots and undo become possible features. With that as a foundation, a collaborative framework could be built such that document state could easily replicate to players. However, many games are competitive which requires privacy. The moment privacy became a concern for the domain specific language was a feature cascade as the language became mostly turing complete.

As the language to transform the document emerged, so to moved all the board game logic. Sitting in a private place is a complete back-end for BSG which I can't release (sad face).

Adama as a head-less Excel

The interesting thing is that privacy of state also begs the question of how to have private computations for individual viewers, and this injected reactivity into the language which was yet another productivity boon which made board games exceptionally easier to build.

The moment you have reactivity, you have the foundation that makes Excel powerful. Suddenly, computing is more accessible and Adama greatly simplifies the burden of building multiplayer experiences.

All you have to do is accept messages, change the document, and computational changes flow to users in a differentiable form.

Adama as serverless multiplayer game hosting

There is a spectrum of multiplayer experiences from board games to MMOs or FPS, and Adama can provide infrastructure for the metaverse where all games share common infrastructure. Game developers can focus on their game logic within Adama and build exceptional experiences using any game engine.

Adama as a low-code collaboration storage and networking engine

A trivial consequence of bringing people together around board games is that normal applications benefit from the investments. Whether an application requires conflict-free replicated data types or operational transforms, Adama provides a medium for applications to share state via a document abstraction. All the burdens of networking, synchronization, and reliable communication are simplified as Adama steps in as a message broker and document store.

Beyond sharing state, computations allow that state to become much more than dead bytes.

Adama as a real-time data store

Adama can be used to build publisher/subscriber systems, presence, live-anything, games. Adama leverages a socket-first approach because reality is real-time, and goal of having games on Adama means gamer demands will drive the platform towards excellence.

Adama as a SMB application provider

With Adama providing both state and compute along with a fundamentally simpler networking idiom, entire applications can be built with just Adama as the back-end. The beauty of these apps is that they don't require complex setups nor configuration. Simply point a UI at a document, and boom you have an app which you can host on the Adama Platform or on-site with your own machines.

It's worth noting that a document within Adama can be massive (as large as the Java heap) because state is persisted via change deltas, and this can scale up to many businesses for their entire lifetime.

Adama as a multi-tenant SMB platform

Since a single document can be used to power an entire application, Adama allows businesses to spawn off clients within their own documents. As most tenants fit within a single machine, they get fantastic properties around data and privacy regulations since the model is easily isolated from other businesses across different geopolitical boundaries.

Adama as a garbage collecting storage proxy

Blobs of data can be attached to an Adama document, and these blobs are called assets (which can be thought of as attachments). These blobs are stored within Amazon S3. As documents change over time with assets coming and going, Adama presents a unique opportunity to precisely control storage by garbage collecting assets against what is stored in the Adama document's history.

Adama as a web hosting provider

Since Adama has a web-server built-into it (for websockets and assets), it was low-hanging fruit to connect HTTP to an Adama document. This is a work in progress, but the vertical integration of the Adama platform will include many of key features required to host both static sites and then provide dynamic services.

Adama as a web hook listener

Since Adama supports HTTP verbs routed to documents, various services can talk to Adama like twilio, discord, or slack. Any webhook provider can feed data into an Adama document.

Adama as a massively scalable "real-time" data-base

As a future possibility, Adama can support a large number of writers (~1K) with infinite readers having personalized views. This is possible due to the fact that Adama documents are tiny databases which emit change logs, and change logs can be shipped across regions naturally. The moment you have a replication topology built from change logs, you can achieve infinite read scale.

Adama as a cron-service

Each Adama document has a state machine loop which can be leveraged to sleep for periods of time, do stuff, and then go back to sleep with a timer to wake up again.

Adama as a workflow coordinator

Adama can reach out to people and await a result. This essential feature for board games allows coordination between customers and staff for something like order fulfillment. A small restaurant could be backed by a single Adama document such that customers place orders, staff are then alerted about the order, and document can block until the staff acknowledge the order.

Adama as a personalized queue

The state machine loop is fundamentally a queue which can be leveraged to do stuff over time such that a mainline path is unblocked. Instead of a massive queue for an entire enterprise, the queue is per document such that precision and insight is available.

Adama as a durable application gateway

Adama can call other services in a reliable manner and durable manner. Beyond allowing Adama to broker a request to another back-end, it can make multiple requests to multiple back-ends without worry of partial failure. This means that Adama prevents torn-writes as Adama has a state machine and a queue to broker the idempotent requests.

Adama as an edge compute serverless function

Adama provides arbitrary almost-turing-complete capabilities, and as such can be used to host both stateful and stateless functions which do things. With massive scale potential, documents can be replicated close to users such that the associated behaviors can be executed very quickly.

Adama as a privacy-aware CDN

Adama providing HTTP, assets as arbitrary blobs of data, and massive scalable potential enables Adama to act as a privacy aware CDN such that resources are not blindly cached at the edge for any passerby lucky enough to get a URL. Instead, the CDN can execute privacy logic close to users such that assets are vended securely.

Adama as a "bit much"

At this point, Adama is looking up from a capabilities perspective. Clearly, there has been a tremendous amount of scope creep, but many of these aspects are consequential of having a turing complete language and sane data model.

Tutorial

Let's get hands on with Adama!

The key steps are going to be

  • Installing the CLI tool which is the primary way to interact with the Adama platform. This tutorial will help you understand the dependencies and validate that the tool is appropriately setup.
  • Initializing your developer account will get you on-boarded to production Adama fleet. This tutorial will walk you through account creation.
  • Creating a space will get you started with a template space and start making an experience. This tutorial will show the essential operations to create, deploy, and use a space.
  • Bringing users into your documents will explain how to invent users on your existing infrastructure or enable social login for your space. This tutorial will explain how to bring people in to use your Adama documents.
  • Using the JavaScript client will walk you through how to use your new space with the JavaScript client. This tutorial will explain how to use the JavaScript API to build applications with Adama.

Installing the tool

First thing you need to do is install Java. Either use your distribution's version of Java 11+, or please refer to Oracle's website for how to install Java 17. You can check that you ready when the command:

java -version

shows something like

openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)

That's all you need for Adama to work. Once Java is working, you can download the latest jar using wget download directly from github.

wget https://github.com/mathgladiator/adama-lang/releases/download/nightly/adama.jar
java -jar adama.jar

or (if you lack wget)

curl -fSLO https://github.com/mathgladiator/adama-lang/releases/download/nightly/adama.jar
java -jar adama.jar

to get help on how to use the jar. The next step is to initialize your developer account.

Initializing your developer account

java -jar adama.jar init

Please note the notice about early access!

This will then prompt you with a blurb of text that outlines that providing your email will be implicit acceptance to Adama's terms, conditions, and privacy policy. You should read them!

Once we have your email, we will send you a verification email with a code. Please copy and paste that code into the terminal, and your account will be setup.

Oh, and if you want to revoke other machines, then this is a great time to do it by inputting Y when asked to revoke.

This tool will drop a file (.adama) within your home directory to act as your default config. You can, of course, override this with the --config parameter.

For now, let's move on towards creating a space...

Creating a space

Fundamentally, a space is a namespace/container for Adama documents which share a common document script.

Spaces are global resources, so you may need to be a bit clever with how you name them as conflicts between other developers can happen. This is the rough spot of the tutorial since I can't tell you exactly what to type as you will have to invent a name. However, I can provide an example of creating a space.

java -jar adama.jar space create --space chat001

And, if you don't see a bloody mess of an error message, then your space is created! Huzzah!

You can poke around the space sub command as well. For instance, you can investigate your options by invoking the help on the space sub command via

java -jar adama.jar space help

And one option available to you is to list all your spaces via

java -jar adama.jar space list

For me, using my freshly made account to test this tooling and your experience, produced a list of JSON object containing

{
  "space" : "chat001",
  "role" : "owner",
  "created" : "2022-02-09",
  "enabled" : true,
  "storage-bytes" : 0
}

This object reveals the name of the space, role of the person doing the listing, date when the space was created, whether or not the space is currently enabled, and finally the total storage used by the space.

The space is now created in an empty state, so let's create some people to leverage it.

Bringing existing users into Adama

The Adama Platform use JWT tokens to authenticate people via their devices, and all Adama developers can use their credentials to talk to the Adama Platform and their documents. It would be an exceptionally limited (yet mephistophelian) requirement for all users to be an Adama developer, so the Adama Platform allows developers to manage public keys. This is done by the developer creating an authority:

java -jar adama.jar authority create

This will return a document like

{
  "authority" : "Z2YISR3YMJN29XZ2"
}

The Z2YISR3YMJN29XZ2 is a unique key for developers to use to identify their users. If you accidentally clear your terminal or lose that id, then you can list your authorities via:

java -jar adama.jar authority list

With the name of the authority in hand, we will use the tool to create a keystore

java -jar adama.jar authority create-local \
 --authority Z2YISR3YMJN29XZ2 \
 --keystore my.keystore.json \
 --private first.private.key.json

This will create two files within your working directory:

  • my.keystore.json is a collection of public keys used by Adama to validate a private
  • first.private.key.json is a private key used by your software to sign your users' id. This requires safe-keeping!

This keystore and private key were created entirely locally on your machine (for exceptional security), and now you upload only the keystore with:

java -jar adama.jar authority set \
  --authority Z2YISR3YMJN29XZ2 \
  --keystore my.keystore.json

This will allow the users signed by that private key into Adama. Consuming the private key will require some crypto libraries in some infrastructure that you manage, but we can get started by using the Adama tooling to create an identity today!

java -jar adama.jar authority sign \
  --key first.private.key.json \
  --agent user001 

which will dump out a JWT token with the agent 'user001' as the subject:

eyJhbFUzNiJ9.eyJdWIiTjI5WFoyIn0.TQZbOkE9abE24_8w

This is the string that you use as the identity parameter with the Client API. For now, let's create a second token.

java -jar adama.jar authority sign \
  --key first.private.key.json \
  --agent user002

We will use the corresponding tokens for user001 and user002 to chat with each other by configuring the space.

Writing and deploying Adama code to a space

With the space created, we can now deploy some code to it! Create a file called chat.adama

The first thing we need to do is some ceremony around who can create documents.

@static {
  // only allow users from the authority we just created
  create {
    return @who.fromAuthority("Z2YISR3YMJN29XZ2");
  }

  // Here "Inventing" is the act of creating the
  // document on demand with connect with no need 
  // for a create(...) call
  invent {
    return @who.fromAuthority("Z2YISR3YMJN29XZ2");
  }
}

Please note that when you copy-pasta this, the Z2YISR3YMJN29XZ2 will need to be replaced with the authority used to generate your initial users from the prior step.

Now, empty documents can be created with the above policy work. Unfortunately, there is no data and no one can connect. Let's enable connections

// let anyone into the document
@connected {
  return @who.fromAuthority("Z2YISR3YMJN29XZ2");
}

Now, people within the developer's authority can connect to the sadly empty document. So, let's add some data.

// `who said `what `when
record Line {
  public principal who;
  public string what;
  public long when;
}

// a table will privately store messages
table<Line> _chat;

// since we want all connected parties to
// see everything, just reactively expose it
public formula chat = iterate _chat;

The document has structure, so let's enable users to populate the chat.

// what users will say stored in a message
message Say {
  string what;
}

// the "channel" enables someone to send a message
// bound to some code
channel say(Say what) {
  // ingest the line into the chat
  _chat <- {who:@who, what:what.what, when: Time.now()};
  
  // since you are paying for the chat, let's cap the 
  // size to 50 total messages.
  (iterate _chat order by when desc offset 50).delete();
}

At this point, the backend for chat is done and we can upload it via:

java -jar adama.jar spaces deploy --space chat001 --file chat.adama

With the space uploaded, you can now build a UI with only HTML.

Using the JavaScript client

The stage is set! Let's use some vanilla.js to craft a new browser experience with Adama powering the back-end. Note, this example is derived from the chat example available from github..

<!DOCTYPE html>
  <html>
    <head>
      <title>Adama Vanilla.JS Chat</title>
      <script src="https://aws-us-east-2.adama-platform.com/libadama.js"></script>
    </head>
  <body>
  <div id="status"></div>
  <table border="0">
    <tr>
      <td colspan="2" id="setup">
        <fieldset>
          <legend>Inputs: Space, Key, and Identities</legend>
          <label for="space">Space (valid characters are a-z, 0-9, - and _)</label>
          <input type="text" id="space" name="space" value="chat000" size="100"/>
          <br />
          <label for="key">Key (valid characters are a-z, 0-9, -, ., and _)</label>
          <input type="text" id="key" name="key" value="room-as-key" size="100"/>
          <br />
          <label for="identity-user-1">User 1</label>
          <input type="text" id="identity-user-1" name="identity-user-1" size="100"/>
          <br />
          <label for="identity-user-2">User 2</label>
          <input type="text" id="identity-user-2" name="identity-user-2" size="100"/>
          <br />
          <button id="connect">Connect both users</button>
          </fieldset>
        </td>
    </tr>
    <tr>
      <td>
        <fieldset>
          <legend>Chat Log (User 1)</legend>
          <div id="chat-output-1"></div>
          <label for="speak-user-1">User 1 Says What</label>
          <input type="text" id="speak-user-1" size="25"/>
          <br />
          <button id="send-1">Speak</button>
        </fieldset>
      </td>
      <td>
        <fieldset>
          <legend>Chat Log (User 2)</legend>
          <div id="chat-output-2"></div>
          <label for="speak-user-2">User 2 Says What</label>
          <input type="text" id="speak-user-2" size="25"/>
          <br />
          <button id="send-2">Speak</button>
        </fieldset>
      </td>
    </tr>
  </table>
</body>
  <script>
     // INSERT CODE BELOW HERE
  </script>
</html>

For your own personal sake, it would be useful to replace chat000 with whatever name you choose for a space. This mess of old-school HTML is a skeleton to demonstrate the basics, so let's connect to Adama.

// connect to Adama
var connection = new Adama.Connection(Adama.Production);
connection.start();

// wait until we are connected
document.getElementById("status").innerHTML = "Connecting to production...";
connection.wait_connected().then(function() {
  document.getElementById("status").innerHTML = "Connected!!!";
});

Before we can make the connect button work, we will create a handler for dealing with document deltas. At core, the below code bridges how data from Adama flows into the DOM.

// write chat changes to the DOM
function makeBoundTree(outputId) {
  var tree = new AdamaTree();
  tree.subscribe({chat: function(chat) {
      var lines = [];
      lines.push("<table border=\"1\"><thead><tr><th>Who</th><th>Said</th></tr></thead><tbody>");
      for (var k = 0; k < chat.length; k++) {
        lines.push("<tr><td>" + chat[k].who.agent + "</td><td>" + chat[k].what + "</td></tr>");
      }
      lines.push("</tbody></table>");
      document.getElementById(outputId).innerHTML = lines.join("");
    }});
  return {
    next: function(payload) {
      if ('delta' in payload) {
        var delta = payload.delta;
        if ('data' in delta) {
          tree.update(delta.data);
        }
      }
    },
    complete: function() {
      document.getElementById(outputId).innerHTML = "chat completed";
    },
    failure: function(reason) {
      document.getElementById(outputId).innerHTML = "Failed: " + reason;
    }
  };
}

// log send errors to console.log
function failureToConsoleLog(prefix) {
  return {
    success: function() {},
    failure: function(reason) {
      console.log(prefix + reason);
    }
  };
}

The above code will simply manifest changes of the following chat formula from chat.adama into the DOM.

public formula chat = iterate _chat;

It does this by creating a tree which will absorb data differentials and rebuild the chat lines. Now we make the connect button work!

document.getElementById("connect").onclick = function() {
  // fetch the input values
  var space = document.getElementById('space').value;
  var key = document.getElementById('key').value;
  var identity1 = document.getElementById('identity-user-1').value;
  var identity2 = document.getElementById('identity-user-2').value;

  // create the connections to the document and bind them to the DOM
  var connection1 = connection.ConnectionCreate(
    identity1, space, key, {}, makeBoundTree('chat-output-1'));
  var connection2 = connection.ConnectionCreate(
    identity2, space, key, {}, makeBoundTree('chat-output-2'));

  // hook up the buttons to send messages to the say channel per user
  document.getElementById("send-1").onclick = function() {
    connection1.send("say",
      {what:document.getElementById("speak-user-1").value}, failureToConsoleLog("user-1 send:"));
  }
  document.getElementById("send-2").onclick = function() {
    connection2.send("say",
      {what:document.getElementById("speak-user-2").value}, failureToConsoleLog("user-2 send:"));
  }

  // remove the setup html
  document.getElementById("setup").innerHTML = "";
}

This will connect each user's identity to appropriate window and make the buttons work. Here, we can observe that reactivity is no longer a client concern. Instead, we have very simple JavaScript with data bound to a tree.

At this point, the tutorial is over which is sad. However, there are examples to inspect. Given the early release nature of this, questions and feedback are welcomed!

The best place for help is to join the discord channel!

Language Tour

Since the central way of interacting with documents held within a space is via the Adama language, let's take a tour.

This document is a lightweight tour of the core features and ideas that make Adama a somewhat novel data-centric programming language. It is important to remember that Adama is not a general purpose language. It’s for board games (and maybe more... much more).

Defining State Layout

We start by defining global fields with default values. Laying out and structuring data is arguably the most important activity in building software, so it must be simple and convenient for developers to define their data. Here, we will define a document with just a name and score field:

public string name = "You";
private int score = 100;

This fairly minimal Adama code defines a document. The backend data for this script is a document represented via the following JSON:

{"name":"You", "score":100}

However, a human viewer (such as yourself or myself) of the document will only see:

{"name":"You"}

This is because the score field is defined with the private modifier. By modifying a field with private, only the code within the document can operate on the score field. This is useful, for example, to define secrets in a game. A key function in board games is the need for secrets (i.e. the contents of your hand) or an unrevealed state of objects such as the ordering of cards within a deck.

The following diagram visualizes the Adama environment and architecture:

Architecture diagram showing you, the data you see, how it connects to a store which runs the Adama code

Here is a brief overview of the Adama working environment:

  • People connect (via a client) to the Adama Platform with a persistent connection.
  • Adama will then send to you a private and personalized version of the document.
  • People send messages to the document, and Adama will run code on the message to validate and change the document.
  • Adama will send updates while respecting the privacy based on directives (e.g., the private modifier sets the score variable as private in the above example).

Adama is not only a data-centric programming language, but a privacy-focused language such that secrets between players (i.e. individual hands) and the universe (i.e. decks) are not disclosed. This environment is essential for games requiring secrets so that other gamers do not gain an unfair advantage from "hacking" environment variables.

Organizing the Chaos Induced by Globals

Having a giant pool of global fields will lead to chaos and copypasta, so we introduce records as a way combining fields around an entity.

record Card {
  public int suit;
  public int rank;
}

public Card a;
public Card b;

A record is a structure that defines one or more named typed fields under a single type name. In the above example, the structure Card is the combination of suit and rank integer fields. These structures can then be used to create instances within the document of that type. The above code backend would have the following JSON:

{
  "a":{"suit":0, "rank":0},
  "b":{"suit":0, "rank":0}
}

The above example is great for cleaning up patterns within the global document, but this is insufficient for non-trivial games. The next step is to introduce a collection of records. Adama provides the notion of a table, and the above record can be used to create a table named deck.

table<Card> deck;

But this begs the question: how do records flow into the deck? Adama uses events that can be associated with developer code which is evaluated when events trigger. This code can then manipulate the document.

One example of an event is the creation of the document. The event is created via a constructor (using the @construct identifier). This constructor can be used with an "ingestion" operator (<-) and some C style for loops. The following Adama code builds a table of Card records based on the JSON document structure:

@construct {
  for (int s = 0; s < 4; s++) {
    for (int r = 0; r < 13; r++) {
      deck <- {suit:s, rank:r};
    }
  }
}

The above code will construct the state of the document representing a typical deck of cards containing 52 cards, 4 suits and 13 cards per suit. Tables are always private in Adama, so viewers of the document will not see table structures. However, the data contained within the table will be viewable. Queries against the table expose selected data to people such as players. As an example, the following code will let everyone know the size of the deck:

public formula deck_size = deck.size();

The above formula variable represents Adama's reactive programming language. As the deck undergoes changes during gameplay, the formula variables depending on that deck will be recomputed and updates will be sent to viewers such as players in the game. For efficiency, this is done once message processing stops.

Because Adama continually updates the state of document, the connection from your device to the Adama Document Store uses a socket. The socket provides a way for the server to know the state of the client, and then minimize the compute overhead on the server. This enables small data changes to manifest in small compute changes that translate to less network usage. Less network usage translates to less client device compute overhead, and this manifests into less battery consumption for the end-user. Board games can last for hours when they leverage Adama's reduction in battery power consumption.

A table is an exceptionally powerful tool, and Adama uses language integrated query (LINQ) to query data. Using the Card structure, the following example adds a client type to the Card record to indicate possession of the card:

record Card {
  public int suit;
  public int rank;
  public principal owner;
}

The above Card record allows us to share how many cards are unassigned in the deck via a formula. The code to do this is below:

public formula deck_remaining = (iterate deck where owner == @no_one).size()

Here @no_one is a special default value for the client type which indicates that cards are unassigned. We can leverage a bubble to share a viewer's hand (if they are a player and not a random observer).

bubble hand = iterate deck where owner == @who;

The bubble is special type of formula which allows data to be computed based on who is viewing the document. This allows people to have a personalized view of the document such as being able to see their hand. As the deck and rows within the deck experience change, the formulas update automatically based on precise static analysis. These changes propagate to all viewers in a predictable way.

Messages from Devices to the Document

Changing the document is done via people sending messages to the document.

Adama acts as a message receiver of messages sent by the client. We can model a message similar to a record. For instance, we can design a message that says "I wish to draw $count cards" demonstrated below:

message Draw {
  int count;
}

This message encodes the product intent, and we can associate code to that message via a channel.

channel draw_cards(Draw d) {
  (iterate deck where owner == @no_one shuffle limit d.count).owner = @who;
}

This channel will allow messages of type Draw to flow from the client to the code outlined above. In this case, the code uses a LINQ query to find at most d.count available random cards to associate to the Draw message sender.

Messages alone create a nice theoretical framework, but they may not be practical for games. This messaging works great for things like chat, but it offloads a great deal of burden to both the message handler and the client. For instance, in a game, when can someone draw cards? Can they draw cards at any time? Or during a specific game phase?

Let the Server Take Control!

To control message flow, Adama uses an incomplete channel identifier. An incomplete channel is like a promise that indicates clients may provide a message of a specific type, but only when the document asks for it.

Adama uses a third party to broker the communication between players. That is, it determines who is asking players for messages. This is where the document's finite state machine comes into play. The document can be in exactly one state at any time, and states are represented via hashtags. For instance, #mylabel is a state machine label used to denote a potential state of the document.

We can associate code to a state machine label directly and set the document to that state via the transition keyword.

@construct { // this could also be a message sent after all players are ready
  transition #round;
}

#round {
  // code to run
}

In this example, the associated code attached to #round will run after the constructor has run and the document has been persisted. An important property of the state machine is that it defines an atomic boundary for both persisting to a durable store and when to share changes to the document.

Only the transition keyword can set the document's next state label to run. For instance, the following is an infinite state machine:

public int turn;

#round {
  turn++;
  transition #round;
}

The reason we took this detour is to have a third party be able to use the incomplete channel. For instance, the document somehow learns of two players within a game; these players' associated clients are stored within the document via:

private principal player1;
private principal player2;

Now, we can define an incomplete channel for the document to ask players for cards.

channel<Draw> how_many_cards;

This incomplete channel will accept messages only from code via a fetch method on the channel. We can leverage the state machine code to ask players for the number of cards they wish to draw using the following Adama code:

#round {
  future<Draw> f1 = how_many_cards.fetch(player1);
  Draw d1 = f1.await();

  future<Draw> f2 = how_many_cards.fetch(player2);
  Draw d2 = f2.await();
}

This is a productivity win with respect to board games because it inverts the control model away from the client towards the server as synchronous code. This is the key to enforce rules in a coherent way and keep control of the implicit state machine formed as rules compound in complexity.

Time to Reflect

This document took you on a tour of a few of the core ideas found within the Adama programming, and while this is not a comprehensive review it does address some of the novel aspects. The key is that you focus on the data at hand for a single game, and then outline all the ways the game state may change. The rules of the game can be written in a synchronous manner which mirrors how they are executed live with people.

How-to Guides

How to create a Tic Tac Toe Game using Adama Platform

written by David Asaolu

Building an efficient board game can be tedious if you're not using the programming language and resources best suited for creating such gaming applications. In this article, I'll guide you through building a Tic Tac Toe game with Adama, a programming language that allows you to create board games easily.

Adama is a reactive programming language that utilizes an event-driven architecture that enables us to build scalable and efficient applications. Adama started as a tool that provides a better way of representing states in an application, then gradually grew into a fully-fledged programming language. Adama uses a serverless infrastructure whereby a single file can contain an infinite space of documents acting like tiny computers with storage and networking.

Before we go further, let's learn why you should choose Adama when building your applications.

Why choose Adama?

In this section, you'll learn about some of the features provided by Adama that enable us to build efficient real-time applications.

Fast compilation and deployment
Programming in Adama is fast, and compilation and deployment happen immediately after initiating the actions. Adama is a language designed to achieve more functionalities with less effort, cost, and time. In Adama, the validator runs in a single digit millisecond for a moderately large code base.

Backward compatibility
Adama is an innovative language that can run early versions of the program in newer environments without errors. Programs written in the older versions of Adama can run efficiently without issues.

Excellent tool for creating efficient applications
Being a reactive programming language, Adama handles events asynchronously; this enables your program to process real-time updates efficiently and accommodate many users at a time. Adama started as a tool for representing states conveniently before becoming a programming language. Adama aims to make application development easy and even provide more capabilities that will enable you to build affordable and reliable applications.

How to start building with Adama

Here, I will guide you through setting up Adama and how you can start creating efficient applications with Adama. Before building with Adama, you need to install Java on your computer. Head over to Oracle's website and install Java 17.
Once you have completed the installation process, run the code below in your terminal to confirm if the installation was successful.

java -version

It should return something similar to the code below.

openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)

Next, download the latest Adama jar file from GitHub by running this code.

wget https://github.com/mathgladiator/adama-lang/releases/download/nightly/adama.jar

Create an Adama developer account by running the code below. Read through the information and supply your email address. Enter the verification code sent to your email in your terminal.

java -jar adama.jar init

Run the code below to create a space for your Adama document. Adama space is similar to buckets in AWS. It is the container for your Adama documents.

java -jar adama.jar space create --space <your_space_name>

Congratulations! You've just created a space for your Adama document. You can now start creating the backend for the Tic Tac Toe game.

Building the backend for your Tic Tac Toe game

In this section, we'll leverage the tools provided by Adama to build a Tic Tac Toe game. Tic Tac Toe is a game that consists of two users, one is X, and the other is O. The player that succeeds in placing three of its symbols horizontally, vertically, or diagonally is the winner.

Before writing Adama's code, we need to state the document's policy. The code snippet below allows anyone to create a document.

@static {
  // This makes it possible for everyone to create a document.
  create { return true; }
  invent { return true; }

  // As this will spawn on demand, let's clean up when the viewer goes away
  delete_on_close = true;
}  

Below the document's policy, declare the states of each square box - empty or contains the player X or O. From the code snippet below, we created an enum variable type that represents all the three possible states of the application.

// What is the state of a square
enum SquareState { Open, X, O }

Next, let's declare a public variable representing each player. In Adama, the client keyword is assigned to users and contains information related to the user; @no_one is its default value.

// The two players
public principal playerX;
public principal playerO;

// The current player
public principal current;

Create another set of variables containing the draws and win in the game.

// how many wins
public int wins_X;
public int wins_O;

// how many stalemates or player draws
public int stalemates;

Assign roles to each player. In Adama, there is a data type called bubble whose values change depending on the connected user viewing the document; this allows users to have a personalized view of the document.

// personalized data for the connected player:
// show the player their role, a signal if it is their turn, and their wins
bubble your_role = playerX == @who ? "X" : (playerO == @who ? "O" : "Observer");
bubble your_turn = current == @who;
bubble your_wins = playerX == @who ? wins_X : (playerO == @who ? wins_O : 0);

From the code snippet above, your_role assigns the current player viewing the document, "X" and the other "O". The your_turn variable is equal to whether the user viewing the document is the current player. The your_wins variable contains the number of times the current user wins the game.

Next, let's create a record containing every data in each square block. Record is a variable that groups variables related to an entity under a single variable.

// a record of the data in the square
record Square {
  public int id;
  public int x;
  public int y;
  public SquareState state;
}

Since we've been able to represent each box in the tic-tac-toe grid as records, create a table containing every box on the tic-tac-toe table.

// the collection of all square boxes
table<Square> _squares;  

From the code snippet above, the table is the keyword for creating a table. The angle brackets accept the record type as a parameter and convert it to a table. _squares represent the name of the table.

Convert the table to a list and separate the board by using its rows.

// converts the table to a list using the iterate keyword
public formula board = iterate _squares;

// breaks the square into its rows
public formula row1 = iterate _squares where y == 0;
public formula row2 = iterate _squares where y == 1;
public formula row3 = iterate _squares where y == 2;

From the code snippet above, the formula identifier enables us to write an expression on the right side of the equal-to sign. Each row is differentiated using the variable y, which represents the y-axis of the table.

Add the code below to the document. The code snippet loops through the x and y variable from the Square record and saves the result into the _squares table.

@construct {
  for (int y = 0; y < 3; y++) {
    for (int x = 0; x < 3; x++) {
      _squares <- { x:x, y:y, state: SquareState::Open };
    }
  }
  wins_X = 0;
  wins_O = 0;
  stalemates = 0;
}

Note that the @construct event will be fired once after creating the document. Initially, the number of wins for the X, O players, and draws is 0.

Update the document by adding the code below. The code snippet assigns the current player an X or O. After giving both values to the players, the game starts. This event returns true indicating that the player is allowed in the game to either play or observe.

@connected {
  if (playerX == @no_one) { //if no one has been assigned playerX
    playerX = @who;     //assign playerX to the current user
    if (playerO != @no_one) { //if playerO has been assigned to a user
      transition #initiate; //start the game
    }
  } else if (playerO == @no_one) { //if no one has been assigned playerO
    playerO = @who;  //assign playerO to the current user viewing the document
    if (playerX != @no_one) { //if playerX has been assigned to a user
      transition #initiate;   //start the game
    }
  }
  return true; //The user has been successfully connected
}

// the game is afoot
#initiate {
  current = playerX; //playerX is the first person to play
  transition #turn;
}

Create the turn state. The turn state checks for empty spaces in the square table; if there are none, the game records a stalemate, and the stalemates variable is increased by one before it ends.

#turn {
  // find the open spaces
  list<Square> open = iterate _squares where state == SquareState::Open;
  if (open.size() == 0) {
    stalemates++;
    transition #end;
    return;
  }
}

Create a channel to enable players to move between each square box via its id.

// open a channel for players to select a move
message Play { int id; }
channel<Play> play;

Next, add a procedure that determines if there is a win. The code snippet below accepts the values in each square box and returns true if the vertical, horizontal, and diagonal spaces contain the same values of either X or O.

// test if the placed square produced a winning combination
procedure test_placed_for_victory(SquareState placed) -> bool {
  for (int k = 0; k < 3; k++) {
    // vertical lines
    if ( (iterate _squares where x == k && state == placed).size() == 3) {
      return true;
    }
    // horizontal lines
    if ( (iterate _squares where y == k && state == placed).size() == 3) {
      return true;
    }
  }
  // diagonals
  if ( (iterate _squares where y == x && state == placed).size() == 3 ||
       (iterate _squares where y == 2 - x && state == placed).size() == 3 ) {
    return true;
  }
  return false;
}

Update the turn state by copying the code below. The play channel created earlier allows the player to select an open space until one of the players wins or when there is no empty space.

#turn {
  // find the open spaces
  list<Square> open = iterate _squares where state == SquareState::Open;
  if (open.size() == 0) {
    stalemates++;
    transition #end;
    return;
  }

  // ask the current play to choose an open space
  if (play.decide(current, @convert<Play>(open)).await() as pick) {
    // assign the open space to the player
    let placed = playerX == current ? SquareState::X : SquareState::O;;
    (iterate _squares where id == pick.id).state = placed;
    if (test_placed_for_victory(placed)) {
      if (playerX == current) {
        wins_X++;
      } else {
        wins_O++;
      }
      transition #end;
    } else {
      transition #turn;
    }
    current = playerX == current ? playerO : playerX;
  }
}

Finally, create the end state and make all the square boxes empty and ready for another round of play.

#end {
  (iterate _squares).state = SquareState::Open;
  transition #turn;
}

Conclusion

In this article, you've learnt about the different features Adama provides, how you can start building with Adama, and how to build the backend of a Tic Tac Toe game using Adama.

Adama is a programming language that leverages the reactive property to enable us to build efficient and scalable real-time applications at a minimal cost. Adama provides a fun way of building applications. If you are looking forward to building an efficient real-time gaming application, Adama is an excellent choice.

Thank you for reading!

Examples - Get Playful

Tic Tac Toe

Back-end

@static {
  // As this is going to be a live home-page sample, let anyone create
  create { return true; }
  invent { return true; }

  // As this will spawn on demand, let's clean up when the viewer goes away
  delete_on_close = true;
}

// What is the state of a square
enum SquareState { Open, X, O }

// who are the two players
public principal playerX;
public principal playerO;

// who is the current player
public principal current;

// how many wins per player
public int wins_X;
public int wins_O;

// how many stalemates
public int stalemates;

// personalized data for the connected player:
// show the player their role, a signal if it is their turn, and their wins
bubble your_role = playerX == @who ? "X" : (playerO == @who ? "O" : "Observer");
bubble your_turn = current == @who;
bubble your_wins = playerX == @who ? wins_X : (playerO == @who ? wins_O : 0);

// a record of the data in the square
record Square {
  public int id;
  public int x;
  public int y;
  public SquareState state;
}

// the collection of all squares
table<Square> _squares;

// show the board to all players
public formula board = iterate _squares;

// for visualization, we break the squares into rows
public formula row1 = iterate _squares where y == 0;
public formula row2 = iterate _squares where y == 1;
public formula row3 = iterate _squares where y == 2;

// when the document is created, initialize the squares and zero out the totals
@construct {
  for (int y = 0; y < 3; y++) {
    for (int x = 0; x < 3; x++) {
      _squares <- { x:x, y:y, state: SquareState::Open };
    }
  }
  wins_X = 0;
  wins_O = 0;
  stalemates = 0;
}

// when a player connects, assign them to either the X or O role. If there are more than two players, then they can observe.
@connected {
  if (playerX == @no_one) {
    playerX = @who;
    if (playerO != @no_one) {
      transition #initiate;
    }
  } else if (playerO == @no_one) {
    playerO = @who;
    if (playerX != @no_one) {
      transition #initiate;
    }
  }
  return true;
}

// open a channel for players to select a move
message Play { int id; }
channel<Play> play;

// the game is afoot
#initiate {
  current = playerX;
  transition #turn;
}

// test if the placed square produced a winning combination
procedure test_placed_for_victory(SquareState placed) -> bool {
  for (int k = 0; k < 3; k++) {
    // vertical lines
    if ( (iterate _squares where x == k && state == placed).size() == 3) {
      return true;
    }
    // horizontal lines
    if ( (iterate _squares where y == k && state == placed).size() == 3) {
      return true;
    }
  }
  // diagonals
  if ( (iterate _squares where y == x && state == placed).size() == 3 || (iterate _squares where y == 2 - x && state == placed).size() == 3 ) {
    return true;
  }
  return false;
}

#turn {
  // find the open spaces
  list<Square> open = iterate _squares where state == SquareState::Open;
  if (open.size() == 0) {
    stalemates++;
    transition #end;
    return;
  }
  // ask the current play to choose an open space
  if (play.decide(current, @convert<Play>(open)).await() as pick) {
    // assign the open space to the player
    let placed = playerX == current ? SquareState::X : SquareState::O;;
    (iterate _squares where id == pick.id).state = placed;
    if (test_placed_for_victory(placed)) {
      if (playerX == current) {
        wins_X++;
      } else {
        wins_O++;
      }
      transition #end;
    } else {
      transition #turn;
    }
    current = playerX == current ? playerO : playerX;
  }
}

#end {
  (iterate _squares).state = SquareState::Open;
  transition #turn;
}

Front-end using RxHTML

<forest>
  <template name="cell">
    <div rx:switch="state">
      <div rx:case="0">
        <decide channel="play">
          <button>Play here</button>
        </decide>
      </div>
      <div rx:case="1">X</div>
      <div rx:case="2">O</div>
    </div>
  </template>
  <template name="game">
    <table>
      <tr>
        <td>Role</td><td><lookup path="your_role" /></td>
        <td>Wins</td><td><lookup path="your_wins" /></td>
      </tr>
    </table>
    <div>
      <test path="your_turn">
        <div>
          It is your turn!
        </div>
      </test>
    </div>
    <div class="[your_turn]text-indigo-600[#your_turn]text-gray-900[/your_turn]">
      CHANGE
    </div>
    <table border="1">
      <tr rx:iterate="row1">
        <td>
          <use name="cell" />
        </td>
      </tr>
      <tr rx:iterate="row2">
        <td>
          <use name="cell" />
        </td>
      </tr>
      <tr rx:iterate="row3">
        <td>
          <use name="cell" />
        </td>
      </tr>
    </table>
  </template>
  <page uri="/#game">
    <connection name="player1" identity="eyJhbGciOiJFUzI1NiJ9.eyJzdWIiOiJ1c2VyMDAxIiwiaXNzIjoiWUlTUjNZTUpSSzNHMlo2MkFWWVdCWUNITjI5WFoyIn0.oKZOXHJUFPyxMT7j6X4WQRLy4VVeGGOvZgqMS2hsU6W1lALW-teOdoHAj2t5K3oHDBj6zH_3NFt6fR6fthfyzA" space="tic-tac-toe" key="demo" random-key-suffix>
      <use name="game" />
    </connection>
    <connection name="player2" identity="eyJhbGciOiJFUzI1NiJ9.eyJzdWIiOiJ1c2VyMDAyIiwiaXNzIjoiWUlTUjNZTUpSSzNHMlo2MkFWWVdCWUNITjI5WFoyIn0.uS2LyhmDh1gg35Zpa1yd-JKxxu4EjzggQlL9tc2zFxZYPD0SZykgtjvL0PeKH0X67ot84Xb6Hk9mmMpRqDyRMA" space="tic-tac-toe" key="demo" random-key-suffix>
      <use name="game" />
    </connection>
  </page>
</forest>

Durable PubSub


@static {
  // anyone can create
  create { return true; }
}

@connected {
   // let everyone connect; sure, what can go wrong
  return true;
}

// we build a table of publishes with who published it and when they did so
record Publish {
  public principal who;
  public long when;
  public string payload;
}

table<Publish> _publishes;

// since tables are private, we expose all publishes to all connected people
public formula publishes = iterate _publishes order by when asc;

// we wrap a payload inside a message
message PublishMessage {
  string payload;
}

// and then open a channel to accept the publish from any connected client
channel publish(PublishMessage message) {
  _publishes <- {who: @who, when: Time.now(), payload: message.payload };

  // At this point, we encounter a key problem with maintaining a
  // log of publishes. Namely, the log is potentially infinite, so
  // we have to leverage some product intelligence to reduce it to
  // a reasonably finite set which is important for the product.

  // First, we age out publishes too old (sad face)
  (iterate _publishes
     where when < Time.now() - 60000).delete();

  // Second, we hard cap the publishes biasing younger ones
  (iterate _publishes
     order by when desc
     offset 100).delete();
}

Maximum Number


@static {
  create { return true; }
}

@connected {
  return true;
}

public int max_db_seq = 0;

message NotifyWrite {
  int db_seq;
}

channel notify(NotifyWrite message) {
  if (message.db_seq > max_db_seq) {
    max_db_seq = message.db_seq;
  }
}

Hearts

Many of the bugs have been fixed, this is from an old version.


@static {
  // anyone can create
  create { return true; }
  invent { return true; }
}

// we define the suit of a card
enum Suit {
  Clubs:1,
  Hearts:2,
  Spades:3,
  Diamonds:4,
}

// the rank of a card
enum Rank {
  Two:2,
  Three:3,
  Four:4,
  Five:5,
  Six:6,
  Seven:7,
  Eight:8,
  Nine:9,
  Ten:10,
  Jack:11,
  Queen:12,
  King:13,
  Ace:14,
}

// where can a card be
enum Place {
  Deck:1,
  Hand:2,
  InPlay:3,
  Taken:4
}

// model the card and its location and ownership
record Card {
  public int id;
  public Suit suit;
  public Rank rank;
  private principal owner;
  private int ordering;
  private Place place;
  private auto points = suit == Suit::Hearts ? 1 : (suit == Suit::Spades && rank==Rank::Queen ? 13 : 0);

  // define a policy as to who can see the card
  policy p {
    // if it is in hand on in the pot, then only the owner of the card can see it
    // the rules of hearts have cards face down
    if (place == Place::Hand || place == Place::Taken) {
      return @who == owner;
    }
    // if it is in the pot or in play, then anyone can see it
    if (place==Place::InPlay) {
      return true;
    }
    // otherwise, it is in the deck and thus not visible
    return false;
  }

  method reset() {
    ordering = Random.genInt();
    owner = @no_one;
    place = Place::Hand;
  }

  require p;
}

// the entire deck of cards
table<Card> deck;

// show the player hand (and let the privacy policy filter out by person)
bubble hand = iterate deck where place == Place::Hand where owner == @who order by id asc;

// show all cards in the pot (this would be a different way of defining hand)
bubble my_take = iterate deck where place == Place::Taken && owner == @who;

// no real constructor
message Empty {}

principal owner;

record Player {
  public int id;
  public principal link;
  public int points;
  viewer_is<link> int play_order;
}

table<Player> players;

@connected {
  if ((iterate players where link==@who).size() > 0) {
    return true;
  }
  if (players.size() < 4) {
    players <- {
      link:@who,
      play_order: players.size(),
      points:0
    };
    if (players.size() == 4) {
      transition #setup;
    }
    return true;
  }
  return false;
}

// everyone in the game
public auto people = iterate players order by play_order;

// the players by their ordering
public auto players_ordered = iterate players order by play_order;

// are we actually playing the game?
public bool playing = false;

// how setup the game state
#setup {
  // build the deck
  foreach (s in Suit::*) {
    foreach (r in Rank::*) {
      deck <- {rank:r, suit:s, place:Place::Deck};
    }
  }

  // normalize the players from 0 to 3
  int normativeOrder = 0;
  (iterate players order by play_order asc).play_order = normativeOrder++;

  // shuffle and distribute the cards
  transition #shuffle_and_distribute;
}

enum PassingMode { Across:0, ToLeft:1, ToRight:2, None:3 }

public PassingMode passing_mode;

#shuffle_and_distribute {
  // it may be useful to allow methods on a record, fuck
  (iterate deck).reset();

  // distribute cards to players
  Player[] op = (iterate players order by play_order).toArray();
  for (int k = 0; k < 4; k++) {
    if (op[k] as player) {
      (iterate deck where owner == @no_one order by ordering limit 13).owner = player.link;
    }
  }
  transition #pass;
}

message CardDecision {
  int id;
}

channel<CardDecision[]> pass_channel;

// this is wanky, need arrays at a top level that are finite to help...
principal player1;
principal player2;
principal player3;
principal player4;

principal current;

#pass {
  if (passing_mode == PassingMode::None) {
    transition #start_play;
    return;
  }

  // this is wanky as fuck, and I don't like it. We have this fundamental problem of what if there are not enough players, then how does this fail...
  // we should consider a @fatal keyword to signal that a game is just fucked

  Player[] op = (iterate players order by play_order).toArray();
  if (op[0] as player) {
    player1 = player.link;
  }
  if (op[1] as player) {
    player2 = player.link;
  }
  if (op[2] as player) {
    player3 = player.link;
  }
  if (op[3] as player) {
    player4 = player.link;
  }
  // what does an await on no_one mean, it means the whole thing is fucked

  // we really need a future array since this has some awkward stuff
  future<maybe<CardDecision[]>> pass1 = pass_channel.choose(player1, @convert<CardDecision>(iterate deck where owner==player1), 3);
  future<maybe<CardDecision[]>> pass2 = pass_channel.choose(player2, @convert<CardDecision>(iterate deck where owner==player2), 3);
  future<maybe<CardDecision[]>> pass3 = pass_channel.choose(player3, @convert<CardDecision>(iterate deck where owner==player3), 3);
  future<maybe<CardDecision[]>> pass4 = pass_channel.choose(player4, @convert<CardDecision>(iterate deck where owner==player4), 3);

  // the reason we do the futures above and then await them below like this is so all players can pass at the same time.
  // the problem at hand is that the await will consume, so non-awaited futures will cause the client to sit dumbly... this can be fixed easily I think
  // by having the make_future<> check the stream and pre-drain the queue and allow the await to short-circuit with the provide option

  if (pass1.await() as decision1) {
  if (pass2.await() as decision2) {
  if (pass3.await() as decision3) {
  if (pass4.await() as decision4) {

  if (passing_mode == PassingMode::ToRight) {
    foreach (dec in decision1) {
      (iterate deck where id == dec.id).owner = player2;
    }
    foreach (dec in decision2) {
      (iterate deck where id == dec.id).owner = player3;
    }
    foreach (dec in decision3) {
      (iterate deck where id == dec.id).owner = player4;
    }
    foreach (dec in decision4) {
      (iterate deck where id == dec.id).owner = player1;
    }
  } else if (passing_mode == PassingMode::ToLeft) {
    foreach (dec in decision1) {
      (iterate deck where id == dec.id).owner = player4;
    }
    foreach (dec in decision2) {
      (iterate deck where id == dec.id).owner = player1;
    }
    foreach (dec in decision3) {
      (iterate deck where id == dec.id).owner = player2;
    }
    foreach (dec in decision4) {
      (iterate deck where id == dec.id).owner = player3;
    }
  } else if (passing_mode == PassingMode::Across) {
    foreach (dec in decision1) {
      (iterate deck where id == dec.id).owner = player3;
    }
    foreach (dec in decision2) {
      (iterate deck where id == dec.id).owner = player4;
    }
    foreach (dec in decision3) {
      (iterate deck where id == dec.id).owner = player1;
    }
    foreach (dec in decision4) {
      (iterate deck where id == dec.id).owner = player2;
    }
  }

  }}}}
  transition #start_play;
}

public int played = 0;
public Suit suit_in_play;
public bool points_played = false;
public auto in_play = iterate deck where place == Place::InPlay order by rank desc;

#start_play {
  // no cards hae been played
  played = 0;
  points_played = false;

  // assign a player to current
  current = player1;
  if ( (iterate deck where rank == Rank::Two && suit == Suit::Clubs)[0] as two_clubs) {
    current = two_clubs.owner;
  } // otherwise, @fatal

  transition #play;
}

channel<CardDecision> single_play;

// how to attribute this to a person

public principal last_winner;

#play {
  list<Card> choices = iterate deck where owner == current && place == Place::Hand && rank == Rank::Two && suit == Suit::Clubs;
  if (choices.size() == 0) {
    choices = iterate deck where owner==current && place == Place::Hand && (
      played == 0 && (points_played || points == 0) ||
      played > 0 && suit_in_play == suit
    );
  }
  if (choices.size() == 0) { // anything in hand
    choices = iterate deck where owner==current && place == Place::Hand;
  }
  if (choices.size() == 0) {
    transition #score;
    return;
  }
  future<maybe<CardDecision>> playX = single_play.decide(current, @convert<CardDecision>(choices));
  if (playX.await() as dec) {
    if ((iterate deck where id == dec.id)[0] as cardPlayed) {
      cardPlayed.place = Place::InPlay;
      if (cardPlayed.points > 0) {
        points_played = true;
      }
      if (played == 0) {
        suit_in_play = cardPlayed.suit;
      }
    }
  }


  // if the number of cards played is less than 4, then next player; otherwise, decide winner of pot and award points

  // TODO: need finite arrays and cyclic integers
  if (current == player1) {
    current = player2;
  } else if (current == player2) {
    current = player3;
  } else if (current == player3) {
    current = player4;
  } else if (current == player4) {
    current = player1;
  }

  if (played == 3) {
    if ( (iterate deck where place == Place::InPlay && suit == suit_in_play order by rank desc limit 1)[0] as winner) {
      (iterate deck where place == Place::InPlay).owner = winner.owner;
      last_winner = winner.owner;
    }
    (iterate deck where place == Place::InPlay).place = Place::Taken;
    played = 0;
    current = last_winner;
    if( (iterate deck where owner == current && place == Place::Hand).size() == 0) {
      transition #score;
      return;
    }
  } else {
    played++;
  }
  transition #play;
}

public int points_awarded = 0;

#score {
  // award points
  foreach(p in iterate players) {
    int local_points = 0;
    foreach(c in iterate deck where owner == p.link && place == Place::Taken) {
      local_points += c.points;
    }
    if (local_points == 26) {
      foreach(p2 in iterate players where link != p.link) {
        p2.points += 26;
        points_awarded += 26;
      }
    } else {
      p.points += local_points;
      points_awarded += local_points;
    }
  }

  passing_mode = passing_mode.next();
  transition #shuffle_and_distribute;
}



Chat


@static {
  // anyone can create
  create { return true; }
  invent { return true; }
  maximum_history = 250;
}

// let anyone into the document
@connected {
  return true;
}

// the lines of chat
record Line {
  public principal who;
  public string what;
  public long when;
}

// the chat table
table<Line> _chat;

// how someone communicates to the document
message Say {
  string what;
}

// the "channel" which enables someone to say something
channel say(Say what) {
  // ingest the line into the chat
  _chat <- {who:@who, what:what.what, when: Time.now()};

  (iterate _chat order by when desc offset 50).delete();
}

// emit the data out
public formula chat = iterate _chat;

SMS Bot

Back-end (Draft)

This is as draft example

@static {
  create {
    return true;
  }
}

public principal owner;

@connected {
  return true;
}

@construct {
  owner = @who;
}

record Entry {
  public int id;
  public dynamic parameters;
  public map<string, string> headers;
  public string from;
  public string body;
}

table<Entry> _entries;

public formula entries = iterate _entries order by id desc;

message TwilioSMS {
  string From;
  string Body;
}

@web put /webhook (TwilioSMS sm) {
  _entries <- {
    from:sm.From,
    body:sm.Body,
    parameters:@parameters,
    headers:@headers
  };
  return {
    xml: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Message>Thank you for the data. NOM. NOM.</Message></Response>"
  };
}

A Silly Example to Show Off

// 1/6: Layout state with a schema
public principal owner;
public string name;
public string description;
public int viewers;

record AddOn {
  public string name;
  public string description;
}

// tables within documents!
table<AddOn> _addons;

// 2/6: create documents with constructors
@static {
  // policy: who can create? anyone!
  create { return true; }
}

message Arg {
  string name;
  string description;
}

@construct(Arg arg) {
  owner = @who;
  name = arg.name;
  description = arg.description;
}

// 3/6: people connect with WebSocket
@connected {
  viewers++;
  return true;
}

@disconnected {
  viewers--;
}

// 4/6: people manipulate document via messages
message AddAddOn {
  string name;
  string description;
}

channel create_new_add_on(AddAddOn arg) {
  // tables can "ingest" data easily
  _addons <- arg;
}

// 5/6: deletes are handled within document (for safety!)
message Nothing {
}

channel delete(Nothing arg) {
  if (owner == @who) {
    Document.destroy();
  }
}

// 6/6: reactive formulas (just like a spreadsheet)
public formula addons = iterate _addons order by name asc;
public formula addons_size = addons.size();
public formula name_uppercase = name.upper();

Language Guide

In static policies and document events, you will learn how to authorize and allow people to connect to documents.

Static policies and document events

A document will contain state, and it is vital to protect that state from unauthorized access or malicous actors. In this section, we will go through the details of access control and answer these questions:

  • Who can create documents?
  • Who can invent documents? And what does document invention mean?
  • How do I tell when a document is loaded? How to upgrade document state?
  • Who can connect to a document?
  • Who can delete a document?
  • Who can attach resources (i.e. files) to documents?
  • What resource got attached? And when do assets get deleted?

Unlike other document stores, access control is done within the platform at the lowest level. This is the first step in building anything with Adama because access control and privacy are important.

Static policies

Static policies are evaluated without any state precisely because that state is not available. For instance, the ability to create a document requires a policy, so we introduce the @static {} construct which denotes a block of policies. Within the @static block are policies like create and invent.

Answer: Who can create documents within your space?

@static {
  create {
    return true;
  }
}

The above policy allows anyone to create a document within your space (#rude), so you will want to lock it down. For instance, we can lock it down to just adama developers.

@static {
  create {
    return @who.isAdamaDeveloper();
  }
}

We can also validate the user is one of your people using a public key you provided via authorities.

@static {
  create {
    return @who.fromAuthority("Z2YISR3YMJN29XZ2");
  }
}

The static policy also has context which provides location about where the user is connecting from using @context constant.

@static {
  create {
    return @who.isAdamaDeveloper() &&
           @context.origin == "https://www.adama-platform.com";
  }
}

From a security perspective, origin can't be trusted from malicious actors writing bots as they can forge the Origin header. However, the origin header is sufficient for preventing XSS attacks and limiting accidental usage across the web.

Answer: What is invention and who can invent documents within your space?

Document invention happens when a user attempts to connect to a document that doesn't exist. If the document requires no constructor then the invent policy is evaluated. If then invent policy returns true, then the document is created and the connect request is tried again. This is useful for introducing real-time scenarios to an already mature user-base as construction can happen on-demand.

The invent policy works similar to create.

@static {
  invent {
    return who.isAdamaDeveloper();
  }
}

The logic within invent can mirror that of create.

Static properties

With the @static block, we can also configure fixed properties that balance cost versus features or enable new modes. For instance, the maximum_history property will inform the system how much history to preserve.

@static {
  maximum_history = 500
}

Here, at least 500 changes to the document will be kept active. We also have delete_on_close which will automatically delete the document once the document is closed. This is useful for ephemeral experiences like typing indicators.

@static {
  delete_on_close = true;
}

Document events

Document events are evaluated against a specific instance of a document. These events happen when a document is created, when a client connects or disconnects to a document, and when an attachment is added to a document.

Note: at this time, document events don't support @context. See issue #118

Construction: How to initialize a document?

Fields within a document can be initialized with a default value or via code within the @construct event.

// y has a default value of 42
public int y = 42;
// x has a default value = 0
public int x;
// who owns the document
public principal owner;

@construct {
  owner = @who;
  // overwrite the default value of x with 42
  x = 42;
}

From a security perspective, it is exceptionally important to capture the creator (via the @who constant) for access control towards establishing an owner of the document as this enables more stringent access control policies.

The @construct event will fire exactly once when the document is constructed. As documents can only be constructed once, this enables documents to have an authoritative owner which can be used within privacy policies and access control.

Load: how to run code when a document is loaded?

Since documents may need to experience radical change, the @load events provides an opportunity to upgrade data when loaded.

@load {
  if (version < 2) {
    // transfer data to new world
  }
  if (version < 3) {
    // transfer data to super new world
  }
  version = 3;
}

Connected and Disconnected

The primary mechanism for users to get access to the document is via a persistent connection (vis-à-vis WebSocket) and then send messages to the document. Before messages can be sent, the connection must be authorized by the document and this is done via the @connected event.

@connected {
  return true;
}

The @connected event is the primary place to enforce document-level access control. For example, we can combine the constructor with the @connected event.

public principal owner;
public bool open_to_public;

@construct {
  owner = @who;
  open_to_public = false;
}

@connected {
  if (owner == @who) {
    return true;
  }
  return open_to_public;
}

If the return value is true then the connection is established. Since the @connected event runs within the document scope, it can both mutate the document and lookup data within the document. A more stringent policy would have a table with who can access the document. Above, we always allow the owner to connect and a boolean controls whether a random person can connect.

Not only can they connect, but they also naturally disconnect (either on purpose or due to technical difficulties). The disconnect signal is an informational event only, and is available via the @disconnected event.

@disconnected {	
}

This allows us to mutate the document on a person disconnecting.

Delete

When document deletion is requested from via the API, the @delete event runs to validate the principal can delete the document.

@delete {
  return @who == owner; 
}

Asset Attachments

Adama allows you to attach assets to documents. Assets are essentially files or binary blobs that are associated with the document. Since the storage and association of assets is non-trivial, there are two document events for assets.

Answer: Who can attach a file to your document?

The @can_attach document event works much like @connnected in that the code is ran and the person attempting to upload is validated. If the event returns true, then the user is allowed to attach the file.

  return @who.isAdamaDeveloper();
}

This event primarily exists to protect user devices from erroneously uploading attachments. After this event returns true, the user will begin the upload which will durably store the file as an asset.

Answer: What resource was attached

Once the asset is durable stored, the @attached event is run with the asset as a parameter.

public asset most_recent_file;

@attached (what) {
  most_recent_file = what;
}

The type of the what variable is asset which has the following methods.

methodtypewhat it is
name()stringThe name provided by the uploader (i.e. file name)
id()stringA unique id to denote the asset
size()longThe number of bytes of the asset
type()stringThe Content type of the asset (
valid()boolThe asset is real or not. The default value for an asset is @nothing

Note: at this time, using assets is a pain. See issue #120

Answer: How do assets leave

Since there is a maximum history for the lifetime of a document, assets are periodically collected and then garbage collected. Once an asset object leaves the document and the stored history, the associated bytes are cleaned up.

Note: this is not true at this time. See issue #121

Document layout

As Adama is a data-centric programming language, the core game to play is organizing your data for state management. At the document level, state is laid out as a series of fields. For instance, the below Adama code outlines three fields:

public string output;
private double balance;
int count;

These three fields will establish a persistent document in JSON:

{"output":"","balance":0.0,"count":0}

The public and private modifiers control what users will see, and the omission of either results in private by default. In this case, users will see:

{"output":""}

when they connect to the document.

The language has many types to leverage along with a more ways to expressive privacy rules.

Furthermore, state can be laid out with records, collected into tables, and computations exposed via formulas.

Details

The syntax which Adama parses for this is as follows:

(privacy)? type name (= expression)?;

such that

  • privacy when set may be private, public, or anything outlined in the privacy section. In this context, private means only the system and code within Adama can see the field while public means the system, the code, and any human viewer may see the field. The privacy section will outline other ways for humans to see the field. When privacy is omitted results in a default value of private which means no users can see the field.
  • type is the type of the data. See types for more details.
  • expression when provided will be computed at the construction of the document.

Types

Adama has many built-in primal types! The following tables outline which types are available.

typecontentsdefaultfun example
boolbool can have one of the two values true or false.falsetrue
intint is a signed integer number that uses 32-bits. This results in valid values between −2,147,483,648 and 2,147,483,647.042
longlong is a signed integer number that uses 64-bits. This results in valid values between -9,223,372,036,854,775,808 and +9,223,372,036,854,775,807.042
doubledouble is a floating-point type which uses 64-bit IEEE754. This results in a range of 1.7E +/- 308 (15 digits).0.03.15
complexcomplex is a tuple of two doubles under the complex field of numbers0 + 0 * @i@i
stringstring is a utf-8 encoded collection of code-points."" (empty string)"Hello World"
labellabel is a pointer to a block of code which is used by the state machine,# (the no-state)#hello
principalprincipal is a reference to a connected person, and the backing data establishes who they are. This is used for acquiring data and decisions from people,@no_one@no_one
dynamica blob of JSON@null@null

Call-out to other types

The above built-in types are building blocks for richer types, and the below table provides callouts to other type mechanisms. Not all types are valid at the document level.

TypeQuick call outApplicable to document/record
enumAn enumeration is a type that consists of a finite set of named constants.yes
messagesA message is a collection of variables grouped under one name used for communication via channels.only via a formula
recordsA record is a collection of variables grouped under one name used for persistence.yes
maybeSometimes things didn't or can't happen, and we use maybe to express that absence rather than null. Monads for the win!yes (only for applicable types)
tableA table forms the ultimate collection enabling maps, lists, sets, and more. Tables use records to persist information in a structured way.yes
channelChannels enable communication between the document and people via handlers and futures.only root document
futureA future is a result that will arrive in the future.no
mapsA map enables associating keys to values, but they can also be the result of a reduction.yes
listsA list is created by using language integrated query on a tableyes
arraysAn array is a read-only finite collection of a adjacent itemsonly via a formula
resultA result is the ongoing progress of a service call made to a remote serviceonly via a formula
serviceA service is a way to reach beyond the document to a remote resourcesonly root document

Rich Types

Complex Numbers

Adama supports complex numbers out-of-the-box with the traditional mathematical operations.

Methods

methodbehavior
lenreturns the length of the complex number

maybe<double>

dynamic

lists

transforms

operations

Comments are good for your health

We humans need to "enhance" our logic with prose and explanation for others humans (and future versions of ourselves). Since code should be manageable and understandable over time via different people, Adama supports C style comments.

// This is a comment with a single line which terminates at the ->\n

/* This is a comment with multiple lines

It can have oh so many lines.

* <- why not? it's freeform madness!
*/

int x /* and can be embedded anywhere */ = 123;

This enables comments to be sprinkled liberally within Adama code.

A Tiny (kind of useless) Technical Detail

Comments are part of the syntax tree for Adama as part of hidden meta-data on tokens. This is so that comments are available to tooling such that error messages can more helpful, but also so documentation can be generated and shared with document consumers.

Comments associate with tokens can be either in a forwards or backwards manner. Comments on multiple lines such as:

/**
 * foo
 */

always associate forward. For instance the comment in

/* age of the user */
int age;

is associated to the 'int' token. Whereas single line comment like

int age; // age of the user

usually associate backwards. Here, the comment is associated with the ';' token. Sometimes, the single line comment will associate forward but only after a multi-line comment has been introduced. For instance

/* the age */ // of the user
int age;

will associate both comments to the 'int' token.

Constants

You want numbers? We got numbers. You want strings? We got strings. You want complex numbers, we got complex numbers! Constants are a fast way to place data into a document. For instance, the following code outlines some basic constants:

#we_got_constants {
  int x = 123;
  int y = 0x04;
  double z = 3.14;
  bool b = true;
  principal c = @no_one;
  complex cx = 1 + @i;
}

Details

There are a variety of ways to conjure up constants. The following table illustrates examples:

typesyntaxexamples
bool(true, false)false, true
int[0-9]+42, 123, 0
int0x[0-9a-fA-F]+0xff, 0xdeadbeef
double[0-9].?[0-9]([eE](0-9)+)?3.14, 10e19, 2.72e10
string"(^"escape)*"
label#[a-z]+#foo, #start
principal@no_one@no_one
maybe<?>@maybe<Type>@maybe<int>
maybe(?)@maybe(Expr)@maybe(123)
complex@i1 + @i

@who is executing

The @who constant refers to the principal that is executing the current code.

@context of the caller

Within static policies, document constructors and events, and message handlers there is constant @context which is useful for getting access to the origin and ip or the caller.

@web's @parameters and @headers

The web handler has access to the parsed query string and HTTP headers respectively via the @parameters and @headers constants. @parameters is a dynamic with the query string converted to JSON. @headers is a map<string,string>.

Bubble's @viewer

A privacy bubble can access viewer state via the @viewer which behaves like a message.

Dynamic @null

The default dynamic type has a value of "null" which is represented via @null.

String escaping

A character following a backslash (\) is an escape sequence within a string, and it is an escape for the parser to inject special characters. For instance, if you are parsing a string initiated by double quotes, then how does one get a double quote into the string? Well, that's what the escape is for. Adama's strings support the following escape sequences:

escape codebehavior
\ta tab character
\ba backspace character
\na newline
\ra carriage return
\fa form feed
"a double quote (")
\\a backslash character (\)
\uABCDa unicode character formed by the four adjoined hex characters after the \u

Records

A record is a collection of privacy-enforced typed data elements grouped under one name. For instance, we can define a Person record with the following code:

record Person {
  public string name;
  private int age;
  private double balance;
}

The data elements mirror how data is spelled out within the root document; however, a record can be used in multiple ways. For example, a record can be held within another record.

record Relationship {
  public Person a;
  public Person b;
}

This simple re-use becomes the foundation for building composite types and collections.

See:

  • types for more information on which types can go within records.
  • privacy policies for how privacy per field is specified within a record.
  • bubbles for how data is exposed based on the viewer from the record.
  • reactive formulas for how to expose reactively compute data to the viewer of record.

Records also have methods, can declare privacy policies, and hint at indexing for tables. If a record is used within a table, an implicit field called id is created with integer type. Since communication is done with messages, conversion to a message is provided as a free helper.

Methods

We can associate code to a record via a method. For example, a method may mutate a record which is useful for consolidating how records change.

record R {
  public int score;
  
  method zero() {
    score = 0;
  }
}

Methods can be marked as read-only such that they are not allowed to mutate the document and thus become available for reactive formulas.

record R {
  public int score;
  
  method double_score() -> int readonly {
    return score * 2;
  }

  public formula ds = double_score();
}

Policies

Records can express policies which are bits of code associated to the record along with @who.

record R {
  private principal owner;
  
  policy is_owner {
    return owner == @who;
  }
  
}

A policy can be used to protect fields within a record.

record R {
  private principal owner;
  
  use_policy<is_owner> int balance;
  
  policy is_owner {
    return owner == @who;
  }
  
}

Alternatively, a policy may be used to protect the entire record.

record R {
  private principal owner;
  
  public int balance;

  policy is_owner {
    return owner == @who;
  }
 
  require is_owner;
}

Indexing tables

The best mental model for a record is a row within a table. By default, a row has a primary key index on id which has a type of int. Additional fields within a record can be indexed to speed up queries.

record R {
  private int key;
  
  index key;
}

table<R> _table;

The index keyword will inform _table that it can group records by the key field to reduce the number of candiates considered during a where clause. This will introduce both memory and computational overhead to maintain.

Easily convert to a message for communication

Given a message or record, we can convert it into a message type via the @convert keyword.

record R {
  public int x;
}
R r;

message M {
  int x;
}

#sm {
 M m = @convert<M>(r);
}

This conversion is useful with channels and futures are outlined as we sometimes want to present people with a list of options derived from a table. It is also useful for sending records to another service.

Messages

A message is similar to a record except without any privacy awareness or privacy concerns. Unlike records, messages lack formulas and bubbles and have a limited form of methods.

All fields within a message are public, and the expectation is that messages come from users. The following defines a real-world message:

message JoinGroup {
  string name;
}

Most types that can be defined within code can be defined within a message. Some exceptions are channels, services, and futures; spiritually, all data defined within a message must be complete in a serialized form. Messages can also be constructed anonymously on the fly.

#yo {
  let msg = {x:1, y:2};  
}

It is worth noting that the type of messages undergo static type rectification, so the above is 100% statically typed. It also leverages a simplified form of type inference such that messages of a known type can be constructed.

message M {
  int x;
  int y;
}

@construct {
  M m = {x:1, y:1};
}

Methods

Messages may have methods, but they are more constrained as they can only reference data from within the message as messages.

message X {
  int val;
  method succ() -> int {
    return val + 1;
  }
}

A collection of messages is a native table

Messages, like records, can be put into a table within code.

message M {
  int x;
  int y;
}

#turn {
  table<M> tbl;
  tbl <- {x:1, y:1};  
  tbl <- {x:1, y:2};  
}

Easy copying and type conversion (@convert)

Given a message or record, we can convert it into a message type via the @convert keyword.

record R {
  public int x;
}

R r;

message M {
  int x;
}

#sm {
 M m = @convert<M>(r);
}

The usefulness of this conversion will become clear when channels and futures are outlined.

Local variables and assignment

While document variables are persisted, variables can be defined locally within code blocks (like constructors to be used to compute things. These variables are only used to compute.

private int score;

@construct {  
  int temp = 123;
  temp = 42;
}

Define by type

Native types can be defined within code without a value as many of them have defaults:

#transition {
  int local;
  local = 42;
  string str = "hello";
}

The default values follow the principle of least surprise.

typedefault value
boolfalse
int0
long0L
double0.0
string""
list<T>empty list
table<T>empty table
maybe<T>unset maybe
T[]empty array

readonly keyword

A local variable can be annotated as readonly, meaning it can not be assigned.

#transition {
  readonly int local = 42;
}

This is fairly verbose, so we introduce the let keyword.

Define via the "let" keyword and type inference

Instead of leading with the readonly and the type, you can simply say "let" and allow the translator to precisely infer the type and mark the variable as readonly.

#transition {
  let local = 42;
}

This simplifies the code and the aesthetics.

Math-based assignment, increment, decrement

Numerical types provide the ability to add, subtract, and multiply the value by a right-hand side. Please note: division and modulus are not available as there is the potential for division by zero, and division by zero is bad.

#transition {
  int x = 3; // 3
  x *= 4; // 12
  x--; // 11
  x += 10; // 21
  x *= 2; // 42, the cosmos are revealed
  x++;  
}

List-based bulk assignment

Lists derived from tables within the document provide a bulk assignment via '=' and the various math based assignments (+=, -=, *=, ++, --).

record R {
  int x;
}
table<R> _records;

procedure reset() {
  (iterate _records).x = 0;
}

procedure bump() {
  (iterate _records).x++;
}

Doing math

Adama lets you do math, and that's awesome! It has the typical operations that you would expect, and then some fun twists. So, let's get into it.

Operators

Parentheses: ( expr )

You can wrap any expression with parentheses, and parentheses will alter the precedence of evaluation. See operator precedence section for details about operator precedence.

Sample Code

public int z;
@construct {
  z = (1 + 2) * 3;
}

Result

  {"z":6}

Typing: The resulting type is the type of the sub-expression.

Unary numeric negation: - expr

When you want to turn a positive into a negative, a negative into a positive, a smile into a frown, a frown upside down, or reflect a value over the y-axis. This is done with the subtract symbol prefixing any expression.

Sample Code

public int z;
public formula neg_z = -z;
@construct {
  z = 42;
}

Result

{"z":42,"neg_z":-42}

Typing: The sub-expression type must be an int, long, or double. The resulting type is the type of the sub-expression.

Unary boolean negation / not / logical compliment: ! expr

Turn false into true, and true into false. This is the power of the unary Not operator using the exclamation point !. Money back if it doesn't invert those boolean values!

Sample Code

public bool a;
public formula not_a = !a;
@construct {
  a = true;
}

Result

{"a":true,"not_a":false}

Typing: The sub-expression type must be a bool, and the resulting type is also bool.

Addition: expr + expr

You'll need to use addition to count the phat stacks of money you'll earn with a computer science degree.

Sample Code

public int a;
public formula b = a + 10;
public formula c = b + 100;
@construct {
  a = 1;
}

Result

{"a":1,"b":11,"c":111}

Typing: The addition operator is commonly used to add two numbers, but it can also be used with strings and lists. The following table summarizes the typing and behavior:

left typeright typeresult typebehavior
intintintinteger addition
doubledoubledoublefloating point addition
doubleintdoublefloating point addition
intdoubledoublefloating point addition
longlonglonginteger addition
longintlonginteger addition
intlonglonginteger addition
stringstringstringconcatenation
intstringstringconcatenation
longstringstringconcatenation
doublestringstringconcatenation
boolstringstringconcatenation
stringintstringconcatenation
stringlongstringconcatenation
stringdoublestringconcatenation
stringboolstringconcatenation
list<int>intlist<int>integer addition on each element
list<int>longlist<long>integer addition on each element
list<int>doublelist<double>floating point addition on each element
list<long>intlist<int>integer addition on each element
list<long>longlist<long>integer addition on each element
list<double>intlist<double>floating point addition on each element
list<double>doublelist<double>floating point addition on each element
list<string>intlist<string>concatenation on each element
list<string>longlist<string>concatenation on each element
list<string>doublelist<string>concatenation on each element
list<string>boollist<string>concatenation on each element

Subtraction: expr - expr

You'll need the subtract operator to remove taxes from your phat stack of dolla-bills.

Sample Code

public int a;
public formula b = a - 10;
public formula c = b - 100;
@construct {
  a = 1000;
}

Result

{"a":1000,"b":990,"c":890}

Typing:

The subtraction operator is commonly used to subtract a number from another number, but it can also be used with lists. The following table summarizes the typing and behavior.

left typeright typeresult typebehavior
intintintinteger subtraction
intdoubledoublefloating point subtraction
doubledoubledoublefloating point subtraction
doubleintdoublefloating point subtraction
longintintinteger subtraction
longlonglonginteger subtraction
intlongintinteger subtraction
list<int>intlist<int>integer subtraction on each element
list<int>longlist<long>integer subtraction on each element
list<int>doublelist<double>floating point subtraction on each element
list<long>intlist<int>integer subtraction on each element
list<long>longlist<long>integer subtraction on each element
list<double>intlist<double>floating point subtraction on each element
list<double>doublelist<double>floating point subtraction on each element

Multiplication: expr * expr

TODO: something pithy about multiplication.

Sample Code

public int a;
public formula b = a * 2;
public formula c = b * 3;
@construct {
  a = 7;
}

Result

{"a":7,"b":14,"c":42}

Typing: TODO | int | int | int | integer subtraction | | int | double | double | floating point subtraction | | double | double | double | floating point subtraction | | double | int | double | floating point subtraction | | long | int | int | integer subtraction | | long | long | long | integer subtraction | | int | long | int | integer subtraction |

Division: expr / expr

TODO: something pithy about division.

Sample Code

public double a;
public formula b = a / 2;
public formula c = b / 10;
@construct {
  a = 20;
}

Result

{"a":20.0,"b":10.0,"c":1.0}

Typing:

In Adama, division always results in a maybe<> due to a division by zero.

left typeright typeresult type
intintmaybe
doubleintmaybe
intdoublemaybe
doubledoublemaybe

Modulus: expr % expr

TODO: something pithy about Modulus and remainders.

Sample Code

public int a;
public formula b = a % 2;
public formula c = a % 4;
@construct {
  a = 7;
}

Result

{"a":7,"b":1,"c":3}

Typing: The left and right side must be integral (i.e. int or long), and the result is integral as well. The following table summarizes the logic precisely.

left typeright typeresult type
intintint
longintint
intlongint
longlonglong

Less than: expr < expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula cmp1 = a < b;
public formula cmp2 = b < a;
@construct {
  a = 1;
  b = 2;
}

Result

{"a":1,"b":2,"cmp1":true,"cmp2":false}

Typing: The left and right must be comparable, and the resulting type is always bool. The following table outlines comparability

left typeright type
intint
intlong
intdouble
longint
longlong
longdouble
doubleint
doublelong
doubledouble
stringstring

TODO: do maybes play into this?

Greater than: expr > expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula cmp1 = a > b;
public formula cmp2 = b > a;
@construct {
  a = 1;
  b = 2;
}

Result

{"a":1,"b":2,"cmp1":false,"cmp2":true}

Typing: This has the same typing as <

Less than or equal to: expr <= expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula cmp1 = a <= b;
public formula cmp2 = b <= a;
public formula cmp3 = a + 1 <= b;
@construct {
  a = 1;
  b = 2;
}

Result

{"a":1,"b":2,"cmp1":true,"cmp2":false,"cmp3":true}

Typing: This has the same typing as <

Greater than or equal to: expr >= expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula cmp1 = a >= b;
public formula cmp2 = b >= a;
public formula cmp3 = a + 1 >= b;
@construct {
  a = 1;
  b = 2;
}

Typing: This has the same typing as <

Result

{"a":1,"b":2,"cmp1":false,"cmp2":true,"cmp3":true}

Equality: expr == expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula eq1 = a == b;
public formula eq2 = a + 1 == b;
@construct {
  a = 1;
  b = 2;
}

Result

{"a":1,"b":2,"eq1":false,"eq2":true}

Typing: If a left and right type are comparable (see table under &lt), then the types can be tested for equality. The resulting type is a bool. Beyond comparable left and right types, the following table

Inequality: expr != expr

TODO: something pithy

Sample Code

public int a;
public int b;
public formula neq1 = a != b;
public formula neq2 = a + 1 != b;
@construct {
  a = 1;
  b = 2;
}

Result

{"a":1,"b":2,"neq1":true,"neq2":false}

Typing: This has the same typing as =;

Logical and: expr && expr

TODO: something pithy

Truth Table

leftrightresult
falsefalsefalse
truefalsefalse
falsetruefalse
truetruetrue

Sample Code

public bool a;
public bool b;
public formula and = a && b;
@construct {
  a = true;
  b = false;  
}

Result

{"a":true,"b":false,"and":false}

Typing: The left and right expressions must have a type of bool, and the resulting type is bool.

Logical or: expr || expr

TODO: something pithy

Truth Table

leftrightresult
falsefalsefalse
truefalsetrue
falsetruetrue
truetruetrue

Sample Code

public bool a;
public bool b;
public formula or = a || b;
@construct {
  a = true;
  b = false;  
}

Result

{"a":true,"b":false,"or":true}

Typing: The left and right expressions must have a type of bool, and the resulting type is bool.

Conditional / Ternary: expr ? expr : expr

Sample Code

public bool cond;
public formula inline = a ? 5 : 10;
@construct {
  cond = true;
}

Result

{"cond":true,"inline":5}

Typing: The first expression must have a type of bool, and the second and third type must be the compatible under type rectification. The result is the rectified type. For more information about type rectification, see anonymous messages and arrays.

Operator precedence

When multiple operators operate on different operands (i.e. things), the order of operators (or operator precedence) is used to break ambiguities such that the final result is deterministic. This avoids confusion, and operators with a higher level must evaluate first.

When there are multiple operators at the same level, then an associativity rule indicates how to evaluate those operations. Typically, this is left to right, but some operators may not be associative or be right to left.

leveloperator(s)descriptionassociativity
11expr._ident
expr[expr]
exprF(expr0,...,exprN)
(expr)
field dereference
index lookup
functional application
parentheses
left to right
10expr++
expr--
post-increment
post-decrement
not associative
9++expr
--expr
pre-increment
pre-decrement
not associative
8-expr
!expr
unary negationnot associative
7expr*expr
expr/expr
expr%expr
multiply
divide
modulo
left to right
6expr+expr
expr-expr
addition
subtraction
left to right
5expr<expr
expr>expr
expr<=expr
expr>=expr
less than
greater than
less than or equal to
greater than or equal to
not associative
4expr==expr
expr!=expr
equality
inequality
not associative
3expr&&exprlogical andleft to right
2expr||exprlogical orleft to right
1expr?expr:_exprinline conditional / ternaryright to left

Anonymous messages and arrays

Adama allows messages and arrays to be constructed as literals similar to how JavaScript allows objects and arrays to be written.

Messages

Anonymous messages (or message literals) are a convenient way to construct messages without explicitly defining a type beforehand. The way to create an anonymous message in Adama is similar to JavaScript/JSON in that braces are used to indicate the beginning and end of an object. For instance, a simple message can be constructed using the following:

@construct {
  let m = {cost:123, name:"Cake Ninja"};
}

The following syntax is the only way to instantiate a message with a named type. For instance,

message M {
  int cost;
  string name;
}

@construct {
  M m = {cost:123, name:"Cake Ninja"};
}

This is done via type rectification. Type rectification is the process of taking two values of two types, then finding (or creating) a type which allows both of them to fit together. For instance, the rectified type of int and double is double because double can hold both values. The rectified type of messages with distinct fields is a message with all the fields with the types wrapped in maybes.

Arrays

Adama supports anonymous arrays as well via the brackets similar in syntax to JavaScript. For instance, the following code produces an array:

@construct {
  let a = [1, 2, 3];
}

It is worth noting that these arrays are statically typed, and the elements within must have a compatible type under "type rectification". An interesting example for the need of type rectification is the following snippet:

@construct {
  let a = [{x:1}, {y:2}, {z:3}];
}

Here, the elements in the array have the same type, and the above is equal to:

@construct {
  let a = [{x:1,y:0,z:0}, {x:0,y:2,z:0}, {x:0,y:0,z:3}];
}

Working with arrays

When you have an array like:

@construct {
  int[] a = [1, 2, 3];
}

The primary way of getting access to a particular element is via the index lookup operator ([]). The result of the index operator has a particular twist on the result type. For instance, the following code:

@construct {
  int[] a = [1, 2, 3];
  maybe<int> second = a[1];
}

The result type is maybe of whatever the element type is, and this both forces the checking of range and contends with invalid ranges. The only way to really know the second type is via an if statement like so:

@construct {
  int[] a = [1, 2, 3];
  maybe<int> second = a[1];
  if (second as actual_second) {
    // ok, do whatever you want
  } else {
    // range failure
  }
}

While the above sample is trivial, this construct enforces the appropriate type and range to disallow bad things to go bump in the night and force an error when things are incorrect.

Maybe some data, maybe not

Too many times, a value could not be found nor make sense to compute with the data at hand. The lack of a value is something to contend with, and failing to contend with it well has proven to be a billion dollar mistake. Adama uses the maybe (or optional) pattern. For example, the following defines an age which may or may not be available:

public maybe<int> age;

// how to write
#sm1 {
 age = 40;
}

// how to read
#sm2 {
 if (age as a) {
  // so something with a if it exists
 } else {
  // age has no value, :(
 }
}

Philosophy

The concept of maybe<> enforces better coding-discipline by leveraging the type system as a forcing function to prevent bad things from happening. Segmentation fault, NullPointerException, and index out of range exceptions are avoided entirely. Within Adama, a failure feels catastrophic as a failure signals the end of the game. This is a core motivation why Adama is a closed ecosystem (i.e. no disk or networking) such that the failures are limited to logic bugs or division by zero (and the jury is out as to whether or not division should result in a maybe<double> or not)

Using maybes

Data can always freely enter a maybe using regular assignment.

maybe<int> key;

#sm {
  key = 123;
}

To safely retrieve data, the safe way is using an if ... as statement.

maybe<int> key;

#sm {
  if (key as k) {
  	// yay, I have the value
  } else {
  	// nay, I don't have a value
  }
}

Maybe expressions

An instance of a maybe with a given type can be generated on the fly via maybe<Type>. Example:

#sm {
	let key = @maybe<int>;
}

And an instance of a maybe with a given value can be generated via maybe(Expr). Example:

#sm {
	let key = @maybe(123);
}

Tables and integrated query

Given a record such as:

record Rec {
  public int id;
  public string name;
  public int age;
  public int score;
}

This record can be used with the table:

table<Rec> _records;

This table is a way of organizing information per given record type. In general, the table is a useful construct which enables many common operations found in data structures. The above record would create a table like:

idnameagescore
1Joe451012
2Bryan49423
3Jamie42892
4Jordan527231

Diving Into Details

A table in and of itself requires a toolkit to handle it, and we introduce a variant of SQL in the form a language integrated query (LINQ). It is a variant in many ways, and we will introduce the mechanics.

Reactive lists

Lists of records can be filtered, ordered, sequenced, and limited via language integrated query (LINQ).

iterate

First, the iterate keyword will convert the table<Rec> into a list<Rec>.

public formula all_records = iterate _records;

Now, by itself, it will list the records in their canonical ordering (by id). It is important to note that the list is lazily constructed up until the time that it is materialized by a consumer, and this enables some query optimizations to happen on the fly.

where

We can suffix a LINQ expression with where to filter items.

public formula young_records = iterate _records where age < 18;

indexing!

Yes, we can make things faster by indexing our tables. The index keyword within a record will indicate how tables should index the record.

public formula lucky_people = iterate _records where age == 42;

This will accelerate the performance of where expressions when expressions like age == 42 are detected via analysis.

shuffle

The canonical ordering by id is not great for card games, so we can randomize the order of the list. Now, this will materialize the list.

public formula random_people = iterate _records shuffle;
public formula random_young_people = iterate _records where age < 18 shuffle;

order

Since the canonical ordering by id is the insertion/creation ordering, order allows you to reorder any list.

public formula people_by_age = iterate _records order by age asc;

limit

public formula youngest_person = iterate _records order by age asc limit 1;

offset

With offset, you can skip the first entries with a query.

public formula next_youngest_person = iterate _records order by age asc offset 1 limit 1;

Bulk Assignments

A novel aspect of a reactive list is bulk field assignment, and this allows us to do some nice things. Take the following definition of a Card table representing a deck of cards:

record Card {
  public int id;
  public int value;
  public principal owner;
  public int ordering;
}

table<Card> deck;

We can shuffle the deck using shuffle and bulk assignment.

procedure shuffle() {
  int ordering = 0;
  (iterate deck shuffle).ordering = ordering++;
}

This assignment of ordering will memorize the results from shuffling. With a single statement, we can deal cards by assigning ownership.

procedure deal_cards(int count) {
  (iterate deck             // look at the deck
    where owner == @no_one  // for each card that isn't own
    order by ordering asc   // follow the memoized ordering
    limit count             // deal only $count cards
    ).owner = @who;          // for each card, assign an owner to the card
}

This ability makes it simple to update a single field, but it also applies to method invocation as well.

Bulk method execution

Bulk Deletes

Maps and reduce

Standard control

Adama has many control and loop structures similar to C-like languages like:

And we introduce two non-traditional ones:

Diving Into Details

if

if statements are straightforward ways of controlling the flow of execution, and Adama's if behaves like most other languages.

public int x;

@construct {
   if (true) {
     x = 123;
   } else {
   	 x = 42;
   }
}

if-as

Unlike most languages, Adama has a special extension to the if statement which is used for maybe. This allows safely extracting values out from the maybe.

int x;
@construct {
   maybe<int> m_value = 123;
   if (m_value as value) {
     x = value;
   }
}

while

while statements are a straightforward way to iterate while a condition is true.

int x = 10;
int y = 0;
while (x > 0) {
  x --;
  y += x;
}

do-while

do-while statements are a way to run code at least once.

int x = 10;
int y = 0;
do {
  x--;
  y += x;
} while (x > 0);

for

for statements are a common shorthand for while loops to initialize and step.

int y = 0;
for(int x = 10; x > 0; x--) {
  y += x;
}

foreach

foreach statements are a shorthand for iterating over arrays

int y;
foreach (x in [1,2,3,4,5,6,7,8,9,10]) {
  y += x;
}

Functions, procedures, and methods

Adama has two forms of colloquial functions.

Functions in Adama are pure in that they have no side effects and also are context-free.

function square(int x) -> int {
  return x * x;
}
int x;
procedure square_of_x() -> int {
  return x * x;
}

Enumerations

Enumerations in Adama are simply ways of associating integers to names.

enum Suit { Hearts:1, Spades:2, Clubs:3, Diamonds:4 }

We can refer to a single value via :: by

Suit x = Suit::Hearts;

Enumeration collections

We can build an array of all the values within an enumeration using the * symbol. For example, we can build an array of all the suit types via:

Suit[] all = Suit::*;

which is a handy. We can also build an array of all enumeration values which share a prefix. For instance, given the enum

enum Role { CoreLeft, CoreRight, Normal };

then we can refer to the collection of all values that start with Core via

Role[] core = Role::Core*;

This is a handy way to build a strict taxonomy within an enumeration.

Dispatch

Async with channels, futures, and handlers

Handling messages directly

When users connect, the primary way they can interact with the document is to send messages to the document. This requires a message like:

public string output = "Hello World";

message ChangeOutput {
  string new_output;
}

This establishes the shape of the communication, and we leverage a channel to open a pathway for messages to execute code. One option is to add a message handler:

channel change_output(principal sender, ChangeOutput change) {
  output = change.new_output;
}

This enables users to send messages via the change_output channel which will execute the associated code. In this example, 'change_output' is the name of the channel which clients will annotate their message with to execute the associated code. Nothing stops you from introducing multiple channels with the same message type.

channel change_output(principal sender, ChangeOutput change) {
  output = change.new_output;
}
channel set_output(principal sender, ChangeOutput change) {
  output = change.new_output;
}

Waiting for users via futures

While accepting messages indiscriminately is great for applications such as chat, you need a way to combine multiple messages in an orderly fashion. Remember, Adama was designed for board games, and a good pattern to use is async/await pattern within code. Here, we define a message and an incomplete channel.

message SomeDecision {
  string text;
}

channel<SomeDecision> decision;

And any state machine transition block can then leverage the channel to pull data from a user.

private principal player1;
private principal player2;

#getsome {
  future<SomeDecision> sd1 = decision.fetch(player1);
  future<SomeDecision> sd2 = decision.fetch(player2);
  string text1 = sd1.await().text;
  string text2 = sd2.await().text;
  // do something with text1 and text2 to decide a winner, I guess?
}

Note, these two users can come up with the message independently and this decoupling of asking a person (i.e. the fetch) and getting the data in code (i.e. the await) enables concurrency.

channel<T>.fetch(who)

channels have a fetch method that will result in a single message but has the semantics that the document can block forever. There is a mechanism called preempt which can unblock the state machine, but this is an experimental feature.

channel<T>.decide(who, T[])

Similar to fetch, the decide function enables something very cool and is the recommended approach for board games. Decide will ask the user to pick exactly one item from the given array of options. This is exciting as this enables AI to autoplay games, but this is outside the topic of the documentation... for now.

The state machine

Each document acts like a state machine where there is a single state label indicating the major state of the document. The rest of the document is considered supporting or related state included within the state machine. The game this induces on the code is very simple. First, you associate code to a state machine label:

#start {
  /* fun times */
}

Second, you transition into that state somehow (i.e. constructor or message handler):

@construct {
  transition #state;
}

Finally, the state transitions can happen over time via the in keyword on the transition keyword.

bool done;
@construct {
  done = false;
  transition #state;
}

#start {
  transition #end in 60;
}

#end {
  done = true;
}

Diving Into Details

Outline Code

Relationship to Messages & Futures

Transition

Pre-emption

Reactive formulas

Adama is inspired by spreadsheets. Beyond capturing data in tables, spreadsheets have formulas to enable various useful forms of computation. This idea is a key part behind the reactive programming model in Adama. It's easy to define a formula, and here is an example:


public int x;
public int y;

public formula len = Math.sqrt(x * x + y * y);

Diving Into Details

The formula identifier is used like a type but enables the right-hand side of the '=' to be an expression combining any previously defined state (or other formulas) in a glorious mathematical expression. There are two key rules important to address:

  • The right-hand side must be "pure" in a side effect free kind of way. That is, it is unable to mutate the document. Practically, this means that only math and functions can be used. Procedures are strictly forbidden.
  • The right-hand side must only refer to a state defined before the introduction of the formula. This prevents circular logic.

With regards to performance and semantics, formulas are 100% lazy. No computation is done until the data is asked for, and the result of the computation is cached until underlying data is invalidated. When data changes happen, the result will be thrown away until the next time the data is called upon.

Privacy and bubbles

The document and records contain fields and formulas that are privacy checked before they can be viewed by users. This section outlines how data is exposed to users. At the core, each field is prefixed with a privacy modifier.

ModifierEffect
publicAnyone can see it
privateNo one can see it
viewer_is<field>Only the viewer indicated by the given field is able to see it
use_policy<policy>Validate the viewer can witness the value via code; policies are defined within documents and records via the policy keyword.

These modifiers are great for revealing simple data common between all viewers of the document. Viewer-dependent data is achievable via a privacy bubble using the bubble keyword.

record Row {
  // it's public
  public int pub;

  // it's private, no one can see it
  private int pri;

  // a private person
  private principal who;

  // data that is only visible to the who
  viewer_is<who> int whos_age;

  // a custom policy based on code
  use_policy<my_policy> int custom;

  // defining the policy
  policy my_policy(c) {
    return pub < pri;
  }

  require p1;
}

table<Row> tbl;

// reveal mine via a formula where me represents the client viewing the document
bubble mine = iterate tbl where who == @who;

Diving Into Details

public/private

The private modifier hides data from users. The public modifier discloses data to users. If no modifier is specified, the default is private.

viewer_is<>

Inside the angle brackets denotes a variable local to the document or record which must be of type client. For instance:

principal owner;
viewer_is<owner> int data_only_for_owner;

Here, the field owner is referenced via the privacy modifier for data_only_for_owner such that only the device/client authenticated can see that data.

use_policy<> & policy

As visibility may depend on some intrinsic logic or internal state, use_policy will leverage code outlined via a policy. This code is then run when the client wishes to see the data.

record Card {
  // some internal state
  private bool played;

  // who owns the card
  private principal owner;

  // the value of the card
  use_policy<is_in_play> value; // 0 to 51 for a standard playing deck

  // who can see the card
  policy is_in_play(who) {
  	// if it has been played, then everyone knows
  	// otherwise, only the owner can see it
  	return played || owner == who;
  }
}

bubble<>

While privacy policies ensure compliance, we can leverage bubbles to efficiently query the document based on the viewer.

table<Card> deck;

bubble hand = iterate deck where owner == @who;

Web processing

Your adama script is a webserver! Each Adama document is an opinionated webserver supporting GET, PUT, OPTIONS, and DELETE methods. These methods can return HTML, JSON, or XML.

Documents are addressable within a region via a URL of the form:

https://$region.adama-platform.com/$space/$key/$path

Where $region is the region to request the data from, $space is the adama's script name, $key is the document key, and the $path is then handed over to the specified document.

GET

@web get / {
  return {
    html: "Oh, Hello there! this is the root document"
  };
}

OPTIONS (Cors Preflight)

@web options / {
  return {
    cors: true
  };
}

PUT (& POST)

Adama normalizes url-encoded bodies into JSON objects, and converts POST into PUT.

message TwilioSMS {
  string From;
  string Body;
}

@web put /webhook (TwilioSMS sm) {
  return {
    xml: "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Message>Thank you for the data. NOM. NOM.</Message></Response>"
  };
}

DELETE

@web delete / {
  return {
    json: {}
  };
}

Query parameters

The @web handler has @parameters (a special constant) for accessing the query parameters as a dynamic.

Headers

The @web handler has @headers (a special constant) for accessing the headers parameters as a map<string,string>.

Responding

The return value on a @web is a message that is compile-time interpreted with rules. This message controls the behavior of the web server

The body

The returned message within a @web may contain at most one body field.

fieldvalue typebehavior
xmlstringthe string is sent to the client with content type 'application/xml'
jsonmessagethe message is converted to JSON and sent to the client with content type 'application/json'
htmlstringthe string is sent to the client with the content type 'text/html'
assetassetthe asset is downloaded and sent to the client with the appropriate content type

Cross origin resource sharing

The returned message within a @web may set a 'cors' field to true. This will allow the request to be visible to a browser.

@web get / {
  return {cors: true}
}

Caching

Not implemented yet

Interacting with remote services

TODO

Reference

The Standard Library outlines the built-in functions that Adama provides developers without the need to reach out to remote services.

The Javascript API Reference details the various methods found on the JavaScript client library once connected, and it's helpful to refer to JavaScript Connection and Authentication to setup a connection and how to authenticate each method call. If you are using a language other than JavaScript, then the JSON Delta Format is useful for understanding how to reconstruct JSON objects for document connections.

Standard Library

Strings

The core string type is extended with a variety of methods which can be invoked on string objects. For example,

  public string x;
  formula x_x = (x + x).reverse();

Methods

MethodDescriptionResult type
length()Returns the length of a stringint
split(string word)Splits the string into a list of parts seperated by the given wordlist<string>
split(maybe<string> word)Splits the string into a list of parts seperated by the given word if the word is availablemaybe<list<string>>
contains(string word)Tests if the string contains the given wordbool
contains(maybe<string> word)Tests if the string contains the given word if the word is availablemaybe<bool>
indexOf(string word)Returns the position of the given wordint
indexOf(maybe<string> word)Returns the position of the given word if the word is availablemaybe<int>
indexOf(string word, int offset)Returns the position of the given word after the given offsetint
indexOf(maybe<string> word, int offset)Returns the position of the given word if the word is available after the given offsetmaybe<int>
trim()Returns a new version of the string with whitespace removed from both the head and tailstring
trimLeft()Returns a new version of the string with whitespace removed from the head/start/left-side of the stringstring
trimRight()Returns a new version of the string with whitespace removed from the tail/end/right-side of the stringstring
upper()Returns a new version of the string with all upper case charactersstring
lower()Returns a new version of the string with all lower case charactersstring
mid(int start, int length)Returns the part of the string starting at the indicated start position with the length provided.maybe<string>
left(int start)Returns the part of the string starting at the indicated start position until the end of the stringmaybe<string>
right(int length)Returns a string with the indicated length coming from the end of the stringmaybe<string>
substr(int start, int end)Returns a string that starts at the given position and ends with the given positionmaybe<string>
startsWith(string prefix)Returns whether or not string is prefixed by the given stringbool
endsWith(string prefix)Returns whether or not string is suffixed by the given stringbool
multiply(int n)Returns the concatentation of the string n timesstring
reverse()Returns a copy of the string with the characters reversedstring

Functions

FunctionDescriptionResult type
String.charOf(int ch)Returns a string with the give integer character converted into a stringstring

Math

The math library adds methods to the primitive data types of int, double, long, and complex.

For example, instead of

  public int x;
  formula a_x = Math.abs(x);

developers can instead use

  public int x;
  formula a_x = x.abs();

Also, many math functions also work on maybe types since some mathematical operators may be undefined (i.e. division by zero). Operating on maybe types, while inefficient, allows for expressive compute.

Type: int

MethodDescriptionResult type
abs()Returns the absolute value of the given integer.int

Type: long

MethodDescriptionResult type
abs()Returns the absolute value of the given long.long

Type: double, maybe<double>

MethodDescriptionResult type
abs()Returns the absolute value of the given double.double
sqrt()Returns the square rootcomplex
ceil()-double
floor()-double
ceil(double precision)-double
floor(double precision)-double
round()-double
round(double precision)-double
roundTo(int digits)-double

Note; while many math functions are supported; they don't yet operate on maybe types; see #124

Type: complex, maybe<compex>

MethodDescriptionResult type
conj()Returns the complex conjugate.complex
length()Returns the length of the complex number per the pythagorean theorem.double

Statistics

Principals

The principal represents an agent on behalf of an authority. The default authority is 'adama' which represents Adama Developers authenticated via the Adama Platform. The principal has a unique facet where only the platform is allowed to create them as they represent identity, and this identity can be used for access control and privacy.

Part of access control is also validating that a user is from the right place which is where the standard library provides some simple functions to check a few things.

Type: principal

MethodDescriptionResult type
isAdamaDeveloper()Returns whether the principal is an Adama Developerbool
fromAuthority(string authority)Returns whether the principal was derived from the given authority. See authentication for how to bring your own authentication.bool

JavaScript Connection and Authentication

Linking

Connecting

The built-in tooling

The tutorial lays out the basics, but we can do a bit more with the tooling.

TODO: illustrate more about the tooling

java -jar adama.jar authority append-local \
  --authority Z2YISR3YMJRYCHN29XZ2 \
  --keystore my.keystore.json
  --private second.private.key.json

The java library

Other libraries

Deployment Plan

Since the configuration behind a space is fundamentally code, it is valuable to change it slowly for a variety reasons. For instance, it is wise to gate a deployment to a small population to test and gather metrics. It may also be wise to go slow to avoid an expensive bill.

The deployment plan has three root fields:

  • versions providing a mapping of id to versions.
  • default is an id to use when the plan fails to pick an id
  • plan is a list of rules to pick a version id

The versions mapping and the default id.

The versions object is a mapping of ids to versions of the script to use. The range of the mapping is either a string or an object. For example,

{
  "versions": {
    "x": "public int x;"
  },
  "default": "x",
  "plan": []
}

is a minimal deployment plan with a single version containing a single source script. The "x" field within "versions" is a string representing adama code to compile, and it is the default version to use. However, there are more features to building an Adama script, and the "x" field may also be an object.

{
  "versions": {
    "x": {
      "main": "public int x; @include xyz;",
      "includes": {
        "xyz" : "public int y;"
      },
      "rxhtml": "<forest>...</forest>"
    }
  },
  "default": "x",
  "plan": []
}

Now the "x" field has an object with a main which is the primary Adama script which may include other scripts which are stored within the "includes" object. Furthermore, RxHTML can be included to augment the web capabilities of Adama.

Safety happens with planning

Since change is a source of engineering pain, the plan field within a deployment plan is a simple rule engine. Simply, it's a list of objects where each object represents the parameters to a decision to route to a specific version.

For example,

{
  "versions": {
    "x": "public int x;",
    "y": "public int x; public int y;"
  },
  "default": "x",
  "plan": [
    {
      "version": "y",
      "keys": ["1", "2"],
      "percent": 1,
      "prefix": "new",
      "seed": "xyz"
    }
  ]
}

The version field within a plan's object is simply a key within the versions object in the deployment plan. The optional keys field is a hard-coded list of which keys must go to the indicated version. If the percent is 100 or more and the key shares the key shares the prefix, then that key will go to the indicated version. If the percent is less than 100 and the key shares the prefix, then the given key is hashed with the seed as a value between 0 and 100. If the value is less than percent, then the key will used the indicated version. The first rule to pick a version wins, and if no rule matches then the default is used.

The above prose represents the below algorithm

public String pickVersion(String key) {
  for (Stage stage : stages) {
    if (stage.keys != null) {
      if (stage.keys.contains(key)) {
        return stage.version;
      }
    }
    if (key.startsWith(stage.prefix)) {
      if (stage.percent >= 100) {
        return stage.version;
      }
      if (hash(stage.seed, key) <= stage.percent) {
        return stage.version;
      }
    }
  }
  return defaultVersion;
}

By fully utilizing the deployment plan, changes can be made exceptionally safe.

API Reference

Methods: InitSetupAccount, InitConvertGoogleUser, InitCompleteAccount, AccountSetPassword, AccountGetPaymentPlan, AccountLogin, Probe, AuthorityCreate, AuthoritySet, AuthorityGet, AuthorityList, AuthorityDestroy, SpaceCreate, SpaceGenerateKey, SpaceUsage, SpaceGet, SpaceSet, SpaceSetRxhtml, SpaceGetRxhtml, SpaceDelete, SpaceSetRole, SpaceReflect, SpaceList, DomainMap, DomainList, DomainUnmap, DomainGet, DocumentCreate, DocumentDelete, DocumentList, ConnectionCreate, ConnectionSend, ConnectionSendOnce, ConnectionCanAttach, ConnectionAttach, ConnectionUpdate, ConnectionEnd, ConfigureMakeOrGetAssetKey, AttachmentStart, AttachmentAppend, AttachmentFinish, SuperCheckIn, SuperListAutomaticDomains, SuperSetDomainCertificate

Method: InitSetupAccount

This initiates developer machine via email verification.

Parameters

namerequiredtypedocumentation
emailyesStringThe email of an Adama developer.

Template

connection.InitSetupAccount(email, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: InitConvertGoogleUser

The converts and validates a google token into an Adama token.

Parameters

namerequiredtypedocumentation
access-tokenyesStringA token from a third party authorization service.

Template

connection.InitConvertGoogleUser(access-token, {
  success: function(response) {
    // response.identity
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
identityStringA private token used to authenticate to Adama.

Method: InitCompleteAccount

This establishes a developer machine via email verification.

Copy the code from the email into this request.

The server will generate a key-pair and send the secret to the client to stash within their config, and the public key will be stored to validate future requests made by this developer machine.

A public key will be held onto for 30 days.

Parameters

namerequiredtypedocumentation
emailyesStringThe email of an Adama developer.
revokenoBooleanA flag to indicate wiping out previously granted tokens.
codeyesStringA randomly (secure) generated code to validate a user via 2FA auth (via email).

Template

connection.InitCompleteAccount(email, revoke, code, {
  success: function(response) {
    // response.identity
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
identityStringA private token used to authenticate to Adama.

Method: AccountSetPassword

Set the password for an Adama developer.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
passwordyesStringThe password for your account.

Template

connection.AccountSetPassword(identity, password, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: AccountGetPaymentPlan

Get the payment plan information for the developer.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.AccountGetPaymentPlan(identity, {
  success: function(response) {
    // response.paymentPlan
    // response.publishableKey
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
payment-planStringPayment plan name. The current default is "none" which can be upgraded to "public".
publishable-keyStringThe public key from the merchant provider.

Method: AccountLogin

Sign an Adama developer in with an email and password pair.

Parameters

namerequiredtypedocumentation
emailyesStringThe email of an Adama developer.
passwordyesStringThe password for your account.

Template

connection.AccountLogin(email, password, {
  success: function(response) {
    // response.identity
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
identityStringA private token used to authenticate to Adama.

Method: Probe

This is useful to validate an identity without executing anything.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.Probe(identity, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: AuthorityCreate

Create an authority. See Authentication for more details.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.AuthorityCreate(identity, {
  success: function(response) {
    // response.authority
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
authorityStringAn authority is collection of third party users authenticated via a public keystore.

Method: AuthoritySet

Set the public keystore for the authority.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
authorityyesStringAn authority is collection of users held together via a key store.
key-storeyesObjectNodeA collection of public keys used to validate an identity within an authority.

Template

connection.AuthoritySet(identity, authority, key-store, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: AuthorityGet

Get the public keystore for the authority.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
authorityyesStringAn authority is collection of users held together via a key store.

Template

connection.AuthorityGet(identity, authority, {
  success: function(response) {
    // response.keystore
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
keystoreObjectNodeA bunch of public keys to validate tokens for an authority.

Method: AuthorityList

List authorities for the given developer.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.AuthorityList(identity, {
  next: function(payload) {
    // payload.authority
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
authorityStringAn authority is collection of third party users authenticated via a public keystore.

Method: AuthorityDestroy

Destroy an authority.

This is exceptionally dangerous as it will break authentication for any users that have tokens based on that authority.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
authorityyesStringAn authority is collection of users held together via a key store.

Template

connection.AuthorityDestroy(identity, authority, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceCreate

Create a space.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
templatenoStringWhen creating a space, the template is a known special identifier for how to bootstrap the defaults. Examples: none (default when template parameter not present).

Template

connection.SpaceCreate(identity, space, template, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceGenerateKey

Generate a secret key for a space.

First party and third party services require secrets such as api tokens or credentials.

These credentials must be encrypted within the Adama document using a public-private key, and the secret is derived via a key exchange. Here, the server will generate a public/private key pair and store the private key securely and give the developer a public key. The developer then generates a public/private key, encrypts the token with the private key, throws away the private key, and then embeds the key id, the developer's public key, and the encrypted credential within the adama source code.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'

Template

connection.SpaceGenerateKey(identity, space, {
  success: function(response) {
    // response.keyId
    // response.publicKey
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
key-idIntegerUnique id of the private-key used for a secret.
public-keyStringA public key to decrypt a secret with key arrangement.

Method: SpaceUsage

Get the most recent space usage in terms of billable hours.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
limitnoIntegerMaximum number of items to return during a streaming list.

Template

connection.SpaceUsage(identity, space, limit, {
  next: function(payload) {
    // payload.hour
    // payload.cpu
    // payload.memory
    // payload.connections
    // payload.documents
    // payload.messages
    // payload.storageBytes
    // payload.bandwidth
    // payload.firstPartyServiceCalls
    // payload.thirdPartyServiceCalls
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
hourIntegerThe hour of billing.
cpuLongCpu (in Adama ticks) used within the hour.
memoryLongMemory (in bytes) used within the hour.
connectionsIntegerp95 connections for the hour.
documentsIntegerp95 documents for the hour.
messagesIntegerMessages sent within the hour.
storage-bytesLongThe storage used.
bandwidthLongBytes used to transmit.
first-party-service-callsLongNumber of services calls made (managed by platform).
third-party-service-callsLongNumber of services calls made (managed by developers).

Method: SpaceGet

Get the deployment plan for a space.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'

Template

connection.SpaceGet(identity, space, {
  success: function(response) {
    // response.plan
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
planObjectNodeA plan is a predictable mapping of keys to implementations. The core reason for having multiple concurrent implementations is to have a smooth and orderly deployment. See deployment plans for more information.

Method: SpaceSet

Set the deployment plan for a space.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
planyesObjectNodeThis 'plan' parameter contains multiple Adama scripts all gated on various rules. These rules allow for a migration to happen slowly on your schedule. Note: this value will validated such that the scripts are valid, compile, and will not have any major regressions during role out.

Template

connection.SpaceSet(identity, space, plan, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceSetRxhtml

Set the RxHTML forest for the space when viewed via a domain name.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
rxhtmlyesStringA RxHTML forest which provides simplified web hosting.

Template

connection.SpaceSetRxhtml(identity, space, rxhtml, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceGetRxhtml

Get the RxHTML forest for the space when viewed via a domain name.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'

Template

connection.SpaceGetRxhtml(identity, space, {
  success: function(response) {
    // response.rxhtml
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
rxhtmlStringThe RxHTML forest for a space.

Method: SpaceDelete

Delete a space.

This requires no documents to be within the space, and this removes the space from use until garbage collection ensures no documents were created for that space after deletion. A space may be reserved for 90 minutes until the system is absolutely sure no documents will leak.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'

Template

connection.SpaceDelete(identity, space, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceSetRole

Set the role of an Adama developer for a particular space.

Spaces can be shared among Adama developers.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
emailyesStringThe email of an Adama developer.
roleyesStringThe role of a user may determine their capabilities to perform actions.

Template

connection.SpaceSetRole(identity, space, email, role, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SpaceReflect

Get a schema for the space.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
keyyesStringWithin a space, documents are organized within a map and the 'key' parameter will uniquely identify documents. Keys are lower case ASCII using the regex [a-z0-9._-]* for validation

Template

connection.SpaceReflect(identity, space, key, {
  success: function(response) {
    // response.reflection
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
reflectionObjectNodeSchema of a document.

Method: SpaceList

List the spaces available to the user.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
markernoStringA key to skip ahead a listing. When iterating, values will be returned that are after marker. To paginate an entire list, pick the last key or name returned and use it as the next marker.
limitnoIntegerMaximum number of items to return during a streaming list.

Template

connection.SpaceList(identity, marker, limit, {
  next: function(payload) {
    // payload.space
    // payload.role
    // payload.created
    // payload.enabled
    // payload.storageBytes
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
spaceStringA space which is a collection of documents with a common Adama schema.
roleStringEach developer has a role to a document.
createdStringWhen the item was created.
enabledBooleanIs the item in question enabled.
storage-bytesLongThe storage used.

Method: DomainMap

Map a domain to a space.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
domainyesStringA domain name.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
certificatenoStringA TLS/SSL Certificate encoded as json.

Template

connection.DomainMap(identity, domain, space, certificate, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: DomainList

List the domains for the given developer

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.DomainList(identity, {
  next: function(payload) {
    // payload.domain
    // payload.space
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
domainStringA domain name.
spaceStringA space which is a collection of documents with a common Adama schema.

Method: DomainUnmap

Unmap a domain

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
domainyesStringA domain name.

Template

connection.DomainUnmap(identity, domain, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: DomainGet

Get the domain mapping

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
domainyesStringA domain name.

Template

connection.DomainGet(identity, domain, {
  success: function(response) {
    // response.space
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
spaceStringA space which is a collection of documents with a common Adama schema.

Method: DocumentCreate

Create a document.

The entropy allows the randomization of the document to be fixed at construction time.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
keyyesStringWithin a space, documents are organized within a map and the 'key' parameter will uniquely identify documents. Keys are lower case ASCII using the regex [a-z0-9._-]* for validation
entropynoStringEach document has a random number generator. When 'entropy' is present, it will seed the random number generate such that the randomness is now deterministic at the start.
argyesObjectNodeThe parameter for a document's @construct event.

Template

connection.DocumentCreate(identity, space, key, entropy, arg, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: DocumentDelete

Delete a document (invokes the @delete document policy).

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
keyyesStringWithin a space, documents are organized within a map and the 'key' parameter will uniquely identify documents. Keys are lower case ASCII using the regex [a-z0-9._-]* for validation

Template

connection.DocumentDelete(identity, space, key, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: DocumentList

List documents within a space which are after the given marker.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
markernoStringA key to skip ahead a listing. When iterating, values will be returned that are after marker. To paginate an entire list, pick the last key or name returned and use it as the next marker.
limitnoIntegerMaximum number of items to return during a streaming list.

Template

connection.DocumentList(identity, space, marker, limit, {
  next: function(payload) {
    // payload.key
    // payload.created
    // payload.updated
    // payload.seq
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
keyStringThe key.
createdStringWhen the item was created.
updatedStringWhen the item was last updated.
seqIntegerThe sequencer for the item.

Method: ConnectionCreate

Create a connection to a document.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
keyyesStringWithin a space, documents are organized within a map and the 'key' parameter will uniquely identify documents. Keys are lower case ASCII using the regex [a-z0-9._-]* for validation
viewer-statenoObjectNodeA connection to a document has a side-channel for passing information about the client's view into the evaluation of bubbles. This allows for developers to implement real-time queries and pagination.

Template

connection.ConnectionCreate(identity, space, key, viewer-state, {
  next: function(payload) {
    // payload.delta
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
deltaObjectNodeA json delta representing a change of data. See the delta format for more information.

Method: ConnectionSend

Send a message to the document on the given channel.

Parameters

namerequiredtypedocumentation
channelyesStringEach document has multiple channels available to send messages too.
messageyesJsonNodeThe object sent to a document which will be the parameter for a channel handler.

Template

stream.Send(channel, message, {
  success: function(response) {
    // response.seq
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
seqIntegerThe sequencer for the item.

Method: ConnectionSendOnce

Send a message to the document on the given channel with a dedupe key such that sending happens at most once.

Parameters

namerequiredtypedocumentation
channelyesStringEach document has multiple channels available to send messages too.
dedupenoStringA key used to dedupe request such that at-most once processing is used.
messageyesJsonNodeThe object sent to a document which will be the parameter for a channel handler.

Template

stream.SendOnce(channel, dedupe, message, {
  success: function(response) {
    // response.seq
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
seqIntegerThe sequencer for the item.

Method: ConnectionCanAttach

Ask whether the connection can have attachments attached.

Parameters

namerequiredtypedocumentation

Template

stream.CanAttach({
  success: function(response) {
    // response.yes
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
yesBooleanThe result of a boolean question.

Method: ConnectionAttach

This is an internal API used only by Adama for multi-region support.

Start an upload for the given document with the given filename and content type.

Parameters

namerequiredtypedocumentation
asset-idyesStringThe id of an asset.
filenameyesStringA filename is a nice description of the asset being uploaded.
content-typeyesStringThe MIME type like text/json or video/mp4.
sizeyesLongThe size of an attachment.
digest-md5yesStringThe MD5 of an attachment.
digest-sha384yesStringThe SHA384 of an attachment.

Template

stream.Attach(asset-id, filename, content-type, size, digest-md5, digest-sha384, {
  success: function(response) {
    // response.seq
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
seqIntegerThe sequencer for the item.

Method: ConnectionUpdate

Update the viewer state of the document.

The viewer state is accessible to bubbles to provide view restriction and filtering. For example, the viewer state is how a document can provide real-time search or pagination.

Parameters

namerequiredtypedocumentation
viewer-statenoObjectNodeA connection to a document has a side-channel for passing information about the client's view into the evaluation of bubbles. This allows for developers to implement real-time queries and pagination.

Template

stream.Update(viewer-state, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: ConnectionEnd

Disconnect from the document document.

Parameters

namerequiredtypedocumentation

Template

stream.End({
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: ConfigureMakeOrGetAssetKey

Here, we ask if the connection if it has an asset key already. If not, then it will generate one and send it along. Otherwise, it will return the key bound to the connection.

This is allows anyone to have access to assets which are not exposed directly via a web handler should they see the asset within their document view. This method has no parameters.

Template

connection.ConfigureMakeOrGetAssetKey({
  success: function(response) {
    // response.assetKey
  },
  failure: function(reason) {
  }
});

Request response fields

nametypedocumentation
asset-keyStringA key used to connect the dots from the connection to assets to a browser. This is a session-based encryption scheme to protect assets from leaking outside the browser.

Method: AttachmentStart

Start an upload for the given document with the given filename and content type.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
spaceyesStringA 'space' is a collection of documents with the same schema and logic, and the 'space' parameter is used to denote the name of that collection. Spaces are lower case ASCII using the regex a-z[a-z0-9-]* to validation with a minimum length of three characters. The space name must also not contain a '--'
keyyesStringWithin a space, documents are organized within a map and the 'key' parameter will uniquely identify documents. Keys are lower case ASCII using the regex [a-z0-9._-]* for validation
filenameyesStringA filename is a nice description of the asset being uploaded.
content-typeyesStringThe MIME type like text/json or video/mp4.

Template

connection.AttachmentStart(identity, space, key, filename, content-type, {
  next: function(payload) {
    // payload.chunk_request_size
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
chunk_request_sizeIntegerThe attachment uploader is asking for a chunk size. Using the WebSocket leverages a flow control based uploader such that contention on the WebSocket is minimized.

Method: AttachmentAppend

Append a chunk with an MD5 to ensure data integrity.

Parameters

namerequiredtypedocumentation
chunk-md5yesStringA md5 hash of a chunk being uploaded. This provides uploads with end-to-end data-integrity.
base64-bytesyesStringBytes encoded in base64.

Template

stream.Append(chunk-md5, base64-bytes, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: AttachmentFinish

Finishing uploading the attachment upload.

Parameters

namerequiredtypedocumentation

Template

stream.Finish({
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SuperCheckIn

The super agent periodically checks in.

This is to bring the external highly secure location into the monitoring system via a sentinel.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.

Template

connection.SuperCheckIn(identity, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

Method: SuperListAutomaticDomains

The super agent downloads all domains that are ready to be renewed.

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
timestampyesLongA system timestamp used for filtering items by time

Template

connection.SuperListAutomaticDomains(identity, timestamp, {
  next: function(payload) {
    // payload.domain
    // payload.timestamp
  },
  complete: function() {
  },
  failure: function(reason) {
  }
});

Streaming payload fields

nametypedocumentation
domainStringA domain name.
timestampLongA system timestamp for actions

Method: SuperSetDomainCertificate

The super agent will set a domain

Parameters

namerequiredtypedocumentation
identityyesStringIdentity is a token to authenticate a user.
domainyesStringA domain name.
certificatenoStringA TLS/SSL Certificate encoded as json.
timestampyesLongA system timestamp used for filtering items by time

Template

connection.SuperSetDomainCertificate(identity, domain, certificate, timestamp, {
  success: function() {
  },
  failure: function(reason) {
  }
});

This method simply returns void.

JSON Delta Format

The core algorithm used for deltas is JSON under the merge operation as defined by RFC 7386 with one big exception around arrays.

Arrays are special because RFC 7386 does not have a great way to merge arrays when small changes occur within an element. Adama works around this by converting arrays to objects, and then reconstruct the arrays with two hidden

During the merge, we detect that an object is an array if:

  • the prior version is an array
  • the object contains an '@o' field
  • the object contains an '@s' field.

Beyond arrays, the delta format also supports document based operational transforms which support collaborative text editing (For example, using Code Mirror 6). The text documents are converted to objects with three special fields: '$s', '$g', and '$i'.

Handling '@o' field (ordering operational transform)

The '@o' is an instruction to build the array from keys within the object. For instance, if the '@o' contains

['1', 'x', 'y', '2']

then the array formed is

  [
    obj['1'],
    obj['x'],
    obj['y'],
    obj['2']
  ] 

This allows small changes within elements to occur as '@o' is only transmitted when elements are added, removed, or re-ordered within the array. However, for small insertions within an array, then the format is rather excessive as it transmit keys. The '@o' array can leverage a prior version of the object. For example, if a delta contains an '@' with value:

  [[0,3],'z']

then the array is formed as

  [
    prior[0],
    prior[1],
    prior[2],
    prior[3],
    obj['z']
  ] 

This operational transform allows minimal reconstruction of the array such that insertions happen at any frequency in any place without much cost beyond a full client side reconstruction.

Handling '@s' (size setter transform)

This is a simpler array construction signal which assumes the keys are integers starting at 0. A '@s' value of 5 would construct an array of

  [
    obj[0],
    obj[1],
    obj[2],
    obj[3],
    obj[4]
  ] 

Handling '$s', '$g', and '$i' keys for operational transform

fieldmeaning
$ssequencer
$ggeneration
$iinitial value

The generation of a text field represent a unique lifetime for the document, and changes with '$g' imply a complete change in both '$i' and '$s' signalling a different document.

The initial value is used to initialize the collaborative editor at construction, and the sequencer indicates that changes are available within the document. For example, if the value of '$s' changed from 3 to 5, then the following updates can be handed over to the editor.

changes = [obj[3], obj[4], obj[5]].

For more information, please see code mirror's collaborative editing sample

RxHTML

The forest of root elements

With RxHTML, the root element is no longer <html>. Instead, it is a <forest> with multiple pages (via <page> element) and templates (via <template> element). While there are a few custom elements like <connection> (connect the children html to an Adama document) and <lookup> (pull data from the Adama document into a text node), the bulk of RxHTML rests with special attributes like rx:if, rx:ifnot, rx:iterate, rx:switch, rx:template, and more. These attributes reactively bind the HTML document to an Adama tree such that changes in the Adama tree manifest in DOM changes.

<template name="$name">

A template is fundamentally a way to wrap a shared bit of RxHTML within a name. At core, it's a function with no parameters.

<forest>
  <template name="template_name">
    <nav>
      It's common to use template for components
      that are repeated heavily like headers, 
      components, widgets, etc...
    </nav>
  </template>
</forest>

However, there are parameters via the <fragment> element and the rx:case attribute. See below for more details.

<page uri="$uri">

A page is a full-page document that is routable via a uri.

<forest>
    <page uri="/">
        <h1>Hello World</h1>
    </page>
</forest>

Beyond the uri, a page may also require authentication.

Data binding with <connection space="$space" key="$key" >

Data can be pulled into HTML via a connection to an Adama document given the document coordinates (space and key).

<forest>
    <page uri="/">
        <connection space="my-space" key="my-key">
            ... use data from the document ...
        </connection>
    </page>
</forest>

Pathing; understanding what $path values

The connection is ultimately providing an object, and various elements and attributes will utilize a value as a path. This path may be a simple field within the current object, or it can be a complex expression to navigate the object structure.

rulewhatexample
/$navigate to the document's root>lookup path="/title" />
../$navigate up to the parent's (if it exists)>lookup path="../name" />
child/$navigate within a child object>lookup path="info/name" />

Viewstate versus Data

At any time, there are two sources of data. There is the data channel which comes from adama, and there is the view channel which is the view state.

The view state is information that is controlled by the view to define the focus of the viewer, and it is sent to Adama asynchronously.

You can prefix a path with "view:" or "data:" to pull either source, and in most situations, the default is "data:".

Using data: pulling in a text node via <lookup path="$path" >

<forest>
    <page uri="/">
        <connection space="my-space" key="my-key">
            <h1><lookup path="title" /></h1>
            <h2><lookup path="byline" /></h2>
            <p>
                <lookup path="intro" /><
            </p>
        </connection>
    </page>
</forest>

Using data: connecting data to attributes

Attributes have a mini-language for building up attribute values using variables pulled from the document or conditions which control the output. | syntax | what | | --- | --- | | {var} | embed the text behind the variable into the string | | [b]other[/b] | embed the stuff between the brackets if the evaluation of the variable b is true | | [b]true branch[#b]false branch[/b] | embed the stuff between the brackets based of the evaluation of b |

```

Attribute extensions to existing HTML elements

A Guiding philosophy of RxHTML is to minimally extend existing HTML elements with new attributes which bind the HTML tree to a JSON tree

<tag ... rx:iterate="$path" ... >

The rx:iterate will iterate the elements of a list/array and scope into each element. This attribute works best when there is a single HTML child of the tag placed, and it will insert a div if there isn't a single child.

<table>
    <tbody rx:iterate="employees">
        <tr>
            <td><lookup path="name" /></td>
            <td><lookup path="level" /></td>
            <td><lookup path="email" /></td>
        </tr>
    </tbody>
</table>

<tag ... rx:if="$path" ... >

The rx:if will test a path for true or the presence of an object. If there is an object, then it will scope into it.

<div rx:if="active">
    Show this if active.
</div>

<tag ... rx:ifnot="$path" ... >

Similar to rx:if, rx:ifnot will test the absense of an object or if the value is false.

<div rx:ifnot="active">
    Show this if not active.
</div>

<tag ... rx:else ... >

Within a tag that has rx:if or rx:ifnot, the rx:else indicates that this element should be within the child if the condition on the parent is the opposite specified.

<div rx:if="active">
    Show this if active.
    <span rx:else>Show this if not active</span>
</div>

<tag ... rx:switch="$path" ... >

A wee bit more complicated than rx:if, but instead of testing if a value is true will instead select cases based on a string value. Children the element are selected based on their rx:case attribute.

<div rx:switch="type">
    Your card is a
    <div rx:case="face_card">
         face card named <lookup path="name"/>
    </div>
    <div rx:case="digit">
         numbered card with value of <lookup path="value"/>
    </div>
</div>

<tag ... rx:case="$value" ... >

Part of rx:switch, this attribute identifies the case that the element belongs too. See rx::switch for an example.

<tag ... rx:template="$name" ... >

The children of the element with rx:template are stored as a fragment and then replaced the the children from the <template name=$name>...</template>. The original children be used within the template via <fragment />

<template name="header">
  <header>
  </header>
  <div class="blah-header">
    <h1><fragment /></h1>
  </div>
</template>
<page ur="/">
  <div rx:template="header">
    Home
  </div>
<page>

<tag ... rx:scope="$path" ... >

Enter an object assuming it is present. This is a much more efficient, yet risky, rx:if

<form ... rx:action="$action" ... >

Forms that talk to Adama can use a variety of built-in actions like

rx:actionbehavior
adama:sign-insign in as an adama developer
adama:sign-upsign up as an adama developer
adama:set-passwordchange your adama developer password
send:$channelsend a message
copy:$pathmerge the form into the viewstate
custom:$verbrun custom logic

<form ... rx:$event="$commands" ... >

commandbehavior
toggle:$pathtoggle a boolean within the viewstate at the given path
inc:$pathincrease a numeric value within the viewstate at the given path
dec:$pathdecrease a numeric value within the viewstate at the given path
custom:$verbrun a custom verb
set:$path=valueset a string to a value within the viewstate at the given path
raise:$pathset a boolean to true within the viewstate at the given path
lower:$pathset a boolean to true within the viewstate at the given path
decide:channelresponse with a decision on the given channel pulling (see decisions)

<custom- ... rx:link="$value" ... >

<tag ... rx:wrap="$value" ... >

<input ... rx:sync="$path" ... >, <textarea ... rx:sync="$path" ... >, <select ... rx:sync="$path" ... >, oh my

Synchronize the input's value to the view state at the given path.

<input type="text" rx:sync="search_filter" />

Decisions

Todo

  • customdata
  • pick
  • cases
  • transforms
  • wrapping
  • decisions

History and Motivations behind Adama (i.e. Why!?!)

"Why" is a very hard (and sometimes painful) question to answer because it cuts so deep, and the answer boils down to the a motivating problem, some values held by the inventor, and a whole lot of frustration.

It all starts with the original design to bring friends together around an online Battlestar Galatica (the board game). However, the mind breaking complexity of the interplay of all the rules and player capabilities required a new kind of thing. That new thing had to manifest some values, and those values are simply:

  • Laziness which makes the job easier to get done with more capabilities.
  • Affordability because the solution shouldn't break the bank and good things happen when costs trend towards zero.
  • Stability because a thing built today should work tomorrow until the heat death of the universe.
  • Fun because building should be fun.

Now, the funny bit is that this all started with a couple of tools to make representing state easier, and then it turned into a giant document store with a full programming language. It wasn't planned, but it's pretty neat. This lead to a post-hoc attempt to define what this thing even is:

Origin Story

This programming language was born from a desire to bring a great game board-game online: Battlestar Galactica. I absolutely love board games. Unfortunately, every time I tried to implement it or any other board game, I find myself broken, hating everything about the technology I use.

At core, I believe board games represent a limit point of both technical and product complexity where traditional web techniques break down. However, old-school gaming techniques work better, but these old-school techniques have their own issues. The first motivation of this project is to bridge how web and old-school gaming techniques can work together in a cohesive way.

The complexity of a board game manifests when describing the implicit state machine required for people to communicate and execute complex rules which interact among players within a single conceptual "turn". We take day to day conversations for granted, and the best way to overcome this technically is to eschew classical HTTP/1.1 and use WebSockets (or just a socket). However, the moment you embrace WebSockets, you have a whole new world of hurt because the internet is not reliable enough for a six-hour board game.

Furthermore, I believe good things happen when compute and storage come together. I've worked and tinkered in this space for a while, and Adama is the culmination of 20 years of problems and experience into a single language, runtime, and platform.

Now, I admit, things will not be perfect. This language is intended to be niche and limited, and I want to be exceptionally upfront about this. I don't want to over-promise that this language will cure cancer or any fanciful claims of grandeur, but I do believe we are living in a dark ages of sort with these machines. Ultimately, I strongly believe we can do better, but doing better requires putting a stake in the ground as to what better looks like.

This project is a stake in the ground.

About the name

Adama was a special Lamancha goat that my wife and I raised until he passed due to calcium stones that blocked his urethra. He was an adorable goat that would love to cuddle, and I named this project after him.

Embracing Laziness

A core principle within Adama is that we should off-load as much work as possible to the language and runtime. We should be lazy and allow the language to do some valuable heavy lifting for us. The question then is "how can the language help with building real things?".

Well, the challenge is that most of the "serious programmer" languages these days are within a narrow band and do similar enough things. Worse yet, we can be rather discriminatory when it comes to languages. For instance, consider Dijkstra once wrote:

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers, they are mentally mutilated beyond hope of regeneration.

Now, he was railing against a different BASIC than what you or I may know. I was raised with QBasic, which I would then leverage in Visual Basic for Applications. If you are a professional software engineer, then you may even sneer at VBA. However, we should ask why VBA is still used and learned. Is it just inertia? Or, is it that it is simple enough for simple tasks, and "non-serious programmer" people can understand it. Not only can "non-serious programmer" understand it, they can hack it!

Don't get me wrong, there are great reasons to hate on Basic/VBA; however, the antidote is not to force people to contend with category theory. Adama aims to bridge this in a reasonable and clear enough way, and the key is to be lazy. Specifically, Adama intends to co-opt the most accessible programming model ever devised: Excel.

With Excel in mind, Adama will do some heavy lifting for product builders. Adama developers will never ever need to think about:

  • Serializing data from structures to byte arrays
  • Using a network that will fail
  • Using a disks that may fail
  • Think about synchronizing state

Excel is the rallying cry for Adama because Excel is just so fucking awesome at being productive. The challenge with Excel is that shipping products with it is not competitive with shipping software in "serious languages" from "serious programmers" to build web products. Adama hopes is to be as productive as Excel except with pure text files.

Sadly, Just being super productive is not enough for this world. Adama must also support some super powers...

Affordability (i.e. Cheap)

We live in magical times when you can spin up a Linux machine for less than $5/mo. For the average individual, the power of that machine is unimaginable (when used efficiently). Unfortunately, this creates an illusion where things are cheap.

That $5/mo machine could power thousands of games, and you could turn that $5 into way more than $5 with a reasonable business model. You will have to, since there is a human cost for managing that $5/mo machine as machines fail. But, what happens when the revenue dips below the $5 + human cost? The game is shutdown, and a core group of fans are disenfranchised. You need not look further than archive.org to know that people generally have a hard time letting go of things.

Fixing this requires a new type of cloud which amortizes both the machine and human cost down. Furthermore, if the cost of a game can be driven so low, then investments could be leveraged to convert the ongoing cost to a single up-front payment.

Let's look at Amazon S3 as a model. As of 4/11/2020, Amazon S3 charges $0.023/mo per gigabyte. For modeling purposes, let's use $0.15/mo which was the price in 2010. Furthermore, embrace pessimism and assume there exists an investment vehicle which will only be able to guarantee 1% in a year forever. For the low up-front cost of $180 ($0.15 * 12 / 0.01), a single gigabyte could exist for as long as Amazon S3 exists.

This $180 price tags feels reasonable for an important document, but a gigabyte is a lot of data. A board game at an extreme maximum may use 100KB of storage. This represents paying two cents to keep that board game's source code alive forever.

Sadly, Amazon S3 only provides storage. This is where Adama comes into the picture for compute. Adama allows a storage service to perform arbitrary-enough compute. In essence, Adama is the spark that turns dead durable storage into lively durable compute storage.

What this means is that not only will the code for a game be able to live forever, but also players will be able to play any game ever written in Adama forever. Future players will still be on the hook to pay for playing a game, but the cost of a single game now is already ridiculously cheap if you can leverage a $5/month machine to run thousands of distinct games.

Stable computing

For a variety of reasons, code requires maintenance. Furthermore, software people are expensive (really expensive), so maintenance is expensive. This is a problem for crafting software because opportunity cost has to come in balance with keeping the lights on.

In theory, some things can be done right, but it is so hard that we can have scope the conditions of the game being played. Now, Adama, the language and runtime, has the goal to provide a stable platform. The back-end for a board game written in 2021 with Adama must still function in the year 3025. Why is this a goal? Well, it sounds fun, that's why.

Adama aims for long term stability, and solving this requirement requires understanding why code requires maintenance in the first place.

First, it is hard to get code right, so we have to fix issues. The good news is that this isn't the language's problem; it's your problem. If the code is wrong, then stability would mean it will stay wrong... forever. Keep in mind, COBOL still exists. Code broken in specific ways should remain broken.

Second, code does not exist in a vacuum. The code must run on a machine, and the code must leverage a runtime. Achieving stability requires defining how to leverage a machine, and then being exceptionally careful with the runtime.

Machines are, more or less, stable enough. Fortunately, machines can be emulated with virtual machines. Adama is initially targeting a virtual machine which should last 1000 years: the JVM. The JVM can run code from the 90s, but there can be breakage. For instance, that breakage generally happens at the edge which are explored due to performance. For Adama, we will therefore drop extreme performance as a requirement. Performance is a nice to have, but it is more important to focus on conceptual simplicity and keeping the math at the elementary level.

Runtimes require great care to build to remain stable. This means when something is added to the runtime, then it should remain functional for a very long time. With Performance a non-goal, the goal is for the runtime to only expose deterministic and simple features.

Achieving a stable runtime is a serious burden. Every API in Adama gets a serious amount of formal and rigorous scrutiny. As an example, there are no plans to add any "get the current time" function. The core reason is that function doesn't exist in a mathematical sense, and it is not deterministic and stable. For productivity, Adama will import some libraries, but only a minimal number of them are high quality and can adapt standards into the minimal interfaces that Adama supports (i.e. Netty for the server). Adama's code base will mostly be a monolith.

Third, stability requires a closed ecosystem. Adama will not support direct networking or disk access. This is why many games from the 90s can be emulated with great success, and this is why some MMOs are gone forever.

Fourth, stability requires a lack of growth. Adama and the runtime are limited such that no more than 1,000 people can play a single game at a time. There, growth solved!

The reality of this desire to achieve stability is a disciplined approach. This project is not going to be done in a weekend, but a decade.

Slow is smooth, smooth is fast.

Fun as a goal

Writing code can be fun, but there are many obstacles to achieving the orgasmic dopamine hits possible when things are working well. When progress is being made, good times are ahead. More often than not, coding can be exceptionally frustrating. Write code, then compile, wait, wait some more, oh crap, and things are still broken. Maybe write this code here, refresh the page, it isn't working. Why not?

Adama takes the position that tooling should be mostly in your face and out of your way. Adama is being designed where productivity and diagnostics are not just add-ons, but central features. Fun comes from speed and no waiting.

  • Adama's validator and code compiler must run in single digit milliseconds for moderately large game back-ends.
  • Adama's deployment must run in a sub-second timeframe.
  • Tests written within Adama run in a sub-millisecond timeframe.

With such speed, each keystroke can be validated, and if things are not working then real-time diagnostics will tell you why (with your text editor of choice via LSP). If validation worked, then a deployment will happen, tests are run and reported (via LSP), and results are visible within the product. Keep in mind, Adama is aiming to be competitive with Excel. Typing in Excel produces results!

The same must be true in Adama. However, going fast sometimes causes a new set of problems. For instance, after a deployment, human interaction in a game may be needed to test it. Well, this interaction can change the state in a bad way. Thus, Adama must enable time travel!

Since Adama takes care of all state management, it also provides coherent ways of rewinding state and snapshotting state. Tests can be written against snapshotted state, and the test results can be reported in real-time.

Going fast, in a fun way, enables products to achieve results that are beyond "good enough". So much so, Adama unlocks new ways of thinking about automation.

What is Adama?

Beyond an adorable Lamancha goat who was named after William Adama from the amazing Battlestar Galatica show, much of what Adama is from an accidental journey of discovery by building tools to build an online board game. There was no specific goal, and the only semblance of cohesion is from the craftsman sensibilities to polish and reduce. After the fact, the question became "Wait, what even is this thing?"

One idea that emerged was the concept of a Living Document, and the short version is a document that can modify itself based on external signals (i.e. messages) and time.

In the language of games, Adama also represents the Dungeon Master who has the job of telling a story and coordinating players.

A Hacker News comment (which I'm unable to find) from Carl Hewitt indicated that everyone invents the actor model sometime in their career, and Adama is my version of a partial actor model.

Since Microsoft Excel were Lotus 1-2-3 were an influece on how to think with computers (and because reactivity is just so cool), Adama is a reactive compute engine that stores everything in a document. However, instead of infinite grids, Adama embraces tables within documents.

A key problem that reactivity makes simple is privacy where what people see is computed based on them rather than the sender. This allows for next level privacy gurantees should regulations intensify.

As time moved on with using the tool, the idea of single file infrastructure emerged emerged where that single Adama file allowed me to define the backend for a complete product.

What is a Living Document?

Let's start by defining a living document as opposed to a dead document. Take a look at the following example JSON document:

{
  "name": "Jeff",	
  "state": "Washington",
  "title": "Dark Lord"
}

This example JSON is a representation of a person living in the great state of Washington with an awkward title (it's me, tee hee), but this JSON is dead. It requires external stimulus to change. If you put this document inside a typical document store without any additional updates, then it will remain that way for as long as that document store exists.

A living document is the opposite of a dead document. A living document can be put within a living document store, and the document will update and change on its own over time. Take for instance the following Adama code:

public int ticks;

@construct {
  ticks = 0;
  transition #tick in 1;
}

#tick {
  ticks++;
  transition #tick in 1;
}

This Adama code defines:

  • a publicly visible integer field called ticks (a singleton value scoped to the document).
  • a constructor which runs only once when the document is created to initialize ticks and kick off the state machine
  • a state machine transition that runs every second and increments the ticks field

In effect, a living document is just a state machine on top of a JSON document that can transition in three ways: time, messages from people, or shared data changes between other living documents. The above example illustrates time. See Actors and Dungeon Master for more details about how messages come into the picture. Shared data changes are a work in progress, so ping me if you want more details.

Alternatively, a living document is a tiny persistent server.

Mental Model: Tiny Persistent Servers

An exceptionally tiny persistent server is an alternative view of this concept. This is what the merging of state and compute looks like with Adama. With Adama, you outline the shape of your state and then open up mechanisms for how that state changes. This is comparable to building a server in whatever language you want except the discipline to correctly persist state outside of the server is handled for you by the runtime.

For the target domain of board games, this is exceptionally useful because representing the state of board games is a difficult task within itself. With Adama, a single document represents the entirety of a single game's state via a singular definition.

Now, representing the state inside a single document provides an exceptional array of features: versioning, debugging via time travel, an imagination for androids, atomic and consistent boundaries, locality homing, and more. These features will be documented in the future.

Actors, Actors, Actors

The actor model is worth considering as it expresses many ideas core to computing. The living document that Adama defines is a limited actor of sorts. Namely, it can only receive messages from users (or devices) and make local decisions and updates based on those messages. Currently, it is unable to create more actors or send messages to other actors. However, this basic and reduced actor model enables the living document to be useful.

Adama allows developers to define messages:

message MyName {
  string name;
}

This Adama code outlines a structure with just a name field. Messages are used exclusively for messaging between the users and the living document. The next step is then to ingest a single message on a named channel:

public string name;

channel my_channel(MyName msg) {
  name = msg.name;
}

This Adama code defines:

  • a publicly visible string field called name (a singleton value scoped to the document).
  • a channel named my_channel that outlines a function
  • a variable within the function called who which represents a connection with a user (via the client type)
  • a variable containing the message (msg) sent by the user
  • a block of code to execute when the message arrives, and this code will associate the singleton name field name with the value coming from the human msg.name

This is the primary way for the living document to learn about the outside world. Internally, the following JSON represents the message sent on a persistent connection from a device:

{
  "name": "Cake Ninja"
}

This message, when sent to the living document via channel "my_channel", will run the handler code and change the top level field name to "Cake Ninja".

The most important property of messages is that they are atomically integrated. This means that if a message handler aborts (via the @abort statement), then all changes up to the abort are reverted. For instance:

public string name;

channel my_channel(MyName msg) {
  name = msg.name;
  if (name == "newbie") {
    @abort;
  }
}

The above code will temporarily set the top level name field to "n00b" which is visible only to the running code, but the @abort will roll back all changes encountered, and it will be as if the message never happened. This is super important for ensuring the document is never inconsistent or torn.

Mental Model: Old School Chat Room

The mental model for the document is a receiver of messages from clients connected via a persistent connection. This persistent connection has signals based on client connectivity, and these meta signals are exposed via @connected and @disconnected events which are exceptionally useful.

@connected {
  // keep track of who
}

@disconnected {
 // remove who
}

These signals enable the document to identify who is connected and provide the features found in classical old school chat rooms: "Jeff has entered the conversation" and "Jeff has left the conversation".

Dungeon Master

With time and messages driving changes to the living document, the next challenge is organizing messages from multiple clients. This is where we combine the state machine model with messaging to turn Adama code into a "dungeon master" (or, a workflow coordinator).

This is accomplished by creating "incomplete" channels that yield futures. First, you define a message:

message PickANumber {
  int number;
}

Then, you define the incomplete channel:

channel<PickANumber> decide_number;

Here, we see an incomplete channel which has no code associated to it, so what good is it? Well, the idea is to delay message delivery until something else asks for it. That is, other code is able to fetch a value from this incomplete channel when it makes sense.

Let's imagine a simple game of "who can pick a bigger number?". Two people must contribute a number, those numbers are compared, and the winner is given a score point. Play continues forever as this game is a game with no end.

In just this trivial game, there is a need to build a state machine formed by product questions:

  • When do player 1 and player 2 learn they need to provide a number?
  • How do we ask players to contribute?
  • Do we ask players to contribute in sequence, or in parallel?
  • If parallel, then how do we handle the ordering of contributions from players?
  • How do we deal with duplicates from players?
  • How do players deal with failure of sending a message?
  • What happens if a connection to a player is lost?

This all manifests from failures and the difficulty of network programming. Network programming is exceptionally hard, but this is where Adama comes in to save the day. This game can be implemented with following logic:

// somehow, the document learned of the two people playing the game
principal player1;
principal player2;

// scores for the players
int score_player1;
int score_player2;

// somehow, the document got into this state, but when this runs
#play {
  // both players are asked for a number
  future<PickANumber> a = decide_number.fetch(player1);
  future<PickANumber> b = decide_number.fetch(player2);

  // we then await the numbers to be able to compare and score them
  if (a.await().number > b.await().number) {
  	score_player1++;
  } else {
  	score_player2++;
  }

  // let's play again for all time.
  transition #play;
}

The above Adama code performs several actions, so there are comments to explain the more mundane elements. Critically important for this discussion, there are two key elements to focus on. First, this code is responsible for asking players for their numbers:

  future<PickANumber> a = decide_number.fetch(player1);
  future<PickANumber> b = decide_number.fetch(player2);

The fetch method on a channel will reach out to the client and ask for a specific type of message for delivery on that channel. This fetch returns a future which can be awaited to return the message from the user. Notice, concurrency is built into this model and both players can contribute their number independently at the same time. A future represents a value which will arrive... in the future.

The second key element is found in the following code to get the contributions from the players:

  if (a.await().number > b.await().number) {

The await invocation here will block execution until the value arrives from the associated clients. Now, this is nothing new in terms of programming languages. However, this is a fundamental game changer for data storage, and this is the key element that simplifies board game back-ends.

This allows the state machine of interaction between users to be constructed with the flow of code rather than modelling an explicit state machine. With Adama, this implicit-code-flow-based state machine is also durable such that the server running the code can change without users noticing.

The server is in control of who is responsible for producing data and when, and failures don't manifest in the experience (beyond elevated latency).

Mental Model: Restaurant Ordering System

The mental model for the document is to behave as a broker between parties.

For instance, if you order food online for delivery, then you really hope the chef gets the order. This requires the technology to monitor a transaction across human-scale timeframes. It may take minutes for the hostess or chef to commit to the execution of the order, or to provide feedback about the viability of the order.

This timeframe elevates the challenge as the reliability and latency of the signaling between these two parties is critical. Adama greatly simplifies this challenge with low cognitive load.

Mixed Storage and Compute Engine

Doing stuff based on time, reacting to messages, and coordinating human behavior brings compute to the table in a fun and almost complete way. Previously, we've mentioned global fields within the living document, but we need more. We need containers of data, and the best container of data ever is: the table.

In Praise of Databases

The way a table works is you first define a record

record MyRecord {
  public string name;
}

and then define a table:

table<MyRecord> my_records;

Tables are not directly exported to people, and instead require a formula to yield data. We can do that via the "iterate" expression

public formula records_by_name = iterate my_records order by name asc;

We leverage the messaging abilities to ingest to the table. The "<-" operator ingests data into tables and free form records.

message AddRecord {
  string name;
}

channel my_channel(AddRecord msg) {
  my_records <- msg;
}

Mental Model: Tiny Personal Databases

The key idea here is that data within a living document is akin to a personal database within a single file. This allows developers to leverage an appropriately scaled database per user or game whilst using NoSQL scaling techniques. This eliminates cross-document data bleed, limiting inter-document communication, but that is a worthy limitation for building a new private world.

Privacy respecting storage

So, exposing data from a table opens up the challenge of "well, wait, who can see that data?". One solution is to ensure all queries are respectful of the board game rules, but this requires ensuring all leafs in a forest are doing the right job. It would be better if the actual data could be encoded with who can see it. We start by defining the simplest model. So, here, we define an Account:

record Account {
  public string name;
  private string bank_account;
}

table<Account> accounts;

Notice that the public and private modifiers have been hijacked to mean "which humans can see this data!", and this mirrors my understanding of field visibility when I was a kid.

This was born from how to represent board game state such that secrets do not flow incorrectly to the wrong people. After all, if I can see your hand in poker, then you lose.

Classical databases were built around security within an organization and the granularity within the database is too coarse-grained. It's up to application developers to protect data between people, and this is a heavy burden. This burden was inherited by NoSQL. Instead, Adama believes that a document should contain all access rules to data, and the language aims to simplify that process.

The end goal here is to have a language which prevents information leakage to unintended parties, so developers are not in a position to accidentally leak data as reading data is entirely controlled by the document. The document is the source of truth with regards to privacy.

Mental Model: VIP Club

The document is an exclusive club, and you have an id card. That id card is checked when entering the club via security, and it grants you access to parts of the club.

Single file infrastructure

The state of the world

Building a web product is a complete pain in the ass. First, you must build the UI and then have the UI be hosted on a server which is usually stateless. The server then must implement an API to expose data to the UI, so the UI can be useful to enable people to do stuff. The API then must coordinate state with a database or other data stores, and if you want any modern features you will need a queue or use a messaging stack. The messaging stack must inter-op with the API server, and that stack also will reach out to a variety of notification services. All this then requires some form of infrastructure orchestration because you will inevitably give up on building a monolith and get a microservice architecture, and then reliability becomes a hard task because you will rely on the network to behave. The network will mostly behave, but everyone will periodically be forced to (re)learn how to deal with queueing theory.

Shit will happen, and services will fail due to growth. Holy crap.

There are so many things behind most websites, that it is a crazy mess. It's MADNESS, and it sure isn't fine. Is it any wonder that I'm full of hate?

A bold new world

Adama enables a different way of thinking such that a single file can define an entire product infrastructure. The UI still exists as a separate build, but the UI only really needs to talk to a single piece of infrastructure. This greatly simplifies the process of building software, but there are no silver bullets. This power is available because scale has been scoped and traded for ergonomics, so let's talk about limits.

A single living document can only live authoritatively on a single host. This limits two things: the update rate (CPU) and size (memory). The update rate must be balanced with the replication cost, and Adama is designed such that small updates manifest in small replication changes and minimal ingestion. This implies durability can be affordable, and it also implies that the ability to consume a living document can scale infinitely. The key limits at play are the update-rate and size.

The big missing piece as of this writing is the inability for documents to connect together, so there is no glue or index between documents. This, however, is a solvable problem.

Mental Model: The Tiny-House for User Data

As a builder, both relational databases and NoSQL stores will shard user data across various machines. As a somewhat privacy concerned user, this terrifies me as all my data is potentially on hundreds of machines. With Adama, all the logic and data can exist within a singular conceptually atomic unit of a document. It's like a tiny house of all your data, and it moves with you. This container model of all your data makes it simpler to comply with emerging regulations since both developers and users can trust where data actually is.

Cheatsheet

Types

string, bool, int, double, complex, long, principal, maybe, table

Privacy

public, private, viewer_is<$variable>

Compute

(privacy)? formula name = foo();
bubble name = foo();