Skip to content

Init system design documentation

Jump to bottom Edit New page
Romāns Volosatovs edited this page Jun 4, 2016 · 7 revisions

Purpose:

Provide an init system that can be used to manage in-browser services used by web applications in a declarative manner, using systemd unit files.

Overview:

Browsix provides the ability to run traditionally server-side services in-browser. Both web services run as Unix processes and those run in containers under systems like Docker, Kubernetes, Mesos provide a way to specify the command used to start the executable, along with the dependencies of that executable. We aim to provide the same functionaly in-browser, under Browsix, ideally with source-level compatability.

Use case:

A developer has a meme creator that they have developed as a traditional client/server web application. The client is written in JavaScript, and the server in Go, and the server relies on redis to store data about recently created memes. We want to be able to run redis + the Go-based web server in-browser, without having to write a lot of Browsix-specific JavaScript to start up services in the right order, restart them if things go wrong, and notify the main JavaScript client when the in-Browsix service is ready to handle requests.

Background:

An example is:

https://meme.bpowers.net/

This is a little web application that has a JavaScript/HTML UI, and talks to a Go HTTP server to generate images. The client-side JavaScript can route requests to 1 of 2 places: an amd64 binary running on Amazon EC2, or to a Browsix process running in the browser. I've got some custom JavaScript to start up the server under browsix ( https://github.com/plasma-umass/browsix/blob/master/examples/meme-service/frontend/app/scripts/main.js#L104 ), but its sort of hacky, and I had to write a systemd unit file anyway to have it run serverside ( https://github.com/plasma-umass/browsix/blob/master/examples/meme-service/server.service.in ).

Doing things manually (and different for server vs. Browsix) like this works ok, but if that server depended on anything else (like a database, or another small HTTP server), it would get annoying quick. I would have to manually make sure each dependency was started, and each dependency of a dependency, before starting the HTTP server - which is the sort of service supervision/dependency management thing init systems do. So the goal of an init system here is to provide the same sort of process supervision + runtime dependency management that applications can depend on when they deploy to the cloud, in the browser. Ideally in such a way that deploying a web application to the cloud uses as much of same process/file formats as makes sense for deploying it to Browsix.

With that in mind, It isn't that I think shell scripts are intrinsically bad, shell scripts are fine, I write scripts almost every day. It is more that I think they're overkill for what we need, and add unnecessary complexity. The key difference to is is that one is a declarative config file (systemd unit files), vs. an imperative, turing complete programming language.

Requirements:

  • An init system that runs in-Browsix, with unit tests that run in-Browsix and on Linux. Optional: init system runs on stock Linux, for testing in a VM/container.

  • A systemctl executable which implements the major commands of the systemd provided one, including: start, stop, reload, restart, status, isolate, list-units. This allows for introspection and manipulation of the environment by developers and programmatically.

  • Support for unit files, including the Important Sections of unit files listed in Tables 8.9, 8.10, 8.11, here.

  • Support for simple, forking, and oneshot unit types as lited in Table 8.10, here.

  • Support for Targets, as described here.

  • Tests for all major functionality, such as properly reading in unit files, building a dependency graph, ensuring services are started/restarted, etc. If there are existing test-suites out there that cover some or all of this, maybe they can be adopted as-is.

Work Plan:

This will be implemented in Go.

Milestones:

  1. init can parse a unit file and consequently start or stop the service specified.
  2. systemctl can be used to start and stop services
  3. systemctl provides info about all running services
  4. systemctl status shows a graph similar to the one in upstream version

Details:

Package unit containing all the unit type declarations, capable of creating an instance of a determined unit type out of definition

Package system capable of finding units on FS or by-name in-memory and supervising them(start/stop/return status, etc.)

  • Iterates over (possible unit) files in several directories (/usr/lib/systemd/system, /etc/systemd/system, etc)

Package lib containing 'shared' resources - defined states, errors, status declarations

Main package capable of booting the system, listening for JSON on a socket

Package systemctl capable of communicating with the 'init' via JSON messages on a socket

Systemctl that can:

  • start a unit systemctl start x
  • stop a unit systemctl stop x
  • show all active units: systemctl

Tests:

  • how do we test all of this, so that we're confident that we don't break functionality (without noticing) later in the summer?
  • e.g.: unit tests for: parsing unit files correctly, making sure incorrect files are handled gracefully.

Tests are going to be written for testing of init system as standalone and as a part of Browsix:

  • The former is mainly consist of testing the exported functions of each package utilising the testing package from standard library.
  • For latter I propose using Mocha.js, which is already used for the shell command tests. The test would create a unit file for each 'type', some of which would be incorrect, enable them, attempt to boot Browsix and compare the output of systemctl status of each unit to the expected one. (wouldn't that work?)

Steps:

  1. Parsing of unit files
  2. Service unit types 'simple' and 'oneshot' - Start(), Stop()
  3. Unit queue, dependency resolution
  4. Implement systemctl (using Cobra), providing access to already implemented functions in unit package
  5. 'Reload', 'Restart', 'Status', 'List-units'
  6. Work on implementing the missing unit types and options
Add a custom footer
Add a custom sidebar
Clone this wiki locally