gophercon-2019.slide

Simple, Portable and Efficient Graphical Interfaces in Go
Gophercon 2019

Elias Naur
mail@eliasnaur.com
@eliasnaur
https://gioui.org
https://scatter.im

* Go 2018 Survey Results

.image survey_obstacles.svg.png 900 _

: According to the Go survey, writing GUI programs is one of the top 5 challenges for Go programmers.

: This talk is about that challenge, and about making Go a good choice for writing GUI programs.

: I want to make a serious dent in fixing that and I want to make Go a natural and even obvious choice for writing GUI programs with.

: Some of you have probably seen this figure before. It's from the Go 2018 survey results, and shows the top responses to the question "What is the biggest obstacle you personally face using Go today?"

: As you may know, the top complaint, packaging and modules, is being taken care of.

: The second, familarity; I suppose that's going to work itself out over time.

: Number 3, generics, is also well covered; there's even a talk about it tomorrow by Ian Taylor.

: So that's the top 3. And it seems momentum is building for doing something about the fifth entry, error handling.

: That leaves #4 open, GUI development. This talk is about making a serious dent in that.



* Introduction

Gio - [[https://gioui.org][gioui.org]]

Gio is a simple Go module for writing portable and fast graphical interfaces.

Scatter - [[https://scatter.im][scatter.im]]

Scatter is a Gio program for end-to-end encrypted messaging over email. 

: So what's in it for you?

: In this talk I'm going to introduce Gio, a set of packages for writing simple and portable graphical user interfaces in Go.

: It is written from scratch in Go and only depend on a few system libraries.

: I've spent almost 2 years designing and developing it, and it's been my dog food for the last month or so while developing Scatter.

: It really does feel much simpler and faster to write user interfaces with Gio compared to my years of experience with other libraries and frameworks.





* Demo - Scatter

: Gio is a comprehensive module and I have a lot to cover in my time slot. However, as a motivating example of what can be achieved today, here's Scatter which is a small program for sending encrypted messages over email.

: Scatter runs on the desktop as well as on the two mobile platforms.

: It asks for login details to a SMTP and IMAP host, after which you can send or receive invitations to messaging using the Signal protocol.

: I've filled a few demo contacts and message threads to give you a feel of the program.

: [demo scatter]



* Features

- Immediate mode design.
- Only depends on lowest-level platform libraries.
- GPU accelerated vector and text rendering.
- No garbage generated in drawing or layout code.
- Cross platform (macOS, Linux, Windows, Android, iOS, tvOS, Webassembly).
- Core is 100% Go. OS-specific native interfaces are optional.

: At the time I decided to write Gio I simply wanted to write mobile apps in pure Go.

: Then, as I considered the various designs, Google's at that time up and coming Flutter framework nudged me to take the clean slate approach. Rather than interfacing with some existing toolkit and suffering the maintenance load of keeping the two worlds cooperating, Flutter does everything internally: drawing, input handling, layout, animation and so on. It takes much longer to build, but the result is much more maintainable.

: The popularity of Flutter is a great boon to Gio; side-stepping a platform's native UI toolkit is risky, but Flutter is proof that the approach is feasible and reasonable.

: Finally, for the efficient drawing of vector graphics, text in particular, Gio uses the same approach as Pathfinder, a Rust library for drawing vector graphics on the GPU.

: With Pathfinder, there is no need to pre-bake vector outlines into images before drawing. For example, all text in Gio is rendered from the truetype font character outlines every frame.


* Immediate mode UI

- UI state is owned by the program. Even your layout and widget tree.
- No callbacks. Events are handled while drawing.

: .image tribaltrouble.jpg 370 _

: I haven't written a line of Dear ImGui code, but reading about it and watching a presentation by fellow game developer Casey Muratori, I went further that even Flutter and abandoned the traditional implicit widget hierarchy and state tracking of other frameworks. Even React doesn't go as far as Dear ImGui and Gio.

: What is an immediate mode UI?

: When I wrote the custom GUI code for the 3D multiplayer game Tribal Trouble 15 years ago, I instinctively chose a design similar to Dear ImGui.


* Blank window

.play blank.go

* Blank window

.play blank.go HLmain

* Blank window

.play blank.go HLeventloop

: I tried to fit a complete hello world example on a slide, but failed. Here's a minimal program for a blank window instead.

: This is a minimal complete Gio program ready to run; it displays an empty window, and is otherwise fully functional.

: * Blank window

: .code blank.go HLinitfunc

: A few surprising details arise from the listing. The first is that the window is created and run from a goroutine started in an init function, not from main. That's because on Android, all non-Java code must be loaded from a C library. And in Go, libraries don't run main functions.



* Hello, World

.play helloworld.go /START OMIT/,/END OMIT/ HLdraw

: This the proverbial Hello, world program written with Gio. It's little bigger but still fit on a slide after leaving out the package statement and package imports.

: Compared to the blank window from earlier, hello world loads a font, initializes a few support variables and draws a label.

: There are no shortcuts in the program, perhaps except from the type assertion that in a larger program will be a type switch for the various event types.


* Running Gio programs

* Linux, macOS, Windows

Enable modules

	export GO111MODULE=on

Build, install or run the program

	go build gioui.org/example/hello
	go install scatter.im/cmd/scatter
	go run helloworld.go

: Running Gio programs is as straightforward as any other program, at least on desktop systems. Linux require a C compiler and a few development libraries, while macOS requires Xcode. There are no build dependencies for Windows, but Gio currently need the ANGLE OpenGL ES emulator to run.

: I recommend enabling module mode so Gio is automatically downloaded for you and to shield yourself from the still frequent API changes.



* Android

Install the gogio tool

	go install gioui.org/cmd/gogio
	$GOBIN/gogio -target android -o hello.apk helloworld.go

Install on a connected device or emulator with adb

	adb install hello.apk

: The mobile platforms are of course more troublesome. The Gio project include a tool that can package a Gio program suitable for installation to a mobile device, an emulator or for including in an existing project.

: To build for Android you need the Android SDK and NDK installed. The
gogio tool can produce an apk file that can be installed with the Android adb tool.



* iOS/tvOS

For iOS/tvOS devices:

	$GOBIN/gogio -target <ios|tvos> -o hello.ipa -appid <bundle id> helloworld.go

Use the .app file extension for simulators:

	$GOBIN/gogio -target <ios|tvos> -o hello.app helloworld.go

Install on a running simulator

	xcrun simctl install booted hello.app

: To build for iOS and tvOS you need Xcode and the bundle id from a valid provisioning profile. A good way to acquire that is to set up a sample project in Xcode and run it once on your device.

: The iSimulators don't accept ipa packages, but rather raw .app
directories. The gogio tool use the output extension to distinguish.

: Please note that tvOS support is very early. For example, there is currenctly no API exposed for the remote control. I added support because it was easy and as a proof of concept.



* Browsers

To output a directory ready to serve:

	$GOBIN/gogio -target js -o www helloworld.go

Use a webserver or goexec to serve it:

	go run github.com/shurcooL/goexec 'http.ListenAndServe(":8080", http.FileServer(http.Dir("www")))'

: The gogio tool also build a webassembly version of your program. It includes the required support files and a basic HTML host page so the program is ready to serve.

: You can use any webserver that serves static files; I use Dmitri's goexec tool for demos and tests. Goexec can run a complete webserver in a single line if you have Go modules enabled.

: Note that running in the browser works but is slow. Partly because Gio invokes JavaScript functions in a straightforward but suboptimal way, partly because the Go runtime is not yet a great match for webassembly. I hope that future webassembly improvements will drastically speed up Go in the browser.

: Also note that Gio use a WebGL Canvas element for display and input. You don't get the convenience of HTML DOM elements. At least not yet.

: [go run helloworld.go]
: [gogio -target android -o android.apk helloworld.go]
: [gogio -target ios -o hello.app helloworld.go]
: [xcrun simctl install booted hello.app]
: [adb install hello.apk]
: [go run webassembly.go]



* Operations

* Operations

Serializing operations

	import "gioui.org/op" // Pure Go

	var ops op.Ops
	// Add operations to ops
	op.InvalidateOp{}.Add(ops)
	...

Only the app package depends on platform libraries

	import "gioui.org/io/system"

	var e system.FrameEvent
	e.Frame(&ops)


: In Gio, the basic building block is the operation. There are operations for clipping, transforming and drawing as well as ops for controlling input flow and requesting a redraw for animation.

: This is a list of drawing operations.

: Immediate mode UI libraries redraw everything and Gio is no different; if your program state changes, you simply request a Redraw and redraw everything.

: This is actually more efficient than it might sound, both through the sheer power of modern GPUs but also through designing operations to be efficient.

: Operations are carefully designed such that they generate no garbage when drawing. The underlying Ops buffer is reused and the Add method of every op is written so the op itself never escapes to the heap.

: For example, the label from the helloworld example is drawn garbage free, as long as your string is constant and re-use the font cache.

* Operations

Position other operations

	import "gioui.org/op"
	import "gioui.org/f32"

	op.TransformOp{}.Offset(f32.Point{...}).Add(ops)

Request a redraw

	ui.InvalidateOp{}.Add(ops) // Immediate
	ui.InvalidateOp{At: ...}.Add(ops) // Delayed

* Drawing operations

Set current color or image

	import "gioui.org/op/paint"

	paint.ColorOp{Color: color.RGBA{...}}.Add(ops)
	paint.ImageOp{Src: ..., Rect: ...}.Add(ops)

Draw with the current color or image

	paint.PaintOp{Rect: ...}.Add(ops)


* Clip operations

Clip drawing to a rectangle

	import "gioui.org/op/clip"

	clip.Rect{Rect: image.Rectangle{...}}.Op(ops).Add(ops)

Or to an outline

	var b paint.Path
	b.Begin(ops)
	b.Line(...)
	b.Quad(...) // Quadratic Beziér curve
	b.Cube(...) // Cubic Beziér curve
	b.End().Add(ops)

: The major parts of any UI program are drawing, layout and input handling.

: We'll tackle drawing first.


* Input operations

Keyboard and text input

	import "gioui.org/io/key"

	// Declare key handler.
	key.InputOp{Key: handler, Focus: true/false}.Add(ops)

	// Hide soft keyboard.
	key.HideInputOp{}.Add(ops)

Mouse and touch input

	import "gioui.org/io/pointer"

	// Define hit area.
	pointer.Rect(...).Add(ops)
	pointer.Ellipse().Add(ops)

	// Declare pointer handler.
	pointer.InputOp{Key: c, Grab true/false}



* Drawing


* Drawing (and animating)

Drawing and animating a clipped square

.play animatedclipping.go /START OMIT/,/END OMIT/

: This is another animated example, this time with a clip that changes over time.

: [go run animatedpaths.go]



* Layout

: Layout is the positioning, sizing and layering of widgets.

: Some framework add properties or special widgets for laying out other widgets.

: Layouts in Gio is done through transient helper objects, except for the scrollable list which is user controlled.


* Constraints and dimensions

Constraints are input

	package layout // import gioui.org/layout

	type Constraints struct {
		Width  Constraint
		Height Constraint
	}

	type Constraint struct {
		Min, Max int
	}

Dimensions are output

	type Dimensions struct {
		Size     image.Point
		Baseline int
	}

* Constraints and dimensions

Widgets accept constraints, output dimensions, stored in
layout.Context.

	package text // import gioui.org/text

	func (l Label) Layout(gtx *layout.Context)
	func (e *Editor) Layout(gtx *layout.Context)

	package widget // import gioui.org/widget

	func (im Image) Layout(gtx *layout.Context)


: Gio borrows Constraints and Dimensions from the Flutter framework.

: The idea is that a widgets is given a range of accepted sizes, and return its actual "layout" size.

: Note that some widgets may choose to draw outside its layout rectangle.

: By convention, widgets have a Layout method that take the constraints as input and return the dimensions.

: Label is no different




* Example - two labels

.play twolabels2.go /START DRAW OMIT/,/END DRAW OMIT/ HLdraw

: The constraints given to us in drawLabel are rigid, and forces the dimensions of the labels to take up the whole window.

: So first we loosen the constraints by setting the minimum height to 0.

: Then, we use the dimensions of the first label to offset the second.

: [go run twolabels2.go]



* Layout helpers

Aligning

	var gtx *layout.Context

	layout.Center.Layout(gtx, func() {
		someWidget.Layout(gtx, ...) // Draw widget
	})
	dimensions := gtx.Dimensions

Insetting

	inset := layout.Inset{Top: ui.Dp(8), ...} // 8dp top inset
	inset.Layout(gtx, func() {
		anotherWidget.Layout(gtx, ...) // Draw widget
	})
	dimensions := gtx.Dimensions

: Centering is a special case of aligning widgets. Aligning and insetting widgets are so common that Gio provides two helper
: types in the layout package, Align and Inset.

: Both Align and Inset are designed to be garbage free. Go is clever enough to allocate them on the stack even though their Begin methods mutate them.

: Note that the layout package represent distances with device independent points instead of pixels to ensure your program looks the
: same regardless of your monitor's pixel density.



* Flex layout

Lay out widgets on an axis.

.play flex.go /START OMIT/,/END OMIT/ HLflex

: A more complex layout in Gio is the Flex. It mimics the flex layout from Flutter and is used for laying out widgets along an axis.

: The program shown on this slide demonstrates a weighted layout, where the first rectangle takes up half the available space, while the two last
: share the other half.

: [go run flex.go]



* Stack layout

.play stack.go /START OMIT/,/END OMIT/ HLstack

: Stack is another layout in Gio which as the name implies is used for layering widget on top of each other.

: In this example, stack is used to stack a list of rectangles with increasing insets.

: [go run stack.go]



* List layout

.code list.go /START INIT OMIT/,/END INIT OMIT/ HLlist

.play list.go /START OMIT/,/END OMIT/ HLlist

: The most complex layout in Gio is the scrollable List. It is also the first stateful widget you've seen, in that it can accept user input and scroll its content to match.

: Even though the list is declared to be a million elements long, the list only lays out the currently visible elements.

: List does that by asking your program to lay out a particular index, returned by the Index method.

: [go run list.go]



* Input

: I came up with the name Gio as an acronym for "graphical input/output". So far we've only seen output: drawing and positioning drawings.

: Let's move on to the other side, input.



* Input queue and handler keys

	// Queue maps an event handler key to the events
	// available to the handler.
	type Queue interface {
		Events(k Key) []Event
	}

	// Key is the stable identifier for an event handler.
	// For a handler h, the key is typically &h.
	type Key interface{}

: One of my favorite features of immediate mode UI programs is the absence of callbacks.

: In Gio all events from input sources such your mouse, finger, keyboard are distributed to handlers through the input.Queue.

: Queue is an interface with just a single method, Next, returning all events available in this frame for a given
: handler.

: A Key is the identifier for a handler. Any value will do, as long as it is stable across frames,
: and can be used as a map key.

: Typically the address of the handler is used as its Key.



* Pointer event handling

.play pointer.go /START OMIT/,/END OMIT/ HLevent

: Let's say you have a Button widget. It has only one field, the pressed boolean.

: The first part of the button layout method updates the pressed state. It runs through the available events and set the pressed to true for press events and false for release events.

: Then, it registers the handler by specifying a hit area, in this case a rectangle, and a InputOp for specifying its own key.

: Note how there are no callbacks involved and that there is no way or need to unregister a handler; all handler registrations are cleared

: at the beginning of the next frame.

: Registration and handling of events are separate and can be done in any order, because event handling is for the current set of events, while registration is for delivering events between frames.

: [go run pointer.go]



* Window event queue

The Window's Queue method returns an ui.Queue for OS events.

	package app // import gioui.org/app

	func (w *Window) Queue() *Queue



* Gestures

	import "gioui.org/op"
	import "gioui.org/gesture"

Detect clicks

	var queue op.Queue
	var c gesture.Click
	for _, event := range c.Events(queue) {
		// event is a gesture.ClickEvent, not a raw pointer.Event.
	}

Determine scroll distance from mouse wheel or touch drag/fling

	var cfg ui.Config
	var s gesture.Scroll

	distance := s.Scroll(cfg, queue, gesture.Vertical)

: The gesture package help your program recognize higher level gestures from low level pointer events. Examples shown are click and scroll.

: In a sense, gestures are state machines that are updated by raw pointers events.

: The Click state machine record the current pressed state, which the Scroll state machine track touch velocity, while animating it on touch release.


* Widgets

* Widgets - the Editor

Initialize the editor

	import "gioui.org/text"

.code editor.go /START INIT OMIT/,/END INIT OMIT/

Draw, layout and handle input in one call.

.play editor.go /START OMIT/,/END OMIT/

* Why Gio?



* Why Gio?

Gio is

- Simple. Immediate mode design, no hidden state.
- Portable. The core of Gio is all Go.
- Fast. GPU accelerated, very little per-frame garbage.
- Convenient. Develop on desktop, deploy on mobile.
- Public domain source (UNLICENCE). Dual licenced MIT to please your lawyers.

Most importantly, Gio needs your help to succeed!

: So if you want to write your user interface in Go,

: and care about as simple and straightforward programming interface.

: Or if you like myself, loathe to repeat yourself, Gio runs on all the popular platforms.

: Portability is also very convenient during development. I usually test on my desktop system while only occasionally deploying to an emulator or device for verification or for mobile specific features.

: That's it for a short introduction to the major parts of the Gio library as it looks like today.

: Perhaps you're wondering whether you should spend time on Gio.

: I put a lot of effort into ensuring Gio's design follows that of Go: simple, orthogonal and scalable. Not just for speed but for programmer productivity.

: Simplicity. The immediate mode design makes it much less frustrating to debug and develop UI programs.

: Portability, because it's written in Go and because its only non-Go dependencies are low level system libraries for window management, input handling, and access to the GPU.

: Gio is fast because Go is fast and because it's designed not to generate garbage during a frame. And the heavy lifting is done by your GPU.

: Convenient. Gio enjoys the very low compilation times and because it runs on your desktop you don't need a device or emulator for much of your mobile development.

: And finally, Gio is unlicenced so you can take whatever you like from Gio and use it in your own project if you choose. You don't need to pollute your code with attribution or licence.

: Let's make Go a natural and even an obvious choice for writing GUI programs!


: What about the future of Gio?

: Well, I've already spent over a year on it, so according to the Lindy effect I'm likely to spend at least another year on it.

: I'm going to spend that time bringing Gio closer to a 1.0 version and on Gio programs that interest me, such as Scatter.

: I very much hope you will join me in the future development of Gio. It most likely won't succeed without your help.

: If you have any questions or comments I'm available during the entire Gophercon.

: Thank you for listening.