Thursday, December 1, 2016

Ubuntu Core Gadget Snaps

Gagdet snaps, the somewhat mysterious part of snappy that few people grok. Being a distinct snap type, next to kernel, os and the most common app types, it gets some special roles. If you are on a classic system like Ubuntu, Debian or Fedora you don't really need or have one yet. Looking at all-snap core devices you will always see one. In fact, each snappy reference platform has one. But where are they?

Up until now the gadget snaps were a bit hard to find. They were out there but you had to have a good amount of luck and twist your tongue at the right angle to find them. That's all changed now. If you look a https://github.com/snapcore you will see a nice, familiar pattern of devicename-gadget. Each repository is dedicated to one device so you will see a gadget snap for Raspberry Pi 2 or Pi 3, for example.

But there's more! Each of those github repositories is linked to a launchpad project that automatically mirrors the git repository, builds the snap and uploads it to the store and publishes the snap to the edge channel!

The work isn't over, as you will see the gadget snaps are mostly in binary form, hand-made to work but still a bit too mysterious. The Canonical Foundations team is working on building them in a way that is friendlier to community and easier to trace back to their source code origins.

If you'd like to learn more about this topic then have a look at the snapd wiki page for gadget snaps.

Saturday, September 3, 2016

Gentoo overlay for snapd is now available

Thanks to a fantastic community contribution from  Clayton "kefnab" Dobbs we now have a Gentoo overlay containing snapd [1].

I just wanted to shout a big thank you to mr Clayton. I'm very glad to see gentoo community interested in snapd and taking active participation in the project.

I will work on updating that to the latest version and as well as extending it to include the new snapd-xdg-open application.

[1] https://github.com/zyga/gentoo-snappy

Tuesday, August 9, 2016

Creating your first snappy interface


Today is a day I've been waiting for a long time. We now have enough knowledge to create our first real interface from scratch. To really understand this content you need to be familiar with parts [1], [2], [3] and [4].

We will go all the way, from branching snapd all the way to running a program that uses our new interface. We will focus on the ancillary tasks this time, the actual interface will be rather basic. Still, this knowledge will be invaluable next time where we will try to do something more complicated.

Adding the new "hello" interface


Let's get started. It all begins with snapd. If you didn't already, fork snapd and clone your fork locally. You may find this small guide that I wrote earlier useful. It goes through all those steps in detail. At the end of the exercise you should be able to build your fork of snapd (make sure it is really your fork, not the upstream version!)

Let's look around. Each time a new interface is added, the following files are modified:
  • The file interfaces/builtin/foo{,_test}.go contains the actual interface
  • The file interfaces/builtin/all{,_test}.go contains tiny change that is used to register a new interface
In addition, if the interface slot should implicitly show up on the core snap we need to change the file snap/implicit{,_test}.go. We will get back to this.

So let's create our new interface now. As mentioned in part [3] this entails implementing a new type with a few methods. Let's do that now:

Using your favorite code editor create a new file and save it as interfaces/builtin/hello.go. Paste the following code inside:


// -*- Mode: Go; indent-tabs-mode: t -*-

/*
 * Copyright (C) 2016 Canonical Ltd
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License version 3 as
 * published by the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 *
 */

package builtin

import (
 "fmt"

 "github.com/snapcore/snapd/interfaces"
)

// HelloInterface is the hello interface for a tutorial.
type HelloInterface struct{}

// String returns the same value as Name().
func (iface *HelloInterface) Name() string {
 return "hello"
}

// SanitizeSlot checks and possibly modifies a slot.
func (iface *HelloInterface) SanitizeSlot(slot *interfaces.Slot) error {
 if iface.Name() != slot.Interface {
  panic(fmt.Sprintf("slot is not of interface %q", iface))
 }
 // NOTE: currently we don't check anything on the slot side.
 return nil
}

// SanitizePlug checks and possibly modifies a plug.
func (iface *HelloInterface) SanitizePlug(plug *interfaces.Plug) error {
 if iface.Name() != plug.Interface {
  panic(fmt.Sprintf("plug is not of interface %q", iface))
 }
 // NOTE: currently we don't check anything on the plug side.
 return nil
}

// ConnectedSlotSnippet returns security snippet specific to a given connection between the hello slot and some plug.
func (iface *HelloInterface) ConnectedSlotSnippet(plug *interfaces.Plug, slot *interfaces.Slot, securitySystem interfaces.SecuritySystem) ([]byte, error) {
 switch securitySystem {
 case interfaces.SecurityAppArmor:
  return nil, nil
 case interfaces.SecuritySecComp:
  return nil, nil
 case interfaces.SecurityDBus:
  return nil, nil
 case interfaces.SecurityUDev:
  return nil, nil
 case interfaces.SecurityMount:
  return nil, nil
 default:
  return nil, interfaces.ErrUnknownSecurity
 }
}

// PermanentSlotSnippet returns security snippet permanently granted to hello slots.
func (iface *HelloInterface) PermanentSlotSnippet(slot *interfaces.Slot, securitySystem interfaces.SecuritySystem) ([]byte, error) {
 switch securitySystem {
 case interfaces.SecurityAppArmor:
  return nil, nil
 case interfaces.SecuritySecComp:
  return nil, nil
 case interfaces.SecurityDBus:
  return nil, nil
 case interfaces.SecurityUDev:
  return nil, nil
 case interfaces.SecurityMount:
  return nil, nil
 default:
  return nil, interfaces.ErrUnknownSecurity
 }
}

// ConnectedPlugSnippet returns security snippet specific to a given connection between the hello plug and some slot.
func (iface *HelloInterface) ConnectedPlugSnippet(plug *interfaces.Plug, slot *interfaces.Slot, securitySystem interfaces.SecuritySystem) ([]byte, error) {
 switch securitySystem {
 case interfaces.SecurityAppArmor:
  return nil, nil
 case interfaces.SecuritySecComp:
  return nil, nil
 case interfaces.SecurityDBus:
  return nil, nil
 case interfaces.SecurityUDev:
  return nil, nil
 case interfaces.SecurityMount:
  return nil, nil
 default:
  return nil, interfaces.ErrUnknownSecurity
 }
}

// PermanentPlugSnippet returns the configuration snippet required to use a hello interface.
func (iface *HelloInterface) PermanentPlugSnippet(plug *interfaces.Plug, securitySystem interfaces.SecuritySystem) ([]byte, error) {
 switch securitySystem {
 case interfaces.SecurityAppArmor:
  return nil, nil
 case interfaces.SecuritySecComp:
  return nil, nil
 case interfaces.SecurityDBus:
  return nil, nil
 case interfaces.SecurityUDev:
  return nil, nil
 case interfaces.SecurityMount:
  return nil, nil
 default:
  return nil, interfaces.ErrUnknownSecurity
 }
}

// AutoConnect returns true if plugs and slots should be implicitly
// auto-connected when an unambiguous connection candidate is available.
//
// This interface does not auto-connect.
func (iface *HelloInterface) AutoConnect() bool {
 return false
}

TIP: Any time you are making code changes use go fmt to re-format all of the code in the current working directory to the go formatting standards. Static analysis checkers in the snappy tree enforce this so your code won't be able to land without first being formatted correctly.

This code is perfectly fine, if a little verbose (we'll get it shorter eventually, I promise). Now let's make one more change to ensure the interface known to snapd. All we need to do is to add it to the allInterfaces list in the same golang package. I've decided to just use an init() function so that all of the changes are in one file and cause less conflicts for other developers creating their interfaces. You can see my patch below.

diff --git a/interfaces/builtin/all_test.go b/interfaces/builtin/all_test.go
index 46ca587..86c8fad 100644
--- a/interfaces/builtin/all_test.go
+++ b/interfaces/builtin/all_test.go
@@ -62,4 +62,5 @@ func (s *AllSuite) TestInterfaces(c *C) {
        c.Check(all, DeepContains, builtin.NewCupsControlInterface())
        c.Check(all, DeepContains, builtin.NewOpticalDriveInterface())
        c.Check(all, DeepContains, builtin.NewCameraInterface())
+       c.Check(all, Contains, &builtin.HelloInterface{})
 }
diff --git a/interfaces/builtin/hello.go b/interfaces/builtin/hello.go
index d791fc5..616985e 100644
--- a/interfaces/builtin/hello.go
+++ b/interfaces/builtin/hello.go
@@ -130,3 +130,7 @@ func (iface *HelloInterface) PermanentPlugSnippet(plug *interfaces.Plug, securit
 func (iface *HelloInterface) AutoConnect() bool {
        return false
 }
+
+func init() {
+       allInterfaces = append(allInterfaces, &HelloInterface{})
+}


Now switch to the snap/ directory, edit the file implicit.go and add "hello" to implicitClassicSlots. You can see the whole change below. I also updated the test that checks the number of implicitly added slots. This change will make snapd create a hello slot on the core snap automatically. As you can see by looking at the file, there are many interfaces that take advantage of this little trick.

diff --git a/snap/implicit.go b/snap/implicit.go
index 3df6810..098b312 100644
--- a/snap/implicit.go
+++ b/snap/implicit.go
@@ -60,6 +60,7 @@ var implicitClassicSlots = []string{
        "pulseaudio",
        "unity7",
        "x11",
+       "hello",
 }
 
 // AddImplicitSlots adds implicitly defined slots to a given snap.
diff --git a/snap/implicit_test.go b/snap/implicit_test.go
index e9c4b07..364a6ef 100644
--- a/snap/implicit_test.go
+++ b/snap/implicit_test.go
@@ -56,7 +56,7 @@ func (s *InfoSnapYamlTestSuite) TestAddImplicitSlotsOnClassic(c *C) {
        c.Assert(info.Slots["unity7"].Interface, Equals, "unity7")
        c.Assert(info.Slots["unity7"].Name, Equals, "unity7")
        c.Assert(info.Slots["unity7"].Snap, Equals, info)
-       c.Assert(info.Slots, HasLen, 29)
+       c.Assert(info.Slots, HasLen, 30)
 }
 
 func (s *InfoSnapYamlTestSuite) TestImplicitSlotsAreRealInterfaces(c *C) {


Now let's see what we have. You should have three patches:
  1. The first one adding the dummy hello interface
  2. The second one registering it with allInterfaces
  3. The third one adding it to implicit slots on the core snap, on classic

This is how it looked like for me. You can also see the particular commit message style I was using. All the commits are prefixed with the directory where the changes were made, followed by a colon and by the summary of the change. Normally I would also add longer descriptions but here this is not required as the changes are trivial.




Seeing our interface for the first time


Let's run our changed code with
devtools. Switch to the devtools directory and run:
./refresh-bits snapd setup run-snapd restore

This roughly means:
  1. Build snapd from source (based on correctly set $GOPATH)
  2. Prepare for running locally built snapd
  3. Run locally built snapd
  4. Restore regular version of snapd

The script will prompt you for your password. This is required to perform the system-wide operations. It will also block and wait until you interrupt it by pressing control-c. Please don't do this yet though, we want to check if our changes really made it through.




To check this we can simply list the available interfaces with

sudo snap interfaces | grep hello

Why sudo? Because of the particular peculiarity of how refresh-bits works. Normally it is not required. Did it work? It did for me:



TIP: If it didn't work for you and you didn't get the hello interface then the most likely cause of the issue is that you were editing your own fork but refresh-bits still built the vanilla upstream version that is checked out somewhere else.

Go to $GOPATH/src/github.com/snapcore/snapd and ensure that this is indeed the fork you were expecting. If not just remove this directory and move your fork (that you may have cloned elsewhere) here and try again.


Great. Now we are in business. Let's recap what we did so far:
  • We added a whole new interface by dropping boilerplate code into interfaces/builtin/hello.go
  • We registered the interface in the list of allInterfaces
  • We made snapd inject an implicit (internally defined) slot on the core snap when running on classic
  • We used refresh-bits to run our locally built version and confirmed it really works

Granting permissions through interfaces


With the scaffolding in place we can now work on using our new interface to actually grant additional permissions. When designing real interfaces you have to think about what kind of action should grant extra permissions. Recall from part [3] that you can freely associate extra security snippets that encapsulate permissions either permanent or connection-based snippets on both plugs and slots.

A permanent snippet is always there as long as a plug or a slot exists. Typically this is used on snaps other than the core snap (e.g. a service provider that is packaged as a snap) to allow the service to run. Examples of such services could include network-manager, modem-manager, docker, lxd, x11 any anything like that. The granted permissions allow the service to operate as well as to create a communication channel (typically a socket or a DBus bus name) that clients can connect to.

Connection based snippet is only applied when a connection (interface connection) is made. This permission is given to the client and typically involves basic IPC system calls as well as a permission to open a given socket (e.g. the X11 socket) or use specific DBus methods and objects.

A special class of connection snippets exist for things that don't involve a client application talking to a server application but rather a program using given kernel services. The prime example of this use case is the network interface which grants access to several network system calls. This interface grants nothing to the slot, just to connected plugs.

Today we will explore that last case. Our hello interface will give us access to a system call that is usually forbidden, reboot

The graceful-reboot snap


For the purpose of this exercise I wrote a tiny C program that uses the reboot() function. The whole program is reproduced below, along with appropriate build support for snapcraft.

graceful-reboot.c


#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/reboot.h>
#include <linux reboot.h>
#include <errno.h>


int main() {
    sync();
    if (reboot(LINUX_REBOOT_CMD_RESTART) != 0) {
        switch (errno) {
            case EPERM:
                printf("Insufficient permissions to reboot the system\n");
                break;
            default:
                perror("reboot()");
                break;
        }
        return EXIT_FAILURE;
    }
    printf("Reboot requested\n");
    return EXIT_SUCCESS;
}

Makefile


TIP: Makefiles rely on differences between tabs and spaces. When copy pasting this sample you need to ensure that tabs are preserved in the clean and install rules

CFLAGS += -Wall
.PHONY: all
all: graceful-reboot
.PHONY: clean
clean:
	rm -f graceful-reboot
graceful-reboot: graceful-reboot.c
.PHONY: install
install: graceful-reboot
	install -d $(DESTDIR)/usr/bin
	install -m 0755 graceful-reboot $(DESTDIR)/usr/bin/graceful-reboot

snapcraft.yaml


name: graceful-reboot
version: 1
summary: Reboots the system gracefully
description: |
    This snap contains a graceful-reboot application that requests the system
    to reboot by talking to the init daemon. The application uses a custom
    "hello" interface that is developed as a part of a tutorial.
confinement: strict
apps:
    graceful-reboot:
        command: graceful-reboot
        plugs: [hello]
parts:
    main:
        plugin: make
        source: .

We now have everything required. Now let's use snapcraft to build this code and get our snap ready. I'd like to point out that we use confinement: strict. This tells snapcraft and snapd that we really really don't want to run this snap in devmode where all sandboxing is off. Doing so would defeat the purpose of adding the new interface.

Let's install the snap and try it out:

$ snapcraft

$ sudo snap install ./graceful-reboot_1_amd64.snap

Ready? Let's run the command now (don't worry, it won't reboot)

$ graceful-reboot
Bad system call

Aha! That's what we are here to fix. As we know from part [4] that seccomp, which is a part of the sandbox system, blocks all system calls that are not explicitly allowed. We can check the system log to see what the actual error message was to figure out which system call need to be added.

Looking through journalctl -n 100 (last 100 messages) I found this:

sie 10 09:47:02 x200t audit[13864]: SECCOMP auid=1000 uid=1000 gid=1000 ses=2 pid=13864 comm="graceful-reboot" exe="/snap/graceful-reboot/x1/usr/bin/graceful-reboot" sig=31 arch=c000003e syscall=169 compat=0 ip=0x7f7ef30dcfd6 code=0x0

Decoding this is rather cryptic message is actually pretty easy. The key fact we are after is the system call number. Here it is 169. Which symbolic system call is that? We can use the scmp_sys_resolver program to find out.

$ scmp_sys_resolver 169
reboot

Bingo. While this case was pretty obvious, library functions don't always map to system call names directly. There are often cases where system calls evolve to gain additional arguments, flags and what not and the name of the library function is not changed.

Adjusting the hello^Hreboot interface


At this stage we can iterate on our interface. We have a test case (the snap we just wrote), we have the means to change the code (editing the hello.go file) and to make it live in the system (the refresh-bits command).

Let's patch our interface to be a little bit more meaningful now. We will rename it from hello to reboot. This will affect the file name, the type name and the string returned from Name(). We will also grant a connected plug permission, through the seccomp security backend, to the use the reboot system call. You can try to make the necessary changes yourself but I provide the essential part of the  patch below. The key thing to keep in mind is that ConnectedPlugSnippet method must return, when asked about SecuritySecComp, the name of the system call to allow, like this []byte(`reboot`).

diff --git a/interfaces/builtin/reboot.go b/interfaces/builtin/reboot.go
index 91962e1..ba7a9e3 100644
--- a/interfaces/builtin/reboot.go
+++ b/interfaces/builtin/reboot.go
@@ -91,7 +91,7 @@ func (iface *RebootInterface) PermanentSlotSnippet(slot *interfaces.Slot, securi
 func (iface *RebootInterface) ConnectedPlugSnippet(plug *interfaces.Plug, slot *interfaces.Slot, securitySystem interfaces.SecuritySystem) ([]byte, error) {
        switch securitySystem {
        case interfaces.SecurityAppArmor:
                return nil, nil
        case interfaces.SecuritySecComp:
-               return nil, nil
+ return []byte(`reboot`), nil
case interfaces.SecurityDBus:

We can now return to the terminal that had refresh-bits running, interrupt the running command with control-c and simply re-start the whole command again. This will build the updated copy of snapd.

Surely enough, if we now ask snap about known interfaces we will see the reboot interface.

Let's now update the graceful-reboot snap to talk about the reboot interface and re-install it (note that you don't have to change the version string at all, just re-install the rebuilt snap). If we ask snapd about interfaces now we will see that the core snap exposes the reboot slot and the graceful-reboot snap has a reboot plug but they are disconnected.



Let's connect them now.

sudo snap connect graceful-reboot:reboot ubuntu-core:reboot

Let's ensure that it worked by running the interfaces command again:

$ sudo snap interfaces | grep reboot
:reboot              graceful-reboot

Success!

Now before we give it a try, let's have a look at the generated seccomp profile. The profile is derived from the base seccomp template that is a part of snapd source code and the set of plugs, slots and connections made to or from this snap. Each application of each snap gets a profile, we can see that for ourselves by looking at
/var/lib/snapd/seccomp/profiles/snap.graceful-reboot.graceful-reboot

$ grep reboot /var/lib/snapd/seccomp/profiles/snap.graceful-reboot.graceful-reboot
reboot

There are a few gotchas that we should point out though:

  • Security profiles are derived from interfaces but are only changed when a new connection is made (and that connection affects a particular snap), when the snap is initially installed or every time it is updated.
  • In practice we will either disconnect / reconnect the hello interface or reinstall the snap (whichever is more convenient)
  • Snapd remembers connections that were made explicitly and will re-establish them across snap updates. If you rename an interface while working on it, snapd may print a message (to system log, not to the console) about being unable to reconnect the "hello" interface because that interface no longer exists in snapd. To make snapd forget all those connections simply remove and reinstall the affected snap.
  • You can experiment by editing seccomp profiles directly. Just edit the file mentioned above and add additional system calls. Once you are happy with the result you can adjust snapd source code to match.
  • You can also do that with apparmor profiles but you have to re-load the profile into the kernel each time using the command apparmor_parser -r /path/to/the/profile

Okay, so did it work? Let's try the graceful-reboot command again.

$ graceful-reboot
Insufficient permissions to reboot the system

The process didn't get killed straight away but the reboot call didn't work yet either. Let's look at the system log and see if there are any hints.

sie 10 10:25:55 x200t kernel: audit: type=1400 audit(1470817555.936:47): apparmor="DENIED" operation="capable" profile="snap.graceful-reboot.graceful-reboot" pid=14867 comm="graceful-reboot" capability=22  capname="sys_boot"

This time we could use the system call but apparmor stepped in and said that we need the sys_boot capability. Let's modify the interface to allow that then. The precise apparmor syntax for this is "capability sys_boot,". You absolutely have to include the trailing comma. It was my most common mistake when hacking on apparmor profiles.

Let's patch the interface and re-run refresh-bits and re-install the snap. You can see the patch I applied below

diff --git a/interfaces/builtin/reboot.go b/interfaces/builtin/reboot.go
index 91962e1..ba7a9e3 100644
--- a/interfaces/builtin/reboot.go
+++ b/interfaces/builtin/reboot.go
@@ -91,7 +91,7 @@ func (iface *RebootInterface) PermanentSlotSnippet(slot *interfaces.Slot, securi
 func (iface *RebootInterface) ConnectedPlugSnippet(plug *interfaces.Plug, slot *interfaces.Slot, securitySystem interfaces.SecuritySystem) ([]byte, error) {
        switch securitySystem {
        case interfaces.SecurityAppArmor:
-               return nil, nil
+               return []byte(`capability sys_boot,`), nil
        case interfaces.SecuritySecComp:
                return []byte(`reboot`), nil
        case interfaces.SecurityDBus:

TIP: I'm using sudo with a full path because of a bug in sudo where /snap/bin is not kept on the path.

Warning: the next command will reboot your system straight away!

$ sudo /snap/bin/graceful-reboot

Final touches


Since interfaces like this are very common, snapd has some support for writing less repetitive code. The exactly identical interface can be written with just this short snippet

// NewRebootInterface returns a new "reboot" interface.
func NewRebootInterface() interfaces.Interface {
	return &commonInterface{
		name: "reboot",
		connectedPlugAppArmor: `capability sys_boot,`,
		connectedPlugSecComp:  `reboot`,
		reservedForOS:         true,
	}
}

Everywhere where we used to say &RebootInterface{} we can now say NewRebootInterface(). The abbreviation helps in code reviews and in just conveying the meaning and intent of the interface.

You can find the source code of this interface in my github repository. The code of the graceful reboot snap is here. Feel free to comment below, or on Google+ or ask me questions directly.

Monday, August 8, 2016

Snap execution environment


This is the fourth article in the series about snappy interfaces. You can check out articles one, two and three though they are not directly required.

In this installment we will explore the layout and properties of the file system at the time snap application is executed. From the point of view of the user nothing special is happening. The user either runs an application by clicking on a desktop icon or by running a shell command. Internally, snapd uses a series of steps (which I will not explain today as they are largely an implementation detail) to configure the application process.

The (ch)root filesystem and a bit of magic


Let’s start with the most important fact: the root filesystem is not the filesystem of the host distribution. Using the host filesystem would lead to lots of inconsistencies. Those are rather obvious: different base libraries, potentially different filesystem layout, etc. At snap application runtime the root filesystem is changed to the core snap itself. You can think of this as a kind of chroot. Obviously the chroot itself would be insufficient as snaps are read only filesystems and the core snap is no different.

Certain directories in the core snap are bind mounted (you can think of this as a special type of a symbolic link or a hard link to a directory though neither are fully accurate) to locations on the host file system. This includes the /home directory, the /run directory and a few others (see Appendix A for the full list). Notably this does not include the /usr/lib or /usr/bin. If a snap needs a library or an executable to function, that library or executable has to be present in the snap itself. The only exception to that are very low level libraries like libc that are present in the core snap.

TIP: explore the core snap to see what is there. Having installed at least one snap you can go to /snap/ubuntu-core/current to see the list of files provided there.

With all those mounts and chroots in place one might wonder how mounts look like for all the other processes in the system? The answer is simple, they look as if nothing special related to snappy was happening.

To understand the answer you have to know a little about Linux namespaces. Namespaces are a way to create a separate view of a given aspect of a Linux system at a per-process level. Unlike full-blown virtual machines (where you run a whole emulated computer with a potentially different operating system kernel and emulated peripheral devices) namespaces are fine grained. Snappy uses just one of the available namespaces, the mount namespace. Now I won’t fool you, while the idea seems simple “mounts in the namespace are isolated from the mounts outside of the namespace” the reality is far more complex because of the so-called shared-subtrees. One interesting consequence is that mounts performed after a snap application is stated (e.g. in the /media directory) are visible to the said application (e.g. to VLC) while the reverse is not true. If a malicious snap tries (and manages despite various defenses put in place) to mount something in say, /usr/ that change will be visible only to the snap application process.

Don’t worry if you don’t fully understand this topic. The main point is that your application sees a different view of the filesystem but that view is consistent across distributions.

TIP: you may have seen the core snap as it looks like on disk if you followed the earlier tip. Now see the real file system at runtime! Install the snapd-hacker-toolbelt snap and run snapd-hacker-toolbelt.busybox sh. This will give you a shell with many of the familiar commands that let you peek and poke at the environment.
Now there are a few more tweaks I should point out but won’t go into too much detail:


  • Each process gets a private /tmp directory with a fresh tmpfs mounted there. This is a security measure. One simple consequence is that you cannot expect to share files by dropping them there and that you cannot create arbitrarily large files there since tmpfs is backed by a fraction of available system memory.
  • There’s also a private instance of /dev/pts directory with pseudo terminal emulators. This is an another security measure. In practice you will not care about this much. It’s just a part of the Linux plumbing that has to be setup in a given way.
  • The whole host filesystem is mounted at /var/lib/snapd/hostfs. This can be used by interfaces similar to the content sharing interface, for example. This is super interesting and we will devote a whole article to using this later on.
  • There’s special code that exists to support Nvidia proprietary drivers. I will discuss this with a separate installment that may be of interest to game developers.
  • The current working directory may be impossible to preserve across the whole chroot and mount and bind mount magic. The easiest way to experience this is to create a directory in /tmp (e.g. /tmp/foo) and try to run any snap command there. Because of the private (and empty) /tmp directory the /tmp/foo directory does not exist for the snap application process. Snap-confine will print an informative error message and refuse to run.

This now much more comfortable. Many of the usual places exist and contain the data the applications are familiar with. This doesn’t mean those directories are readable or writable to the application process, they are just present. Confinement and interfaces decide if something is readable or writable. This brings us to the second big part of snap-confine

Process confinement

Snap-confine (as of version 1.0.39) supports two sandboxing technologies: seccomp and apparmor.

Seccomp is used to constrain system calls that an application can use. System calls are the interface between the kernel and user space. If you are unfamiliar with the concept then don’t worry. In very rough terms some of the functions of your programming language are implemented as as system calls and seccomp is the linux subsystem that is responsible for mediating access to them.

Right now when your application runs in devmode you will not get any advice on the system calls you are relying on that are not allowed by the set of used interfaces. The so-called complain mode of seccomp is being actively developed upstream so the situation may change by the time you are reading this.

In strict confinement any attempt to use a disallowed system call will instantly kill the offending process. This is accompanied by a rather cryptic message that you can see in the system log:
sie 08 12:36:53 gateway kernel: audit: type=1326 audit(1470652613.076:27): auid=1000 uid=1000 gid=1000 ses=63 pid=66834 comm="links" exe="/snap/links/2/usr/bin/links" sig=31 arch=c000003e syscall=54 compat=0 ip=0x7f8dcb8ffc8a code=0x0
What this tells us is that process ID 66834 was killed with signal 31 (SIGSYS) because it tried to use system call 54 (setsockopt) Note that system call numbers are architecture specific. The output above was from an amd64 machine.

Even when a system call is allowed the particular operation may be intercepted and denied by apparmor. For example, the sandbox is setup so that applications can freely write to $SNAP_USER_DATA (or $SNAP_DATA for services) but cannot, by default, either read or write from the real home directory.
sie 08 12:56:40 gateway kernel: audit: type=1400 audit(1470653800.724:28): apparmor="DENIED" operation="open" profile="snap.snapd-hacker-toolbelt.busybox" name="/home/zyga/.ssh/authorized_keys" pid=67013 comm="busybox" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000

Here we see that process ID 67013 tried to “open” /home/zyga/.ssh/authorized_keys and that the “r” (read) mask was denied. In devmode that is obviously allowed but is accompanied with an appropriate warning message instead.

TIP: Whenever you run into problems like this you should give the snappy-debug snap a try. It is designed to read and understand messages like that and give you useful advice.

Apparmor has much wider feature set and can perform checks for linux capabilities, traditional UNIX IPC like signals and sockets, DBus messages (including details of the object and method invoked). The vast majority of the current snap confinement is made with apparmor profiles. We will look at all the features in greater detail in the next few articles in this series where we will actively implement simple new interfaces from scratch.

There's one last thing that snap-confine does, in some cases is creates...

A device control group

In certain cases a device control group is created and the application process is moved there. The control group has only a small set of typical device nodes (e.g. /dev/null) and explicitly defined additional devices. This is done by using udev rules to tag appropriate devices. This is listed for completeness, you should never worry about or even think about this topic. Once the need arises we will explore it and how it can be used to simplify handling of certain situations. Again, by default there is no device control group being used.

Putting it all together

With all of those changes in place snap-confine executes (using execv) a wrapper script corresponding to the application command entry in the snapcraft.yaml file. This happens each time you run an application.

If you are interested in learning more about snap-confine I encourage you to check out its manual page (snap-confine.5) and source code. If you have any questions please feel free to ask at the snapcraft.io mailing list or using comments on this blog below.

Next time we will look at creating our first, extremely simple, interface in practice.

Appendix A: List of host directories that are bind-mounted.

NOTE: This list will slowly get less and less accurate as more of the mount points become dynamic and controlled by available interfaces.
  • /dev
  • /etc (except for /etc/alternatives)
  • /home
  • /root
  • /proc
  • /snap
  • /sys
  • /var/snap
  • /var/lib/snapd
  • /var/tmp
  • /var/log
  • /run
  • /media
  • /lib/modules
  • /usr/src

Tuesday, July 5, 2016

Fresh bite-sized bugs in snappy

Do you remember my previous post about bite-sized bugs in snappy? Those are bugs that are small and trivial to fix. A perfect way to get involved in a project or to get your feet wet in a new programming language.

I've just noticed that there are plenty of new bugs that have been tagged  "bitesized" in Snapcraft. As before I'd like to extend my invitation to the the community to have a look and send patches, pull requests or just a ping if they are interested in fixing them.

It usually takes just a few moments to understand the bug and a few moments to come up with a fix locally. While you do that you'll learn a lot about how the project works and, perhaps, you will make your first public contribution to a free software project (I remember how the journey started for me years ago :-)

Please give it a try, you can always ping us in #snappy on Freenode with any questions.

Monday, July 4, 2016

snapd 2.0.10 released to Fedora COPR

Hey there snappers

Fedora users can now get snapd 2.0.10 from the COPR repository. There are many bug fixes and new features in this release.

  • interfaces: also allow @{PROC}/@{pid}/mountinfo and
  • @{PROC}/@{pid}/mountstats
  • interfaces: allow read access to /etc/machine-id and
  • @{PROC}/@{pid}/smaps
  • interfaces: miscelleneous policy updates for default, log-observe and system-observe
  • snapstate: add logging after a successful doLinkSnap
  • tests, integration-tests: port try tests to spread
  • store, cmd/snapd: send a basic user-agent to the store
  • store: add buy method
  • client: retry on failed GETs
  • tests: actual refresh test
  • docs: REST API update
  • interfaces: add mount support for hooks.
  • interfaces: add udev support for hooks.
  • interfaces: add dbus support for hooks.
  • tests, integration-tests: port refresh test to spread
  • tests, integration-tests: port change errors test to spread
  • overlord/ifacestate: don't retry snap security setup
  • integration-tests: remove unused file
  • tests: manage the socket unit when reseting state
  • overlord: improve organization of state patches
  • tests: wait for snapd listening after reset
  • interfaces/builtin: allow other sr*/scd* optical devices
  • systemd: add support for squashfuse
  • snap: make snaps vanishing less fatal for the system
  • snap-exec: os.Exec() needs argv0 in the args[] slice too
  • many: add new `create-user` command
  • interfaces: auto-connect content interfaces with the same content and developer
  • snapstate: add Current revision to SnapState
  • readme: tweak readme blurb
  • integration-tests: wait for listening port instead of active
  • service reported by systemd
  • many: rename Current -> {CurrentSideInfo,CurrentInfo}
  • spread: fix home interface test after suite move
  • many: name unversioned data.
  • interfaces: add "content" interface
  • overlord/snapstate: defaultBackend can go away now
  • debian: comment to remember why the timer is setup like it is
  • tests,spread.yaml: introduce an upgrade test, support/split into two suites for this
  • overlord,overlord/snapstate: ensure we keep snap type in snapstate of each snap
  • many: rework the firstboot support
  • integration-tests: fix test failure
  • spread: keep core on suite restore
  • tests: temporary fix for state reset
  • overlord: add infrastructure for simple state format/content migrations
  • interfaces: add seccomp support for hooks.
  • interfaces: allow gvfs shares in home and temporarily allow
  • socketcall by default (LP: #1592901, LP: #1594675)
  • tests, integration-tests: port network-bind interface tests to spread
  • snap,snap/snaptest: use PopulateDir/MakeTestSnapWithFiles directly and remove MockSnapWithHooks
  • interfaces: add mpris interface
  • tests: enable `snap run` on i386
  • tests, integration-tests: port network interface test to spread
  • tests, integration-tests: port interfaces cli to spread
  • tests, integration-tests: port leftover install tests to spread
  • interfaces: add apparmor support for hooks.
  • tests, integration-tests: port log-observe interface tests to spread
  • asserts: improve Decode doc comment about assertion format
  • tests: moved snaps to lib
  • many: add the camera interface
  • many: add optical-drive interface
  • interfaces: auto-connect home if running on classic
  • spread: bump gccgo test timeout
  • interfaces: use security tags to index security snippets.
  • daemon, overlord/snapstate, store: send confinement header to the store for install
  • spread: run tests on 16.04 i386 concurrently
  • tests,integration-tests: port install error tests to spread
  • interfaces: add a serial-port interface
  • tests, integration-tests, debian: port sideload install tests to spread
  • interfaces: add new bind security backend and refactor backendtests
  • snap: load and validate implicit hooks.
  • tests: add a build/run test for gccgo in spread
  • cmd/snap/cmd_login: Adjust message after adding support for wheel group
  • tests, integration-tests: ported install from store tests ton spread
  • snap: make `snap change <taskid>` show task progress
  • tests, integration-tests: port search tests to spread
  • overlord/state,daemon: make abort proceed immediately, fix doc comment, improve tests
  • daemon: extend privileged access to users in "wheel" group
  • snap: tweak `snap refresh` and `snap refresh --list` outputTiny
  • interfaces: refactor auto-connection candidate check
  • snap: add support for snap {install,refresh} --{edge,beta,candidate,stable}
  • release: don't force KDE Neon into devmode.

Snappy in Arch moved to community repo

Hey there snappers!

I’d like to announce something that you may have noticed during the last update of snapd to version 2.0.10. The AUR package is no longer there, instead you can now get and update snappy on Arch simply by running this one-liner:

„pacman -S snapd"

That’s right, snapd and snap-confine have now moved to the official community repository. This means that the barrier to entry is now significantly lower and that installation is even faster than before. You still want to read the snapd wiki page to know the details about various post-install activities.