Difference between revisions of "Docker notes"

From Helpful
Jump to: navigation, search
m (habits and tips)
m (How does storage work?)
(15 intermediate revisions by the same user not shown)
Line 7: Line 7:
  
 
They are more lightweight than running a classical VM
 
They are more lightweight than running a classical VM
: in part because they virtualize the OS, not the hardware
+
: in part because they virtualize the OS (interface), not the hardware
 
: in part because of practical details, e.g. how the images are created/used
 
: in part because of practical details, e.g. how the images are created/used
 
: in part because persistent storage is intentionally separated
 
: in part because persistent storage is intentionally separated
  
  
Actually, the isolation is mostly recent kernel features, Docker is just one of various toolkits on top that make it practical,
+
Actually, the isolation is mostly recent kernel features; Docker is one specific product/toolkit on top that make it practical to actually use.
and with its own take on it. For some more background and comparison, see [[Virtualization,_emulation,_simulation#Linux_containers]].
+
  
There are comparable things, e.g. [[SmartOS]] had been doing this before Linux, though more for reasons of operational efficiency and security,
+
Docker has its own take on it.
whereas docker started more with the microservice angle.
+
For some more background and comparison, see [[Virtualization,_emulation,_simulation#Linux_containers]].
 +
 
 +
There are comparable things, e.g. [[SmartOS]] had been doing this before Linux, though more for reasons of operational efficiency and security, whereas docker focuses more on the microservice angle.
  
 
===What's it useful for?===
 
===What's it useful for?===
Line 55: Line 56:
  
  
'''Useful layer of abstraction'''
+
'''Useful layer of abstraction for software packages'''
  
 
The tech for OS containers had existed in a usable state for over decade.  
 
The tech for OS containers had existed in a usable state for over decade.  
Line 267: Line 268:
 
* it found an image called ubuntu (downloaded it from docker hub if not present yet)
 
* it found an image called ubuntu (downloaded it from docker hub if not present yet)
 
* instantiated it, with bash as entry point / main process
 
* instantiated it, with bash as entry point / main process
* you ran ps within it. (Yes, the only processes inside right then are that shell and that command)
+
* you manually ran ps within it. (Yes, the only processes inside right then are that shell and that command)
  
  
Line 288: Line 289:
  
 
* in many cases, you can also use the name, but note that these need to be unique
 
* in many cases, you can also use the name, but note that these need to be unique
===Status===
 
  
To see what's running (or, continuing the exampe above, ''that'' it's running), run {{inlinecode|docker ps}} on the host, which shows something like:
+
===Status and cleanup, of images and instances===
CONTAINER ID    IMAGE            COMMAND      CREATED          STATUS          PORTS      NAMES
+
3b66e09b0fa2    ubuntu:latest    "bash"        9 seconds ago    Up 8 seconds                drunk_mestorf
+
  
  
 +
'''Status'''
  
* {{inlinecode|docker ps}} to list running containers
+
See '''existing images''' on the host:
 +
* {{inlinecode|docker images}} lists present images
 +
* {{inlinecode|docker images -a}} includes non-named ones,
  
* {{inlinecode|docker ps -a}} to list running '''and old''' containers
+
  REPOSITORY            TAG                IMAGE ID            CREATED            SIZE
 +
<none>                <none>              eda5290e904b        2 months ago        70.5MB
 +
ubuntu                bionic-20190515    7698f282e524        3 months ago        69.9MB
  
  
* {{inlinecode|docker images}} lists present images
 
  
* {{inlinecode|docker images -a}} includes non-named ones,
+
See '''running container instances''' on the host:
 +
* {{inlinecode|docker ps}}  to list running containers
 +
* {{inlinecode|docker ps -a}} to list running '''and old''' containers
  
===Cleanup===
+
CONTAINER ID    IMAGE            COMMAND      CREATED          STATUS          PORTS      NAMES
 +
3b66e09b0fa2    ubuntu:latest    "bash"        9 seconds ago    Up 8 seconds                drunk_mestorf
  
In default config, '''container state''' sticks around after the container stops.
 
This can be very useful for debug, but you do eventually want to clean them.
 
  
Roughly:
 
* {{inlinecode|docker rm ''ids''}} to remove their state (e.g. based on {{inlinecode|docker ps -a}})
 
 
 
If you don't care about post-mortem debugging, you can start containers with {{inlinecode|--rm}} to always have it immediately  clean up after itself.
 
  
  
 +
'''Cleanup'''
  
 
If you're done with ''images'':
 
If you're done with ''images'':
Line 324: Line 323:
  
 
* {{inlinecode|docker image prune}} should clean up dangling images (or, with -a, ''unused'')
 
* {{inlinecode|docker image prune}} should clean up dangling images (or, with -a, ''unused'')
: dangling images are mostly build layers that nothing refers to (no [[#repository and tag|repository and tag]])
+
: dangling images are mostly build layers are no longer referred to (no [[#repository and tag|repository and tag]])
  
===Storage===
 
  
You may much prefer to keep all interesting data in a database (some NoSQL may be much nicer for general-purpose storage than RDBMSes).
 
  
 +
In default config, container instance state sticks around after the container stops.
  
If you want things you create in a container to persist, look at data volumes. [[#How_does_storage_work.3F]]
+
This can be very useful for debug during development, but you do eventually want to clean them:
 +
* {{inlinecode|docker rm ''contid''}}
  
Keep in mind that these are not necessarily the answer to anything you want to deploy, scale, manage, or be portable.
+
 
 +
If you don't care about post-mortem debugging, you can start containers with {{inlinecode|--rm}} to always have it immediately  clean up after itself.
 +
 
 +
 
 +
<!--
 +
 
 +
Stop all containers
 +
docker stop $(docker ps -a -q)
 +
Cleanup all containers:
 +
docker rm $(docker ps -a -q)
 +
 
 +
 
 +
Remove all images (WARNING: be sure that's what you wanted):
 +
docker rmi $(docker images -a -q)
 +
 
 +
-->
  
 
==On image building==
 
==On image building==
  
===The two basic ways===
 
  
 
There are two basic ways to build an image:
 
There are two basic ways to build an image:
 
* manually: start with something close to what you want, make the changes you want
 
* manually: start with something close to what you want, make the changes you want
: saving this container saves all filesystem changes within it
+
: saving this container = saving all filesystem changes within it
 
: good for a one-time quick fix, less ideal for some habits you'll need at larger scale
 
: good for a one-time quick fix, less ideal for some habits you'll need at larger scale
  
Line 349: Line 362:
 
: https://docs.docker.com/engine/reference/builder/
 
: https://docs.docker.com/engine/reference/builder/
  
<!--
+
 
'''Manual example'''
+
 
 +
===The hard way: manually===
  
 
Say we want to build on the ubuntu image to make a web server image
 
Say we want to build on the ubuntu image to make a web server image
Line 368: Line 382:
  
 
Notes:
 
Notes:
* commit is to your own repository. You won't accidentally write things on docker hub (or other linked site) until you are registered, and linked, and do a <tt>docker ''push''</tt>
+
* commits are to your local repository. You won't accidentally write things on docker hub (or other linked site) until you are registered, you are linked, and do a <tt>docker ''push''</tt>
 
: actually a commit is more like a snapshot / layer
 
: actually a commit is more like a snapshot / layer
 
: if you want things reusable and configurable, there are various good practices about how configuration and environment is best handled. You'll read up when you need it.
 
: if you want things reusable and configurable, there are various good practices about how configuration and environment is best handled. You'll read up when you need it.
Line 384: Line 398:
  
 
Eventually, we'll want to docker run the container, and we want it to  
 
Eventually, we'll want to docker run the container, and we want it to  
 +
-->
  
 +
 +
===The usual way: Dockerfiles===
 +
<!--
 +
 +
You build an image like:
 +
docker build .
 +
This picks up a Dockerfile and a context, where that context contains the files you may be including.
  
  
Line 392: Line 414:
 
FROM phusion/baseimage
 
FROM phusion/baseimage
 
# Which is ubuntu-based, see https://github.com/phusion/baseimage-docker
 
# Which is ubuntu-based, see https://github.com/phusion/baseimage-docker
 +
 
RUN apt-get update                                                                                                                   
 
RUN apt-get update                                                                                                                   
 +
 
RUN apt-get install -y apache2 libapache2-mod-php
 
RUN apt-get install -y apache2 libapache2-mod-php
  
Line 400: Line 424:
  
  
# The point this fragment is from is to use sofware (isolating stupid dependencies)
+
# The point this Dockerfile is software intended to be interactive via SSH.  
#  so be interactive via SSH. This stuff is specific to phusion/baseimage
+
# Enable SSH  (specific to phusion/baseimage)
# Enable SSH
+
 
RUN rm -f /etc/service/sshd/down
 
RUN rm -f /etc/service/sshd/down
 
# generate SSH keys (so it'll changes every build. leaving this out would do it each boot
 
# generate SSH keys (so it'll changes every build. leaving this out would do it each boot
Line 414: Line 437:
 
Given this is in a subdirectory called foo, with any absolute-path-referenced files:
 
Given this is in a subdirectory called foo, with any absolute-path-referenced files:
 
  docker build foo
 
  docker build foo
 +
 +
 +
 +
 +
 +
-->
 +
 +
====multi-stage builds====
 +
<!--
 +
 +
Multi-stage builds basically mean a Dockerfile that has multiple sections.
 +
 +
Roughly, the idea is that each starts with FROM,
 +
the last sections works as usual,
 +
and all sections before them build software with you copying out just what you need.
 +
 +
 +
 +
The main upside to this is probably that it lets you do more complex builds,
 +
avoiding bloat while including things from already-built/already-published images.
 +
 +
 +
It's an alternative to another approach people have asked for,
 +
namely to allow dockerfiles to include other dockerfiles.
 +
 +
This is conceptually simpler in terms of building related images,
 +
though would make it a more complex build system
 +
 +
 +
 +
It also ''halfway'' solves cases where you'ld like to compose a larger image from
 +
parts that you might also build in steps.
 +
 +
 +
 +
 +
https://docs.docker.com/develop/develop-images/multistage-build/
 +
  
 
-->
 
-->
  
=== habits and tips ===
+
====habits and tips====
 
<!--
 
<!--
 
Security:
 
Security:
Line 445: Line 506:
 
: can also be a reason to avoid use of COPY/ADD
 
: can also be a reason to avoid use of COPY/ADD
 
: ...and arguably to do installs in simple scripts
 
: ...and arguably to do installs in simple scripts
 +
: that means the example somewhere above, with {{inlinecode|RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*}} at the end, is only masking, and it would be better to stick {{inlinecode|&& rm -rf /var/lib/apt/lists/*}} at the end of each apt command. {{comment|(Note: this cleans up package metadata. This makes builds slower, yes, but images slimmer as you will/should never use that information after image build)}}
 +
* something similar goes for package downloads, but this is apparently hacked in [https://github.com/moby/moby/blob/03e2923e42446dbb830c654d0eec323a0b4ef02a/contrib/mkimage/debootstrap#L82-L105 via this]
 +
 +
* use .dockerignore files can help
 +
  
  
Line 484: Line 550:
  
  
The "keeps things simple" is more true when your single responsibility happens to be a single ''process''.
+
The "keeps things simple" is more true when your single responsibility happens to be a single ''process'' - but this is sometimes overstated.
  
Another way of putting that is "if there is no functional value to shoving things together, it's bother with no upside"
 
  
Consider for example that docker watches the main process for failure, and will itself signal and/or restart.
+
Another way of putting that is "if there is no functional value to shoving things together, the shoving them together eventually amounts to bother with no upside"
Also the logs will that process's, tied to this container instance, so you can find them easily and there's a decent chance they are immediately useful.
+
  
 +
Consider for example that docker watches the main process for failure, and will itself signal for restart, or do restart itself.
  
 +
Also the logs docker fishes out will be the main process's, tied to this container instance, so you can find them easily, and there's a decent chance they are immediately useful.
  
The security argument is often largely that parts only reveal what they explicitly communicate,
 
and you tend to know your APIs better than your hosts. That said, not everyone is considering what exploits within containers really mean.
 
  
 +
There's also some amount of security argument, e.g. that if a single program is exploted, that now means only revealing the information that that one container instance is explicitly communicating. Just how much of an improvement that really is, though, largely depends on whether both dev and ops are really thinking about this.
  
  
That said, often enough a responsibility may be a set of tied processes. In this case, you'll need to add some monitoring inside (look at monit, runit, supervisord. Some of these add things that are useful enough that you may want them ''anyway'')
 
  
Also, some apps expect a bit of OS around them - think services, cron
+
 
 +
That said, sometimes it makes more sense to have a bunch of processes together fulfil a purpose.
 +
 
 +
This now means docker can't monitor it fully so you want some of your own monitoring inside the container (look at monit, runit, supervisord. Some of these add things that are useful enough that you may want them ''anyway'')
 +
 
 +
 
 +
Also, some apps expect a bit of OS around them - things like services, cron
 
: if you remove too much, you may get a system that looks right, buy does not act right
 
: if you remove too much, you may get a system that looks right, buy does not act right
  
 
* if you need non-trivial logging, that may imply a service and network logging
 
* if you need non-trivial logging, that may imply a service and network logging
  
* if subprocesses are created, you must consider the orphan-process reaping problem
+
* if you ever create subprocesses, you must consider the orphan-process reaping problem
: tl;dr: Your PID 1 gets these, and must be able to clean these up.
+
: tl;dr: Your PID 1 gets these, and must be able to clean these up - in the 'you are running exactly one process' view you are foregoing init so you have effectively ''disabled'' this
 
: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
 
: https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
  
  
Phusion/baseimage solves some of these details, so can be a useful-yet-still-pretty-minimal basis.
+
There are some images, like phusion/baseimage, which have addressed these details,
 +
so can be a useful-yet-still-pretty-minimal basis.
 +
 
  
  
Line 522: Line 594:
 
===="avoid sshd"====
 
===="avoid sshd"====
 
<!--
 
<!--
tl;dr:
 
: Nothing inherently evil about it
 
: Yet you need to manage it when it's there
 
: So when it's unnecessary, why do it?
 
  
 +
Roughly:
 +
* Nothing inherently evil about it
  
Installing sshd introduces a bunch of security and management considerations:
+
* If you install it, you introduce security and management considerations:
 
: where do you keep the config
 
: where do you keep the config
 
: where do you keep the keys/password (baked into the image? volume on top?)
 
: where do you keep the keys/password (baked into the image? volume on top?)
 
: how will you patch vulnerabilities?
 
: how will you patch vulnerabilities?
So there are upsides to going without it.
 
  
 +
* So when it's unnecessary, why do it?
  
It's ''often'' unnecessary. Consider for example:
 
  
* entering a running container (e.g. for image/config changes): {{inlinecode|docker exec -i -t ''nameorid'' bash}} gets you in (seems to be based on the earlier {{inlinecode|nsenter}}), and for some purposes {{inlinecode|docker attach}}
+
It's unnecessary for ''some'' of the cases you're currently thinking of. Consider for example:
 +
 
 +
* entering a running container: (e.g. for image/config changes): {{inlinecode|docker exec -i bash}} gets you in (seems to be based on the earlier {{inlinecode|nsenter}}), and for some purposes {{inlinecode|docker attach}}
  
 
* restart services - You can usually do that with signals (via docker exec)
 
* restart services - You can usually do that with signals (via docker exec)
Line 546: Line 617:
  
 
* read logs - If they are important, you should put them on a volume (or do network logging)
 
* read logs - If they are important, you should put them on a volume (or do network logging)
 +
  
  
Line 552: Line 624:
 
: note: take a look at LXC
 
: note: take a look at LXC
  
* where ypical use means occasional container login (and...)
+
* cases where typical use means occasional container login (in particular when...)
  
 
* You don't have access to the docker host OS (to run docker exec and such)
 
* You don't have access to the docker host OS (to run docker exec and such)
Line 568: Line 640:
 
===How does storage work?===
 
===How does storage work?===
 
<!--
 
<!--
 +
 
tl;dr:
 
tl;dr:
* don't use containers to store anything you want to keep
+
* don't use container state to store anything you want to keep
: The container has some contents, but they belong to the lifecycle of the container (which is great for volatile stuff like /var/run to be cleaned up)
+
: The container has its own content, yes, but they belong to the lifecycle of the container {{comment|(which is great for volatile stuff like /var/run to be implicitly cleaned up)}}
 
: also means that if you want logs
 
: also means that if you want logs
 
:: logging to a network logger is probably handiest  
 
:: logging to a network logger is probably handiest  
 
:: ...or set the containers to ''not'' be cleaned up after they stop, but that means you need to keep watching such leftover state
 
:: ...or set the containers to ''not'' be cleaned up after they stop, but that means you need to keep watching such leftover state
 +
: also note that anything you add needs to be backed by the host
 +
:: There are default limits here, e.g. 10GB per container and 100GB total, to not trample it immediately
 +
:: https://bobcares.com/blog/docker-container-size/
 +
  
 
* you can map a directory from the host in the container (called data volumes)
 
* you can map a directory from the host in the container (called data volumes)
: which is effectively just local access, so just as fast
+
: which is effectively just local access, so just as fast (or as slow, if over-shared)
: but not portable, in that the directory is easily be specific per node
+
: data volumes are not necessaririly the answer to anything you want to deploy, scale, manage, or be portable.
  
 
* consider (distributed) network filesystems
 
* consider (distributed) network filesystems
Line 583: Line 660:
 
: from the host or within the container, whatever makes more sense for that filesystem
 
: from the host or within the container, whatever makes more sense for that filesystem
  
* (you could also bind-mount a filesystem - but this is less flexible than volumes (/ network filesystems))
+
* (you could also bind-mount a filesystem - but this is less flexible than volumes (or network filesystems))
  
* you could consider a docker a temporary space
 
: https://bobcares.com/blog/docker-container-size/
 
  
  
  
Note that if databases can be your only storage,
 
that simplifies the issue for the worker containers to "connect to the database",
 
and you can put more specific thought into how to do that storage (whether dockerized or not).
 
(Keep in mind that most RDBMSes would be a single instance tied to its backing data, and will likely be a bottleneck)
 
  
 +
'''Data volumes'''
  
'''More technically''', the filesystem you see within a container  
+
Just mean a directory inside a container that points to a directory actually outside the container.
is a union of
+
 
 +
You can back this with local disk, which will be fast (if not over-shared) -- but also mean the container ''must'' be run on a specific node.
 +
 
 +
So at scale, you might back this with network mounts.
 +
Which begs the question whether you wouldn't do a network mount from within the container.
 +
 
 +
 
 +
 
 +
'''Databases'''
 +
 
 +
If you put databases in docker, you will be doing it for the database server executable, database server config, and/or client config, and not for the data itself.
 +
 
 +
 
 +
That data will be on volumes. ''You'' will have to decide how that is best deployed, because if perforance is paramount, that means tying database instances to the docker host that stores the actual data.
 +
 
 +
But also that at large enough scale, that single host will most likely be your bottleneck.
 +
Note that NoSQL, while not quite as safe as RDBMSes, may be a lot easier to scale among more hosts (possibly all docker hosts).
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
 
 +
'''At a more technical level'''
 +
 
 +
The filesystem you see within a container is:
 
* all the image's layers, union-mounted on top of each other in order
 
* all the image's layers, union-mounted on top of each other in order
 
: read-only in itself {{comment|(which is why starting a container needs almost no IO)}}. The contents are copy-on-write, in that writes to these files will mean a copy goes to....
 
: read-only in itself {{comment|(which is why starting a container needs almost no IO)}}. The contents are copy-on-write, in that writes to these files will mean a copy goes to....
  
* a write layer over that
+
* a writeable layer over that
:: consider this part of the running container, not of the image
+
: consider this part of the running container, not of the image
:: writing to an existing file works in a copy-on-write way
+
: writing to an existing file works in a copy-on-write way
:: You ''can'' commit changes, but this is only really useful when tweaking the image.
+
: You ''can'' commit changes, but this is only really useful when tweaking the image.
:: space comes from the hosting filesystem, exactly how it is stored depends on the storage plugin {{comment|(probably somewhere in <tt>/var/lib/docker</tt> though)}}
+
: space comes from the hosting filesystem, exactly how it is stored depends on the storage plugin {{comment|(probably somewhere in <tt>/var/lib/docker</tt> though)}}
:: by default this is not cleaned up, which is fine for development, and long-running jobs, where you want to inspect it. You can tell docker to clean up all container state if you want to keep things clean (good when using containers for short tasks, and when all useful logging is stored on a volume anyway)
+
: by default this is not cleaned up, which is fine for development, and long-running jobs, where you want to inspect it. You can tell docker to clean up all container state if you want to keep things clean (good when using containers for short tasks, and when all useful logging is stored on a volume anyway)
  
  
Line 633: Line 735:
 
* http://blog.goranrakic.com/archives/2014/06/understanding_docker_volumes.html
 
* http://blog.goranrakic.com/archives/2014/06/understanding_docker_volumes.html
  
 +
-->
  
 
===How does networking work?===
 
===How does networking work?===
Line 737: Line 840:
  
  
If you're doing it for the don't-break-my-libraries reasons, then you may want to consider displaying directly to the host's X server.
+
If you're doing it just for the don't-break-my-libraries reasons, then you may want to consider displaying directly to the host's X server. (Also, take a look at [[snap]])
  
 
+
Since X is a networked protocol in the first place, that connection is not very complex - the most you have to worry about is X authentication, and potentially about X extensions. <!--  
Since X is a networked protocol in the first place, that connection is not very hard - the most you have to worry about is X authentication, and potentially about X extensions.
+
 
+
Keep in mind that sound is a little extra work.
+
 
+
As is GPU access
+
 
+
 
+
<!--  
+
 
You don't need to route it, mind - locally, if you share the X11 socket most things will be happy.
 
You don't need to route it, mind - locally, if you share the X11 socket most things will be happy.
 
http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/  
 
http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/  
 
-->
 
-->
  
 +
 +
Keep in mind that sound is a little extra work.  As is using your GPU.
  
 
<!--
 
<!--
Line 840: Line 937:
 
{{stub}}
 
{{stub}}
  
Okay, so this one's confusing.
+
So, this one's confusing. As far as I can tell, so far...
  
As far as I can tell, so far...
 
  
 +
Originally, docker on windows meant "Sure you can run linux containers on windows: Install VirtualBox on windows, run linux in that, and docker in that".  This was somewhat awkward, but worked, and potentially quite valid: the VM's overhead appears only once, the instances within it work as you'ld expect,  [https://docs.docker.com/toolbox/toolbox_install_windows/ tooling] was introduced to manage this from the windows side, meaning you could have linux docker basically as-is on windows {{comment|(and of course run windows apps (or VMs) on the same metal and OS)}}.
  
Originally, docker on windows meant "Sure you can run linux containers on windows: Install VirtualBox on windows, run linux in that, and docker in that". This was somewhat awkward, but only somewhat, and quite valid: the VM's overhead appears only once and the instances within it work as usual, and [https://docs.docker.com/toolbox/toolbox_install_windows/ tooling] was introduced to manage this from the windows side, meaning you could have linux docker basically as-is on windows {{comment|(and of course run windows apps (or VMs) on the same metal and OS)}}.
+
{{comment|(Though you do need windows Pro or Server to do this, because you'll want Hyper-V which is missing in the basic windows versions)}}
 
+
{{comment|(Though you do need windows Pro, or Server, because you'll want Hyper-V which is missing in the most basic windows versions)}}
+
  
  
Line 854: Line 949:
 
and 'does not require VirtualBox'.
 
and 'does not require VirtualBox'.
  
Because what it actually means is "we have made our own thing that runs only on windows".
+
Because what it basically means is "we have made our own thing that runs only on windows".
 +
It primarily points as some in-kernel isolation (that imitates what linux did), which is useful.
  
  
And called it docker, because you can build these with dockerfiles.
+
The confusing part is that they called it docker.
 +
Sure you can build these images with dockerfiles, but that's where the similarity begins and ends.
  
But the way these are ''run'' are completely unrelated to linux docker.
+
The way these are ''run'' are completely unrelated to linux docker.
 
There is zero technical overlap.
 
There is zero technical overlap.
  
  
Sure, docker for windows is useful in itself.
+
Again, the 'run things on the same metal' implication is useful,
Technically it's mostly some in-kernel isolation that imitates what linux did, which is useful.
+
which now amounts to having docker for linux in a VM as described earlier,
 +
and adding native windows containers alongside.
  
 +
And it's nice that the tooling was updated to manage ''both'' these systems.
  
The ability to use windows containers alongside linux-in-VM is useful too,
 
as is the fact that the tooling makes it easy to manage ''both'' these systems,
 
making it easier to manage a mix on the same metal,
 
and in the same swarm.
 
  
  
It just makes a lot more sense to see docker windows as an alternative to Windows VMs,
+
But the containers are fundamentally and thoroughly incompatible,
than to compare it to docker, or say it ''is'' docker.
+
so it's misleading to use Docker as an umbella terms when this very introduction
 +
included a lot of conditionals about how to run it.
  
It is confusing to call it the same thing ''just'' because the tools can handle both,
+
...but the tooling's the same. Well, on one system.
particularly because that name by its nature indicates containers, portability, not having VM overhead, and such.
+
  
Well, used to. It now is an umbrella including some things you can ''only ever do'' via VMs, which is weird and confusing to the point it seems like bad marketing, or maybe really good marketing.
 
  
Confusing, because you'll notice we've extended the situation to half a dozen possible distinct concepts:
+
You'll notice we've extended the situation to half a dozen possible distinct concepts:
 
* running linux containers on linux (= docker)
 
* running linux containers on linux (= docker)
 
* running linux containers on windows via a VM (= typical if you want the combination)
 
* running linux containers on windows via a VM (= typical if you want the combination)
* running linux containers on windows natively (MS have basically said this won't happen)
+
* running linux containers on windows natively (MS have basically said this will never happen)
  
 
* running windows containers on windows (= "docker windows")
 
* running windows containers on windows (= "docker windows")
Line 891: Line 985:
  
  
 +
 +
It's confusing to the point it seems like bad marketing -- or maybe really intentional marketing.
  
  
People have raised the question whta MS is working towards in the long term.
+
People have raised the question what MS is working towards in the long term.
  
 
It's not that MS doesn't understand the concept of an lightweight secure VM.
 
It's not that MS doesn't understand the concept of an lightweight secure VM.
Line 903: Line 999:
  
 
So many have suggested this it's also a clever but surreptitious marketing strategy:
 
So many have suggested this it's also a clever but surreptitious marketing strategy:
You get people to think they are not committing to a specific tech -- while leading them to do so.
+
You get people thinking they are not committing to a specific tech -- while leading them to do so:
  
 
If you get people to see docker as a ''mixed-OS'' thing, mainly due to the tooling,
 
If you get people to see docker as a ''mixed-OS'' thing, mainly due to the tooling,

Revision as of 16:00, 14 August 2019

Notes related to (mostly whole-computer) virtualization, emulation and simulation.

Some overview · Docker notes · Qemu notes

Intro

What?

Not a separated machine, but processes within a host linux kernel (since approx 3.13) -- that happen to be isolated in all the ways that matter.

They are more lightweight than running a classical VM

in part because they virtualize the OS (interface), not the hardware
in part because of practical details, e.g. how the images are created/used
in part because persistent storage is intentionally separated


Actually, the isolation is mostly recent kernel features; Docker is one specific product/toolkit on top that make it practical to actually use.

Docker has its own take on it. For some more background and comparison, see Virtualization,_emulation,_simulation#Linux_containers.

There are comparable things, e.g. SmartOS had been doing this before Linux, though more for reasons of operational efficiency and security, whereas docker focuses more on the microservice angle.

What's it useful for?

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

Depends on who you ask.


Angles include:


Separating apps things to relieve dependency hell

Where historically, executables were often more monolithic things that controlled everything, modern systems tend to have applications live in an ecosystem of libraries and more, which keep working by merit of that environment staying sane.

The larger said ecosystem, the more friction and fragility there is, and harder to administer, e.g. more chance for "oh god I tried to update X and now everything is in a weird sorta-broken state that I don't understand.".


Keeping things mixed works out as a good solution for the OS itself, largely because there are people putting time into keeping that well defined.

Apps on the other hand do what they want, and are sometimes best kept isolated from not only each other but also from the OS, basically whenever that ends up simplifying the management.


Note that docker's a little overkill for just this - there's other ways to do this, some simpler, some newer.


Portability

The just-mentioned separation also means each container'd app will run the same regardless of your hardware or OS. Same as a VM, really.


Useful layer of abstraction for software packages

The tech for OS containers had existed in a usable state for over decade. Docker just made them a lot easier to actually use.

Or, as some other people put it, "docker is to apt what apt is to tar."

On top of that there are things like Docker Hub, repositories to share images people build, which makes it a software-service repository.

(Which confuses the above analogy in that it's like apt for docker. Also docker images are tarballs. Am I helping yet? :) )



Development and automating deployment

It's not too hard to set up so that a code push automatically triggers: testing, building an image, and starting it on a dev server.

Easy and fast container startup is pretty convenient to automate testing of a new version, both of the unit/regression sort, as of some amount of larger deployment. But also valid on just a single host.

docker diff can sometimes be rather useful in debugging works-for-me issues


Large-scale deployment

In clusters it's very handy to not have exactly reproducable environments, without fine management of each node and with minimal overhead.

When is it less useful?

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)
  • When there is no justification reason to use it.
because you'ld be adding complexity without any reason.
and distributed software is, by nature, harder to design and a lot harder to debug than a monolithic setup.
  • if you think it's the scalability band-aid
Docker only makes the deployment step easier, but efficiency and scalability are properties of design
  • takes some relearning on how to do things like least-privilege access, restorable backups, monitoring,
  • if security is your main interest
you get isolation, yes, but it's not the primary goal, and there are footnotes you must know
VMs are arguably better there, at least until security becomes a central focus for docker
  • some potential upsides, like automated setup and provisioning, aren't automatic.
they only really happen when you think about how to do them for your case, and it has to make sense for your use case.

Single-purpose

Security model

Good and bad ideas

Technical Intro

Some concepts

An image file is a completely specified environment that can be instantiated.

It is basically a snapshot of a filesystem.
They are often layered on top of other images
which makes it easier to build, and easier to version-control each layer
and since layers they are references (to things you also have) rather than copies, it makes heavily layered images much smaller
For example's sake you can rely on the default fetching of image files from Docker Hub.
(you can create image files yourself, and will eventually want to)


A container is a instance of an image file - either running, or stopped and not cleaned up yet.

Note that the changes to files from the image are not written to that image. They are copy-on-write to container-specific state.



You can refer to images either with identifier (full or shortened hex form), or with an alias.

repository:tag
is mostly just human-readable aliases for an image id.

And yes, the naming is confusing:

  • alias and tag sometimes mean the same thing, tag also to the post-column part (that is usually used for versioning)
  • repository is just a name, often grouping similar builds (and does not refer to the store of images as a whole, networkwise or not)


You can add tags with docker tag. Note that removing images by name will only actually remove the image if this alias was the only thing pointing at the underlying identifier (you can consider them similar to filesystem hardlinks).


Also note that omitting the tag implies :latest, which is generally not what you want.

...because it refers to the last build(/tag?) command without an explicit tag (or the). This is going to bite you if you don't always specify the tag. Simplest solution: always specify a tag.


For example, when docker ps shows me I have

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ubuntu              xenial              d355ed3537e9        4 weeks ago         119 MB
I can run it with
docker run -it ubuntu:xenial bash
or
docker run -it d355ed3537e9 bash
, but
ubuntu
will cause a download because it implies
ubuntu:latest
which is not in the list.

Introduction by example

Install docker. Via package management often does 90% of the setup work.

You may need/want to give a specific users extra rights, so that you don't need to be root or use sudo all the time. But in a pinch that works too.


Instantiating things

root@host# docker run -i -t ubuntu /bin/bash
root@3b66e09b0fa2:/# ps faux
root         1  0.0  0.0  18164  1984 ?        Ss   12:09   0:00 bash
root        15  0.0  0.0  15560  1104 ?        R+   12:11   0:00 ps faux

What happened:

  • it found an image called ubuntu (downloaded it from docker hub if not present yet)
  • instantiated it, with bash as entry point / main process
  • you manually ran ps within it. (Yes, the only processes inside right then are that shell and that command)


Notes:

  • The entry point is the main process, and also determines the container lifetime.
You would generally use an independent, long-running command. Exactly what is up to you.
In the microservice philosophy often the actual service (sometimes a process monitor)
note that the main process's stdout will go to the logs
It is also valid to see a container as half of an OS, with a stack of software, plus perhaps a ssh daemon. It's often cleaner to avoid that if you can, but it can sometimes make a lot of sense.
note:
docker kill
kills the main process, and thereby the container.
In this example the main process is bash, purely as a "prove it's a separate thing" example and means the container only lives until you log out.
also note that
-i -t
, for 'interactive' and 'allocte a tty' are only necessary because we want an interactive shell, which is not typical
  • If you didn't have the
    ubuntu
    image, then ubuntu:latest would have been downloaded from docker hub.
  • by default, the container id also becomes its hostname
  • the full container id is long. Most places where docker prints one or wants one needs only a few bytes (docker usually shows six bytes, twelve hex characters), because that's almost always unique
  • in many cases, you can also use the name, but note that these need to be unique

Status and cleanup, of images and instances

Status

See existing images on the host:

  • docker images
    lists present images
  • docker images -a
    includes non-named ones,
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
<none>                <none>              eda5290e904b        2 months ago        70.5MB
ubuntu                bionic-20190515     7698f282e524        3 months ago        69.9MB


See running container instances on the host:

  • docker ps
    to list running containers
  • docker ps -a
    to list running and old containers
CONTAINER ID     IMAGE            COMMAND       CREATED          STATUS           PORTS      NAMES
3b66e09b0fa2     ubuntu:latest    "bash"        9 seconds ago    Up 8 seconds                drunk_mestorf



Cleanup

If you're done with images:

  • docker rmi ids
    to remove them, e.g. based on
    docker images
    .
It'll refuse when something currently depends on it.
  • docker image prune
    should clean up dangling images (or, with -a, unused)
dangling images are mostly build layers are no longer referred to (no repository and tag)


In default config, container instance state sticks around after the container stops.

This can be very useful for debug during development, but you do eventually want to clean them:

  • docker rm contid


If you don't care about post-mortem debugging, you can start containers with
--rm
to always have it immediately clean up after itself.


On image building

There are two basic ways to build an image:

  • manually: start with something close to what you want, make the changes you want
saving this container = saving all filesystem changes within it
good for a one-time quick fix, less ideal for some habits you'll need at larger scale
  • automate: write a dockerfile
docker build creates an image from a dockerfile - basically from a series of commands
faster to transfer, since images are cached
https://docs.docker.com/engine/reference/builder/


The hard way: manually

Say we want to build on the ubuntu image to make a web server image

root@host# docker run -i -t ubuntu /bin/bash
root@3b66e09b0fa2:/# apt-get install apache2
# ...and whatever else

To verify the difference to the original image, we can do:

root@host@ docker diff 3b66e

Which'll print ~1200 filenames. Fine, let's roll that into a new image

root@host# docker commit 3b66e my_ubuntu_apache
cbbb61030ba24dda25f2cb27af41cc7a96a5ad9d23ef2bb500e9eaa2e16aa44d

and check that it's now known for us:

root@host# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              VIRTUAL SIZE
my_ubuntu_apache    latest              cbbb61030ba2        About a minute ago   202.7 MB
ubuntu              latest              b7cf8f0d9e82        2 weeks ago          188.3 MB

Notes:

  • commits are to your local repository. You won't accidentally write things on docker hub (or other linked site) until you are registered, you are linked, and do a docker push
actually a commit is more like a snapshot / layer
if you want things reusable and configurable, there are various good practices about how configuration and environment is best handled. You'll read up when you need it.


Okay, say we want to check the new image works - we can quit the old one and continue with the new one

root@3b66e09b0fa2:/# exit
root@host# docker run -it my_ubuntu_apache bash
root@d9bc62e40440:/# apt-get install apache2
Reading package lists... Done
Building dependency tree
Reading state information... Done
apache2 is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Eventually, we'll want to docker run the container, and we want it to -->


The usual way: Dockerfiles

multi-stage builds

habits and tips

The microservice philosophy

"avoid sshd"

Resource management

How does storage work?

How does networking work?

Limiting CPU, memory

Various commands

Starting, stopping

Pratical sides to

docker security

GUI apps in docker

You can host a VNC, RDP, or similar screen if you want it to be and entirely independent instance.


If you're doing it just for the don't-break-my-libraries reasons, then you may want to consider displaying directly to the host's X server. (Also, take a look at snap)

Since X is a networked protocol in the first place, that connection is not very complex - the most you have to worry about is X authentication, and potentially about X extensions.


Keep in mind that sound is a little extra work. As is using your GPU.


microservices in docker

SELinux and docker

Semi-sorted

"Cannot connect to the Docker daemon. Is the docker daemon running on this host?" or "Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock"

Typically means you don't have permissions to
/var/run/docker.sock
.

Which is often owned by root:docker, so while sudo works, the cleaner solution is to add the user (that you want to do this management) to the docker group.



Docker for windows

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

So, this one's confusing. As far as I can tell, so far...


Originally, docker on windows meant "Sure you can run linux containers on windows: Install VirtualBox on windows, run linux in that, and docker in that". This was somewhat awkward, but worked, and potentially quite valid: the VM's overhead appears only once, the instances within it work as you'ld expect, tooling was introduced to manage this from the windows side, meaning you could have linux docker basically as-is on windows (and of course run windows apps (or VMs) on the same metal and OS).

(Though you do need windows Pro or Server to do this, because you'll want Hyper-V which is missing in the basic windows versions)


The confusion came later, when MS said "Now we can run docker containers natively" and 'does not require VirtualBox'.

Because what it basically means is "we have made our own thing that runs only on windows". It primarily points as some in-kernel isolation (that imitates what linux did), which is useful.


The confusing part is that they called it docker. Sure you can build these images with dockerfiles, but that's where the similarity begins and ends.

The way these are run are completely unrelated to linux docker. There is zero technical overlap.


Again, the 'run things on the same metal' implication is useful, which now amounts to having docker for linux in a VM as described earlier, and adding native windows containers alongside.

And it's nice that the tooling was updated to manage both these systems.


But the containers are fundamentally and thoroughly incompatible, so it's misleading to use Docker as an umbella terms when this very introduction included a lot of conditionals about how to run it.

...but the tooling's the same. Well, on one system.


You'll notice we've extended the situation to half a dozen possible distinct concepts:

  • running linux containers on linux (= docker)
  • running linux containers on windows via a VM (= typical if you want the combination)
  • running linux containers on windows natively (MS have basically said this will never happen)
  • running windows containers on windows (= "docker windows")
  • running windows containers on linux natively (no, WSL cannot do that. And it can't really happen unless MS is fully on board with the idea, mainly because licenses)
  • running windows containers on linux via a windows VM (you can, but I don't suspect many people do this)


It's confusing to the point it seems like bad marketing -- or maybe really intentional marketing.


People have raised the question what MS is working towards in the long term.

It's not that MS doesn't understand the concept of an lightweight secure VM.

It's not that MS are bandwagoning for some open-source street cred.

They may just want to make sure they're somehow a player in a type of market that might exclude them.


So many have suggested this it's also a clever but surreptitious marketing strategy: You get people thinking they are not committing to a specific tech -- while leading them to do so:

If you get people to see docker as a mixed-OS thing, mainly due to the tooling, then if you squint you can say say they do it a little better.

Something you can't say if it a case of "windows also does something similar" more so because it'd make it clearer just how little this is about interoperability.


Also, if you get people to see docker as a windows-based application packager, then the next server your business buys may be a licensed windows server for "you never know, it will run all these hip dockery things" reasons.

Unsorted