Docker and containers (generally) and persistence

aplatypus's picture

Hello everybody ...

Not too long ago the blue-moon developer who co-habits my desktop looked at using Docker for a database hosting.  The (then) use-case(s) were ...

  • messing about with idea-s, in a kind of database-sandbox world
  • swapping between sandbox-stores -- In the sense of one application many databases "verification"
    • For example; Regression testing of database", consider running the same application against database-backup-v01 and database-backup-v02
  • establishing reusable base-linexit
  • e databases for automated testing and even prototyping "stuff" (e.g. ETL)
  • purposed (personal) information servers (say like a wiki) as 'appliances'
    • e.g development tools such as the SCM appliance.

I'm sure other possibilities were flying about in my head, at that time.  By and large they each and every instance more or less stood or fell-flat pivoted around the notion of some method of decoupling and (re-)connecting data with application(-container).  There are and were then various "cook book" ideas on this theme. 

After taking a good look, I haven't found how turnkey linux container configurations (in general) or Docker (specifically) address the challenges of persistent storage.  There is a comparison to be made between a virtualisation model.  A VM has a virtual-harddrive or you may use a (mapped) external folder.  Both are straightforward concepts. 

Containers are 'more efficient' than virtual-machines and I don't need to donate disk-space for latent virtual-harddrives when things are no required (esp. testing 'machines', etc.).  But they normally come shrink-wrapped with data included.  When a container finishes any data or state generated is lost unless it can be exported or a storage solution is used.

Usual a turnkey linux appliance comes as a virtual-diskdrive and it is de facto persistent because the virtual machine (and virtual-disk) are persistient ... Where as my encounters with Docker 'appliances' have been as ephemeral clones for 'application' instances.

Do Turnkey Linux containers keep their data/state in the same way as a VM?  And which mechanism(s) are available/used to implement persistent storage.  Not every method available is as convient, and each has implictions to how an appliance is set-up and how its hosted context is organised.

Thanks in advacne.

Kind regards,

     William

Jeremy Davis's picture

So TurnKey currently provides 2 main "container" builds. One is the LXC build (which BTW is mostly used either within our LXC appliance or via Proxmox). In theory it should also work on any vanilla LXC host.

LXC containers are actually much more like a VM than the Docker container model. Although from another perspective, they are really just a chroot on steroids. On our LXC appliance (and probably on vanilla LXC host), as per a chroot, a sub-directory of the host's filesystem is used. The amount of storage allowed to be used by the guest is controlled by a filesystem quota (same way that you set/control how much storage space individual users can chew up).

Proxmox do it a little differently. I'm not 100% sure on the rationale, but I suspect that it's security/isolation related. In Proxmox a filesystem-in-a-file is set up for each container and that is mounted within the container.

As per the subject line, our current Docker build is a hack. The TurnKey appliance model provides a whole OS, whereas that's not really how Docker is designed to be used. So what we've done is built in a helper script to start services etc (when you 'docker run ...'). That allows us to create Docker compatible containers, which run just like the LXC builds. So essentially it stores all the data within it's base directory.

As I noted in response to your other post, in the future, we plan to move to a fully containerized model. Those containers will adhere to the "proper" Docker container style model and each individual Docker style container will use a shared (or not) volume storage.

Re-reading your question, I still don't think I've adequately answered it, but TBH I have very little experience with Docker style containers. I'm a big fan of LXC (actually was a bigger fan of OpenVZ before it, but unfortunately that died) although the majority of my usage is limited to Proxmox which makes it oh so easy...

So I suggest that you have a bit of a play. Also FWIW all our build code is on GitHub. All the separate build types are created by processing the ISO. We call it "buildtasks". The top level LXC container script is called bt-container and the Docker script is bt-docker.

Post new comment