Download here: http://gg.gg/x3tf2
Estimated reading time: 28 minutes
Device Mapper is a kernel-based framework that underpins many advancedvolume management technologies on Linux. Docker’s devicemapper storage driverleverages the thin provisioning and snapshotting capabilities of this frameworkfor image and container management. This article refers to the Device Mapperstorage driver as devicemapper, and the kernel framework as Device Mapper.
The project’s Python Interpreter is set to use Docker Compose. If I point it at a docker-compose service that binds no ports, I cannot connect to the Django server from my web browser. If I add a `ports` section to that docker-compose service, the first thing I start grabs that port, and when I attempt to run other things they fail with ```. Docker compose is much easier to change than images. If we change anything in docker-compose we just need to recreate docker-compose. That is done by executing: docker-compose up -d -force-recreate Docker-sync stopped to sync files¶ Docker-sync suggests (in known issue) to use Docker for Mac in version 17.09.1-ce-mac42 (21090).
Now, connections to localhost:8000 are sent to port 80 in the container. The syntax for -p is HOSTPORT:CLIENTPORT. HTTP/HTTPS Proxy Support. Known limitations, use cases, and workarounds. Following is a summary of current limitations on the Docker Desktop for Mac networking stack, along with some ideas for workarounds. Install a Docker Engine and Docker Compose. Install Docker for Mac, which includes the Docker engine and a recent version of docker-compose out of the box. Once you have Docker for Mac installed, open up the preferences pane for Docker, go to the ’Resources’ section and increase the allocated memory to 6GB. Now, connections to localhost:8000 are sent to port 80 in the container. The syntax for -p is HOSTPORT:CLIENTPORT. HTTP/HTTPS Proxy Support. Known limitations, use cases, and workarounds. Following is a summary of current limitations on the Docker Desktop for Mac networking stack, along with some ideas for workarounds.
For the systems where it is supported, devicemapper support is included inthe Linux kernel. However, specific configuration is required to use it withDocker.
The devicemapper driver uses block devices dedicated to Docker and operates atthe block level, rather than the file level. These devices can be extended byadding physical storage to your Docker host, and they perform better than usinga filesystem at the operating system (OS) level.Prerequisites
*devicemapper is supported on Docker Engine - Community running on CentOS, Fedora,Ubuntu, or Debian.
*devicemapper requires the lvm2 and device-mapper-persistent-data packagesto be installed.
*Changing the storage driver makes any containers you have alreadycreated inaccessible on the local system. Use docker save to save containers,and push existing images to Docker Hub or a private repository, so you donot need to recreate them later.Configure Docker with the devicemapper storage driver
Before following these procedures, you must first meet all theprerequisites.Configure loop-lvm mode for testing
This configuration is only appropriate for testing. The loop-lvm mode makesuse of a ‘loopback’ mechanism that allows files on the local disk to beread from and written to as if they were an actual physical disk or blockdevice.However, the addition of the loopback mechanism, and interaction with the OSfilesystem layer, means that IO operations can be slow and resource-intensive.Use of loopback devices can also introduce race conditions.However, setting up loop-lvm mode can help identify basic issues (such asmissing user space packages, kernel drivers, etc.) ahead of attempting the morecomplex set up required to enable direct-lvm mode. loop-lvm mode shouldtherefore only be used to perform rudimentary testing prior to configuringdirect-lvm.
For production systems, seeConfigure direct-lvm mode for production.
*
Stop Docker.
*
Edit /etc/docker/daemon.json. If it does not yet exist, create it. Assumingthat the file was empty, add the following contents.
See all storage options for each storage driver in thedaemon reference documentation
Docker does not start if the daemon.json file contains badly-formed JSON.
*
Start Docker.
*
Verify that the daemon is using the devicemapper storage driver. Use thedocker info command and look for Storage Driver.
This host is running in loop-lvm mode, which is not supported on production systems. This is indicated by the fact that the Data loop file and a Metadata loop file are on files under /var/lib/docker/devicemapper. These are loopback-mounted sparse files. For production systems, see Configure direct-lvm mode for production.Configure direct-lvm mode for production
Production hosts using the devicemapper storage driver must use direct-lvmmode. This mode uses block devices to create the thin pool. This is faster thanusing loopback devices, uses system resources more efficiently, and blockdevices can grow as needed. However, more setup is required than in loop-lvmmode.
After you have satisfied the prerequisites, follow the stepsbelow to configure Docker to use the devicemapper storage driver indirect-lvm mode.
Warning: Changing the storage driver makes any containers you have already created inaccessible on the local system. Use docker save to save containers, and push existing images to Docker Hub or a private repository, so you do not need to recreate them later.Allow Docker to configure direct-lvm mode
Docker can manage the block device for you, simplifying configuration of direct-lvmmode. This is appropriate for fresh Docker setups only. You can only use asingle block device. If you need to use multiple block devices,configure direct-lvm mode manually instead.The following new configuration options are available:OptionDescriptionRequired?DefaultExampledm.directlvm_deviceThe path to the block device to configure for direct-lvm.Yesdm.directlvm_device=’/dev/xvdf’dm.thinp_percentThe percentage of space to use for storage from the passed in block device.No95dm.thinp_percent=95dm.thinp_metapercentThe percentage of space to use for metadata storage from the passed-in block device.No1dm.thinp_metapercent=1dm.thinp_autoextend_thresholdThe threshold for when lvm should automatically extend the thin pool as a percentage of the total storage space.No80dm.thinp_autoextend_threshold=80dm.thinp_autoextend_percentThe percentage to increase the thin pool by when an autoextend is triggered.No20dm.thinp_autoextend_percent=20dm.directlvm_device_forceWhether to format the block device even if a filesystem already exists on it. If set to false and a filesystem is present, an error is logged and the filesystem is left intact.Nofalsedm.directlvm_device_force=true
Edit the daemon.json file and set the appropriate options, then restart Dockerfor the changes to take effect. The following daemon.json configuration sets all of theoptions in the table above.
See all storage options for each storage driver in thedaemon reference documentation
Restart Docker for the changes to take effect. Docker invokes the commands toconfigure the block device for you.
Warning: Changing these values after Docker has prepared the block devicefor you is not supported and causes an error.
You still need to perform periodic maintenance tasks.Configure direct-lvm mode manually
The procedure below creates a logical volume configured as a thin pool touse as backing for the storage pool. It assumes that you have a spare blockdevice at /dev/xvdf with enough free space to complete the task. The deviceidentifier and volume sizes may be different in your environment and youshould substitute your own values throughout the procedure. The procedure alsoassumes that the Docker daemon is in the stopped state.
*
Identify the block device you want to use. The device is located under/dev/ (such as /dev/xvdf) and needs enough free space to store theimages and container layers for the workloads that host runs.A solid state drive is ideal.
*
Stop Docker.
*
Install the following packages:
*
RHEL / CentOS: device-mapper-persistent-data, lvm2, and alldependencies
*
Ubuntu / Debian: thin-provisioning-tools, lvm2, and alldependencies
*
Create a physical volume on your block device from step 1, using thepvcreate command. Substitute your device name for /dev/xvdf.
Warning: The next few steps are destructive, so be sure that you havespecified the correct device!
*
Create a docker volume group on the same device, using the vgcreatecommand.
*
Create two logical volumes named thinpool and thinpoolmeta using thelvcreate command. The last parameter specifies the amount of free spaceto allow for automatic expanding of the data or metadata if space runs low,as a temporary stop-gap. These are the recommended values.
*
Convert the volumes to a thin pool and a storage location for metadata forthe thin pool, using the lvconvert command.
*
Configure autoextension of thin pools via an lvm profile.
*
Specify thin_pool_autoextend_threshold and thin_pool_autoextend_percentvalues.
thin_pool_autoextend_threshold is the percentage of space used before lvmattempts to autoextend the available space (100 = disabled, not recommended).
thin_pool_autoextend_percent is the amount of space to add to the devicewhen automatically extending (0 = disabled).
The example below adds 20% more capacity when the disk usage reaches80%.
Save the file.
*
Apply the LVM profile, using the lvchange command.
*
Ensure monitoring of the logical volume is enabled.
If the output in the Monitor column reports, as above, that the volume isnot monitored, then monitoring needs to be explicitly enabled. Withoutthis step, automatic extension of the logical volume will not occur,regardless of any settings in the applied profile.
Double check that monitoring is now enabled by running thesudo lvs -o+seg_monitor command a second time. The Monitor columnshould now report the logical volume is being monitored.
*
If you have ever run Docker on this host before, or if /var/lib/docker/exists, move it out of the way so that Docker can use the new LVM pool tostore the contents of image and containers.
If any of the following steps fail and you need to restore, you can remove/var/lib/docker and replace it with /var/lib/docker.bk.
*
Edit /etc/docker/daemon.json and configure the options needed for thedevicemapper storage driver. If the file was previously empty, it shouldnow contain the following contents:
*
Start Docker.
systemd:
service:
*
Verify that Docker is using the new configuration using docker info.
If Docker is configured correctly, the Data file and Metadata file isblank, and the pool name is docker-thinpool.
*
After you have verified that the configuration is correct, you can remove the/var/lib/docker.bk directory which contains the previous configuration.Manage devicemapperMonitor the thin pool
Do not rely on LVM auto-extension alone. The volume groupautomatically extends, but the volume can still fill up. You can monitorfree space on the volume using lvs or lvs -a. Consider using a monitoringtool at the OS level, such as Nagios.
To view the LVM logs, you can use journalctl:
If you run into repeated problems with thin pool, you can set the storage optiondm.min_free_space to a value (representing a percentage) in/etc/docker/daemon.json. For instance, setting it to 10 ensuresthat operations fail with a warning when the free space is at or near 10%.See thestorage driver options in the Engine daemon reference.Increase capacity on a running device
You can increase the capacity of the pool on a running thin-pool device. This isuseful if the data’s logical volume is full and the volume group is at fullcapacity. The specific procedure depends on whether you are using aloop-lvm thin pool or adirect-lvm thin pool.Resize a loop-lvm thin pool
The easiest way to resize a loop-lvm thin pool is touse the device_tool utility,but you can use operating system utilitiesinstead.Use the device_tool utility
A community-contributed script called device_tool.go is available in themoby/mobyGithub repository. You can use this tool to resize a loop-lvm thin pool,avoiding the long process above. This tool is not guaranteed to work, but youshould only be using loop-lvm on non-production systems.
If you do not want to use device_tool, you can resize the thin pool manually instead.
*
To use the tool, clone the Github repository, change to thecontrib/docker-device-tool, and follow the instructions in the README.mdto compile the tool.
*
Use the tool. The following example resizes the thin pool to 200GB.Use operating system utilities
If you do not want to use the device-tool utility,you can resize a loop-lvm thin pool manually using the following procedure.
In loop-lvm mode, a loopback device is used to store the data, and anotherto store the metadata. loop-lvm mode is only supported for testing, becauseit has significant performance and stability drawbacks.
If you are using loop-lvm mode, the output of docker info shows filepaths for Data loop file and Metadata loop file:
Follow these steps to increase the size of the thin pool. In this example, thethin pool is 100 GB, and is increased to 200 GB.
*
List the sizes of the devices.
*
Increase the size of the data file to 200 G using the truncate command,which is used to increase or decrease the size of a file. Note thatdecreasing the size is a destructive operation.
*
Verify the file size changed.
*
The loopback file has changed on disk but not in memory. List the size ofthe loopback device in memory, in GB. Reload it, then list the size again.After the reload, the size is 200 GB.
*
Reload the devicemapper thin pool.
a. Get the pool name first. The pool name is the first field, delimited by ` :`. This command extracts it.
b. Dump the device mapper table for the thin pool.
c. Calculate the total sectors of the thin pool using the second field of the output. The number is expressed in 512-k sectors. A 100G file has 209715200 512-k sectors. If you double this number to 200G, you get 419430400 512-k sectors.
d. Reload the thin pool with the new sector number, using the following three dmsetup commands.Resize a direct-lvm thin pool
To extend a direct-lvm thin pool, you need to first attach a new block deviceto the Docker host, and make note of the name assigned to it by the kernel. Inthis example, the new block device is /dev/xvdg.
Follow this procedure to extend a direct-lvm thin pool, substituting yourblock device and other parameters to suit your situation.
*
Gather information about your volume group.
Use the pvdisplay command to find the physical block devices currently inuse by your thin pool, and the volume group’s name.
In the following steps, substitute your block device or volume group name asappropriate.
*
Extend the volume group, using the vgextend command with the VG Namefrom the previous step, and the name of your new block device.
*
Extend the docker/thinpool logical volume. This command uses 100% of thevolume right away, without auto-extend. To extend the metadata thinpoolinstead, use docker/thinpool_tmeta.
*
Verify the new thin pool size using the Data Space Available field in theoutput of docker info. If you extended the docker/thinpool_tmeta logicalvolume instead, look for Metadata Space Available.Activate the devicemapper after reboot
If you reboot the host and find that the docker service failed to start,look for the error, “Non existing device”. You need to re-activate thelogical volumes with this command:How the devicemapper storage driver works
Warning: Do not directly manipulate any files or directories within/var/lib/docker/. These files and directories are managed by Docker.
Use the lsblk command to see the devices and their pools, from the operatingsystem’s point of view:
Use the mount command to see the mount-point Docker is using:
When you use devicemapper, Docker stores image and layer contents in thethinpool, and exposes them to containers by mounting them undersubdirectories of /var/lib/docker/devicemapper/.Image and container layers on-disk
The /var/lib/docker/devicemapper/metadata/ directory contains metadata aboutthe Devicemapper configuration itself and about each image and container layerthat exist. The devicemapper storage driver uses snapshots, and this metadatainclude information about those snapshots. These files are in JSON format.
The /var/lib/docker/devicemapper/mnt/ directory contains a mount point for each imageand container layer that exists. Image layer mount points are empty, but acontainer’s mount point shows the container’s filesystem as it appears fromwithin the container.Image layering and sharing
The devicemapper storage driver uses dedicated block devices rather thanformatted filesystems, and operates on files at the block level for maximumperformance during copy-on-write (CoW) operations.Snapshots
Another feature of devicemapper is its use of snapshots (also sometimes calledthin devices or virtual devices), which store the differences introduced ineach layer as very small, lightweight thin pools. Snapshots provide manybenefits:
*
Layers which are shared in common between containers are only stored on diskonce, unless they are writable. For instance, if you have 10 differentimages which are all based on alpine, the alpine image and all itsparent images are only stored once each on disk.
*
Snapshots are an implementation of a copy-on-write (CoW) strategy. This meansthat a given file or directory is only copied to the container’s writablelayer when it is modified or deleted by that container.
*
Because devicemapper operates at the block level, multiple blocks in awritable layer can be modified simultaneously.
*
Snapshots can be backed up using standard OS-level backup utilities. Justmake a copy of /var/lib/docker/devicemapper/.Devicemapper workflow
When you start Docker with the devicemapper storage driver, all objectsrelated to image and container layers are stored in/var/lib/docker/devicemapper/, which is backed by one or more block-leveldevices, either loopback devices (testing only) or physical disks.
*
The base device is the lowest-level object. This is the thin pool itself.You can examine it using docker info. It contains a filesystem. This basedevice is the starting point for every image and container layer. The basedevice is a Device Mapper implementation detail, rather than a Docker layer.
*
Metadata about the base device and each image or container layer is stored in/var/lib/docker/devicemapper/metadata/ in JSON format. These layers arecopy-on-write snapshots, which means that they are empty until they divergefrom their parent layers.
*
Each container’s writable layer is mounted on a mountpoint in/var/lib/docker/devicemapper/mnt/. An empty directory exists for eachread-only image layer and each stopped container.
Each image layer is a snapshot of the layer below it. The lowest layer of eachimage is a snapshot of the base device that exists i
https://diarynote-jp.indered.space
Estimated reading time: 28 minutes
Device Mapper is a kernel-based framework that underpins many advancedvolume management technologies on Linux. Docker’s devicemapper storage driverleverages the thin provisioning and snapshotting capabilities of this frameworkfor image and container management. This article refers to the Device Mapperstorage driver as devicemapper, and the kernel framework as Device Mapper.
The project’s Python Interpreter is set to use Docker Compose. If I point it at a docker-compose service that binds no ports, I cannot connect to the Django server from my web browser. If I add a `ports` section to that docker-compose service, the first thing I start grabs that port, and when I attempt to run other things they fail with ```. Docker compose is much easier to change than images. If we change anything in docker-compose we just need to recreate docker-compose. That is done by executing: docker-compose up -d -force-recreate Docker-sync stopped to sync files¶ Docker-sync suggests (in known issue) to use Docker for Mac in version 17.09.1-ce-mac42 (21090).
Now, connections to localhost:8000 are sent to port 80 in the container. The syntax for -p is HOSTPORT:CLIENTPORT. HTTP/HTTPS Proxy Support. Known limitations, use cases, and workarounds. Following is a summary of current limitations on the Docker Desktop for Mac networking stack, along with some ideas for workarounds. Install a Docker Engine and Docker Compose. Install Docker for Mac, which includes the Docker engine and a recent version of docker-compose out of the box. Once you have Docker for Mac installed, open up the preferences pane for Docker, go to the ’Resources’ section and increase the allocated memory to 6GB. Now, connections to localhost:8000 are sent to port 80 in the container. The syntax for -p is HOSTPORT:CLIENTPORT. HTTP/HTTPS Proxy Support. Known limitations, use cases, and workarounds. Following is a summary of current limitations on the Docker Desktop for Mac networking stack, along with some ideas for workarounds.
For the systems where it is supported, devicemapper support is included inthe Linux kernel. However, specific configuration is required to use it withDocker.
The devicemapper driver uses block devices dedicated to Docker and operates atthe block level, rather than the file level. These devices can be extended byadding physical storage to your Docker host, and they perform better than usinga filesystem at the operating system (OS) level.Prerequisites
*devicemapper is supported on Docker Engine - Community running on CentOS, Fedora,Ubuntu, or Debian.
*devicemapper requires the lvm2 and device-mapper-persistent-data packagesto be installed.
*Changing the storage driver makes any containers you have alreadycreated inaccessible on the local system. Use docker save to save containers,and push existing images to Docker Hub or a private repository, so you donot need to recreate them later.Configure Docker with the devicemapper storage driver
Before following these procedures, you must first meet all theprerequisites.Configure loop-lvm mode for testing
This configuration is only appropriate for testing. The loop-lvm mode makesuse of a ‘loopback’ mechanism that allows files on the local disk to beread from and written to as if they were an actual physical disk or blockdevice.However, the addition of the loopback mechanism, and interaction with the OSfilesystem layer, means that IO operations can be slow and resource-intensive.Use of loopback devices can also introduce race conditions.However, setting up loop-lvm mode can help identify basic issues (such asmissing user space packages, kernel drivers, etc.) ahead of attempting the morecomplex set up required to enable direct-lvm mode. loop-lvm mode shouldtherefore only be used to perform rudimentary testing prior to configuringdirect-lvm.
For production systems, seeConfigure direct-lvm mode for production.
*
Stop Docker.
*
Edit /etc/docker/daemon.json. If it does not yet exist, create it. Assumingthat the file was empty, add the following contents.
See all storage options for each storage driver in thedaemon reference documentation
Docker does not start if the daemon.json file contains badly-formed JSON.
*
Start Docker.
*
Verify that the daemon is using the devicemapper storage driver. Use thedocker info command and look for Storage Driver.
This host is running in loop-lvm mode, which is not supported on production systems. This is indicated by the fact that the Data loop file and a Metadata loop file are on files under /var/lib/docker/devicemapper. These are loopback-mounted sparse files. For production systems, see Configure direct-lvm mode for production.Configure direct-lvm mode for production
Production hosts using the devicemapper storage driver must use direct-lvmmode. This mode uses block devices to create the thin pool. This is faster thanusing loopback devices, uses system resources more efficiently, and blockdevices can grow as needed. However, more setup is required than in loop-lvmmode.
After you have satisfied the prerequisites, follow the stepsbelow to configure Docker to use the devicemapper storage driver indirect-lvm mode.
Warning: Changing the storage driver makes any containers you have already created inaccessible on the local system. Use docker save to save containers, and push existing images to Docker Hub or a private repository, so you do not need to recreate them later.Allow Docker to configure direct-lvm mode
Docker can manage the block device for you, simplifying configuration of direct-lvmmode. This is appropriate for fresh Docker setups only. You can only use asingle block device. If you need to use multiple block devices,configure direct-lvm mode manually instead.The following new configuration options are available:OptionDescriptionRequired?DefaultExampledm.directlvm_deviceThe path to the block device to configure for direct-lvm.Yesdm.directlvm_device=’/dev/xvdf’dm.thinp_percentThe percentage of space to use for storage from the passed in block device.No95dm.thinp_percent=95dm.thinp_metapercentThe percentage of space to use for metadata storage from the passed-in block device.No1dm.thinp_metapercent=1dm.thinp_autoextend_thresholdThe threshold for when lvm should automatically extend the thin pool as a percentage of the total storage space.No80dm.thinp_autoextend_threshold=80dm.thinp_autoextend_percentThe percentage to increase the thin pool by when an autoextend is triggered.No20dm.thinp_autoextend_percent=20dm.directlvm_device_forceWhether to format the block device even if a filesystem already exists on it. If set to false and a filesystem is present, an error is logged and the filesystem is left intact.Nofalsedm.directlvm_device_force=true
Edit the daemon.json file and set the appropriate options, then restart Dockerfor the changes to take effect. The following daemon.json configuration sets all of theoptions in the table above.
See all storage options for each storage driver in thedaemon reference documentation
Restart Docker for the changes to take effect. Docker invokes the commands toconfigure the block device for you.
Warning: Changing these values after Docker has prepared the block devicefor you is not supported and causes an error.
You still need to perform periodic maintenance tasks.Configure direct-lvm mode manually
The procedure below creates a logical volume configured as a thin pool touse as backing for the storage pool. It assumes that you have a spare blockdevice at /dev/xvdf with enough free space to complete the task. The deviceidentifier and volume sizes may be different in your environment and youshould substitute your own values throughout the procedure. The procedure alsoassumes that the Docker daemon is in the stopped state.
*
Identify the block device you want to use. The device is located under/dev/ (such as /dev/xvdf) and needs enough free space to store theimages and container layers for the workloads that host runs.A solid state drive is ideal.
*
Stop Docker.
*
Install the following packages:
*
RHEL / CentOS: device-mapper-persistent-data, lvm2, and alldependencies
*
Ubuntu / Debian: thin-provisioning-tools, lvm2, and alldependencies
*
Create a physical volume on your block device from step 1, using thepvcreate command. Substitute your device name for /dev/xvdf.
Warning: The next few steps are destructive, so be sure that you havespecified the correct device!
*
Create a docker volume group on the same device, using the vgcreatecommand.
*
Create two logical volumes named thinpool and thinpoolmeta using thelvcreate command. The last parameter specifies the amount of free spaceto allow for automatic expanding of the data or metadata if space runs low,as a temporary stop-gap. These are the recommended values.
*
Convert the volumes to a thin pool and a storage location for metadata forthe thin pool, using the lvconvert command.
*
Configure autoextension of thin pools via an lvm profile.
*
Specify thin_pool_autoextend_threshold and thin_pool_autoextend_percentvalues.
thin_pool_autoextend_threshold is the percentage of space used before lvmattempts to autoextend the available space (100 = disabled, not recommended).
thin_pool_autoextend_percent is the amount of space to add to the devicewhen automatically extending (0 = disabled).
The example below adds 20% more capacity when the disk usage reaches80%.
Save the file.
*
Apply the LVM profile, using the lvchange command.
*
Ensure monitoring of the logical volume is enabled.
If the output in the Monitor column reports, as above, that the volume isnot monitored, then monitoring needs to be explicitly enabled. Withoutthis step, automatic extension of the logical volume will not occur,regardless of any settings in the applied profile.
Double check that monitoring is now enabled by running thesudo lvs -o+seg_monitor command a second time. The Monitor columnshould now report the logical volume is being monitored.
*
If you have ever run Docker on this host before, or if /var/lib/docker/exists, move it out of the way so that Docker can use the new LVM pool tostore the contents of image and containers.
If any of the following steps fail and you need to restore, you can remove/var/lib/docker and replace it with /var/lib/docker.bk.
*
Edit /etc/docker/daemon.json and configure the options needed for thedevicemapper storage driver. If the file was previously empty, it shouldnow contain the following contents:
*
Start Docker.
systemd:
service:
*
Verify that Docker is using the new configuration using docker info.
If Docker is configured correctly, the Data file and Metadata file isblank, and the pool name is docker-thinpool.
*
After you have verified that the configuration is correct, you can remove the/var/lib/docker.bk directory which contains the previous configuration.Manage devicemapperMonitor the thin pool
Do not rely on LVM auto-extension alone. The volume groupautomatically extends, but the volume can still fill up. You can monitorfree space on the volume using lvs or lvs -a. Consider using a monitoringtool at the OS level, such as Nagios.
To view the LVM logs, you can use journalctl:
If you run into repeated problems with thin pool, you can set the storage optiondm.min_free_space to a value (representing a percentage) in/etc/docker/daemon.json. For instance, setting it to 10 ensuresthat operations fail with a warning when the free space is at or near 10%.See thestorage driver options in the Engine daemon reference.Increase capacity on a running device
You can increase the capacity of the pool on a running thin-pool device. This isuseful if the data’s logical volume is full and the volume group is at fullcapacity. The specific procedure depends on whether you are using aloop-lvm thin pool or adirect-lvm thin pool.Resize a loop-lvm thin pool
The easiest way to resize a loop-lvm thin pool is touse the device_tool utility,but you can use operating system utilitiesinstead.Use the device_tool utility
A community-contributed script called device_tool.go is available in themoby/mobyGithub repository. You can use this tool to resize a loop-lvm thin pool,avoiding the long process above. This tool is not guaranteed to work, but youshould only be using loop-lvm on non-production systems.
If you do not want to use device_tool, you can resize the thin pool manually instead.
*
To use the tool, clone the Github repository, change to thecontrib/docker-device-tool, and follow the instructions in the README.mdto compile the tool.
*
Use the tool. The following example resizes the thin pool to 200GB.Use operating system utilities
If you do not want to use the device-tool utility,you can resize a loop-lvm thin pool manually using the following procedure.
In loop-lvm mode, a loopback device is used to store the data, and anotherto store the metadata. loop-lvm mode is only supported for testing, becauseit has significant performance and stability drawbacks.
If you are using loop-lvm mode, the output of docker info shows filepaths for Data loop file and Metadata loop file:
Follow these steps to increase the size of the thin pool. In this example, thethin pool is 100 GB, and is increased to 200 GB.
*
List the sizes of the devices.
*
Increase the size of the data file to 200 G using the truncate command,which is used to increase or decrease the size of a file. Note thatdecreasing the size is a destructive operation.
*
Verify the file size changed.
*
The loopback file has changed on disk but not in memory. List the size ofthe loopback device in memory, in GB. Reload it, then list the size again.After the reload, the size is 200 GB.
*
Reload the devicemapper thin pool.
a. Get the pool name first. The pool name is the first field, delimited by ` :`. This command extracts it.
b. Dump the device mapper table for the thin pool.
c. Calculate the total sectors of the thin pool using the second field of the output. The number is expressed in 512-k sectors. A 100G file has 209715200 512-k sectors. If you double this number to 200G, you get 419430400 512-k sectors.
d. Reload the thin pool with the new sector number, using the following three dmsetup commands.Resize a direct-lvm thin pool
To extend a direct-lvm thin pool, you need to first attach a new block deviceto the Docker host, and make note of the name assigned to it by the kernel. Inthis example, the new block device is /dev/xvdg.
Follow this procedure to extend a direct-lvm thin pool, substituting yourblock device and other parameters to suit your situation.
*
Gather information about your volume group.
Use the pvdisplay command to find the physical block devices currently inuse by your thin pool, and the volume group’s name.
In the following steps, substitute your block device or volume group name asappropriate.
*
Extend the volume group, using the vgextend command with the VG Namefrom the previous step, and the name of your new block device.
*
Extend the docker/thinpool logical volume. This command uses 100% of thevolume right away, without auto-extend. To extend the metadata thinpoolinstead, use docker/thinpool_tmeta.
*
Verify the new thin pool size using the Data Space Available field in theoutput of docker info. If you extended the docker/thinpool_tmeta logicalvolume instead, look for Metadata Space Available.Activate the devicemapper after reboot
If you reboot the host and find that the docker service failed to start,look for the error, “Non existing device”. You need to re-activate thelogical volumes with this command:How the devicemapper storage driver works
Warning: Do not directly manipulate any files or directories within/var/lib/docker/. These files and directories are managed by Docker.
Use the lsblk command to see the devices and their pools, from the operatingsystem’s point of view:
Use the mount command to see the mount-point Docker is using:
When you use devicemapper, Docker stores image and layer contents in thethinpool, and exposes them to containers by mounting them undersubdirectories of /var/lib/docker/devicemapper/.Image and container layers on-disk
The /var/lib/docker/devicemapper/metadata/ directory contains metadata aboutthe Devicemapper configuration itself and about each image and container layerthat exist. The devicemapper storage driver uses snapshots, and this metadatainclude information about those snapshots. These files are in JSON format.
The /var/lib/docker/devicemapper/mnt/ directory contains a mount point for each imageand container layer that exists. Image layer mount points are empty, but acontainer’s mount point shows the container’s filesystem as it appears fromwithin the container.Image layering and sharing
The devicemapper storage driver uses dedicated block devices rather thanformatted filesystems, and operates on files at the block level for maximumperformance during copy-on-write (CoW) operations.Snapshots
Another feature of devicemapper is its use of snapshots (also sometimes calledthin devices or virtual devices), which store the differences introduced ineach layer as very small, lightweight thin pools. Snapshots provide manybenefits:
*
Layers which are shared in common between containers are only stored on diskonce, unless they are writable. For instance, if you have 10 differentimages which are all based on alpine, the alpine image and all itsparent images are only stored once each on disk.
*
Snapshots are an implementation of a copy-on-write (CoW) strategy. This meansthat a given file or directory is only copied to the container’s writablelayer when it is modified or deleted by that container.
*
Because devicemapper operates at the block level, multiple blocks in awritable layer can be modified simultaneously.
*
Snapshots can be backed up using standard OS-level backup utilities. Justmake a copy of /var/lib/docker/devicemapper/.Devicemapper workflow
When you start Docker with the devicemapper storage driver, all objectsrelated to image and container layers are stored in/var/lib/docker/devicemapper/, which is backed by one or more block-leveldevices, either loopback devices (testing only) or physical disks.
*
The base device is the lowest-level object. This is the thin pool itself.You can examine it using docker info. It contains a filesystem. This basedevice is the starting point for every image and container layer. The basedevice is a Device Mapper implementation detail, rather than a Docker layer.
*
Metadata about the base device and each image or container layer is stored in/var/lib/docker/devicemapper/metadata/ in JSON format. These layers arecopy-on-write snapshots, which means that they are empty until they divergefrom their parent layers.
*
Each container’s writable layer is mounted on a mountpoint in/var/lib/docker/devicemapper/mnt/. An empty directory exists for eachread-only image layer and each stopped container.
Each image layer is a snapshot of the layer below it. The lowest layer of eachimage is a snapshot of the base device that exists i
https://diarynote-jp.indered.space
コメント