Monday, February 15, 2016

Docker Container: How to Check Memory Size

Inside a Docker container, the correct way to check its memory size is not using regular Linux commands such as:

In this article, we will review how to discover runtime metrics inside a container and focus specifically on memory statistics.  Note that the docker version used in this discussion is 1.6.1.

Misleading Metrics


Using either "top" or "free" command, it will report the memory size of 7 GiB instead of 2 GiB (the correct answer) for our container.  Those commands don't know the existence of container and hence report the memory metrics of its host only.

Docker Stats API


One way to find out the correct memory statistics is to use the docker sub-command:
For example, you can type:

# docker ps
CONTAINER ID  
66f4084c6a36  

#docker stats 66f4084c6a36
CONTAINER         CPU %       MEM USAGE/LIMIT    MEM %       NET I/O
66f4084c6a36      0.05%       257.1 MiB/2 GiB    12.55%      198.5 KiB/2.008 MiB


From the above, we can find the max memory size of container is 2 GiB.  Note that the information returned by "top" or "free" command is retrieved from /proc/meminfo:

# cat /proc/meminfo
MemTotal:        7397060 kB

cgroups (or Control Groups)


As described in [3], Docker containers are built on top of cgroups.  For cgroups, runtime metrics are exposed through:
  • Newer builds
    • Control groups are exposed through a pseudo-filesystem named[4]
      • /sys/fs/cgroup
        • /sys/fs/cgroup/memory/docker/
  • Older builds
    • The control groups might be mounted on /cgroup, without distinct hierarchies.
    • To figure out where your control groups are mounted, you can run:
      • $ grep cgroup /proc/mounts
        • /cgroup/memory/docker/
In either newer or older build, you need to first fetch the long-form container ID by typing:

# docker ps --no-trunc
CONTAINER ID  
66f4084c6a3683cc2f41242e4d58a6381072ba64f41ce2e94b75c82099acd732

In our system, we can find memory runtime metrics under the folder:

  • /cgroup/memory/docker/66f4084c6a3683cc2f41242e4d58a6381072ba64f41ce2e94b75c82099acd732

For example, relevant memory runtime metrics can be found as follows:
# cat memory.stat
cache 84217856
rss 186290176
mapped_file 14630912
swap 0
pgpgin 83557
pgpgout 17515
pgfault 78655
pgmajfault 43
inactive_anon 0
active_anon 186290176
inactive_file 68890624
active_file 15327232
unevictable 0
hierarchical_memory_limit 2147483648
hierarchical_memsw_limit 4294967296
total_cache 84217856
total_rss 186290176

In summary, cgroups allow Docker to
  • Group processes and manage their aggregate resource consumption 
  • Share available hardware resources to containers 
  • Limit the memory and CPU consumption of containers 
    • A container can be resized by simply changing the limits of its corresponding cgroup
    • You can gather information about resource usage in the container by examining Linux control groups in /sys/fs/cgroup
  • Provide a reliable way of terminating all processes inside a container.

No comments: