The following commands can help you identify if a specific process is using a lot of memory:

• The show process memory command displays the memory allocation per process.

  
  
  switch# show processes memory
  PID   MemAlloc MemLimit   MemUsed  		StackBase/Ptr     Process
  ----- -------- ---------- ---------- ----------------- ---------
  4662  52756480 562929945  150167552  bfffdf00/bfffd970 netstack
  

  Note	The output of the show process memory command might not provide a completely accurate picture of the current utilization (allocated does not mean in use). This command is useful for determining if a process is approaching its limit.

• The show system internal processes memory command displays the process information in the memory alerts log (if the event occurred).

  To determine how much memory the processes are really using, check the Resident Set Size (RSS). This value will give you a rough indication of the amount of memory (in KB) that is being consumed by the processes. You can gather this information by using the show system internal processes memory command.

  switch# show system internal processes memory
   PID TTY STAT TIME MAJFLT TRS RSS VSZ %MEM COMMAND
   4811 ?        Ssl  00:00:16      0    0 49772 361588  0.3 /isan/bin/routing-sw/clis -cli /isan/etc/routing-sw/cli
   4928 ?        Ssl  00:18:41      0    0 44576 769512  0.2 /isan/bin/routing-sw/netstack /isan/etc/routing-sw/pm.cfg
   4897 ?        Ssl  00:00:18      0    0 42604 602216  0.2 /isan/bin/routing-sw/arp
   4791 ?        Ss   00:00:00      0    0 34384 318856  0.2 /isan/bin/pixm_vl
   4957 ?        Ssl  00:00:26      0    0 30440 592348  0.1 /isan/bin/snmpd -f -s udp:161 udp6:161 tcp:161 tcp6:161
   5097 ?        Ssl  00:06:53      0    0 28052 941880  0.1 /isan/bin/routing-sw/pim -t
   5062 ?        Ss   00:01:00      0    0 27300 310596  0.1 /isan/bin/diag_port_lb
   5087 ?        Ssl  00:03:53      0    0 24988 992756  0.1 /isan/bin/routing-sw/bgp -t 65001
   4792 ?        Ss   00:00:00      0    0 24080 309024  0.1 /isan/bin/pixm_gl
   5063 ?        Ss   00:00:01      0    0 21940 317440  0.1 /isan/bin/ethpm
   5044 ?        Ss   00:00:00      0    0 21700 304032  0.1 /isan/bin/eltm
   5049 ?        Ss   00:00:14      0    0 20592 306156  0.1 /isan/bin/ipqosmgr
   5042 ?        Ssl  00:00:05      0    0 20580 672640  0.1 /isan/bin/routing-sw/igmp
   5082 ?        Ssl  00:00:25      0    0 19948 914088  0.1 /isan/bin/routing-sw/mrib -m 4
   5091 ?        Ssl  00:01:58      0    0 19192 729500  0.1 /isan/bin/routing-sw/ospfv3 -t 8893
   5092 ?        Ssl  00:01:55      0    0 18988 861556  0.1 /isan/bin/routing-sw/ospf -t 6464
   5083 ?        Ss   00:00:06      0    0 18876 309516  0.1 /isan/bin/mfdm
   remaining output omitted
  
  

  If you see an increase in the utilization for a specific process over time, you should gather additional information about the process utilization.

If you have determined that a process is using more memory than expected, it is helpful to investigate how the memory is being used by the process.

• The show system internal sysmgr service pid PID-in-decimal command dumps the service information running the specified PID.

  switch# show system internal sysmgr service pid 4727
  Service "pixm" ("pixm", 109):
  UUID = 0x133, PID = 4727, SAP = 176
  State: SRV_STATE_HANDSHAKED (entered at time Fri May 10 01:42:01 2013).
  Restart count: 1
  Time of last restart: Fri May 10 01:41:11 2013.
  The service never crashed since the last reboot.
  Tag = N/A
  Plugin ID: 1
  
  

  Convert the UUID from the above output to decimal and use in the next command.

  Note	If you are troubleshooting in a lab, you can use Cisco NX-OS hexadecimal/decimal conversion using the following hidden commands: hex <decimal to convert> dec <hexadecimal to convert>

• The show system internal kernel memory uuid uuid-in-decimal command displays the detailed process memory usage including its libraries for a specific UUID in the system (convert UUID from the sysmgr service output).

  switch# show system internal kernel memory uuid 307
   Note: output values in KiloBytes
   Name                          rss   shrd     drt  map    heap   ro  dat   bss stk misc
   ----                          ---   ----    ----  ---    ----   --  ---   --- --- ----
   /isan/bin/pixm                7816  5052    2764    1       0    0    0     0  52    0
   /isan/plugin/1/isan/bin/
  pixm                         115472     0  115472    0  109176  752   28  6268   0   24
   /lib/ld-2.3.3.so                84    76       8    2       0   76    0     0   0    8
   /usr/lib/libz.so.1.2.1.1        16    12       4    1       0   12    4     0   0    0
   /usr/lib/libstdc++.so.6.0.3    296   272      24    1       0  272   20     4   0    0
   /lib/libgcc_s.so.1            1824    12    1812    1    1808   12    4     0   0    0
   /isan/plugin/1/isan/lib/
  libtmifdb.so.0                   12     8       4    1       0    8    4     0   0    0
   /isan/plugin/0/isan/lib
  libtmifdb_stub                   12     8       4    1       0    8    4     0   0    0
   /dev/mts                         0     0       0    0       1    0    0     0   0    0
   /isan/plugin/1/isan/lib/
  libpcm_sdb.so.                   16    12       4    1       0   12    4     0   0    0
   /isan/plugin/1/isan/lib/
  libethpm.so.0.                   76    60      16    1       0   60   16     0   0    0
   /isan/plugin/1/isan/lib
  /libsviifdb.so.                  20     4      16    1      12    4    4     0   0    0
   /usr/lib/libcrypto.so.0.9.7    272   192      80    1       0  192   76     4   0    0
   /isan/plugin/0/isan/lib/
  libeureka_hash                    8     4       4    1       0    4    4     0   0    0
   remaining output omitted
  
  

  This output helps you to determine if a process is holding memory in a specific library and can assist with memory leak identification.

• The show system internal service mem-stats detail command displays the detailed memory utilization including the libraries for a specific service.

  switch# show system internal pixm mem-stats detail
   Private Mem stats for UUID : Malloc track Library(103) Max types: 5
  -----------------------------------------------------------------------------
  TYPE NAME                                       ALLOCS                 BYTES
                                             CURR    MAX       CURR        MAX
     2 MT_MEM_mtrack_hdl                       35     35     132132     149940
     3 MT_MEM_mtrack_info                     598    866       9568      13856
     4 MT_MEM_mtrack_lib_name                 598    866      15860      22970
  -----------------------------------------------------------------------------
  Total bytes: 157560 (153k)
  -----------------------------------------------------------------------------
  
  Private Mem stats for UUID : Non mtrack users(0) Max types: 157
  -----------------------------------------------------------------------------
  TYPE NAME                                       ALLOCS                 BYTES
                                             CURR    MAX       CURR        MAX
     1 [0x41000000]ld-2.15.so                 283    283      48255      48256
     2 [0x41024000]libc-2.15.so               142    144       4979       5587
     8 [0x41241000]libglib-2.0.so.0.3200.3    500    771      10108      15588
    39 [0xf68af000]libindxobj.so                7      7        596        596
    45 [0xf68ca000]libavl.so                   73     73       1440       1440
    67 [0xf71b3000]libsdb.so                   56     58       3670      73278
    75 [0xf7313000]libmpmts.so                 35     37        280        380
    86 [0xf7441000]libutils.so                 23     28       3283       5766
    89 [0xf74bf000]libpss.so                   59     60       8564     483642
    90 [0xf750b000]libmts.so                    7      8        816        828
    92 [0xf754c000]libacfg.so                   0      4          0      51337
  -----------------------------------------------------------------------------
  Total bytes: 82817 (80k)
  --------------------------------------------------------------------------------
   remaining output omitted
  
  

  These outputs are usually requested by the Cisco customer support representative when investigating a potential memory leak in a process or its libraries.

When many features are deployed, baseline memory requires the following thresholds:

• MINOR

• SEVERE

• CRITICAL

Because the default thresholds are calculated on boot up depending on the DRAM size, its value varies depending on the DRAM size that is used on the platform. The thresholds are configurable using the system memory-thresholds minor percentage severe percentage critical percentage command.

Beginning with Cisco NX-OS Release 10.2(4)M, the default system memory thresholds are as follows:

Beginning with Cisco NX-OS Release 10.3(1)F, the default system memory thresholds are as follows:

• Critical: 91

• Severe: 89

• Minor: 88

The show system internal memory-status command allows you to check the current memory alert status.

switch# show system internal memory-status
MemStatus: OK

Switches running scaled deployment, including scaled BGP EVPN VxLAN VNI (please see Cisco Nexus 9000 Series NX-OS Verified Scalability Guide for supported scale), the memory alert may be seen during Non-Disruptive ISSU as the default system memory threshold has been lowered beginning with Cisco NX-OS Release 10.3(3)F release. To avoid system reacting to critical memory alert, before upgrade configure higher value for system memory thresholds. For example: Set system memory thresholds as 90 for minor, 94 for severe, and 95 for critical.

When a memory threshold has been passed (OK -> MINOR, MINOR -> SEVERE, SEVERE -> CRITICAL), the Cisco NX-OS platform manager captures a snapshot of memory utilization and logs an alert to syslog. This snapshot is useful in determining why memory utilization is high (process, page cache, or kernel). The log is generated in the Linux root path (/) and copy is moved to OBFL (/mnt/plog) if possible. This log is very useful for determining if memory utilization is high due to the memory that was consumed by the page cache, kernel, or Cisco NX-OS user processes.

The show system internal memory-alerts-log command displays the memory alerts log.

The memory alerts log consists of the following outputs: