Field Notice: FN74004 - Firepower Software: Some Versions Of Vulnerability Database (VDB) Might Cause Excessive Memory Consumption And Traffic Loss For Some Secure Firewall Devices - Software Upgrade Recommended
Available Languages
Notice
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Products Affected
| Affected Software Product | Affected Release | Affected Release Number | Comments |
|---|---|---|---|
| Firepower Threat Defense (FTD) Software | 6 | 6.2.3, 6.2.3.1, 6.2.3.10, 6.2.3.11, 6.2.3.12, 6.2.3.13, 6.2.3.14, 6.2.3.15, 6.2.3.16, 6.2.3.17, 6.2.3.18, 6.2.3.2, 6.2.3.3, 6.2.3.4, 6.2.3.5, 6.2.3.6, 6.2.3.7, 6.2.3.8, 6.2.3.9, 6.3.0, 6.3.0.1, 6.3.0.2, 6.3.0.3, 6.3.0.4, 6.3.0.5, 6.3.0.6, 6.4.0, 6.4.0.1, 6.4.0.10, 6.4.0.11, 6.4.0.12, 6.4.0.13, 6.4.0.14, 6.4.0.15, 6.4.0.16, 6.4.0.2, 6.4.0.3, 6.4.0.4, 6.4.0.5, 6.4.0.6, 6.4.0.7, 6.4.0.8, 6.4.0.9, 6.5.0, 6.5.0.1, 6.5.0.2, 6.5.0.3, 6.5.0.4, 6.5.0.5, 6.6.0, 6.6.0.1, 6.6.1, 6.6.1.1, 6.6.3, 6.6.4, 6.6.5, 6.6.5.1, 6.6.5.2, 6.6.7, 6.6.7.1, 6.7.0, 6.7.0.1, 6.7.0.2, 6.7.0.3, 6.7.0.4 | |
| Firepower Threat Defense (FTD) Software | 7 | 7.0.0, 7.0.0.1, 7.0.1, 7.0.1.1, 7.0.2, 7.0.2.1, 7.0.3, 7.0.4, 7.0.5, 7.1.0, 7.1.0.1, 7.1.0.2, 7.1.0.3, 7.2.0, 7.2.0.1, 7.2.1, 7.2.2, 7.2.3, 7.3.0, 7.3.1 |
Defect Information
| Defect ID | Headline |
| CSCwf29944 | VDB 363-364 incorrectly intentionally fail to install when there is a deleted low-end device |
| CSCwd55058 | SNORT: Memory pressure in detection cgroup forcing snort into "D" state on some devices |
| CSCwe51219 | VDB install [intentionally] failing on version 363+ when managing lower end platforms running snort2 |
| CSCwd88641 | Deployment changes to push VDB package based on Device model and snort engine |
| CSCwf21682 | VDB should remove itself from the system when it encounters bug CSCwd88641/CSCwe51219 |
| CSCwd70722 | VDB 361 and 362 fail to install on SW versions less than 6.3 |
Problem Description
Some versions of Vulnerability Database (VDB) releases might cause excessive memory consumption and traffic loss for some Secure Firewall devices.
Background
Cisco Firepower Threat Defense (FTD) Software uses the VDB updates to provide protection against known vulnerabilities to which hosts might be susceptible, as well as fingerprints for operating systems, clients, and applications. Customers are encouraged to configure the scheduling of automatic VDB updates to maximize protection against any new attack patterns.
For some models of Cisco Secure Firewall devices that are running Snort2 network intrusion detection software, the system memory may be insufficient to support the number of different fingerprints that are introduced in VDB versions 360 and later. Cisco Secure Firewall devices that are affected by this issue includes all models of the following:
- ASA5506
- ASA5508
- ASA5512
- ASA5515
- ASA5516
- ASA5525
- ASA5545
Starting with VDB version 363, three conditional checks were added to the VDB package to prevent potential memory issues and traffic loss for affected Cisco Secure Firewall devices. Installation of the VDB update will intentionally fail if all conditional checks are met. This will generate a generic failure message in the Cisco Firepower Management Center (MC) UI that requires a review of the log files to confirm the failure reason.
The three conditional checks that are performed for VDB version 363 and later are as follows:
Condition 1: VDB version
VDB version 363 and later
Condition 2: Cisco Firepower Device Manager (FDM) or FMC Software release:
| Cisco Software Release | Affected Releases for VDB Install |
|---|---|
| 6.3 and earlier | All releases |
| 6.4 | 6.4.0.16 and earlier |
| 6.5 | All releases |
| 6.6 | All releases |
| 6.7 | All releases |
| 7.0 | 7.0.5 and earlier |
| 7.1 | All releases |
| 7.2 | 7.2.3 and earlier |
| 7.3 | 7.3.1.0 and earlier |
Condition 3: Managed device check
The device manager—Cisco FMC, FDM or Adaptive Security Device Manager (ASDM)—is managing at least one device that meets both of the following criteria:
- Running Snort2 network intrusion detection software
- The Cisco Secure Firewall device is one of these models: ASA5506, ASA5508, ASA5512, ASA5515, ASA5516, ASA5525 and ASA5545
Fore more information, see Cisco Vulnerability Database (VDB) Release Notes.
Note: VDB versions 363 and later contained a reduced fingerprint set for the Cisco Secure Firewall devices that are affected by this issue.
The reduced fingerprint set prevents Snort from consuming excess system memory for affected Cisco Secure Firewall devices. Fingerprints have been removed for applications that have more than one fingerprint. Fingerprints that trigger on behavior that is least likely to be seen by the application have also been removed. The effect on general security efficacy is low, but in some limited situations it might result in the incorrect access control rule being used for some connections.
The reduced fingerprint set affects only the access control, QoS, and SSL policies because these are the only policies that can control traffic based on application. The change does not affect the Snort Intrusion Prevention System (IPS) system.
Problem Symptom
Symptom 1: D State
To determine if the Cisco Secure Firewall device is affected by the excessive memory consumption due to a VDB update, enter /var/log/top.log.
The Snort processes will show a D state, as highlighted in the example:
top - 2022-11-05 00:01:23 up 7 days, 20:10, 0 users, load average: 3.21, 2.26, PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3596 sfsnort 1 -19 2062m 759m 508 D 23 22.2 1108:07 snort
3594 sfsnort 1 -19 2062m 764m 360 D 23 22.3 2578:38 snort
3595 sfsnort 1 -19 2057m 754m 1248 D 22 22.0 834:58.61 snort
There might be a loss of network traffic, either entirely or partially, during this condition. The device might automatically recover from the Snort D state or a Snort restart might be required. In some cases, Snort may not reply to the restart command and a reboot may be required to fix the issue.
Symptom 2: VDB Update Failed Installation
To determine whether determine the VDB update failed installation, complete the following steps to check the log files:
- SSH to the Cisco Secure Firewall device and enter expert mode.
- Enter the command sudo su to elevate to root.
- Run one of the following commands based on the management platform. Make sure that the VDB version in the log path (shown as vdb* below) matches the VDB version that you are trying to install. For example, for the log path /var/log/sf/vdb-4.5.0-363/pre/005_check_low_end.pl.log, the path vdb-4.5.0-363 is for VDB version is 363.
-
- For devices managed by Cisco FMC or devices running Cisco ASA with FirePOWER Services, enter grep -H 'VDB install cancelled' /var/log/sf/vdb*/pre/005_check_low_end.pl.log
- For devices managed by Cisco FDM, enter grep -H 'VDB install cancelled' /ngfw/var/log/sf/vdb*/pre/005_check_low_end.pl.log
The output of the command will show the following:
VDB install cancelled: insufficient device memory. At least one of your managed devices or for device manager, this device cannot install the full VDB. Before you install VDB 363+, upgrade the management center or device manager. This allows you to install a smaller VDB package on lower memory devices. For more information, see the VDB release notes:'' at pre/005_check_low_end.pl line 64.
Workaround/Solution
Workaround
Rollback to VDB version 362 to prevent memory issues for affected Cisco Secure Firewall devices. This VDB contains reduced fingerprints and is recommended only for the devices that are affected by this issue.
The VDB 362 update is available from Cisco Software Download.
Solution
Cisco recommends upgrading to one of the Cisco Firepower manager software releases shown in the following table. The fixed releases will also prevent the warning message for the installation of VDB versions 363 and later for affected devices,
To avoid installation failure, remove all VDB databases that are version 363 and later before attempting to upgrade system software. For more information, see Cisco Bug ID CSCwf21682.
| Cisco Software Release | First Fixed Release |
|---|---|
| 6.3 and earlier | Migrate to a fixed release. |
| 6.4 | 6.4.0.17 |
| 6.5 | Migrate to a fixed release. |
| 6.6 | 6.6.7.1 with HotFix 6.6.7.1-EB |
| 6.7 | Migrate to a fixed release. |
| 7.0 | 7.0.6 |
| 7.1 | Migrate to a fixed release. |
| 7.2 | 7.2.4 |
| 7.3 | 7.3.1.1 |
For instructions on how to install VDB updates, see the Cisco Firepower Management Center Configuration Guide.
For Cisco legacy firewall devices that are managed by Cisco FMC, see the Cisco Secure Firewall Threat Defense Compatibility Guide to find compatible Cisco FMC Software releases.
If the Cisco Secure Firewall device is managed using Cisco FDM or Cisco ASA with FirePOWER services and a HotFix is required, contact Cisco TAC for instructions on how to obtain the software (HotFix) to fix the issue for affected Cisco Secure Firewall devices.
Revision History
| Version | Description | Section | Date |
| 1.0 | Initial Release | — | 2023-OCT-05 |
For More Information
For further assistance or for more information about this field notice, contact the Cisco Technical Assistance Center (TAC) using one of the following methods:
Receive Email Notification About New Field Notices
To receive email updates about Field Notices (reliability and safety issues), Security Advisories (network security issues), and end-of-life announcements for specific Cisco products, set up a profile in My Notifications
Feedback
Unleash the Power of TAC's Virtual Assistance