Troubleshoot File Rotation Failed

Introduction

This document describes how to troubleshoot a file rotation fail.

File Rotation Failed

ICSeverity

5 - Notice

Impact

Unable to rotate older files to handle future storage needs.

Description

The Binary Tracing (Btrace) manager is designed to deliver highly efficient logging services for the different subsystems and to manage disk space by archiving old log files and creating new ones. The btrace rotation can be done either in-memory or into persistent storage, usually under the tracelogs directory on the bootflash or harddisk. When a log file exceeds a certain size, the system tries to move it to a different location to free up space. If the move operation fails, the system retries up to five times. If all retries fail, the system outputs the message "Move file failed, Delete file," indicating that it could not relocate the file and is considering deletion as a last resort.

The error occurs when the default location for crash files becomes full, leading to a situation where there is no more space to store new logs. This issue can arise not only because the storage location is full, but also due to a corrupted flash, which can prevent file operations from completing successfully. The error message serves as a notification that there is a significant problem with log file management, which requires attention to prevent potential loss of important diagnostic information and to ensure the continued smooth operation of the device.

SyslogMessage

BTRACE_MANAGER-3-BTRT_ERR_MSG

MessageSample

Jan 05 08:45:28 <> %BTRACE_MANAGER-3-BTRT_ERR_MSG: R0/0: btman: Btrace Rotate Error: Move file failed, retry THIS IS A SAMPLE MESSAGE

ProductFamily

• Cisco ASR 1000 Series Aggregation Services Routers
• Cisco Catalyst 9300 Series Switches
• Cisco Catalyst 9400 Series Switches
• Cisco Catalyst 3850 Series Switches
• Cisco Catalyst 9500 Series Switches
• Cisco ASR 900 Series Aggregation Services Routers
• Cisco Catalyst 9200 Series Switches
• Cisco Catalyst 3650 Series Switches
• Cisco Catalyst 9600 Series Switches
• Cisco 4000 Series Integrated Services Routers

Regex

N/A

Recommendation

This message is commonly displayed when there is an issue with the file rotation process in the device.

Please follow these steps to try to remediate the issue:

1. Review the available free space with the command show file systems or show filesystem, depending on the platform, to verify used and free space on flash. Not enough free space is your first check. You need to clear out older files manually if there is insufficient space on the drives. Using the command dir /all displays all the files within the different directories. Router#show file systems File Systems: Size(b) Free(b) Type Flags Prefixes - - opaque rw system: - - opaque rw tmpsys: * 14412443648 8376909824 disk rw bootflash: flash: crashinfo: 8107630592 8015613952 disk ro webui: - - opaque rw null: - - opaque ro tar: - - network rw tftp: - - opaque wo syslog: 33554432 33465889 nvram rw nvram: - - network rw rcp: - - network rw http: - - network rw ftp: - - network rw scp: - - network rw sftp: - - network rw https: - - opaque ro cns:

2. Validate the flash reading and writing capabilities. For example, write a new file test to bootflash by using the command copy running bootflash:test.

3. Read the file test to see if it works by using the command more bootflash:test.

4. Run the command fsck bootflash: or fsck /all and make sure all errors are fixed. You can also run the command fsck ? to display the different filesystems that can be checked by the fsck tool: Router#fsck ? /all. Check all partitions of this disk /force Forc fsck of disk bootflash: Filesystem to be checked crashinfo: Filesystem to be checked flash: Filesystem to be checked.

5. Re-run steps 2 and 3. If issue persists, follow the next step.

6. During a maintenance window, for devices that have hardware redundancy, a fail-over can clear the problem as the former active resets. If this did not fix the issue, or if you do not have redundancy, schedule a reload of the device. This also triggers filesystem check during bootup which can detect bad sectors.

7. Please review the list of known defects, and consider upgrading the software of the Cisco device in question to the recommended/latest version to ensure that most known software fixes are present in the software in use. Cisco bug ID CSCvu89062 : Bootflash showing incorrect available space Platform affected: cEdge devices (ISR1000, ISR4000, and ASR1000), this message and %PLATFORM-1-NOSPACE: bootflash: no space alarm assert seen, even though there appears to be sufficient space on the flash file system. Fixed Version: 16.12.5 and later.

Commands

#show version

#show logging

#show module

#show platform

#show filesystem

#show file systems

#show media

#show license usage