Recovering a Corrupt Cisco IOS Image on a Catalyst Switch

Lab Objectives

•   Boot the switch into SWITCH ROM mode.
•   Initialize the flash file system.
•   OPTIONAL – Format the flash file system if needed.
•   Set the baud rate to 115200 to increase xmodem transfer speed and decrease time required to restore the IOS Image.
•   Copy the Cisco IOS Image via xModem to Flash:
•   Set the boot parameter to the new image name located in Flash:
•   Unset the baud so that it defaults back to 9600.
•   Boot the switch and verify that the switch boots properly.

Lab Instruction

Recovering the Cisco IOS Image on a Cisco Catalyst Series switch is quite a bit different and more of an “old school” way of doing an IOS image recovery. Unfortunately fixed configuration switches that do not support removable Compact Flash cards like the Cisco 4500 and Cisco 6500 Series switches use xModem to restore a corrupt or missing IOS image to flash:

Step 1. Boot your switch into switch ROM mode. This typically involves holding down the STAT button on the front of the switch when plugging in the power.

Once you’ve booted your switch into Switch ROM mode you’ll be see the following dispatch;

Boot Sector Filesystem (bs) installed, fsid: 2 Base ethernet MAC Address: 00:14:f2:d2:41:80 Xmodem file system is available. The password-recovery mechanism is enabled. The system has been interrupted prior to initializing the flash filesystem. The following commands will initialize the flash filesystem, and finish loading the operating system software: flash_init boot switch:

Step 2. Once you’ve booted into Switch ROM mode you’ll need to initialize the flash file system by executing the flash_init command as shown below;

switch: flash_init Initializing Flash... flashfs[0]: 1 files, 1 directories flashfs[0]: 0 orphaned files, 0 orphaned directories flashfs[0]: Total bytes: 15998976 flashfs[0]: Bytes used: 12474880 flashfs[0]: Bytes available: 3524096 flashfs[0]: flashfs fsck took 10 seconds. ...done Initializing Flash. switch:

Step 3 – OPTIONAL – If your image is corrupted you may need to format your flash file system by executing the format flash: command as shown below;

Switch: format flash: Are you sure you want to format "flash:" (all data will be lost) (y/n)?y flashfs[0]: 0 files, 1 directories flashfs[0]: 0 orphaned files, 0 orphaned directories flashfs[0]: Total bytes: 7741440 flashfs[0]: Bytes used: 1024 flashfs[0]: Bytes available: 7740416 flashfs[0]: flashfs fsck took 12 seconds. Filesystem "flash:" formatted Switch:

Step 4. To speed up the xModem file transfer speed you’ll need to set the console line BAUD rate to 115200 by executing the set BAUD 115200. If you do this you’ll need to disconnect your console session and change the serial line speed from 9600 to 115200. However if you do not set the BAUD to 115200 your file transfer can take up to 3 hours.

switch: set BAUD 115200 ÿ

Step 5. Once setting the BAUD rate and reconnecting to the console line of your Cisco switch you’re ready to copy the IOS image via xModem to the switch’s flash memory by executing the copy xmodem: flash:filename.bin as shown below;

switch: copy xmodem: flash:c3560-ipservicesk9-mz.122-53.SE.bin Begin the Xmodem or Xmodem-1K transfer now... CCC Starting xmodem transfer. Press Ctrl+C to cancel. Transferring c3560-ipservicesk9-mz.122-53.SE.bin... 100% 12181 KB 6 KB/s 00:31:56 0 Errors ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................................................................. ................. File "xmodem:" successfully copied to "flash:c3560-ipservicesk9-mz.122-53.SE.bin" switch:

Step 6. Once the IOS image is copied into flash you’ll need to update the BOOT parameter by executing the set BOOT flash:filename.bin command as shown below;

switch: set BOOT flash:c3560-ipservicesk9-mz.122-53.SE.bin

Step 7. Once the boot parameter has been set you can now set the BAUD rate back to the default 9600bps by executing the command unset BAUD. Note when doing this you’ll need to disconnect your current console connection and change the baud speed on your terminal emulator from 115200 back to 9600.

switch: unset BAUD ÿ

Step 8 is the last step which is to boot the switch and verify that the switch successfully boots the IOS image placed into flash from xMODEM. Execute the boot command to load the IOS image specified in the boot parameter as shown below;

switch: boot Loading "flash:/c3560-ipservicesk9-mz.122-53.SE.bin"...@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ File "flash:/c3560-ipservicesk9-mz.122-53.SE.bin" uncompressed and installed, entry point: 0x1000000 executing... Restricted Rights Legend Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c) of the Commercial Computer Software - Restricted Rights clause at FAR sec. 52.227-19 and subparagraph (c) (1) (ii) of the Rights in Technical Data and Computer Software clause at DFARS sec. 252.227-7013. cisco Systems, Inc. 170 West Tasman Drive San Jose, California 95134-1706 Cisco IOS Software, C3560 Software (C3560-IPSERVICESK9-M), Version 12.2(53)SE, RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2009 by Cisco Systems, Inc. Compiled Sun 13-Dec-09 15:45 by prod_rel_team Image text-base: 0x01000000, data-base: 0x02E00000 Initializing flashfs... flashfs[1]: 1 files, 1 directories flashfs[1]: 0 orphaned files, 0 orphaned directories flashfs[1]: Total bytes: 15998976 flashfs[1]: Bytes used: 12474880 flashfs[1]: Bytes available: 3524096 flashfs[1]: flashfs fsck took 1 seconds. flashfs[1]: Initialization complete....done Initializing flashfs. Checking for Bootloader upgrade.. not needed POST: CPU MIC register Tests : Begin POST: CPU MIC register Tests : End, Status Passed POST: PortASIC Memory Tests : Begin POST: PortASIC Memory Tests : End, Status Passed POST: CPU MIC interface Loopback Tests : Begin POST: CPU MIC interface Loopback Tests : End, Status Passed POST: PortASIC RingLoopback Tests : Begin POST: PortASIC RingLoopback Tests : End, Status Passed POST: Inline Power Controller Tests : Begin POST: Inline Power Controller Tests : End, Status Passed POST: PortASIC CAM Subsystem Tests : Begin POST: PortASIC CAM Subsystem Tests : End, Status Passed POST: PortASIC Port Loopback Tests : Begin POST: PortASIC Port Loopback Tests : End, Status Passed Waiting for Port download...Complete This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html If you require further assistance please contact us by sending email to export@cisco.com. cisco WS-C3560-24PS (PowerPC405) processor (revision M0) with 131072K bytes of memory. Processor board ID CAT0928Z2EE Last reset from power-on 1 Virtual Ethernet interface 24 FastEthernet interfaces 2 Gigabit Ethernet interfaces The password-recovery mechanism is enabled. 512K bytes of flash-simulated non-volatile configuration memory. Base ethernet MAC Address : 00:14:F2:D2:D1:AF Motherboard assembly number : 73-9673-06 Power supply part number : 341-0029-03 Motherboard serial number : CAT09880NNZ Power supply serial number : LIT091091ZV Model revision number : M0 Motherboard revision number : A0 Model number : WS-C3560-24PS-S System serial number : CAT0911FAEE Top Assembly Part Number : 800-25861-03 Top Assembly Revision Number : A0 Version ID : V05 CLEI Code Number : COM1X1FARB Hardware Board Revision Number : 0x01 Switch Ports Model SW Version SW Image ------ ----- ----- ---------- ---------- * 1 26 WS-C3560-24PS 12.2(53)SE C3560-IPSERVICESK9-M Press RETURN to get started!

Recovering a Corrupt Cisco IOS Image on a 2600 Series Router

Lab Objectives

•   Boot the Cisco router into ROM mode by breaking the boot sequence using the keystroke CTRL + Pause Break
•   Set the TFTPDNLD variables required to execute the procedure which include IP Address, Subnet Mask, Default Gateway, TFTP Server, TFTP Image name.
•   Execute the TFTPDNLD command and load the image into RAM using the -r switch.
•   Once booted into Cisco IOS, configure a router so that you may copy a Cisco IOS image from a TFTP server to Flash.
•   Once all necessary configuration is done copy the IOS image from the TFTP server to the flash.
•   After the copy has completed, reboot the router and verify that the image boots successfully.

Lab Instruction

As per the prerequisites you’ll need a REAL Cisco 2600 series router or greater as this lab cannot be emulated on the GNS3 application.

Step 1. Assuming that you already have an active console session to the router and the device is powered you should automatically be placed into ROMMON if you’re Cisco IOS image is corrupt or missing. However you have “Press Return to get started” Prompt then you’ve booted into an IOS image, you should power cycle the router and press CTRL+BREAK repeatedly to break the boot sequence and be placed into ROM Monitor mode as shown below;

System Bootstrap, Version 12.2(8r) [cmong 8r], RELEASE SOFTWARE (fc1) Copyright (c) 2003 by cisco Systems, Inc. C2600 platform with 262144 Kbytes of main memory device does not contain a valid magic number boot: cannot open "flash:" boot: cannot determine first file name on device "flash:" rommon 1 >

By executing the help command you can see all the available commands in ROMMON as shown below on the 2651XM;

rommon 1 > help alias set and display aliases command boot boot up an external process break set/show/clear the breakpoint confreg configuration register utility cont continue executing a downloaded image context display the context of a loaded image cookie display contents of cookie PROM in hex dev list the device table dir list files in file system dis display instruction stream dnld serial download a program module frame print out a selected stack frame help monitor builtin command help history monitor command history meminfo main memory information repeat repeat a monitor command reset system reset set display the monitor variables stack produce a stack trace sync write monitor environment to NVRAM sysret print out info from last system return tftpdnld tftp image download unalias unset an alias unset unset a monitor variable xmodem x/ymodem image download rommon 2 >

Step 2. In this lab we are most interested in the tftpdnld command.

This command will copy an IOS image from a TFTP Server into flash or load it directly into RAM. By executing the tftpdnld you can view all required variables for the command to operate properly;

rommon 2 > tftpdnld Missing or illegal ip address for variable IP_ADDRESS Illegal IP address. usage: tftpdnld [-r] Use this command for disaster recovery only to recover an image via TFTP. Monitor variables are used to set up parameters for the transfer. (Syntax: "VARIABLE_NAME=value" and use "set" to show current variables.) "ctrl-c" or "break" stops the transfer before flash erase begins. The following variables are REQUIRED to be set for tftpdnld: IP_ADDRESS: The IP address for this unit IP_SUBNET_MASK: The subnet mask for this unit DEFAULT_GATEWAY: The default gateway for this unit TFTP_SERVER: The IP address of the server to fetch from TFTP_FILE: The filename to fetch The following variables are OPTIONAL: TFTP_VERBOSE: Print setting. 0=quiet, 1=progress(default), 2=verbose TFTP_RETRY_COUNT: Retry count for ARP and TFTP (default=12) TFTP_TIMEOUT: Overall timeout of operation in seconds (default=7200) TFTP_CHECKSUM: Perform checksum test on image, 0=no, 1=yes (default=1) FE_SPEED_MODE: 0=10/hdx, 1=10/fdx, 2=100/hdx, 3=100/fdx, 4=Auto(deflt) Command line options: -r: do not write flash, load to DRAM only and launch image rommon 3 >

As per the objectives, we’re required to set the required variables to execute the tftpdnld command. You can issue the set command to view the current set variables.

rommon 3 > set PS1=rommon ! > BOOT= RET_2_RUTC=0 BSI=0 RANDOM_NUM=1492875412 ROM_PERSISTENT_UTC=1016225763 RET_2_RTS= RET_2_RCALTS= ?=1 rommon 24 > set PS1=rommon ! > BOOT= RET_2_RUTC=0 BSI=0 RANDOM_NUM=1492875412 ROM_PERSISTENT_UTC=1016225763 RET_2_RTS= RET_2_RCALTS= ?=0 rommon 4 >

Shown below is the required variables to execute tftpdnld successfully.

rommon 4 > IP_ADDRESS=10.1.1.10 rommon 5 > IP_SUBNET_MASK=255.255.255.0 rommon 6 > DEFAULT_GATEWAY=10.1.1.254 rommon 7 > TFTP_SERVER=172.16.20.17 rommon 8 > TFTP_FILE=c2600-i-mz.123-26.bin

NOTE: You do not need a default gateway if your TFTP Server is on the same subnet as the Cisco router you’re recovering.

Once you have set the variables you can continue on to objective 3 by executing the tftpdnld command with the -r switch to copy the Cisco IOS image from a TFTP Server and load it into ram directly.

rommon 9 > tftpdnld -r IP_ADDRESS: 10.1.1.10 IP_SUBNET_MASK: 255.255.255.0 DEFAULT_GATEWAY: 10.1.1.254 TFTP_SERVER: 172.16.20.17 TFTP_FILE: c2600-i-mz.123-26.bin ..... Receiving c2600-i-mz.123-26.bin from 172.16.20.17 !!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! File reception completed. program load complete, entry point: 0x80008000, size: 0x765238 Self decompressing the image : ############################################## ####################################################################### [OK] Smart Init is enabled smart init is sizing iomem ID MEMORY_REQ TYPE 00036F 0X00103980 C2651XM Dual Fast Ethernet 000065 0X00031500 Four port Voice PM 0X00098670 public buffer pools 0X00211000 public particle pools TOTAL: 0X003DE4F0 If any of the above Memory Requirements are "UNKNOWN", you may be using an unsupported configuration or there is a software problem and system operation may be compromised. Rounded IOMEM up to: 4Mb. Using 3 percent iomem. [4Mb/128Mb] Restricted Rights Legend Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c) of the Commercial Computer Software - Restricted Rights clause at FAR sec. 52.227-19 and subparagraph (c) (1) (ii) of the Rights in Technical Data and Computer Software clause at DFARS sec. 252.227-7013. cisco Systems, Inc. 170 West Tasman Drive San Jose, California 95134-1706 Cisco Internetwork Operating System Software IOS (tm) C2600 Software (C2600-I-M), Version 12.3(26), RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2008 by cisco Systems, Inc. Compiled Mon 17-Mar-08 15:23 by dchih cisco 2651XM (MPC860P) processor (revision 0x200) with 126976K/4096K bytes of memory. Processor board ID JAE08030QZL (457188033) M860 processor: part number 5, mask 2 Bridging software. X.25 software, Version 3.0.0. 2 FastEthernet/IEEE 802.3 interface(s) 2 Serial network interface(s) 32K bytes of non-volatile configuration memory. 49152K bytes of processor board System flash (Read/Write) --- System Configuration Dialog --- Would you like to enter the initial configuration dialog? [yes/no]:

Once you’ve booted your Cisco router into IOS you may be prompted by the Initial configuration Dialog, type no and press return.

Once you’ve booted the router into IOS you’ll then have to copy an actual image into flash. You’re probably thinking why did we not just copy it using TFTPDNLD and the answer is quite simple. Using TFTPDNLD will take significantly longer to copy an image via TFTP to flash verses booting into RAM and doing a quick ios image recovery as you’ve done in a previous lab.

The next step is to copy the IOS image from the TFTP Server in Cisco IOS. As shown below the typical prerequisite configuration required to copy an IOS image from a tftp server to flash.

Router>enable Router#configure terminal Router(config)#interface fa0/0 Router(config-if)#ip add 10.1.1.10 255.255.255.0 Router(config-if)#no shut Router(config-if)#exit Router(config)#ip route 0.0.0.0 0.0.0.0 10.1.1.254 Router(config)#end Router#

Once you’ve configured the required configuration to copy an ios image via tftp to the router you can execute the copy command as shown below;

Router#copy tftp flash Address or name of remote host []? 172.16.20.17 Source filename []? c2600-adventerprisek9-mz.124-1.bin Destination filename [c2600-adventerprisek9-mz.124-1.bin]? Accessing tftp://172.16.20.17/c2600-adventerprisek9-mz.124-1.bin... Erase flash: before copying? [confirm] Erasing the flash filesystem will remove all files! Continue? [confirm] Erasing device... eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee ...erased Erase of flash: complete Loading c2600-adventerprisek9-mz.124-1.bin from 172.16.20.17 (via FastEthernet0/0): !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [OK - 34634180 bytes] Verifying checksum... OK (0x8E89) 34634180 bytes copied in 279.014 secs (124131 bytes/sec) Router#

Once the new IOS image has copied to flash reload your router and verify that the router reboots.

Router#reload System configuration has been modified. Save? [yes/no]: no Proceed with reload? [confirm] SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command. System Bootstrap, Version 12.2(8r) [cmong 8r], RELEASE SOFTWARE (fc1) Copyright (c) 2003 by cisco Systems, Inc. C2600 platform with 262144 Kbytes of main memory program load complete, entry point: 0x80008000, size: 0x2107824 Self decompressing the image : ###################################### Smart Init is enabled smart init is sizing iomem ID MEMORY_REQ TYPE 00036F 0X00103980 C2651XM Dual Fast Ethernet 000065 0X00031500 Four port Voice PM 0X00098670 public buffer pools 0X00211000 public particle pools TOTAL: 0X003DE4F0 If any of the above Memory Requirements are "UNKNOWN", you may be using an unsupported configuration or there is a software problem and system operation may be compromised. Rounded IOMEM up to: 3Mb. Using 1 percent iomem. [3Mb/256Mb] Increasing IOMEM up to: 8Mb Restricted Rights Legend Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c) of the Commercial Computer Software - Restricted Rights clause at FAR sec. 52.227-19 and subparagraph (c) (1) (ii) of the Rights in Technical Data and Computer Software clause at DFARS sec. 252.227-7013. cisco Systems, Inc. 170 West Tasman Drive San Jose, California 95134-1706 Cisco IOS Software, C2600 Software (C2600-ADVENTERPRISEK9-M), Version 12.4(1), RELEASE SOFTWARE (fc2) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2009 by Cisco Systems, Inc. Compiled Wed 28-Oct-09 18:16 by prod_rel_team Image text-base: 0x800080F8, data-base: 0x83594B3C This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html If you require further assistance please contact us by sending email to export@cisco.com. Cisco 2651XM (MPC860P) processor (revision 2.0) with 253952K/8192K bytes of memory. Processor board ID JAE08030QZL M860 processor: part number 5, mask 2 2 FastEthernet interfaces 2 Serial interfaces 32K bytes of NVRAM. 49152K bytes of processor board System flash (Read/Write) --- System Configuration Dialog --- Would you like to enter the initial configuration dialog? [yes/no]:

Once you’ve verified that the new image will boot successfully on your recovered device you’ve completed the objectives of this lab.