Cisco Cloud Services Router (CSR) 1000v RESTful API won’t start

What if you have a CSR 1000v and you’re trying to start the RESTful API but it won’t start? “show remote-managent status” returns nothing and “show virtual-service detail” doesn’t show the usual process status like nginx, and the restful_api.

CSR1000V#show remote-management status
CSR1000V#
CSR1000V#show virtual-service detail name csr_mgmt

Virtual service csr_mgmt detail
State : Activated
Package information
Name : iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova
Path : bootflash:/iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova
Application
Name : csr_mgmt
Installed version : 1.7.1
Description : CSR-MGMT
Signing
Key type : Cisco development key
Method : SHA-1
Licensing
Name : Not Available
Version : Not Available

Detailed guest status
Information not available
Activated profile name: None
Resource reservation
Disk : 756 MB
Memory : 512 MB
CPU : 30% system CPU

Attached devices
Type Name Alias
 — — — — — — — — — — — — — — — — — — — — — — -
NIC ieobc_1 ieobc
NIC dp_1_33 net2
Disk _rootfs
Disk /opt/var
Disk /opt/va…
Serial/shell serial0
Serial/aux serial1
Serial/Syslog serial2
Serial/Trace serial3
Watchdog watchdo…

Network interfaces
MAC address Attached to interface
 — — — — — — — — — — — — — — — — — — — — — — — — — — — 
54:0E:00:0B:0C:02 ieobc_1
00:50:56:8B:19:8A VirtualPortGroup33

Guest interface
 — -
Information not available
 — -

Guest routes
 — -
Information not available
 — -

Resource admission (without profile) : passed
Disk space : 756MB
Memory : 512MB
CPU : 30% system CPU
VCPUs : Not specified

If you have output similar to the above where show remote-managent status returns nothing and the Guest interface is missing and the Detailed guest status displays Information not available, then its likely something in the remote-mgmt virtual machine got corrupted and you need to uninstall and and re-install it. This is a harmless process. For me this happened because someone tried to take a live VMware snapshot of the CSR1000v and it crashed, corrupting configuration.

You should make sure that your remote-management and virtual-service csr_mgmt configuration are correct and then try to deactivate/reactivate the virtual-service before you uninstall just in case that fixes the issue. If you can reboot the device that is a good troubleshooting step as well.

remote-management
restful-api autosave 300
!
virtual-service csr_mgmt
ip shared host-interface GigabitEthernet1
activate

To deactivate, you run no activate. Then do activate to start the virtual-service again.

CSR1000V#conf t
CSR1000V#(config)#virtual-service csr_mgmt
CSR1000V#(config-virt-serv)#no activate
CSR100V#show virtual-service list
Virtual Service List:

Name Status Package Name
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
csr_mgmt Deactivated iosxe-remote-mgmt.03.16.01a.S.15…

If none of this works to fix your problem then lets uninstall and re-install the csr_mgmt virtual-service. You need to deactivate the service first. Then run the commands below.

CSR1000V#virtual-service uninstall name csr_mgmt
CSR1000V#show virtual-service list
Virtual Service List:

CSR1000V#show log | i VIRT
Feb 15 17:07:52.857: %VIRT_SERVICE-5-INSTALL_STATE: Successfully uninstalled virtual service csr_mgmt

CSR1000V#virtual-service install name csr_mgmt package flash:iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova
Installing package ‘bootflash:/iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova’ for virtual-service ‘csr_mgmt’. Once the install has finished, the VM may be activated. Use ‘show virtual-service list’ for progress.

CSR1000V1#show virtual-service list
Virtual Service List:

Name Status Package Name
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
csr_mgmt Installed iosxe-remote-mgmt.03.16.01a.S.15…

CSR1000V# show log | i VIRT
Feb 15 17:09:13.455: %VIRT_SERVICE-5-INSTALL_STATE: Successfully installed virtual service csr_mgmt

Now activate the virtual-service again and run show remote-managent status and show virtual-service detail. You should see the expected output and the RESTFUL API should now be working.

CSR1000v#show remote-management status
Remote management release version: 1.7.1

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Process Status Uptime # of restarts
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
nginx UP 0Y 0W 0D 0: 1:14 0
climgr UP 0Y 0W 0D 0: 1:14 0
restful_api UP 0Y 0W 0D 0: 1:14 0
fcgicpa Down
pnscag Down
pnscdme Down
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
Feature Status Configuration
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
Restful API Enabled, UP port: 55443
auto-save-timer: 300 seconds
socket: unix:/usr/local/nginx/csrapi-fcgi.sock;

PNSC Disabled, Down

Network stats:
eth0: RX packets:44, TX packets:6
eth1: RX packets:83, TX packets:65

Coredump file(s): lost+found

CSR1000V#show virtual-service detail
Virtual service csr_mgmt detail
State : Activated
Package information
Name : iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova
Path : bootflash:/iosxe-remote-mgmt.03.16.01a.S.155–3.S1a-ext.ova
Application
Name : csr_mgmt
Installed version : 1.7.1
Description : CSR-MGMT
Signing
Key type : Cisco development key
Method : SHA-1
Licensing
Name : Not Available
Version : Not Available

Detailed guest status

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Process Status Uptime # of restarts
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
nginx UP 0Y 0W 0D 0: 0: 3 0
climgr UP 0Y 0W 0D 0: 0: 3 0
restful_api UP 0Y 0W 0D 0: 0: 3 0
fcgicpa Down
pnscag Down
pnscdme Down
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
Feature Status Configuration
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 
Restful API Enabled, UP port: 55443
auto-save-timer: 300 seconds
socket: unix:/usr/local/nginx/csrapi-fcgi.sock;

PNSC Disabled, Down

Network stats:
eth0: RX packets:7, TX packets:4
eth1: RX packets:10, TX packets:8

Coredump file(s): lost+found

Activated profile name: None
Resource reservation
Disk : 756 MB
Memory : 512 MB
CPU : 30% system CPU

Attached devices
Type Name Alias
 — — — — — — — — — — — — — — — — — — — — — — -
NIC ieobc_1 ieobc
NIC dp_1_33 net2
Disk _rootfs
Disk /opt/var
Disk /opt/va…
Serial/shell serial0

Thanks to Cisco TAC for helping me with this issue. I had no idea about the uninstall/install commands.