Troubleshooting VCF Automation Part 1: Log Locations

VCF Automation is the cornerstone component that provides advanced application automation and multi-tenancy capabilities within the VCF platform. The VCF-A system is deployed into a "VMSP" cluster. VMSP stands for VMware Management Services Platform. It is a Kubernetes-native control plane used in modern VMware Cloud Foundation (VCF) deployments to host and manage core private cloud automation and operational services.

Key Aspects of VMSP

• Function: VMSP runs as a cluster of containerized microservices (such as tenant managers, orchestrators, databases, and message queues) that power modern VMware private clouds.

• Architecture: It generally consists of one or more appliance nodes with significant resource requirements (e.g., 24 vCPUs and 96 GB RAM). It deploys onto the management network via Kubernetes namespaces.

• Day-2 Operations: It facilitates critical tasks like automated blueprinting, catalog services, identity management, and logging

As VCF-A is now containerized, there might be situations where you discover some issues with the system which require debugging and troubleshooting. In order to do this, we will need to access the VMSP cluster and run Kubernetes commands to investigate further.

Note: Never make configuration changes to any VCF system components unless explicitly directed to by Broadcom support or documentation.

1.) Accessing VMSP (VCF-A) Node:

ssh vmware-system-user@automation.customerdomain.local
##your_vmware-system-user_pw##
sudo su 

2.) 

Set Kubeconfig:

cd /home/vmware-system-user

export KUBECONFIG=/etc/kubernetes/admin.conf

3.) Set Alias:

alias k="kubectl"

4.) View microservices environment:

k get po -A 

VCF-A "System" Pods will be located within the "prelude" k8s namespace:

The pod name gives you an indication of what service it belongs to, also each of these pods maps to a folder for its logs within: 

/var/log/services-logs/prelude