Log Forwarding and Retention with vRealize Log Insight Cloud

Introduction

vRealize Log Insight Cloud (vRLIC) collects and analyzes logs generated in your SDDC. A trial version of the vRLIC service is enabled by default in a new SDDC. After the trial expires, only 1GB of daily log data is retained by the service. Any logs generated beyond that 1GB are lost unless you switch to the paid version.

Note: VMware has recently rebranded vRLIC to VMware Aria Operations for Logs, you may see this name referred to in some of the documentation. At the time of publication of this article, the VMware Cloud control panel still uses the name vRealize Log Insight Cloud

My customer had two requirements:

1. Configure log forwarding to their SIEM
2. Store as little data as possible in vRLIC

Log Forwarding

It is possible to configure log forwarding to a cloud destination as shown. This is how the UI looks at the time of this post - April 2023.

VMware wrote a detailed blog post several years ago on log forwarding. In this post the product is called Log Intelligence - same product, another different name. The instructions remain accurate, take a look if you're having trouble configuring log forwarding. It is not possible to forward logs without using vRLIC.

Log Retention

Logs ingested by vRLIC are written to a Log Partition. The default log partition has a retention period of 30 days, which cannot be changed. But you can create additional log partitions. Partitions can be indexed, which cost more but are faster to search. Or, they can be non-indexed, which cost less, are slower to search, and also carry a cost to perform searches. Click here for pricing details. I went with indexed to make writing this re:Post faster, but you can pick non-indexed to cut costs. In this scenario you're forwarding logs and don't intend to spend much time in the vRLIC console.

This screenshot shows the default partition as well as a new partition filtering for nsx. You can also consider more granular filtering criteria by looking at KB77537 and this document. The default partition shows a retention period of 30 days, and the new partition shows a retention period of 5 days.

A note on pricing: To be most cost-effective you will need to align your retention periods with VMware's pricing structure.

I created the new partition by clicking on New Partition

I entered a partition name, retention period, and filter.

Exploring Logs

Now when I explore the logs, I see all partitions by default.

I flip to the default partition

I search for nsx and get no results. This is exactly what I wanted, the NSX-related logs are no longer in the default partition!

I flip over to the index-1 partition and I can see all of the NSX logs. These will now be deleted after the 5 day retention period.