Retention not working correctly

0

I have set Kafka retention for 1 hour by the configurations per suggestions on every public doc. But I can't make it work correctly. I can make sense that it is trying to do as per my configurations, like when I set for 1 hour 60-70 percent data seems to get deleted and if I set it to 12 hours, again 60-70 percent get deleted. My requirement is, I want to delete all data and retention should delete all data as per the logic. Any ideas how can I make sure that there are no loose ends? All data should follow retention and not just some percent of it.

Dane
已提問 10 個月前檢視次數 186 次
1 個回答
0

If you're experiencing issues with Kafka retention not deleting all data as expected, there are a few things you can check to ensure there are no loose ends:

  • You may verify that the server/cluster level retention policy parameters are same as topic level configuration because if a topic is created with a different retention setting than the one you intend to use, Kafka will retain data based on the topic's retention policy instead of the server/cluster configuration.

  • Also make sure that the 'log.cleanup.policy' configuration is set to 'delete' (the default) rather than 'compact'. The 'compact' policy retains the latest value for each key and may interfere with data deletion.

If the above suggestions doesn't help you, in that case you may reach to Support team via support case with more details such as current configuration, cluster ARN, which topic's documents are not getting deleted etc. and they will be happy to guide you further.

AWS
支援工程師
已回答 10 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南