Skip to main content
Engineering, Data / ML

Enabling Infinite Retention for Upsert Tables in Apache Pinot

1 October / Global
Featured image for Enabling Infinite Retention for Upsert Tables in Apache Pinot
Image
Figure 1: High-level architecture of upsert in Pinot.
Image
Figure 2: Example explaining the low-level architecture of upsert.
Image
Figure 3: Configuration to enable point-deletes.
Image
Figure 4: Architecture of the point deletes feature.
Image
Figure 5: Configuration to enable retention of metadata of deleted keys.
Image
Figure 6: Low-level architecture of metadata retention of deleted keys.
Image
Figure 7: Pseudo-code for deleted keys TTL flow.
Image
Figure 8: Deleted keys TTL feature impact in production.
Image
Figure 9: Low-level architecture of upsert compaction. 
Image
Figure 10: Impact of upsert compaction task at production scale.
Pratik Tibrewal

Pratik Tibrewal

Pratik Tibrewal is a Sr. Software Engineer at Uber.

Posted by Pratik Tibrewal