Skip to main content
Engineering, Data / ML

Enabling Security for Hadoop Data Lake on Google Cloud Storage

30 July / Global
Featured image for Enabling Security for Hadoop Data Lake on Google Cloud Storage
Image
Figure 1: Cloud Data Lake IAM Systems compared to Existing and GCP Native Stack.
Image
Figure 2: Existing Architecture with HDFS.
Image
Figure 3: HDFS Authentication Flow.
Image
Figure 4: Layered Access Model
Image
Figure 5: Cloud Architecture.
Image
Figure 6: Cloud Authentication Flow.
Image
Figure 8: Multi Level Caching.
Image
Figure 9: Cache Time with respect to Access Token Lifetime.
Image
Figure 10: Average Access Time Calculation.
Image
Figure 11: RPS Served by SAS instances during Benchmarking.
Matt Mathew

Matt Mathew

Matt is a Sr. Staff Engineer on the Engineering Security team at Uber. He currently works on various projects in the security domain. Previously, he led the initiative to containerize and automate Data infrastructure at Uber.

Alexander Gulko

Alexander Gulko

Alexander is a Staff Software Engineer working on the Data Security team based in Seattle, WA. While the team is responsible for all aspects of data security and compliance, he primarily focuses on Authentication and leads various initiatives in that area.

Lei Sun

Lei Sun

Lei Sun is a Tech Lead Manager on the Technical Privacy team at Uber. His team builds highly scalable, highly reliable yet efficient infrastructure with innovative ideas to protect the privacy of customers and empower Uber engineers to seamlessly integrate security, compliance and privacy into their product development lifecycle.

KK Sriramadhesikan

KK Sriramadhesikan

KK Is a Sr Staff Security Engineer at Uber. KK secures Uber’s use of the cloud. He also works on a broad set of cross-functional initiatives on Security & Privacy at Uber.

Alan Cao

Alan Cao

Alan is a Staff Software Engineer on the Core Security Engineering team at Uber. He works on the unified authorization platform for Uber’s services and infrastructure.

Omkar Kakade

Omkar Kakade

Omkar is a Software Engineer working on the Data Security team at Uber. His expertise primarily lies in authorization, where he leads various initiatives to safeguard data integrity.

Posted by Matt Mathew, Alexander Gulko, Lei Sun, KK Sriramadhesikan, Alan Cao, Omkar Kakade