Google Vertex AI SDK Flaw Let Attackers Hijack Model Uploads via Bucket Squatting

A vulnerability in the Google Cloud Vertex AI SDK for Python was discovered, allowing attackers to hijack machine learning model uploads and execute code within Google's infrastructure. This flaw, identified by Palo Alto Networks Unit 42, was dubbed "Pickle in the Middle." Although there have been no reports of this vulnerability being exploited in the wild, Google has since patched the issue. Users of the SDK are advised to update to version 1.148.0 or later to ensure their systems are secure. The attack required minimal resources from the attacker, who only needed a Google Cloud project and the victim's project ID, which is often publicly accessible. Notably, the attacker did not need any credentials or to engage in phishing to exploit this flaw. The vulnerability stemmed from the SDK's method of selecting a temporary Cloud Storage bucket for model uploads, which could be easily predicted and manipulated by an attacker. The SDK generated a predictable bucket name using the project ID and region, such as project-vertex-staging-region. While it checked for the existence of the bucket, it did not verify ownership, allowing attackers to create the bucket first in their own project. Consequently, the victim's model files could be uploaded to the attacker's bucket, where they could be replaced with a malicious model. This vulnerability was particularly concerning because many Python machine learning models are saved using pickle or joblib, which can execute code when loaded. Once the malicious model was loaded by Vertex AI, the attacker's code would run within the serving container. The attack's success hinged on speed, with Unit 42 noting a 2.5-second window between the victim's upload and Vertex AI reading the file. In a proof of concept, the attacker used a Cloud Function to replace the model within 1.4 seconds. The malicious payload was capable of stealing an OAuth token from the serving container's metadata server, which could then be used to access other model artifacts and sensitive data within the Google-managed tenant project. This included access to a full TensorFlow model, BigQuery metadata, access lists, tenant logs, GKE cluster names, and internal container image paths. The attack was only feasible under specific conditions: the victim's default staging bucket did not exist in the region, and the staging_bucket parameter was left unset. These conditions are common for new projects in Vertex AI, particularly when developers rely on the SDK's default settings instead of specifying their own bucket. Unit 42 reported the vulnerability to Google on March 5, 2026, after testing versions 1.139.0 and 1.140.0, both of which were vulnerable. Google initially addressed the issue in version 1.144.0 by adding a random uuid4 to the bucket name and completed the fix in version 1.148.0 with bucket ownership verification. Users are advised to update to the latest version and explicitly set a staging_bucket to a controlled Cloud Storage location. This incident marks the second predictable-bucket-name flaw in Vertex AI this year, following a similar issue patched in February. Unit 42's previous research on Vertex AI's service-agent permissions highlighted potential paths for unauthorized access to customer and tenant data, underscoring the importance of robust security measures in cloud-based AI services.