GCP

GCP - Google Cloud Platform. Sometimes, it is hard to find out what Google uses behind the marketing names. Here is a quick overview of my review on it. It may not be complete or may contain errors, but it helps me if someone comes up with ‘Dataflow’.

Compute Services

  • Compute Engine – Virtual machines (VMs) powered by KVM and Linux-based hypervisors.

  • Kubernetes Engine (GKE) – Managed Kubernetes (K8s) clusters using containerd runtime.

  • App Engine – Serverless PaaS based on Google Borg (the precursor to Kubernetes).

  • Cloud Functions – FaaS (Function-as-a-Service) built on Node.js, Python, Go, Java, and .NET runtime environments.

Storage & Databases

  • Cloud Storage – Distributed object storage using Colossus, GCP’s next-gen file system.

  • BigQuery – Serverless data warehouse powered by Dremel (Google’s columnar storage and query engine).

  • Cloud SQL – Managed PostgreSQL, MySQL, SQL Server using Google’s Spanner replication & HA techniques.

  • Firestore & Datastore – NoSQL databases using Firebase technology and Spanner for ACID compliance.

  • Cloud Spanner – Global-scale relational database using TrueTime API for strong consistency.

Networking

  • Cloud Load Balancing – Uses Maglev (Google’s high-performance load balancing tech).

  • VPC (Virtual Private Cloud) – Built on Andromeda, Google’s software-defined networking (SDN) stack.

  • Cloud CDN – Uses Google’s Edge Network and QUIC protocol for low-latency content delivery.

AI & Machine Learning

  • Vertex AI – AI/ML workflow orchestration using TensorFlow, PyTorch, and JAX.

  • Cloud AI APIs – Pre-trained ML models built on TPUs (Tensor Processing Units).

  • AutoML – Low-code ML based on Neural Architecture Search (NAS) and TPUs.

Security & Identity

  • Cloud IAM – Built on Google’s internal authorization framework (used in G-Suite).

  • Secret Manager – Encrypted storage using Google’s KMS (Key Management Service).

  • Security Command Center – Uses Google Chronicle security analytics and SIEM tools.

DevOps & Monitoring

  • Cloud Build – CI/CD using Buildpacks, Docker, and Kaniko for container builds.

  • Operations Suite (formerly Stackdriver) – Monitoring/logging powered by OpenTelemetry & Prometheus.

  • Artifact Registry – Secure container and package storage supporting Docker, npm, Maven.

Analytics & Data Processing

  • Dataflow – Serverless data processing using Apache Beam on Flink or Spark.

  • Dataproc – Managed Hadoop, Spark, Presto, Hive, and HBase clusters.

  • Pub/Sub – Real-time messaging built on Chubby lock service and Paxos consensus algorithm. (Is often compared to Kafka. While it is not built on Kafka, it serves a similar purpose for real-time messaging and event-driven architectures.)

Now: 2024/01/10