> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Service limits remediation

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Using Console" defaultOpen="true">
        Here’s how to remediate GCP IAM “Service Limits” (quota) issues using the GCP Console, focusing on IAM-related quotas (service accounts, roles, policy bindings, etc.).

        ***

        ## 1. Identify Which IAM Limit You’re Hitting

        1. Go to **IAM & Admin → Quotas** in the GCP Console
           * URL: `https://console.cloud.google.com/iam-admin/quotas`
        2. In the **Services** filter, select:
           * **IAM Service Account Credentials API**, **Identity and Access Management (IAM) API**, and/or **Cloud Resource Manager API**, depending on what you’re working with.
        3. Use the **Metric** filter to locate relevant quotas, for example:
           * *Service accounts per project*
           * *Custom roles per organization/project*
           * *Policy size / policy bindings per resource*
           * *API requests per minute/hour*
        4. Check which quota shows **Usage** close to or at **Limit** or is marked as **Blocked**.

        ***

        ## 2. Short-Term Remediation: Reduce Usage (Clean Up IAM Resources)

        Depending on which limit is impacted:

        ### A. If you hit **service account** limits

        1. Go to **IAM & Admin → Service Accounts**
           * `https://console.cloud.google.com/iam-admin/serviceaccounts`
        2. Sort or filter by **Last used** (if available) or by **Name/Description** to identify unused accounts.
        3. For each unused service account:
           * Click the **three dots (⋮)** → **Delete**.
        4. If apps can share accounts:
           * Consolidate multiple similar service accounts into fewer, well-scoped service accounts.
           * Update workloads (GCE, GKE, Cloud Run, etc.) to use shared accounts with least-privilege roles.

        ### B. If you hit **custom role** limits

        1. Go to **IAM & Admin → Roles**
           * `https://console.cloud.google.com/iam-admin/roles`
        2. Filter for **Custom** roles.
        3. Identify:
           * Roles that are **deprecated**, not used, or duplicate.
        4. For unused custom roles:
           * Click the role → **Delete** (or **Disable** first if you want a safe test).
        5. Merge overlapping custom roles where possible so fewer custom roles are needed.

        ### C. If you hit **policy size / bindings** limits

        1. Go to **IAM & Admin → IAM**
           * `https://console.cloud.google.com/iam-admin/iam`
        2. Look for:
           * Members with many roles, or roles with many members.
        3. Reduce bindings:
           * Replace many individual users with **Google Groups** and assign roles to the group instead.
           * Remove obsolete members or roles that are no longer required.
           * Prefer predefined roles instead of several overlapping custom roles.

        ### D. If you hit **IAM API request rate** limits

        1. Reduce automation chattiness:
           * Batch IAM changes where possible.
           * Avoid scripts that repeatedly call `setIamPolicy` / `getIamPolicy` in tight loops.
        2. Space out calls:
           * Implement backoff and retry in tooling, if you control it.

        ***

        ## 3. Long-Term Remediation: Request a Quota Increase

        1. Go to **IAM & Admin → Quotas**
           * `https://console.cloud.google.com/iam-admin/quotas`
        2. Filter by **Service** and **Metric** to locate the specific quota you’re exceeding.
        3. Check the box next to the quota you want to increase.
        4. At the top, click **EDIT QUOTAS**.
        5. In the side panel:
           * Select a **Support case** channel (you may need a support plan).
           * Fill in:
             * **New limit** you’re requesting.
             * **Justification** (describe workloads, growth, why cleanup alone isn’t enough).
        6. Submit the request and monitor the status in your **Support** section.

        ***

        ## 4. Validate After Remediation

        1. Re-run the action that previously failed (e.g., create service account, add IAM binding).
        2. Revisit **IAM & Admin → Quotas** to confirm:
           * Usage is below limit, or
           * New higher limit is applied.
        3. Optionally, set up internal processes:
           * Periodic review of service accounts, roles, and bindings.
           * Use **Groups** and standardized roles to control IAM sprawl.

        If you share the exact quota metric you’re hitting (e.g., “Service accounts per project” or “Policy size”), I can give exact click-by-click steps tailored to that limit.
      </Accordion>

      <Accordion title="Using CLI">
        For IAM on GCP, “Service Limits” (quotas) can’t actually be raised directly via gcloud; increases are requested via the web console or a support case. With the CLI you can:

        1. Identify what limit you’re hitting
        2. Reduce usage (cleanup / refactor IAM)
        3. Then formally request a quota increase in the console

        Below are the concrete CLI steps to diagnose and remediate IAM-related service limits.

        ***

        ## 1. Identify which IAM limit you’re hitting

        Common IAM limits include:

        * Max bindings per policy (e.g., 1,500 bindings on a project)
        * Max roles per principal
        * Max custom roles per project/org
        * Permissions per custom role

        Use logs or error messages from failed commands (such as `gcloud projects add-iam-policy-binding`, `gcloud iam roles create`, etc.) and note the exact error text, e.g.:

        * `RESOURCE_EXHAUSTED`
        * `Quota exceeded`
        * `exceeds the maximum number of bindings`
        * `Maximum number of custom roles reached`

        There isn’t a direct `gcloud` command to show IAM policy limits, but you can:

        ```bash theme={null}
        # Show current project IAM policy size (bindings count, etc.)
        gcloud projects get-iam-policy PROJECT_ID --format=json > policy.json

        # Count bindings
        jq '.bindings | length' policy.json
        ```

        If the bindings count is near/above 1500, you’re hitting the IAM policy bindings limit.

        ***

        ## 2. Remediate by reducing IAM policy size (clean up via CLI)

        ### 2.1 List IAM policy bindings

        ```bash theme={null}
        gcloud projects get-iam-policy PROJECT_ID \
          --format="table(bindings.role, bindings.members)"
        ```

        This lets you see:

        * Roles that are overused or duplicated
        * Direct user/service account bindings that can be replaced by groups

        ### 2.2 Remove unused or redundant bindings

        1. Get current policy:

        ```bash theme={null}
        gcloud projects get-iam-policy PROJECT_ID --format=json > policy.json
        ```

        2. Edit `policy.json`:
           * Remove bindings you no longer need.
           * Consolidate multiple bindings where possible (e.g., same role with multiple members → one binding with combined members).

        Example binding to remove (in `policy.json`):

        ```json theme={null}
        {
          "role": "roles/viewer",
          "members": [
            "user:olduser@example.com"
          ]
        }
        ```

        Delete that block or remove individual members.

        3. Set the updated policy:

        ```bash theme={null}
        gcloud projects set-iam-policy PROJECT_ID policy.json
        ```

        Repeat the same pattern for:

        * `gcloud organizations get-iam-policy` / `set-iam-policy`
        * `gcloud folders get-iam-policy` / `set-iam-policy`

        ### 2.3 Replace individual bindings with group-based bindings

        1. Create or use an existing Google Group in your org (done in Admin Console, not gcloud).
        2. Replace many individual user bindings with a single group binding.

        Example: Replace:

        ```bash theme={null}
        gcloud projects add-iam-policy-binding PROJECT_ID \
          --member="user:alice@example.com" \
          --role="roles/viewer"

        gcloud projects add-iam-policy-binding PROJECT_ID \
          --member="user:bob@example.com" \
          --role="roles/viewer"
        ```

        With one group binding:

        ```bash theme={null}
        gcloud projects add-iam-policy-binding PROJECT_ID \
          --member="group:viewers@example.com" \
          --role="roles/viewer"
        ```

        Then remove the per-user bindings as in 2.2.

        ### 2.4 Use custom roles to reduce per-binding permission sprawl

        If you’re hitting “permissions per custom role” or similar:

        1. Inspect a custom role:

        ```bash theme={null}
        gcloud iam roles describe ROLE_ID --project=PROJECT_ID
        ```

        2. Reduce permissions if there are many rarely-used ones:

        ```bash theme={null}
        gcloud iam roles update ROLE_ID \
          --project=PROJECT_ID \
          --permissions=perm1,perm2,perm3
        ```

        3. If you have many near-identical custom roles, consolidate them into fewer roles with shared permissions.

        ***

        ## 3. Remediate “too many custom roles” limit

        If error mentions maximum custom roles per project/org:

        1. List all custom roles:

        ```bash theme={null}
        # Project-level
        gcloud iam roles list --project=PROJECT_ID --filter="stage=GA"

        # Org-level
        gcloud iam roles list --organization=ORG_ID --filter="stage=GA"
        ```

        2. Identify unused ones (for example, by last used in your own CMDB / internal docs; GCP doesn’t give per-role usage counts directly via gcloud).

        3. Disable roles you no longer need:

        ```bash theme={null}
        gcloud iam roles update ROLE_ID \
          --project=PROJECT_ID \
          --stage=DISABLED
        ```

        4. After validation, delete them:

        ```bash theme={null}
        gcloud iam roles delete ROLE_ID --project=PROJECT_ID
        ```

        Repeat at org level with `--organization=ORG_ID`.

        ***

        ## 4. Request a quota increase (cannot be done via gcloud)

        Once you’ve optimized, if you still hit limits you must request a quota increase:

        1. Go to:\
           [https://console.cloud.google.com/iam-admin/quotas](https://console.cloud.google.com/iam-admin/quotas)

        2. Filter for:
           * “IAM”
           * Or the specific quota mentioned in your error (e.g., “Policy size”, “Custom roles per project”).

        3. Select the quota → “EDIT QUOTAS” → fill out:
           * Project
           * Contact details
           * New requested limit
           * Justification

        4. Submit and wait for approval.

        If you have a support plan, you can also open a support case and reference the error and project ID.

        ***

        ### Summary (CLI-only actions)

        * Use `gcloud projects/organizations/folders get-iam-policy` to inspect policy size and bindings.
        * Clean up and consolidate IAM bindings by editing JSON and applying via `set-iam-policy`.
        * Use group-based IAM instead of many per-user bindings.
        * Consolidate and clean up custom roles via `gcloud iam roles list/update/delete`.
        * Any actual increase in the service limit must be requested via the GCP Console or support, not gcloud.
      </Accordion>

      <Accordion title="Using Python">
        “Service limits” for GCP IAM usually means you’re hitting or about to hit hard limits like:

        * Max role bindings per policy (e.g., 1,500 bindings)
        * Max members per binding
        * API rate limits / quotas for `iam.googleapis.com`

        You can’t change these limits with code, but you *can* remediate by:

        1. detecting when you’re close to limits
        2. cleaning up unused / duplicate IAM bindings
        3. batching / optimizing calls to stay under API quota

        Below is a practical, step‑by‑step approach using Python.

        ***

        ## 1. Set up Python environment

        ```bash theme={null}
        pip install google-api-python-client google-auth google-auth-httplib2 google-cloud-monitoring
        gcloud auth application-default login
        ```

        Make sure your ADC (Application Default Credentials) has permission:

        * `roles/resourcemanager.projectIamAdmin` for modifying project IAM
        * `roles/monitoring.viewer` (optional for monitoring quotas)

        ***

        ## 2. Detect IAM policy “bloat” (bindings/members near limits)

        Example: check all IAM bindings on a project and warn if near a threshold.

        ```python theme={null}
        from google.oauth2 import service_account
        from googleapiclient import discovery
        import google.auth

        # Use ADC
        credentials, project_id = google.auth.default()

        service = discovery.build('cloudresourcemanager', 'v1', credentials=credentials)

        def get_iam_policy(project_id):
            request = service.projects().getIamPolicy(
                resource=project_id,
                body={"options": {"requestedPolicyVersion": 3}}
            )
            return request.execute()

        def analyze_policy(policy, binding_threshold=1200, members_threshold=100):
            bindings = policy.get("bindings", [])
            num_bindings = len(bindings)
            print(f"Total bindings: {num_bindings}")

            if num_bindings > binding_threshold:
                print(f"WARNING: Total bindings ({num_bindings}) close to limit (e.g. 1500).")

            for b in bindings:
                role = b.get("role")
                members = b.get("members", [])
                if len(members) > members_threshold:
                    print(f"Role {role} has {len(members)} members (high, may hit member limits).")

        policy = get_iam_policy(project_id)
        analyze_policy(policy)
        ```

        This doesn’t fix anything yet; it just tells you where you’re likely to hit service limits.

        ***

        ## 3. Remediate by cleaning up IAM bindings

        Typical cleanups:

        * Remove members that no longer exist (users, service accounts)
        * Remove duplicate members
        * Remove roles that are no longer needed

        Below is a pattern to:

        * remove specific members from specific roles
        * then write back the cleaned policy

        ```python theme={null}
        from googleapiclient.errors import HttpError

        def remove_members_from_roles(policy, members_to_remove_by_role):
            """
            members_to_remove_by_role example:
            {
              "roles/editor": [
                "user:old.user@example.com",
                "serviceAccount:old-sa@PROJECT_ID.iam.gserviceaccount.com",
              ]
            }
            """
            new_bindings = []

            for b in policy.get("bindings", []):
                role = b["role"]
                members = b.get("members", [])

                if role in members_to_remove_by_role:
                    to_remove = set(members_to_remove_by_role[role])
                    new_members = [m for m in members if m not in to_remove]
                    if new_members:
                        new_bindings.append({"role": role, "members": new_members})
                    else:
                        # Entire binding removed (no members left)
                        continue
                else:
                    new_bindings.append(b)

            policy["bindings"] = new_bindings
            return policy

        def set_iam_policy(project_id, policy):
            request = service.projects().setIamPolicy(
                resource=project_id,
                body={"policy": policy}
            )
            return request.execute()

        # EXAMPLE USAGE:
        members_to_remove_by_role = {
            "roles/editor": [
                "user:old.user@example.com",
                "serviceAccount:old-sa@%s.iam.gserviceaccount.com" % project_id
            ]
        }

        policy = get_iam_policy(project_id)
        clean_policy = remove_members_from_roles(policy, members_to_remove_by_role)

        try:
            updated = set_iam_policy(project_id, clean_policy)
            print("Updated IAM policy written.")
        except HttpError as e:
            print(f"Error updating IAM policy: {e}")
        ```

        You can extend this logic to:

        * drop bindings where the role is no longer used in your org
        * move many individual users to a Google group and grant the role to the group instead (reduces members per binding)

        ***

        ## 4. Avoid IAM API quota limits (rate limits)

        If your scanner / automation is hitting IAM rate limits, you need to:

        1. Batch operations when possible
        2. Use exponential backoff on `429` / `503` responses

        Example of a simple exponential backoff wrapper:

        ```python theme={null}
        import time
        from googleapiclient.errors import HttpError

        def with_backoff(callable_fn, max_retries=5, initial_delay=1.0, multiplier=2.0):
            delay = initial_delay
            for i in range(max_retries):
                try:
                    return callable_fn()
                except HttpError as e:
                    if e.resp.status in (429, 500, 503):
                        if i == max_retries - 1:
                            raise
                        time.sleep(delay)
                        delay *= multiplier
                    else:
                        raise

        # Example: safe getIamPolicy
        def safe_get_iam_policy(project_id):
            return with_backoff(
                lambda: service.projects().getIamPolicy(
                    resource=project_id,
                    body={"options": {"requestedPolicyVersion": 3}}
                ).execute()
            )
        ```

        ***

        ## 5. Monitor IAM quotas via Cloud Monitoring (optional)

        You can track metrics like `iam.googleapis.com/quota/allocation/usage` and alert when near quota.

        Basic example to list IAM quota metrics:

        ```python theme={null}
        from google.cloud import monitoring_v3

        client = monitoring_v3.MetricServiceClient()
        project_name = f"projects/{project_id}"

        for descriptor in client.list_metric_descriptors(name=project_name):
            if "iam.googleapis.com" in descriptor.type:
                print(descriptor.type)
        ```

        You would typically configure alerts in the console, but you can automate that with the Monitoring API if desired.

        ***

        ## 6. When you truly hit a hard service limit

        * If you hit **hard IAM structural limits** (e.g., max bindings per policy):
          * Consolidate: use groups instead of many individual members
          * Split: move resources into additional projects / folders so policy is distributed
        * If you hit **quota limits** (API calls, QPS):
          * Optimize code (batch, cache, backoff)
          * Then request a quota increase in the console (IAM & Admin → Quotas), which cannot be done purely via Python.

        ***

        If you can share:

        * which exact IAM service limit you’re hitting (bindings, members, API QPS, etc.)
          I can give you a more targeted Python snippet tailored to that specific limit.
      </Accordion>

      <Accordion title="Using Terraform">
        ```hcl theme={null}
        # There is no Terraform resource that changes IAM-related service limits (quotas) in GCP.
        # Quota increases must be requested via the Google Cloud Console or gcloud, not Terraform.

        # Example: you can define IAM policies and bindings in Terraform, but not the IAM service limits.

        resource "google_project_iam_binding" "example" {
          project = "YOUR_PROJECT_ID"   # Replace with your GCP project ID
          role    = "roles/viewer"

          members = [
            "user:USER_EMAIL@example.com",  # Replace with the user/service account/etc.
          ]
        }
        ```

        GCP IAM service limits (e.g., number of roles per principal, policy size, etc.) are enforced by Google and are not configurable via Terraform or the Google provider; to remediate limit issues you must request a quota/limit increase in the Cloud Console (IAM & Admin → Quotas) or via `gcloud services quotas` where available. `terraform plan` cannot show a quota change because it is outside Terraform’s control.
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>