📋 Recommended Actions

⚠️ Action Required
Immediate action is required for users deploying Istio with Helm v4 (server-side apply) or those with newer Gateway API CRDs. Review the section on Helm failurePolicy to configure base.validationFailurePolicy: Fail as needed. Also, be aware of the new Gateway API CRD maximum version filter, which may ignore newer TLSRoute versions (v1.5.0+). All users should upgrade to benefit from critical security hardening and stability fixes.

📝 Summary

Istio 1.28.6 delivers essential stability fixes and security enhancements, streamlining operations for Gateway API users and Helm deployments. This release fortifies JWT validation with JWKS URI CIDR blocking, preventing key fetches from untrusted networks and bolstering your service mesh’s perimeter. For Helm users leveraging server-side apply, a crucial fix addresses a failurePolicy conflict during upgrades, ensuring smoother rollouts. The Ambient mesh sees a significant stability improvement by resolving a panic related to cross-network WorkloadEntries when multi-network support is disabled. We’ve also corrected the retryBudget default from a subtle 0.2% to its intended 20%, impacting traffic management reliability. Gateway API continues its evolution with enhanced CORS origin wildcard parsing and a fix for TLSRoute hostname constraint bypasses, ensuring consistent traffic behavior. This update also properly handles null resource limits in deployments, preventing validation errors. Review for an immediate upgrade to leverage these critical improvements.

🔐 Enhanced Security for JWT and Debug Endpoints

This release significantly hardens the security posture of your Istio mesh, particularly concerning JWT validation and access to debug endpoints. You can now prevent your Istio gateways and proxies from fetching JSON Web Key Sets (JWKS) from untrusted or internal IP ranges, addressing a potential supply chain risk. Additionally, finer-grained authorization controls are now available for Istiod’s debug endpoints, ensuring sensitive information remains protected. Critical fixes for webhook server timeouts and service account regex matching further enhance the overall security and reliability of your control plane components.

Istio now allows administrators to specify blocked CIDR ranges for JWKS URIs via the BLOCKED_CIDRS_IN_JWKS_URIS environment variable. If any IP resolved from a JWKS URI falls within these ranges, the public key fetch will be skipped, causing JWT tokens to be rejected. This is implemented with a custom DialContext that inspects the resolved IP before connection, ensuring redirects are also honored:

# Example: Block private IP ranges from JWKS fetches
export BLOCKED_CIDRS_IN_JWKS_URIS="10.0.0.0/8,192.168.0.0/16,172.16.0.0/12"

For improved control over Istiod’s debug endpoints, a new DEBUG_ENDPOINT_AUTH_ALLOWED_NAMESPACES environment variable lets you grant access to specific namespaces beyond the system namespace. This ensures that only authorized entities can access potentially sensitive runtime information:

# Allow 'dev-team' and 'ops-team' namespaces to access debug endpoints
export DEBUG_ENDPOINT_AUTH_ALLOWED_NAMESPACES="dev-team,ops-team"

Furthermore, missing ReadHeaderTimeout and IdleTimeout values on the Istiod webhook HTTPS server (port 15017) have been added, aligning with existing HTTP server timeouts and preventing potential resource exhaustion. The service account matcher regex used in AuthorizationPolicy has also been fixed to correctly quote special characters, preventing unintended matches for service accounts with complex names.

Source:

  • pilot/pkg/features/pilot.go (326-350, 102-108)
  • pilot/pkg/model/jwks_resolver.go (596-635)
  • releasenotes/notes/block_cidr_in_jwk_uris.yaml (0-8)
  • releasenotes/notes/jwks-uri-block-follows-redirect.yaml (0-9)
  • pilot/pkg/xds/debug.go (323-328)
  • releasenotes/notes/59238.yaml (0-9)
  • pilot/pkg/bootstrap/webhook.go (63-70)
  • releasenotes/notes/webhook-https-timeouts.yaml (0-7)
  • pilot/pkg/security/authz/model/generator.go (209-211)
  • releasenotes/notes/59700.yaml (0-8)

⚙️ Gateway API Refinements & Helm Deployment Stability

This release brings a suite of critical enhancements to Istio’s Gateway API implementation, ensuring more consistent and predictable behavior for your ingress and mesh gateways. Key improvements include robust handling of CORS wildcard origins, preventing unintended traffic patterns, and a fix for TLSRoute hostname constraints, bringing it fully in line with Gateway API specifications. For deployments, especially those using Helm with server-side apply, a significant update resolves failurePolicy ownership conflicts during upgrades. Furthermore, setting resource limits to null is now correctly interpreted, offering greater flexibility and preventing common validation errors in your deployment manifests.

CORS origin parsing for Gateway API HTTPRoutes has been significantly improved, now correctly interpreting wildcard patterns. Previously, only exact matches were fully supported, but now * or *.example.com patterns are properly translated into corresponding regex or prefix matches for flexible cross-origin resource sharing policies. For example, allowOrigins: ['https://*.xpto.com'] will now correctly match.

http:
  - match:
    - uri:
        prefix: /route-a
    route:
    - destination:
        host: app.com
        port:
          number: 80
      filters:
        - cors:
            allowOrigins:
            - 'https://*.xpto.com'
            allowMethods:
              - GET
            exposeHeaders:
            - "x-header-3"
            maxAge: 50
          type: CORS

A critical fix ensures that TLSRoute hostnames are properly constrained to the intersection with the Gateway listener hostname. This prevents scenarios where a broad *.com TLSRoute attached to a narrower *.example.com listener would incorrectly match the full *.com domain, instead of the specified *.example.com as required by the Gateway API specification.

For Helm users, particularly with server-side apply (Helm v4), a failurePolicy field ownership conflict on ValidatingWebhookConfiguration during helm upgrade has been resolved. This allows for smoother upgrades by omitting the failurePolicy from the webhook template during upgrades, preserving its runtime value. If you use helm template | kubectl apply --server-side, you can now explicitly set base.validationFailurePolicy: Fail to manage this behavior.

Resource limit definitions for proxies and gateways now correctly handle null values. Previously, setting cpu: null or memory: null could lead to validation errors. This update ensures that such values are properly omitted from the generated Kubernetes manifests, providing more flexibility in resource configuration:

global:
  proxy:
    resources:
      limits:
        cpu: null         # Will now be correctly omitted
        memory: 500Mi
      requests:
        cpu: null         # Will now be correctly omitted
        memory: 150Mi

Finally, Istiod will now filter out Gateway API CRDs whose bundle-version annotation is greater than the maximum supported version (e.g., TLSRoute v1.5.0+ will be ignored by this release), preventing startup errors when newer Gateway API CRDs are present in the cluster with an older Istio client.

Source:

  • pilot/pkg/config/kube/gateway/conversion.go (1233-1262, 405-430)
  • pilot/pkg/config/kube/gateway/testdata/http.yaml (337-353)
  • releasenotes/notes/59018.yaml (0-7)
  • releasenotes/notes/59026.yaml (0-6)
  • pilot/pkg/config/kube/gateway/route_collections.go (591-621)
  • releasenotes/notes/fix-tls-route-hostname-bypass.yaml (0-15)
  • manifests/charts/base/templates/defaultrevision-validatingwebhookconfiguration.yaml (46-51)
  • manifests/charts/istio-control/istio-discovery/templates/validatingwebhookconfiguration.yaml (46-51)
  • manifests/charts/istio-control/istio-discovery/values.yaml (551-557)
  • releasenotes/notes/58302.yaml (0-12)
  • releasenotes/notes/webhook-failurepolicy-upgrade.yaml (0-10)
  • pkg/kube/inject/template.go (410-460)
  • manifests/charts/gateway/templates/deployment.yaml (107-107)
  • releasenotes/notes/fix-null-resource-limits.yaml (0-8)
  • pkg/kube/kclient/crdwatcher.go (68-109)
  • releasenotes/notes/59443.yaml (0-8)
  • pilot/pkg/config/kube/gateway/deploymentcontroller.go (477-481)
  • releasenotes/notes/59709.yaml (0-8)

⚡ Core Stability & Ambient Mesh Fixes

This release delivers crucial stability and performance fixes across Istio’s core components, enhancing resilience and addressing specific operational challenges. A significant bug impacting Ambient mesh when handling cross-network WorkloadEntries has been resolved, preventing critical panics. The retryBudget default percentage for DestinationRule is now correctly set, aligning with its intended behavior and improving traffic management. Furthermore, fixes for CNI iptables lock contention and ztunnel’s HTTP/2 ping errors ensure smoother operation and reduce noisy logs, contributing to a more robust and reliable service mesh.

A critical issue causing Istiod to panic when Ambient mesh is enabled without multi-network support (AMBIENT_ENABLE_MULTI_NETWORK is false) and a WorkloadEntry from a different network exists has been fixed. This ensures stability for single-network Ambient deployments interacting with remote WorkloadEntries:

// Before fix: cross-network WE without multi-network enabled could cause panic
// After fix: WorkloadEntry precomputation is skipped for remote networks when coalescing is not enabled

The default percent value for retryBudget in DestinationRule has been corrected. It was previously set to 0.2 (0.2%), but the intended default is 20.0 (20%). This fix ensures that retry budgets behave as expected out-of-the-box, providing more effective traffic management for transient errors.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  trafficPolicy:
    connectionPool:
      http:
        http1MaxPendingRequests: 10
        # Now correctly defaults to 20% if not explicitly set
        retryBudget:
          minRetryConcurrency: 3

The Istio CNI plugin now handles iptables binary detection more robustly by incorporating the --wait flag when supported. This change improves lock contention management for xtables.lock, reducing misleading errors in logs and ensuring more reliable CNI operations. Additionally, a race condition that caused intermittent “proxy::h2 ping error: broken pipe” logs in ztunnel has been resolved, improving the stability of HTTP/2 connections.

Source:

  • pilot/pkg/serviceregistry/kube/controller/ambient/ambientindex_workloadentry_test.go (475-509)
  • pilot/pkg/serviceregistry/kube/controller/ambient/workloads.go (649-709)
  • releasenotes/notes/59321.yaml (0-11)
  • pilot/pkg/networking/core/cluster_traffic_policy.go (187-190)
  • releasenotes/notes/retry-budget-percent.yaml (0-9)
  • tools/istio-iptables/pkg/dependencies/implementation_linux.go (105-111)
  • releasenotes/notes/58507.yaml (0-9)
  • releasenotes/notes/ztunnel-h2-ping-broken-pipe.yaml (0-35)
  • pkg/wasm/httpfetcher.go (175-176)
  • releasenotes/notes/wasm-gzip-decompression-limit.yaml (0-8)

Minor Updates & Housekeeping

This release also includes general dependency updates, such as the BASE_VERSION bump and updates for istio.io/api, istio.io/client-go, proxy, and ztunnel sub-projects. The auto-registration retry assertion in tests was also fixed to reduce flakiness. Additionally, a size limit of 256MB was added for gzip-decompressed WASM binaries fetched over HTTP.