Questions tagged [hpa]
Horizontal Pod Autoscaler is an API resource in the Kubernetes and used to horizontally scale number of kubernetes pods based on metrics values
228
questions
20
votes
9
answers
63k
views
Metrics server is currently unable to handle the request
I am new to kubernetes and was trying to apply horizontal pod autoscaling to my existing application. and after following other stackoverflow details - got to know that I need to install metric-server ...
- 211
12
votes
2
answers
5k
views
Kubernetes Service not distributing the traffic evenly among pods
I am using Kubernetes v1.20.10 baremetal installation. It has one master node and 3 worker nodes. The application simply served HTTP requests.
I am scaling the deployment based on the (HPA) Horizontal ...
- 587
12
votes
2
answers
12k
views
Time-based scaling with Kubernetes CronJob: How to avoid deployments overriding minReplicas
I have a HorizontalPodAutoscalar to scale my pods based on CPU. The minReplicas here is set to 5:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-web
spec:
...
- 1,781
9
votes
2
answers
4k
views
GKE - HPA using custom metrics - unable to fetch metrics
I have custom metrics exported to Google Cloud Monitoring and i want to scale my deployment according to it.
This is my HPA:
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
...
- 2,949
8
votes
1
answer
11k
views
How to implement Kubernetes horizontal pod autoscaling with scale up/down policies?
Kubernetes v1.19 in AWS EKS
I'm trying to implement horizontal pod autoscaling in my EKS cluster, and am trying to mimic what we do now with ECS. With ECS, we do something similar to the following
...
- 15.5k
8
votes
1
answer
5k
views
Kubernetes HPA is flapping replicas regardless of stabilisation window
According to the K8s documentation, to avoid flapping of replicas property stabilizationWindowSeconds can be used
The stabilization window is used to restrict the flapping of replicas when the ...
7
votes
2
answers
2k
views
How Kubernetes computes CPU utilization for HPA?
I want to understand how HPA computes CPU utilization across Pods.
According to this doc it takes the average of CPU utilization of a pod (average across the last 1 minute) divided by the CPU ...
- 3,279
7
votes
1
answer
1k
views
How to prevent scale down of newly scaled up pod for specific period of time which was created by HPA in Kubernetes?
I have a Kubernetes cluster set up in DigitalOcean. The cluster is configured to auto-scale using HPA(Horizontal Pod Autoscaler). I want to prevent termination of a pod that got scaled up in the last ...
- 1,077
6
votes
3
answers
35k
views
Kubernetes HPA -- Unable to get metrics for resource memory: no metrics returned from resource metrics API
Good afternoon
I'm just starting with Kubernetes, and I'm working with HPA (HorizontalPodAutoscaler):
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: find-complementary-...
- 745
6
votes
2
answers
7k
views
HPA on AWS EKS with Fargate
I have AWS EKS cluster with only Fargate profile, no Node Groups.
Is it possible to enable HPA in this case? I tried to enable metric server as described here but pod creation fails with error
0/4 ...
- 377
6
votes
4
answers
4k
views
K8S Ingress: How to limit requests in flight per pod
I am porting an application to run within k8s. I have run into an issue with ingress. I am trying to find a way to limit the number of REST API requests in flight at any given time to each backend ...
- 3,234
5
votes
3
answers
5k
views
Kubernetes HPA disable scale down
By our product's design we would like to disable the scale down in HPA, can it be disabled?
- 159
5
votes
1
answer
994
views
Horizontal scaling based on the number of events published in NATS server
I am new to both Nats and Kubernetes.
I created a python console application that is subscribed to nats-server and wrapped in a pod with a Deployment object in Kubernetes. Now I want to auto-scale the ...
- 803
5
votes
1
answer
785
views
How to supply external metrics into HPA?
Problem setting. Suppose I have 2 pods, A and B. I want to be able to dynamically scale pod A based on some arbitrary number from some arbitrary source. Suppose that pod B is such a source: for ...
- 883
4
votes
2
answers
10k
views
How to get list of metrics available for HPA?
I have GCP cluster which contains GKE application:
I want to scale application using HPA
Based on supporting for metrics
HPA able to read metrics from
metrics.k8s.io (resource metrics)
custom....
- 37k
4
votes
1
answer
3k
views
Kubernetes HPA doesn't scale down after decreasing the loads
the Kubernetes HPA works correctly when load of the pod increased but after the load decreased, the scale of deployment doesn't change. This is my HPA file:
apiVersion: autoscaling/v2beta2
kind: ...
4
votes
1
answer
3k
views
Kubernetes enabling or disabling Keda and HPA
Is there a 'master switch' to enable/disable Keda and HPA?
I can enable/disable scaling rules by editing the replicacount to 0, but is there a main enalbe/disable field?
cooldownPeriod: 1800
...
- 419
4
votes
1
answer
4k
views
Error scaling up in HPA in GKE: apiserver was unable to write a JSON response: http2: stream closed
Following the guide that google made for deploying an HPA in Google Kubernetes Engine: https://cloud.google.com/kubernetes-engine/docs/tutorials/autoscaling-metrics
And adding the right permissions ...
- 372
4
votes
3
answers
13k
views
Unable to fetch metrics from custom metrics API: the server is currently unable to handle the request
I'm using a HPA based on a custom metric on GKE.
The HPA is not working and it's showing me this error log:
unable to fetch metrics from custom metrics API: the server is currently unable to handle ...
4
votes
1
answer
3k
views
Kubernates autoscale memory
I have a java application running on K8s with min:2 and max:6 pods in a deployment.
heap min:256Mb, heap max: 512Mb. request and limit memory is 1Gi
Here is the hpa spec:
spec:
scaleTargetRef:
...
- 493
4
votes
1
answer
2k
views
Enforced scaled to zero with Keda
With plain HPAs it is possible to scale the underlying deployment manually to zero. This is required for eg. maintainance tasks. When scaling back to a value greater zero, the scaling continues as ...
- 10.3k
3
votes
2
answers
7k
views
Kubernetes HPA - Scale up cooldown
I am running a Kubernetes cluster v1.16(currently newest version on GKE) with HPA that scales the deployments base on custom metrics(Specifically rabbitmq messages count fetched from google cloud ...
- 2,949
3
votes
1
answer
3k
views
Is it ok to use Kubernetes autoscaling v2beta2 in production as apiVersion?
I have use case to scale pods (HPA) based on memory (and custom metrics).
However respective apiVersion is in beta (autoscaling/v2beta2).
Not sure when this will be in GA/stable version, hence ...
3
votes
1
answer
745
views
How to view HPA autoscaling definition in GKE console
Using GKE version - 1.18.17-gke.700
I am looking to find an option inside GKE console, where I can find the information of Pods HPA autoscalling information?
We are able to get the HPA info using &...
- 55
3
votes
1
answer
3k
views
Kubernets update minReplicas and maxReplica
I am a newbie on Ops and need to update through Lens the HPA configuration like:
From:
minReplicas: 6
maxReplicas: 10
To:
minReplicas: 4
maxReplicas: 16
My doubt is if the PODs will be ...
- 817
3
votes
2
answers
4k
views
can't get custom metrics for hpa from datadog
hey guys i’m trying to setup datadog as custom metric for my kubernetes hpa using the official guide:
https://docs.datadoghq.com/agent/cluster_agent/external_metrics/?tab=helm
running on EKS 1.18 &...
- 65
3
votes
1
answer
1k
views
How to autoscale Kubernetes Pods based on average memory usage in EKS?
I am running an EKS cluster and I have a HorizontalPodAutoscaler created for autoscaling number of pods based on average CPU utilisation.
How to do the same for Average memory utilization?
Suppose ...
- 33
3
votes
1
answer
2k
views
HPA Keep the oldest pod when scaling down
When the HPA scales up it creates new pods, when it scales down those new pods are terminated and HPA keeps the oldest ones. Can I configure something to say to HPA to terminate the oldest pods and ...
3
votes
2
answers
4k
views
Is it possible to use Kubernetes autoscale on cron job pods
Some context: I have multiple cron jobs running daily, weekly, hourly and some of which require significant processing power.
I would like to add requests and limitations to these container cron pods ...
- 31
3
votes
1
answer
2k
views
HorizontalPodAutoscaler Kubernetes does not keep the minimum number of replicas [closed]
Good afternoon. I am working with HPA (HorizontalPodAutoscaler) for the automatic scaling of replicas of a pods, in this case I am using memory usage as a reference, I declare it as follows:
...
- 745
3
votes
1
answer
905
views
Kubernetes AWS Cloudwatch adapter not fetching custom metric value for EKS HPA autoscaling
I'm trying to enable AWS EKS autoscaling based on a custom Cloudwatch metric via the Kubernetes Cloudwatch adapter. I have pushed custom metrics to AWS Cloudwatch, and validated they appear in ...
- 33
3
votes
2
answers
2k
views
Can anyone explain this Kubernetes HPA behavior?
So this is happening on EKS K8s v1.15. You can see the api version in the describe output. The millicpu hovers between 80 and 120... which does not at ALL match the replica counts coming out of the ...
- 1,010
3
votes
0
answers
5k
views
HorizontalPodAutoscaler scaleDown behavior
We are using v1.18 and ArgoCD.
Desired Behavior: scale down by 1 pod at a time every 5 minutes when usage under 50%
The HPA scales up and down perfectly using default spec.
When we add the custom ...
- 31
3
votes
0
answers
1k
views
Horizontal pod autoscaler not able to get metrics in minikube deployment
This is an open issue: https://github.com/kubernetes/minikube/issues/9370
Steps to reproduce:
$ minikube start — extra-config=controller-manager.horizontal-pod-autoscaler-upscale-delay=1m — extra-...
2
votes
2
answers
7k
views
HPA Scaling even though Current CPU is below Target CPU
I am playing around with the Horizontal Pod Autoscaler in Kubernetes. I've set the HPA to start up new instances once the average CPU Utilization passes 35%. However this does not seem to work as ...
- 147
2
votes
2
answers
11k
views
How to make k8s cpu and memory HPA work together?
I'm using a k8s HPA template for CPU and memory like below:
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: {{.Chart.Name}}-cpu
labels:
app: {{.Chart.Name}}
...
- 51
2
votes
1
answer
1k
views
HPA on EKS-Fargate
this is not a question about how to implement HPA on a EKS cluster running Fargate pods... It´s about if it is necessary to implement HPA along with Fargate, because as far as I know, Fargate is a &...
- 121
2
votes
2
answers
2k
views
the server could not find the metric nginx_vts_server_requests_per_second for pods
I installed the kube-prometheus-0.9.0, and want to deploy a sample application on which to test the Prometheus metrics autoscaling, with the following resource manifest file: (hpa-prome-demo.yaml)
...
- 137
2
votes
1
answer
3k
views
How memory metric is evaluated by Kubernetes HPA
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: backend-hpa
spec:
maxReplicas: 10
minReplicas: 3
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
...
- 9,828
2
votes
1
answer
816
views
k8s HorizontalPodAutoscaler - set target on limit, not request
I have a deployment where cpu request is 500m and cpu limit is 1000m.
I create a hpa as -
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
...
2
votes
1
answer
1k
views
Kubernetes Prometheus metric for HPA (horizontal pod autoscaler) `currentCPUUtilizationPercentage`?
On a Kubernetes cluster with CoreOS Prometheus Operator scraping all standard cluster metrics, what Prometheus metric would show me the currentCPUUtilizationPercentage value for a simple HPA (...
- 19.3k
2
votes
1
answer
546
views
Kubernetes hpa cpu utilization
I found something wrong at HPA for istio gateway.
Why did 10m equal 10%? Wasn't 10m 1%?
Kubernetes version is 1.18.5.
# kubectl get hpa --all-namespaces
NAMESPACE NAME REFERENCE ...
- 155
2
votes
1
answer
540
views
Horizontal pod autoscaling using a logging custom metric in GKE
I would like to scale my deployment based on a custom logging metric, but I'm not able to make that work, I created already the custom metric and I'm also able to see it in the metric explorer but for ...
2
votes
1
answer
1k
views
How to track number of changes for Kubernetes HPA?
Concerning the Kubernetes Horizontal Autoscaler, are there any metrics related to the number of changes between certain time periods?
- 87
2
votes
1
answer
5k
views
getting error while implementing metric server inside the Kubernetes
"Failed to scrape node" `
err="GET \"https://10.128.0.17:10250/stats/summary?
only_cpu_and_memory=true\": bad status code \"403 Forbidden\""
node="gke-...
- 21
2
votes
1
answer
438
views
Kubernetes autoscaling : HPA not working with custom metrics for Java Netty API
I am setting up HPA on custom metrics - basically on no. of threads of a deployment.
I have created a PrometheusRule to get average of threads (5 min. based). On the container, I am doing cont. load ...
- 4,187
2
votes
1
answer
1k
views
How to use K8S HPA and autoscaler when Pods normally need low CPU but periodically scale
I am trying to determine a reliable setup to use with K8S to scale one of my deployments using an HPA and an autoscaler. I want to minimize the amount of resources overcommitted but allow it to scale ...
- 3,234
2
votes
2
answers
2k
views
Kubernetes metrics-server doesn't provide all metrics or scale HPA
Following the example here https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-metrics-not-related-to-kubernetes-objects, I have created installed ...
- 61
2
votes
1
answer
4k
views
HPA says FailedComputeMetricsReplicas
I have deployed an app and exposed it as a loadbalancer service. I added the resource field in the yaml of the deployment to request for 100m cpu. Defined a HPA to scale the app when the CPU goes ...
- 21
2
votes
1
answer
577
views
Kubernetes scaling up not fast enough (for K8 1.17 or below)
Kubernetes 1.18 has a very nice ability to change auto-scaling going up through the behavior and scaleUp set. For 1.17, there's a way to control downscaling. Is there a parallel way to scale up faster?...
- 21