Hello! It’s nice to meet you. We’re Donis and Riley, who oversee the server development inArbeon!

Before we get into the first-ever Tech article by us, the Server team, we would like to introduce you briefly to what we do in the company.

As you can guess from the name “Server,” the word is a compound word of “serve” (action) and “-er” (doer). That is, it means “the person who provides the service.” We, the Server team, create a quick, stable, and expandable environment in advance, predict possible risks and errors in the service, and minimize them in case those issues occur. The stable operation of the Arbeon service is in the hands of the Server team!

That was quite a long introduction, wasn’t it? Wait no more. Let’s get started with our first article right away!

gRPC Load Test by Locust with k8s

Searching for the faster and more stable communication

At the moment, Arbeon’s server application is developed using the go language and composed of the microservice that uses gRPC communication protocol. The Rest API communication that uses the Nest.js framework was previously used to communicate between the clients and the server. But gRPC was selected as the final choice as it ensures faster and more stable communication. As a result, we were able to achieve speed three times faster than the Rest API.

So now, let’s look at how we can load test using the gRPC!

Load Test 

Load test refers to the test that checks the system function and predicts the possible issues by simulating several users and accessing a certain program or app at the same time. The test allows us to increase the load to the set value or threshold to compare the functions of various systems, or accurately measure the function of a single system. Moreover, it is done to measure how well it does in running the actual system.

Load Test Checklist 

• How much user access can it hold?
• Are there any issues with the expandability when the acceptance range is exceeded?
• Is the current infrastructure enough?
• Can the maximum DB and network workload, which are related to the server application, be determined and expanded in the future?
• Are there any errors and issues like memory leak?

Looking into the Tools for the Load Test 

Commonly, in most services, things may go smoothly in the developing environment until the service is run in the actual environment. That’s when unexpected issues and exemptions start to occur. And so, it’s nearly impossible to run a test that predicts every single error and exemption during the developing and coding stage.

For this reason, we looked into several tools to test out the gRPC API we use under conditions very similar to the actual service environment. Comparing each of them, we came to choose Locust because it can be easily tested in a single system based on the test case, particularly the Kubernetes (k8s) environment.

Creating the Load Test Environment

First, the Locust environment must be set to write the test case. Since it should be written in Python, the program and gRPC dependencies library must be installed.

Installing the Python gRPC Dependencies Library

# Python upgrade python >= 3.6
> python3 -m pip install --upgrade pip

# install grpcio 
> python3 -m pip install grpcio

# install grpcio-tools
> python3 -m pip install grpcio-tools

Compiling the Protobuf for gRPC Test Object

# compile proto file

python3 -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. proto/v1/user/user.proto
python3 -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. proto/v1/feed/feed.proto

* If you want to find more about the tutorial, click the link on the right. ▶ Python gRPC Basics tutorial

Writing the Test Case File for gRPC

import sys
import grpc
import inspect
import time
import gevent

# Libs
from locust.contrib.fasthttp import FastHttpUser
from locust import task, events, constant, between
from locust.runners import STATE_STOPPING, STATE_STOPPED, STATE_CLEANUP, WorkerRunner
from proto.v1.user import user_pb2
from proto.v1.user import user_pb2_grpc
from proto.v1.feed import feed_pb2
from proto.v1.feed import feed_pb2_grpc

def stopwatch(func):

    def wrapper(*args, **kwargs):
        # import function name
        previous_frame = inspect.currentframe().f_back
        _, _, task_name, _, _ = inspect.getframeinfo(previous_frame)

        start = time.time()
        result = None
        try:
            result = func(*args, **kwargs)
        except Exception as e:
            total = int((time.time() - start) * 1000)
            events.request_failure.fire(request_type="TYPE",
                                        name=task_name,
                                        response_time=total,
                                        response_length=0,
                                        exception=e)
        else:
            total = int((time.time() - start) * 1000)
            events.request_success.fire(request_type="TYPE",
                                        name=task_name,
                                        response_time=total,
                                        response_length=0)
        return result

    return wrapper


class GRPCMyLocust(FastHttpUser):
    host = 'https://grpc.bla-bla-비밀.com:443'
    wait_time = between(1, 5)
    # we separately added the pem file because we use the TLS for gRPC connection.
    creds = grpc.ssl_channel_credentials(open('./cert.pem', 'rb').read())
    access_token = ""
    try:
        replace_host = host.replace('https://', '')
        channel = grpc.secure_channel(replace_host, creds)
        stub = user_pb2_grpc.UserStub(channel)
        response = stub.DefaultLogin(user_pb2.DefaultLoginRequest(user_id='donis', password='1234'))
        # save after issuing the token
        access_token = response.data.access_token
    except (KeyboardInterrupt, SystemExit):
        sys.exit(0)

    def on_start(self):
        # write here if something needs to be executed before doing the @task.
        pass

    def on_stop(self):
        # write here if you something needs to be executed after the @task is finished.
        pass

    @task
    @stopwatch
    def grpc_client_task(self):
        try:
            # there is no 'http://' or 'https://' protocol name in the front for gRPC connection, so replace them.
            replace_host = self.host.replace('https://', '')
            channel = grpc.secure_channel(replace_host, self.creds)
            feed_stub = feed_pb2_grpc.FeedServiceStub(channel)

            metadata = (('authorization', self.access_token),)
            request = feed_pb2.FeedTimelineRequest(user_id='donis', sort_type='RECENT', last_feed_id='0')
            response = feed_stub.GetFeedTimeline(request=request, metadata=metadata)
            # print(response)
        except (KeyboardInterrupt, SystemExit):
            sys.exit(0)


# quit locust if it goes over fail ratio
def checker(environment):
    while not environment.runner.state in [STATE_STOPPING, STATE_STOPPED, STATE_CLEANUP]:
        time.sleep(1)
        if environment.runner.stats.total.fail_ratio > 0.2:
            print(f"fail ratio was {environment.runner.stats.total.fail_ratio}, quitting")
            environment.runner.quit()
            return


# initial setting
@events.init.add_listener
def on_locust_init(environment, **_kwargs):
    if not isinstance(environment.runner, WorkerRunner):
        gevent.spawn(checker, environment)

• Write locust.py @Task file(gRPC)
  
  

Conducting the Load Distribution Test in k8s Environment

The services of the Arbeon server are composed of Container, which is managed using k8s. In the case of Locust, it is packaged with Helm for an easy deployment in k8s, and so the load test in the k8s environment is simple.

Building the Docker Image for the Locust Test Case

# Dockerfile
FROM locustio/locust:2.10.1
RUN pip3 install grpcio
RUN pip3 install grpcio-tools

COPY . /home/locust/.

• It must be built according to the infrastructure architecture that will be deployed during the Docket Build. Since we are using MacBook M1, we built it as below:

docker docker buildx build --platform linux/amd64 -t locust-grpc:0.0.1 .

Registering the Docker Image in the Repository

• ­Push the built images to AWS ECR or Dockerhub.

docker push public.ecr.aws/blahblah/locust-grpc:0.0.1

Using the Helm Chart

• Once the cluster installment environment of the load test is created using the Helm chart, the locust cluster can be easily installed or deleted.
• Add the deliveryhero/locust available in Delivery Hero in the Helm repository and update.

helm repo add deliveryhero ""
helm repo update

Write the setting file values.yaml and Install the Locust Helm Chart

# values.yaml
loadtest:
name: grpc-loadtest
locust_locustfile: ../../home/locust/locustfile.py
master:
image: public.ecr.aws/blahblah/locust-grpc:0.0.1
worker:
image: public.ecr.aws/blahblah/locust-grpc:0.0.1
hpa:
enabled: true
minReplicas: 5

helm upgrade --install locust deliveryhero/locust -f values.yaml

deliveryhero/locust installed!

 locust access screen

• In Locust, the address starting with “http://” or “https://” must be inserted when entering the host (For this reason, we inserted the logic to replace the host address when we wrote the code for the locust.py test).
  
  
  

Conclusion: The Result of the Load Test

The Test Result of Locust Using MacBook

• The user does not go above 100 in MacBook (CPU M1, Memory 16GB).

The Result of Test Using Locust After Composing the On-Premise k8s 

• The worker node (CPU 6 Core, Memory 32GB) was able to test up to 750 users in 1 single k8s.

We have uploaded the test results as attachments below, so if you want to find out more about it, please see and check them out!

Today, we looked at the gRPC test. We hope that more services will make use of gRPC, and test and verify their apps using this method. If you have comments or questions, or if you found any errors in it, please send an email to media@arbeon.com.

Thank you! 

Ref :
- locust.io
- Example: writing a gRPC User/client
- How to Load Test gRPC Applications With Locust.io in Python
- Load testing your workload running on Amazon EKS with Locust

Test results