ess-etl-go/benchmark_report/Report.md

Benchmark results

I’ve decided to compare same behavior done with 2 different technologies: Go vs NestJS

Context

I’ve worked with NestJS in some projects. We build a DeFi platform using NestJS+TypeORM+Postgres and project was pretty good. We had to use some raw SQL queries to perform some special tasks like obtain full network (referral program). After working with go since April 2020 I’ve found a super fast tool in terms of: 1- development, 2- clean code, 3- performance and list could continue.

Description

I have to perform an ETL taks that will be fetching data from AMS xml API, then parse into some readable Struct/Class therefore store into some database. Database here is really important to acquire a high performance fast-response application.

Development spent time

go

Just using go struct tags I could complete this task in just 4h in just 5 lines of code to load XML and 45 lines to define my domain entities (structs)

I spent around 3-4 days working with Nest+ Apollo+MikroORM and I found really tough this task in terms of development spent time. Every time I need to add 1 console.log I had to restart server and it took around 15 and 20 seconds to execute npm start which is nest start I found this command what it does internally is first to compile typescript into javascript therefore run main.js from dist folder.

nest + graphql

GraphQL api + ETL on Nestjs took me 3-4 days do some pairing with my team mates and
Full restAPI + ETL process Go took me 4h to develop everything without any trouble. I found quite hard using graphQL as I find selected architecture quite hard to comprehend and navigate with decission. I know if you are building a multi-tenant library or service is ok to make do more unit test, to pass mocks and use more interfaces: which is one of the most used things in go.

I also would recommend to read this blog which I found really interesting point of view about graphQL and it's real use case.

XML issues

Standard go library is pretty easy to use as you will see in the following example of the benchmark test application

Performance

startup

NestJS application takes between 15 and 25 seconds to startup. I’ve created a bash script that counts the whole process since you run npm start until API is reachable

go API server startup takes couple milliseconds to start up

ETL

For this test I’ve decided to make 1 AMS API request for 1 employee for 1 month nest I’ve made a jest test: first test is to fetch database. We can see it takes around 10850ms

go With go the whole process takes 2613ms

AMS direct request

I didn’t go further with this comparison as I’ve seen direct request to SITA is not worth. On previous nest test execution we can see how AMS takes 1188ms to retrieve 1 employee’s work information for 1 day.

Process 1000 request sent at same time

I’ve created a go project that is responsible to send 1000 request at same time and wait until last response is sent by server to calculate how much time server took to handle 1000 requests at same time for GET operation. Well, in NestJS case it was POST itself because it is graphql but is a request to retrieve data. I’ve added an operation to write each response into a log file to more or less simulate a real request from a browser client.

As we can see go server served using fiber framework takes 2874ms to handle 1000 requests. NestJS graphql took 40612ms to handle all requests. There is a huge difference: go is 14.13 times faster 40612/2874 than NestJS+Grahpql

Testing each one isolated we can see that: nest takes 16417ms to handle 1000 requests

go takes 1410ms to handle 1000 requests this means go is 11.64 times faster than nest handling requests

Process 5000 request sent at same time

Testing each one isolated we can see that: nest takes 75536ms to handle 5000 requests

go takes 6191ms to handle 5000 requests this means go is 12.200 times faster than nest handling requests

1000 requests per second on 3 seconds duration

nest

go

1000 requests per second on 5 seconds duration

nest it tooks really long to handle all requests. These are some output results. I had to manually stop the test because It was going to crash my machine.

go the first request is just 5 seconds, and then it increases up to 20 seconds due to resources. But handling this traffic with this response times, with 4-6h of development is insane.

1000 requests per second on 10 seconds duration

nest

+---+---------------------------+------------+
| # | DESCRIPTION               | ELAPSED    |
+---+---------------------------+------------+
| 0 | nest handle 1000 response | 9842ms !!! |
+---+---------------------------+------------+
|   | TOTAL                     | 9842MS !!! |
+---+---------------------------+------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 23957ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 23957MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 33049ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 33049MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 34006ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 34006MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 40252ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 40252MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 46852ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 46852MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 47458ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 47458MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 54965ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 54965MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 57436ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 57436MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 59447ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 59447MS !!! |
+---+---------------------------+-------------+
PASS

go

+---+-------------------------+------------+
| # | DESCRIPTION             | ELAPSED    |
+---+-------------------------+------------+
| 0 | go handle 1000 response | 7918ms !!! |
+---+-------------------------+------------+
|   | TOTAL                   | 7918MS !!! |
+---+-------------------------+------------+
+---+-------------------------+------------+
| # | DESCRIPTION             | ELAPSED    |
+---+-------------------------+------------+
| 0 | go handle 1000 response | 7595ms !!! |
+---+-------------------------+------------+
|   | TOTAL                   | 7595MS !!! |
+---+-------------------------+------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 14349ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 14349MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 14500ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 14500MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 13338ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 13338MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 12395ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 12395MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 12966ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 12966MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 16075ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 16075MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 13120ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 13120MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 12161ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 12161MS !!! |
+---+-------------------------+-------------+
PASS

1000 requests per second on 15 seconds duration

nest it took really long to handle all requests. These are some output results.

+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 17291ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 17291MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 38445ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 38445MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 52656ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 52656MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 80340ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 80340MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 88219ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 88219MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 84223ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 84223MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 95468ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 95468MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 96851ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 96851MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 96769ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 96769MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+--------------+
| # | DESCRIPTION               | ELAPSED      |
+---+---------------------------+--------------+
| 0 | nest handle 1000 response | 100684ms !!! |
+---+---------------------------+--------------+
|   | TOTAL                     | 100684MS !!! |
+---+---------------------------+--------------+
+---+---------------------------+--------------+
| # | DESCRIPTION               | ELAPSED      |
+---+---------------------------+--------------+
| 0 | nest handle 1000 response | 102292ms !!! |
+---+---------------------------+--------------+
|   | TOTAL                     | 102292MS !!! |
+---+---------------------------+--------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 99374ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 99374MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+--------------+
| # | DESCRIPTION               | ELAPSED      |
+---+---------------------------+--------------+
| 0 | nest handle 1000 response | 103381ms !!! |
+---+---------------------------+--------------+
|   | TOTAL                     | 103381MS !!! |
+---+---------------------------+--------------+
+---+---------------------------+-------------+
| # | DESCRIPTION               | ELAPSED     |
+---+---------------------------+-------------+
| 0 | nest handle 1000 response | 98271ms !!! |
+---+---------------------------+-------------+
|   | TOTAL                     | 98271MS !!! |
+---+---------------------------+-------------+
+---+---------------------------+--------------+
| # | DESCRIPTION               | ELAPSED      |
+---+---------------------------+--------------+
| 0 | nest handle 1000 response | 100401ms !!! |
+---+---------------------------+--------------+
|   | TOTAL                     | 100401MS !!! |
+---+---------------------------+--------------+
PASS

go the first request is less than 5 seconds, and then it increases up to 20 seconds due to resources. But handling this traffic with this response times, with 4-6h of development is insane.

+---+-------------------------+------------+
| # | DESCRIPTION             | ELAPSED    |
+---+-------------------------+------------+
| 0 | go handle 1000 response | 4760ms !!! |
+---+-------------------------+------------+
|   | TOTAL                   | 4760MS !!! |
+---+-------------------------+------------+
+---+-------------------------+------------+
| # | DESCRIPTION             | ELAPSED    |
+---+-------------------------+------------+
| 0 | go handle 1000 response | 6252ms !!! |
+---+-------------------------+------------+
|   | TOTAL                   | 6252MS !!! |
+---+-------------------------+------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 10400ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 10400MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18328ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18328MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 17876ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 17876MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 20245ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 20245MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18277ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18277MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 17435ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 17435MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18705ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18705MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 19156ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 19156MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18544ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18544MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18976ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18976MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 24030ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 24030MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 18129ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 18129MS !!! |
+---+-------------------------+-------------+
+---+-------------------------+-------------+
| # | DESCRIPTION             | ELAPSED     |
+---+-------------------------+-------------+
| 0 | go handle 1000 response | 17274ms !!! |
+---+-------------------------+-------------+
|   | TOTAL                   | 17274MS !!! |
+---+-------------------------+-------------+
PASS

Other machine

I’ve asked my teammate Jeffer to please run these tests in his machine, to compare different bandwidth latency and machine performance. As we saw he couldn’t launch 1000 requests due to nest was overloading then crashed, but also while go was processing he get some error we didn’t have time to inspect further. Anyway, here are results

300 requests per second on 15 seconds duration

nest

+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 2712ms .!! |
+---+--------------------------+------------+
|   | TOTAL                    | 2712MS .!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 4633ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 4633MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 5346ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 5346MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 4444ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 4444MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 4757ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 4757MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 6506ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 6506MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 7631ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 7631MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 7705ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 7705MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 5511ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 5511MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 6329ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 6329MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 7177ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 7177MS !!! |
+---+--------------------------+------------+
+---+--------------------------+-------------+
| # | DESCRIPTION              | ELAPSED     |
+---+--------------------------+-------------+
| 0 | nest handle 300 response | 11947ms !!! |
+---+--------------------------+-------------+
|   | TOTAL                    | 11947MS !!! |
+---+--------------------------+-------------+
+---+--------------------------+-------------+
| # | DESCRIPTION              | ELAPSED     |
+---+--------------------------+-------------+
| 0 | nest handle 300 response | 11275ms !!! |
+---+--------------------------+-------------+
|   | TOTAL                    | 11275MS !!! |
+---+--------------------------+-------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 8611ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 8611MS !!! |
+---+--------------------------+------------+
+---+--------------------------+------------+
| # | DESCRIPTION              | ELAPSED    |
+---+--------------------------+------------+
| 0 | nest handle 300 response | 7731ms !!! |
+---+--------------------------+------------+
|   | TOTAL                    | 7731MS !!! |
+---+--------------------------+------------+

go

+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 562ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 562MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 471ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 471MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 437ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 437MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 410ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 410MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 469ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 469MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 450ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 450MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 494ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 494MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 616ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 616MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 541ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 541MS ... |
+---+------------------------+-----------+
+---+------------------------+------------+
| # | DESCRIPTION            | ELAPSED    |
+---+------------------------+------------+
| 0 | go handle 300 response | 1398ms ..! |
+---+------------------------+------------+
|   | TOTAL                  | 1398MS ..! |
+---+------------------------+------------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 803ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 803MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 688ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 688MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 626ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 626MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 580ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 580MS ... |
+---+------------------------+-----------+
+---+------------------------+-----------+
| # | DESCRIPTION            | ELAPSED   |
+---+------------------------+-----------+
| 0 | go handle 300 response | 449ms ... |
+---+------------------------+-----------+
|   | TOTAL                  | 449MS ... |
+---+------------------------+-----------+

If we compare results, for example taking last execution time we can see nest takes 7731ms and go takes just 449ms. If we divide 7731/449 we get 17.218 which means time difference between two process.

Summary

As we can see comparing these results that go is the most efficient language to handle this kind of behavior. I would recommend to sit down and rethink about software needs, integrity, scalability, etc…

go benchmark

Using go test -bench mode we are able to test both servers at same time and compare results. We can see go test can handle 1 operation in 4.42ms which is a great response time while nest is 3 times more 13.94ms

Database used

I’ve hardly cannot finish developing with Nest as I have to do a lot of manual class mapping through 3 different classes. I couldn’t spent more time to compare databases as I think could be something to discuss later after some previous requirements has been defined.

Local Mongo

I only had time to test against mongo as it is the fastest to store everything into single table without having to connect to cloud.

Source code

Nest https://github.com/urko-iberia/ess-etl-nest Go https://github.com/urko-iberia/ess-etl-go