The real problem is the choice to use Node.js for a large full-stack project in the first place
I have a mixed relationship with JavaScript. It is crucial for web development, and there is no way around it. One could use it on the backend with Node.js as well. As a result, it is possible to build a full-stack web application from start to finish in the same language. I get the appeal of it.
I could even argue that Node.js is a convenient platform for serverless architectures. For example, backend logic implemented in the form of multiple AWS Lambda functions is a good candidate for JavaScript.
However, complex large backend applications that involve many developers require additional capabilities. TypeScript adds static analysis and strong typing capabilities that JavaScript alone doesn’t have.
Herein lies the problem — The TypeScript compiler itself is written in JavaScript. It has a rich typing system, performs static checking, and transpiles TypeScript to JavaScript. Yet, it is built upon a slow single-threaded platform not meant for CPU-intensive tasks like a compilation.
Anything more extensive or complex than a few thousand lines of code becomes excruciatingly slow to compile by the TypeScript compiler. A large project worked upon by a large development team will soon discover that the TypeScript compiler itself is a massive productivity bottleneck.
I concede that I am not an expert on frontend applications, and I admit that both JavaScript and TypeScript might be the only viable options for any form of cross-platform frontend development. My focus is on complex business logic backends for enterprise applications.
A backend project involving a dozen Node modules and 20000 lines can take minutes to compile using TypeScript. If you add npm install to the mix, you will now be measuring the total time to produce deployable assets in tens of minutes. I dare you to compare that to a similarly sized project in Golang or even Java, which takes seconds to compile.
A search of StackOverflow and Google for ideas on how to speed up TypeScript compilation reveals quite a bit of MacGyvering going on in the developer community. All TypeScript compiler performance tuning techniques appear to revolve around avoiding or disabling certain parts of TypeScript.
People jump through many hoops to make TypeScript perform in 2022 on an M1 processor as the Pascal compiler did on a 386 in 1992. One of the first things that people do is disable type checking. Frontend developers need to see the results of their work, so for faster turnaround in their local IDEs, they demote type checking to a secondary background process.
There are even alternate TypeScript compilers, like swc, that don’t bother with type checking. If you are resorting to avoiding certain parts of the language, and disabling the type checking aspect of TypeScript, dare I ask why you picked TypeScript in the first place for your project?
Local development, however, is only one part of the development lifecycle. Surely, in their local IDE, the developer can rely on syntax and error highlighting to a certain extent. But once the code gets to a CICD pipeline such as Jenkins, it needs to be correctly compiled and statically analyzed, which can take dozens of minutes on a non-trivial project. Relying exclusively on their IDEs to highlight errors, developers frequently break builds and waste everyone’s time.
We need solutions.
Don’t use Node, JavaScript, and TypeScript for backends
It is interesting how Microsoft’s documentation on improving TypeScript performance states upfront that one should consider the limitations of TypeScript for large projects earlier rather than later:
There are easy ways to configure TypeScript to ensure faster compilations and editing experiences. The earlier that these practices can be adopted, the better. Beyond best-practices, there are some common techniques for investigating slow compilations/editing experiences, some common fixes, and some common ways of helping the TypeScript team investigate the issues as a last resort.
I’ll give you an easy way to make backend compilations faster. The earlier you adopt my advice, the better.
My advice: don’t use Node, JavaScript, and TypeScript for backends. I don’t buy the argument that somehow same developers can be used to build the full stack. Full-stack development in Node works for simple projects done by teams of one or two people.
If you are starting a new large project and are thinking of what platform to use for the backend, I would recommend looking into Go. Though, if you want the productivity benefits of an interpreted language and a degree of strong typing, why not Python?
If you made a wise decision not to use Node, JavaScript, or TypeScript on the backend by choosing a proper compiled language, you might stop reading right here. We can be friends for life.
Monoliths are an anti-pattern for both Node and TypeScript
If you are still reading this post, you found yourself stuck using Node and TypeScript. Perhaps like me, you have a hard time saying no to bad ideas thinking that maybe it’s politically expedient to avoid a confrontation. Trust me, I am working on that personal flaw. Sometimes it is better to confront bad ideas upfront than deal with them later.
If you do decide to stick with Node and TypeScript for backend logic, I believe you will pay for your mistake later on. You can make it less painful for yourself and your team.
The biggest problem with TypeScript is large codebases. The performance of the TypeScript compiler declines non-linearly, with each line of code added beyond maybe 5000 lines.
Large codebases are a result of monolithic architecture. We know monoliths are a terrible idea, and the TypeScript compiler makes it abundantly clear just how bad. Between the single-threaded nature of the Node event loop and the non-scalable TypeScript compiler, it is unnatural to build large monolithic backends using those tools.
Since monoliths, in general, are a terrible idea, it is better to think in terms of micro-services and modules, each of which should be no larger than 5000, maybe 10000, lines of code. As the application grows, it is worthwhile to periodically analyze the code commit hotspots and identify which microservices are growing exceedingly large.
If you prefer not to rely on the number of lines of code, I understand. Lines of code don’t reflect complexity. Then try using the time it takes to compile as a rule – if your project takes more than, say, 10 seconds to compile, it means it is getting too large for TypeScript.
Setting things up for optimal microservice performance is a topic for another discussion. I will cover it in a later post.
Final thoughts
I firmly believe that Node should not be used for complex backends. TypeScript helps, but it is lipstick on a pig. If you get stuck with Node and TypeScript on the backend, you need to avoid creating monolithic architectures.
The Dulin Report 