10 August 2022
Target audience: junior-mid engineers. Or senior engineers who just want to learn about my thought process. It's okay to use this document for our interview process. I haven't copied things from Google. Still, keep in mind that I've put quite some thought into this before writing, calm and relaxed at home, so I may not be able to recite on the spot, under pressure, some of the things that I've written. There could also be small inconsistencies because I'm learning all the time and my views change with learning.
Stick with the tech stack you/your team is already used to. That is assuming you haven't picked some unusual stack such as Haskell backend and some transpile-to-js frontend or Arch-based server for production. I like Haskell, I've heard very good things about ClojureScript(transpile-to-js Clojure), and I know that some of the most techie people use Arch distros successfully, without that slowing them down. But I do know for sure that this is not scalable. Not scalable to many people, I can't say it's not scalable to many machines.
DO pick Rust/Haskell/Clojure/OCAML/transpile-to-js/wasm/any of the non-established/less-known tech-stack if you'd like your tech-stack to filter for passionate developers by itself. Or if you are a manager who trusts their engineers, and wants to "spoil" them a little bit.
DO study Rust/Haskell/other tech-stacks that you might not use in production. Look at them as thinking frameworks. E.g. learning about Elixir/Phoenix framework will make you think more about building concurrent backend services. If you're coming from Python and you try Typescript, you will learn how much easier/safer refactoring can be, and vice versa. Reading about Rust might introduce you to some system programming concepts. Learning about Linux might make you appreciate Windows (ha-ha). Learning Golang might make you question what makes a programming language good. There are very interesting discussions e.g. on Quora from functional programmers that give a very solid critique of Golang. At the same time, there are a lot of people with many years of experience, functional programming included, saying that Go is a breath of fresh air. Either way, it will make you understand better programming languages and their ecosystems.
TLDR: Here I've listed repositories that no matter how hard I try to explain with my own words, it's still going to be a massacre compared with their quality. It's good to know your limits :)
You probably shouldn't be worried about performance from the start. There are more important problems to figure out at first. Yet, if you know for sure that your server/app might need some lower-level optimizations, you can still isolate these into separate services and pick a language known for its performance, e.g. Rust, C/++, Go, etc. Still, build your main business logic in the tech-stack that is well-known/high(er) level that is easy to use by more junior developers. Then do REST/RPC/Graphql calls to the high-performance service.
Or if performance is needed for a web-based frontend app, you might try to build that part of the app with e.g Rust(yew framework) and try connecting it via micro-front-ends architecture to your main React/Angular app. I'm not very familiar with the micro-front-ends concept. It may be unreasonable but it's worth exploring before building your whole front-end app in Rust. Btw I like Rust. Or any other language that forces the developer to think hard about what he/she is doing.
TLDR: Stick to the most used/popular editor for your environment.
Neovim, I've spent enough time, around 100h, using/configuring. Now I know that I might come back to use it in the future and that 100h weren't enough to become comfortable using it for my daily work. Though your mileage may vary, I believe that for most people the 100h mark is a sure thing, to be able to understand the editor/configs but not feel comfortable using it for daily work. There are always exceptions, I know.
Emacs, performance is slow for nowadays standards and its dev community is not stable, i.e. slow to fix bugs and introduce new features. I've heard that fewer and fewer people want to work on their codebase.
DO try (neo)Vim extensions for your editor. E.g. VSCodeVim.
Update: I've been actively using LunarVim for a few weeks. I'm slowly becoming more comfortable but I've already invested at least a couple hundred hours in the neo/vim ecosystem. And I still waste a lot of time configuring. Even though Lunarvim comes with saner defaults, once things don't work, you're expected to know even more than if you did a small, simpe config yourself. Compared to VSCode, where I just went into it and understood pretty much everything I needed to in just a few days, the difference is huge.
Update2: No more. 6 months was enough. Biggest time sink, really. Sunk-cost fallacy on steroids. "But, it makes you a better developer", yeah, but you can become a better developer by solving important real world problems with software, and not fighting with your editor. Having fear to udpate because you don't know what will break. Having to read source code for editor packages because things break unexpectedly. Spending half your time in github issues, just not for your own projects but for basic editor features that are supposed to work.
For me, at that point, the conclusion is that Neovim(plus all its frameworks like Lunarvim) are just toys for people who don't have more important things to do. They break way too much and in order to be able to tap into their power, you have to remember a lot. And remember it over time, which is even more difficult. I also found it wasteful because I only have so much memory, and I prefer not to waste it on my editor's internals. One of the worst time waste of my life.
TLDR: when it's not obvious why a technical decision has been made. Not explaining what the code does. Or, explain what code does if it's really difficult to write best_practices_code, e.g. performance optimizations, working around library problems, etc. MUST explain regexes.
TLDR: do not bother switching unless you have at least 20-40 extra hours to play with it for the first few weeks or even months. The benefits come after at least a month or two of an uphill investment with a lot of effort, and unexpected issues. Assume the slogan "things don't work by default" and you'll be well prepared. Yes, I'm talking about stable distros such as Ubuntu, Mint, and related.
How is it better?
Also, it's not hard to find forum threads/youtube videos explaining every little detail of how Linux is better. Read/watch youtube at least some of the most popular ones, until you get a feeling that what you're hearing/reading is repetitive. Then you can kind of say that you know fairly well the benefits. The Linux guys are good at pointing them out. Then it's time to get to know the bad stuff, ha-ha.
How is it worse than Windows?
All that said, it's still useful to at least try Linux through WSL or a virtual machine, even if you're not planning on switching to Linux.
TLDR: it's not. It's bad for the majority of problems/teams because the potential benefits that it introduces rarely outweigh the risks/bugs/problems/incompatibilities with how things are already working, and bad/no documentation for the new features. Say goodbye to Googling because you'll often be the first one to encounter the issues.
A situation to study is Discord migrating one of their services from Golang to Rust(latest). The interesting thing is that in the discussion threads reasonable comments were saying that they could have avoided the migration. And that's for Discord.. with millions of users of their service/s where they might need a specific bleeding edge tech/feature. So, if it had been unreasonable to use bleeding edge tech for an app, that has a lot of unique needs compared to the average app, it's probably unreasonable for most of the other apps, too. It's a generalization, I know. But I find it interesting to think about.
Good reason: the people that work with Java are usually older, with different life priorities than the ambitious, highly motiavated developers. Or in other words, many feel the ecosystem/people there are a little bit stagnant, too bureaucratic, not allowing them to experiment and learn new things. People feeling they don't get to build enough in order to learn. Having a lot of huge codebases to maintain by just changing very little pieces of code.
Reasonable reason: job opportunities don't seem to go in the direction of more Java.
Bad reason: overly eager developers needing to scratch their brain with constantly changing technologies such as those in the front-end ecosystem, not appreciating the stability and maturity of Java, and don't solve important problems but rather like to just build small projects with new tools.
TLDR: When you have a lot of client-side state that is not just a copy/cache of data from the backend. Or state that has a lot of variables that you don't want/need/can't sync often with the backend. When state mostly resembles data from the backend, then ReactQuery, Apollo, or urql should be sufficient with some react context.
Angular is good if you want a lot of tools/structure/opinions out of the box without having to look at many places and wonder if you're missing out on a lot.
Angular is not good for junior(ish) developers that need to be productive fast. Also if you need flexibility, doing non-standard things, plugging a lot of 3rd party libraries. It's not that it cannot be done with Angular, it's just more intuitive/simple to do so with e.g. React. With React this is the way you're expected to work.
The boilerplate is IMO both good and bad. Bad for obvious reasons that there are more things to read and maintain, therefore more things to make mistakes on. Unnecessary if you're building small projects. Good because it's not just random boilerplate, it's usually boilerplate that gives an extra opinionated structure that is helpful as the project scales. Boilerplate that once you begin to think about how to modularize your React app, to make it more scalable, you end up inventing yourself. E.g. all the seemingly unnecessary creating of modules for everything. There is work done to simplify their modules structure to be more like React, more beginner friendly.
The best reason for React nowadays(2022) is that it's become an industry standard. You can't escape it. Not that it's bad. It's just that it's unreasonable to not learn it if you want to do serious front-end work.
Vue is a mixture between React and Angular with the best of both. I can't think of any serious disadvantages from React, except that its ecosystem is smaller, which might be an issue if you stumble upon edge cases. It's said to be more beginner friendly than both React and Angular, although I can't confirm.
Googling react vs vue(reddit) and going through the first google page results can give us good reasons why we might not want to use vue. TLDR: Typescript support feels like an afterthought. Yes, even for vue 3.
TLDR: before .NET core, the main good reason to pick Java over .NET was cross-platform compatibility.
The safest/easiest solution is to use some service like auth0, or Firebase's auth. You don't need to use Firebase to use its auth features. May get costly after a few thousand users, so I'd check their pricing pages. By safest I mean safe from developer mistakes and less maintenance. Nowadays with all the political polarities, such services might not be safe if you don't share their political views.
The second best, and still a standard, is to roll up your own authentication service, using the most established libraries for your ecosystem.
Questions to consider:
TLDR: I think that relational DBs fit enough use-cases in order not to overthink it. Relational DBs can also store unstructured data if it's required. Unless you expect tens of thousands to a million requests per second(very rare), SQL DBs force you to structure your data, give it types, and think about its future use much more than NoSQL. IMO, NoSQL often encourages sloppy work from developers because the initial effort is close to nothing. This leads to more difficult querying down the road, and more difficult ensuring that the data is consistent(because NoSQL doesn't encourage normalization).
And even if you expect millions of requests per second, you can still separate that part of your app with a microservice that uses NoSQL. Just because a few parts of the app need this higher performance, doesn't justify the whole app using NoSQL. In my research, I've found companies with huge user bases scaling SQL DBs for their services. Most companies don't have user bases in the millions/billions. That said...
a good case for NoSQL that I can think of right now is chat apps. They need to be able to handle millions of requests per second. Having the accuracy of financial software is not a priority.
NoSQL is faster both in terms of performance and initial development speed
NoSQL has better horizontal(across many machines) scaling
NoSQL breaks ACID principles. E.g. not suitable for financial software
NewSQL is relational DBs designed to scale horizontally, although I haven't researched how they work. There might be marketing gotchas
TLDR: in the js ecosystem, as of August 2022, I can't find anything better documented than Prisma 2. Its Typescript support is also said to be superior out of the box. Yes, I know about the controversy from their v1 migration, where people were just left there with no (easy) way to migrate. I'd put Objection.js second. It's much more barebones, it's built on top of Knex.js, with better Typescript support out of the box.
I think although ORMs have become very complex, they still save enough boilerplate writing for the majority, let's say 90% of queries. And almost every advanced ORM(not just .js ecosystem) allows its users to fall to raw SQL when needed. A lot of flexibility is lost by not directly writing SQL, so for that, there are barebones ORMs like Knex.js and Objection.js. They still keep most of the flexibility of writing SQL but feel like you're still doing regular programming.
TLDR: Please pick Typescript for any project longer than 5k LOC. Or less than 5k but where you'd expect to come back in the future and remember your objects/functions.
TS' biggest strengths are refactoring and not having to read your colleagues' objects/intentions for what objects/functions are supposed to have and not have. I'd have said intellisense a few years back, but nowadays with VSCode and/or its LSP(Language Server Protocol) with e.g. Neovim, the editor can give you very good hints for .js files as well.
Frontend is easier to begin with and more difficult as the project grows. Engineers are predisposed to not think about the complexity in advance. There are still not a lot of standardized practices, except for maybe Angular on the frontend about how an app should be built.
Backend is much more difficult, to begin with, but once you get the hang of it, it feels a lot more consistent and stable. Easier to scale not because it's easy but because best practices have emerged from decades of tinkering that somehow do NOT change like the ones on the frontend.
DevOps, for me, requires the most upfront investment to grasp. It's a strange mixture of bad coding practices with a lot of trying to get used to commands and their options. That said, DevOps and backend feel like the natural progression of my software engineer interests. It feels like I need to study harder upfront but once I start getting used to it, it becomes easier. In contrast to frontend, it feels very easy at first, but the more I start learning how to do it properly and consistently(professionally), not just the tools, but the ecosystem as a whole, the more difficult it feels. We'll see.
TLDR: learn some FP even though you might not be using it a lot in real projects. A lot of the new ideas in languages come from FP, such as lambda functions, Promise object is also said to originate from FP.
OOP is not dead. Although it's said to be very far from what its originator Alan Kay had in mind, it's still an abstraction that is heavily used in industry. IMHO as far as one values composition over inheritance, it's not too bad. Smalltalk is said to be a good(though old, and not used in production) language to study the original intent behind OOP. Learn Smalltalk in Y minutes. FP repositories I've found interesting
TLDR: Pick Python if you don't want(you/your team) to have to learn and remember a lot of new syntax. Don't pick Python if importing its standard library for very tiny tasks might be an issue e.g. some Raspberry-Pi-like embedded hardware or small Docker instances(though it's still probably fine)
Pick Bash for really small scripts, less than 30 lines of code. Or if you're proficient with it, and know that no other is going to work with your scripts.
Pick Perl if you want a powerful language that can succinctly do a lot within a few lines of code, and you don't mind that it's been used less and less in the industry. Perl can be used as a substitute for Bash because of its succinctness, and it has been used as Bash's substitute for some time in the past. Perl is said to play excellently with regex and plain text manipulations. Perl stands for - Practical extraction and report language.
Pick Zsh if you want a little better command-line completion and awareness. Bash is said that it can imitate a lot/everything that Zsh can do with plugins as well. But even if it weren't, for me, the benefits of Zsh are too few/little to deviate from something as established as Bash. Although Zsh is said to be POSIX compatible and Bash compatible, there are still differences that you have to remember. It may be worth it for sys-admins who spend many hours in the terminal.
Pick Fish, if you want to play with a shell that is better than Bash, often compared in quality to the highly praised Powershell. Not POSIX compatible.
Pick PowerShell if you're going to do Windows sys-admin/DevOps tasks. It works better with Windows(it's designed specifically to do so, while trying to avoid some of the problems with Bash such as input/output being plain next, as opposed to structured objects). IMO, even though it's said the be better than Bash, it still doesn't work as well with Linux as Bash does. The whole Linux ecosystem is expected to work with Bash(something like it). Plus the ecosystem around Bash is much bigger.
With smaller cloud providers pricing is much more straightforward. Much less time is spent on studying their services, more time spent building your product. With AWS and its competitors like Azure and Google Cloud, it's almost overwhelming with all their details and options to choose from, always leaving you slightly insecure if you've picked the right option.
TLDR: as long as you don't have a lot of business logic that should be hidden from the client, BAAS should be sufficient for your needs. If you have a little business logic, you can still use the BAAS provider's cloud functions. Not good if you have a lot of data transformations with the DB data because the transformations will end up being executed in the client(usually a browser with much fewer resources than a dedicated server). You can still use cloud functions for that as well, but IMO, if you end up using BAAS with cloud functions more and more, you're just better off not using BAAS services and building your own thing, be it again deployed with serverless functions or a monolithic server.
TLDR: start with Golang. In case you end up needing to manually manage the memory in some part of the server/app, you can isolate that part into a separate service/s(with e.g. Rust) and still build the majority with Golang, which is fast enough but way less complexity.