Documaster: How to Build an Infrastructure Tech Stack That Scales Well as the Business Grows
Startups launch with one big bag of unknowns and yet these companies are ‘designed to grow fast’ as the American entrepreneur and venture capitalist Paul Graham notes. On the one hand, they have to manage scarce resources and really focus on their day-to-day businesses, but on the other, they cannot afford to lose sight of the big picture and ignore the path to scalability in the long run. When talking about scalable startups, the community is often centering on the business model, which while no doubt important, is only part of the success equation. What happens if a company starts growing extremely fast but its technological infrastructure is not able to support the big demand? Or, if it just turns out that a given tech stack is great for a company with five customers but it will have to evolve significantly in order to support 500?
In this story, together with Documaster, a fast-growing Norwegian-Bulgarian records and document management startup, Trending Topics SEE is going to explore this fundamental question: “How did Documaster’s engineering team manage to build an infrastructure tech stack that scaled well as the company expanded?” We talked to Documaster co-founder and CTO Dimitar Ouzounov and infrastructure lead Delcho Tuhchiev. Among the main discussion-points were the technology decisions at the company’s beginning, which paid off later, how Documaster’s tech stack evolved since its first days in 2014, and the role of automation in helping the company reach the next level of massive scale.
How it all started
“In the earliest days of Documaster, it was just me and one more developer, so we had to focus on the technologies we already knew – Java and PHP/Javascript. Our first customers used to give us access to Linux servers and with every new release, we had to log in, install, and configure the software manually. It worked quite well with the three clients we had back then. However, at one point, we saw that this approach doesn’t scale at all,” remembers Dimitar.
When it comes to infrastructure technology, there are three fundamental decisions the engineering team of Documaster took after that moment, and which yielded great results down the road.
The first one was the realization that the company will need to host the product as a service and ultimately build its own cloud (partly because in Scandinavia data from public institutions need to remain inside the borders of the country but also because public clouds didn’t satisfy all of the non-functional requirements of the software.)
Here the big challenge was finding a knowledgeable partner in Scandinavia to deliver, configure, and operate physical servers and other hardware. After finding a suitable partner, Documaster started experimenting with LXC containers running on top of physical servers.
Fit to requirements
The second important decision was the adoption of suitable production-ready technologies even if they weren’t the most popular option on the market.
“When I started working at Documaster, I was skeptical about LXC containers at first as it was a relatively new technology at the time. However, after doing the research, I quickly changed my mind in regard to its advantages over virtual machines in some scenarios. At that time we decided to also look into LXD containers. We needed to select a technology that’s both stable and reliable but at the same time flexible,” shares Delcho.
They reached the conclusion that LXD containers are better than LXC and a much less resource-intensive alternative to virtual machines – you can put more containers with the same software on one host – respectively the cost of running Documaster instances goes down. In addition, the integration of LXD with the ZFS file system allowed Documaster to develop a powerful mechanism for managing backups.
So, in the end, the external provider was responsible only for the hardware while Documaster put the entire software layer on top – operating system, containers, backups management, etc… thus creating their own private cloud.
“Looking back, if we had chosen to use a public cloud, we wouldn’t have been able to scale as much. In terms of price, quality, management – for us, a private cloud was the best fit. Documaster works with data so it was very important that we had full control over where and how the data is stored, how it is backed up, and whether we can directly take care of any problems that arise,” tells us Dimitar.
With about 700 containers in 2020, Documaster’s private cloud has grown quite a lot over the years. But it wasn’t all that easy in the beginning.
Even the best technology needs the right communication
When non-technical people hear the word ‘cloud’, they often first think of the public cloud – the services provided by AWS, Microsoft Azure, Google. This turns out to be a problem when your customers are public sector institutions for whom the public cloud used to sound like a scary place for document and records management.
Early on, some clients opted to install Documaster on-premises, even though it was better for them to have the software running in the Documaster cloud and provided as a service.
At that moment came important decision number three: the realization that ‘cloud’ is not a term well-perceived by customers. Thus – the introduction of the ‘secure cloud’ concept – having customers connect their premises to a highly secured and dedicated section (sometimes running on dedicated hardware) of the Documaster cloud over a site-to-site VPN.
Ultimately, one by one, many public sector organizations see that all works well in the Documaster cloud and move away from on-premises installations.
Selecting the right stack
“The most important thing we always do when selecting a new technology is to have clear requirements, a vision of what our end goal is and what we want to achieve. From there on it’s mostly a matter of research,” states Delcho.
He and everyone at Documaster are guided by several key principles when doing this research:
- Whether a given technology is a good fit for the problem at hand
- Whether the technology is production-ready and reliable
- And whether it has a big open-source community behind it as this can show whether the technology would still exist in a few years.
“Maturity is also important, some engineers like to throw themselves on bleeding edge tech, but often they don’t think about how it will be used in the years to come. It can be very cool now, but if after 2 years there is no one to support it and the technology dies, then your company will face big problems,” adds Dimitar.
Only with automation can a company truly scale
As the business grew, the team of Documaster also saw the need for faster and more automated processes. The logical choice they made was to implement Ansible to enable automated product deployment and configuration.
“Only with full automation can a company truly scale. If even a small piece of the deployment and configuration process is not automated, then we are still wasting too much time on manual work. At the moment, we are moving towards eliminating the last remnants of things that are done manually,” shares Dimitar.
At the same time, there is still lots of space for innovation. Until recently, Documaster has been focusing on the public sector – currently, the company has over 250 customers from this segment and has grown to address their needs and requirements very well.
Now, Documaster is starting to provide document and records management solutions for businesses as well, and the team is working on developing solutions for new interesting infrastructural challenges – for example, a client may have huge volumes of data or lots of users who need to work with the system at the same time. Moving beyond the first five corporate customers, automation will once again play an important role in helping the business reach the next level of scale.
With that said, given Documaster’s strong growth journey, it looks like there will still be plenty of interesting and meaningful work for infrastructure engineers. As Delcho tells us: “Infrastructure is at the core of a successful business. With a solid foundation, you have an excellent opportunity to expand.”
