Building composable serverless architectures with Fauna, Netlify, and Big Nerd Ranch

This is a community post authored by Richard Flosi, Principal Architect at Big Nerd Ranch, and originally hosted on the Big Nerd Ranch blog.

Serverless solutions are empowering small teams to do more and focus on their business. One of the principles of software development is to keep things DRY, or “Don’t Repeat Yourself.” Software as a Service (SaaS) is an iteration on this principle with additional benefits. By migrating to managed serverless infrastructures, engineering teams can reduce operational management, allowing more time for product development. Serverless infrastructures provide scaling on-demand to grow with business needs and the distributed systems lower network latency for users. These elements work together to create superior end-user experiences since the business can focus on product, has automated scaling, and is delivering content from the user’s region. Clients are requesting serverless architectures to build for scalability and longevity while reduced costs. Composable solutions increase development velocity thus increasing business outcomes while delighting developers and end-users.

Architectural Considerations

When making architectural decisions, engineers need to consider performance impacts as well. Given the on-demand nature of serverless architectures, engineers need to consider cold start times as resources are being provisioned, long running processes that exceed serverless runtime limits, network overhead of requests to external resources, and the possibility of vendor lock-in. Depending on an application’s needs and usage, a dedicated server may be less expensive or more performant for some or all of the infrastructure. When making these decisions consider who uses the application, where they are located, and how these trade-offs impact the business needs.

Common Application Needs

Building and deploying a web application typically requires a host to serve the application code, authentication to allow users into the system, and a database to store and retrieve data from. In the serverless realm, static files from the application build step are often deployed to a CDN to provide high availability to users; application code that needs to run in a server environment can be deployed to serverless functions that run on-demand. Authenticating a user can be solved by integrating an Identity Provider (IdP) to manage users, provide login flows, and return an access token for a user’s session. Serverless databases replace the need for hosting and scaling a data storage solution. From this base, additional composable solutions can be integrated to add functionality. This may include serverless solutions for long running background jobs or scheduled cron jobs, running tests for continuous integration, optimizing images or video, managing content, implementing a search engine, payment processing, geocoding, web sockets, or other services. What these elements have in common is the fact that they are all APIs that can be composed and work together to provide rich user experiences while easing development overhead and getting a product to market sooner.

How to build a fully-composed web application

Building a fully-composed web application will end up looking like a mesh of API calls to external systems, which combine results to implement your business needs. The next few sections will dive into specific technologies that can be used to implement different parts of your system. While there are many options to choose from, in this blog post, I’ll be focusing on the technologies I use on a regular basis.

Deploying static assets to a Content Delivery Network (CDN)

If you are building a purely static website, then your build step will produce a set of static files that can be deployed to a Content Delivery Network (CDN). Unlike a centralized server, a CDN’s content is distributed across the world. This means content is delivered to a user from the closest node on the network, thus increasing the speed of delivery to the user. If you are building a Single Page Application (SPA), then you still have static assets that can be deployed to a CDN, allowing your application to be delivered to users by the closest network node as well. One of the solutions that Netlify provides is deploying your static assets to a CDN for you automatically. After Netlify runs your build command, the assets in your build directory are deployed to Netlify’s CDN. For a more in-depth introduction to Netlify, view Getting Started with Netlify: Creating your first Netlify Site.

Building an API with Serverless Functions

If you are building a SPA or an API without a front-end, then you’ll need somewhere for this code to run. On server-based applications, your API would run on your server(s) and would be limited to the resources of that hardware. You are responsible for managing and scaling the servers to meet the demand. Since the server is always on, this means there are also times when the resources are under-utilized. Serverless Functions provide an on-demand solution for running your code and implementing an API that doesn’t require you to manage or scale servers. Netlify Functions seamlessly deploys your API endpoints to AWS Lambdas during the build step, allowing your API to scale to user demand.

Providing Authentication with an Identity Provider (IdP)

If the site you are building requires distinct users, then you’ll need a way of managing and authenticating those users. An Identity Provider (IdP) provides authentication as a service. Auth0 is an Identity Provider that provides login flows for your application and returns an access token for the authenticated user. The access token can then be included in the Authorization header when making requests to your Serverless Functions and verified by your API.

Providing Dynamic Data with a Serverless Database

A central requirement of most applications is persistence of user data to a database. Databases tend to be a bottleneck, and are challenging to scale if you are managing them yourself with a centralized database. If multiple users are trying to write to the same database record, you may encounter deadlock while each request is waiting on the other to give up the lock preventing your users from using the system. As your database grows larger, you will need to shard or otherwise partition your data, which will make keeping the database consistent more challenging as well as querying the data across database instances. One of the challenges with Serverless Databases is the trade off between consistency and availability. Many Serverless Databases suffer from Eventual Consistency, which means that users may not always get the current version of the data. Eventual Consistency errors can lead to more complexity on the application level and increased user confusion and frustration. Luckily, there is one Serverless Database that remains ACID-compliant—that is, to have the presence of atomicity, consistency, isolation, and durability—and provides Strong Consistency, and that’s Fauna. Fauna is a unique Serverless Database allowing you to implement relational data in a schema-less document datastore while remaining ACID compliant, providing strong consistency, temporal queries, User-Defined Functions, User-Defined Roles, and an Access Provider integration with Identity Providers like Auth0. By integrating with Fauna’s Access Provider you can use the access token from Auth0 to query Fauna as that user. Learn more about Fauna’s unique offerings by reading this Intro to FaunaDB and FQL.

What about handling cron jobs and long-running processes?

Serverless Functions aren’t always enough for your needs. Sometimes you need to run code that takes longer than the 10 seconds typically allotted to Serverless Functions, or you need to run a task on a schedule like a cron job. GitHub Actions is a great option if you are already using GitHub and need this functionality. I’m currently using GitHub Actions to implement cron jobs and to process long running webhooks in the background. Netlify offers Background Functions as a beta feature and recently added Scheduled Functions as an experimental feature. I haven’t used these newer Netlify options, but am excited to try them and look forward to them being production ready.

Continuous Integration

The last piece that most projects need is a Continuous Integration (CI) solution. GitHub Actions fits in nicely here as well, assuming you are already using GitHub for your project. Since GitHub Actions is a GitHub product, it’s easy to trigger your CI workflow from git events like a pull_request or a push. Configuring a GitHub Action to run on a pull_request will also automatically include the CI workflow in the Checks section of your pull request.

What else do you need?

From here, your application needs will probably be more variable based on the project. I’ll highlight additional services that I’ve used or have had my eye on to address additional solutions you made need, but remember there are often many options to choose from and you should choose the solution that meets your needs.

• Cloudinary provides APIs for optimizing your images and videos for delivery to different devices.
• Contentful is a rich Content Management System that allows you to define models and has built-in support for internationalization.
• Algolia Search makes it easy to create a custom search engine for your content.
• Stripe provides APIs for integrating payment processing solutions.
• AWS WebSocket APIs in Amazon API Gateway provides web sockets as a service.
• HERE provides geocoding and other map related APIs.
• Plaid provides access to financial data by connecting bank accounts.

Keeping it DRY with Shared Modules

You may find yourself duplicating code in Netlify Functions, GitHub Actions, your front-end and/or other parts of your composed stack. To mitigate this I suggest creating a shared module that can be used by the various parts of the system instead. It might not be obvious what is needed in a shared module when you are starting off, but if you begin to repeat yourself, make a note of the opportunity to refactor that functionality to a shared module. One way to create such a shared module is in a separate repository. This is particularly useful if you need to share this module across multiple applications. However, having the shared module in another repository can hinder your development workflow, especially in the early stages. I like to get started with a local shared module with a longer vision of moving it to a separate repository once it’s mature enough to do so. In the early stages, it’s convenient to have a local module that doesn’t require additional management. As such, I like to leverage package.json to install a local module with something like "core": "file:./core" in dependencies. I like to keep business logic and other functional utilities in a shared module like this. Just keep in mind that each project’s needs are unique and you’ll need to decide what makes sense for your project. Explore the starter repository linked below to see how this can be used.

Front-end considerations

Likewise, on the front-end I’ve long advocated for using Web Components since they are composable and now have interoperability support in many front-end frameworks. From a composability perspective, I think building components as web components provides for the most flexibility. For additional context, read Building a Design System and Consuming it in Multiple Frameworks using Stencil.js.

Get Started

To get you started with Netlify, Fauna, GitHub Actions, Stencil, and a local shared module, I’ve created a starter repository as an example to build from. The Starter Repository is the code base that will be deployed to Netlify using the Deploy to Netlify button. The Demo Site is an example of the Netlify Site that will be deployed. Follow the instructions in the Starter Repository Read Me for the additional steps needed to create a Fauna database with demo data, generate a Fauna Server Key, and configure an Environment Variable in Netlify with your Fauna Server Key.

Have any questions or want to engage with the Fauna, Netlify, or Big Nerd Ranch communities? Reach out in the Fauna Discord channel or Fauna community forum, and also check out the Netlify Community Forums and Big Nerd Ranch programming forums.